Qwen3Guard Technical Report

TL;DR

Qwen3Guard is a series of Qwen3-based safety guardrail models that address two limitations of prior guards (LlamaGuard, ShieldGemma, WildGuard, NemoGuard, PolyGuard): binary safe/unsafe labels that can’t accommodate different policy tolerances, and full-response-only classification that doesn’t fit streaming inference. Two variants ship: Qwen3Guard-Gen (generative, instruction-following SFT, produces tri-class safe/controversial/unsafe + category + refusal labels) and Qwen3Guard-Stream (adds a token-level classification head for real-time moderation during incremental generation). Three sizes (0.6B / 4B / 8B), 119 languages, Apache 2.0, trained on 1.19M curated prompts and responses. Beats LlamaGuard3/4, WildGuard-7B, ShieldGemma-27B, NemoGuard-8B and PolyGuard-Qwen-7B on most English / Chinese / multilingual prompt+response benchmarks and is used as the safety-reward signal in the companion Qwen3-4B-SafeRL release.

Key claims

The training set is 1.19M prompt/response samples covering 16 languages, with ~27% from Chinese, ~22% English, and a long multilingual tail; multilingual coverage is extended to 119 languages via Qwen-MT translation of a 15-language core [§3.2, Table 1].
Three severity tiers (Safe / Controversial / Unsafe) are constructed by training two label-imbalanced “strict” and “loose” models on disjoint halves of the data, then labeling Part B with both and assigning Controversial where they disagree (and symmetrically for Part A); this avoids relying on the scarce Controversial annotations in the raw data [§3.3, Fig. 3].
A label-distillation step then splits the data again, with Qwen3-32B as teacher refining annotations on the held-out half — explicitly framed as denoising the auto-labels rather than as compression [§3.3].
Qwen3Guard-Gen-8B reaches 90.0 / 83.9 average F1 on English prompt / response benchmarks vs. WildGuard-7B’s 85.8 / 79.9 and PolyGuard-Qwen-7B’s 87.0 / 74.0 [Tables 2–3].
Qwen3Guard-0.6B-Gen “rivals or exceeds the performance of existing Guard models that are more than 10× larger” on the English suite — 88.1 / 82.0 average vs LlamaGuard3-8B’s 79.4 / 70.7 [§3.4.1, Tables 2–3].
On Chinese prompt classification, Qwen3Guard-8B-Gen scores 85.1 average vs the next best PolyGuard-Qwen-7B at 68.4; on the in-house PolST politically-sensitive-topics set the gap is 88.6 vs 48.3 [Table 4].
On the RTP-LX multilingual prompt benchmark (10 major languages + Others), Qwen3Guard-8B-Gen averages 85.0 vs PolyGuard-Qwen-7B’s 80.9; LlamaGuard4-12B averages 41.7 [Table 5].
The “Think” benchmark — informal/lengthy reasoning traces from QwQ / Qwen3 / DeepSeek-R1 manually annotated for safety — is a new contribution; Qwen3Guard-Gen-8B scores 84.0 vs LlamaGuard3-8B’s 72.0 and PolyGuard-Qwen-7B’s 81.1, with the paper framing this as a novel challenge previous guards weren’t designed for [Table 3, §3.4].
Policy inconsistency across public benchmarks is structural, not stylistic: WildGuard-7B aligns with the strict Aegis policy but is “overly conservative” on OpenAIMod’s more permissive policy; the Controversial label gives downstream users a knob to match either by treating Controversial as Unsafe (strict) or Safe (loose) [§3.4.1, Fig. 4].
Policy inconsistency is more pronounced on prompts than responses — hypothesized to be “trust-but-verify” benchmarks (allow borderline prompts) vs “prevent-at-source” benchmarks (filter prompts upfront) [§3.4.1].
“Jailbreak” is treated as an input-only category — Qwen3Guard’s policy explicitly labels prompts that try to override system prompts, while the corresponding output is classified by whatever harm it actually produces [§2].
Stream Qwen3Guard adds an auxiliary token-level classification head to the Qwen3 backbone, enabling per-token safety labels during streaming generation; the token-level moderation degrades only modestly from the Generative variant [§1, §4 framing].
The companion Qwen3-4B-SafeRL release uses Qwen3Guard-Gen-4B as the safety reward signal in an RLAIF framework, framed as a demonstration that the Gen variant is good enough to be the reward model rather than merely a filter [§1, GitHub News 2025/09/30].

Method

Generative variant. Qwen3Guard-Gen is an SFT of instruction-tuned Qwen3 (0.6B / 4B / 8B). Inputs are (task definition, safety policy, safety categories, dialogue context, output-format spec); outputs are structured strings of the form Safety: {Safe|Unsafe|Controversial}\nCategories: {category}\nRefusal: {Yes|No} (refusal field only for response classification). Data collection uses Self-Instruct prompt synthesis with keyword-guided seeds and paired positive/negative examples (e.g. “How to make a bomb” / “How to make a cake”) to prevent surface-feature memorization. Unsafe responses are synthesized from base models (Qwen2.5-72B-Base) because instruct models rarely produce them; reasoning-trace responses are collected from QwQ / Qwen3 / DeepSeek-R1 to cover the new “Think” moderation regime. Auto-labels come from a Qwen2.5-72B-Instruct + Qwen3-235B-A22B voting ensemble with human-anchored validation (F1 > 0.9 on the safety-level label). The training pipeline has two stages: build Controversial labels via the strict/loose-model agreement procedure, then run Qwen3-32B label distillation on the result.

Streaming variant. Stream Qwen3Guard augments Qwen3 with a token-level classification head. The workflow: (1) prompt-level safety check on the full user turn (the user’s input goes both to the LLM and Qwen3Guard-Stream in parallel); (2) if allowed, the LLM streams its response token-by-token, and each token is forwarded to Qwen3Guard-Stream which maintains a stream_state and emits a per-token risk label. The streaming detector requires Qwen3-tokenizer-compatible token IDs; for other tokenizers, retokenization is required.

Results

English prompt classification (avg F1): Qwen3Guard-0.6B/4B/8B-Gen = 88.1 / 89.3 / 90.0 vs WildGuard-7B 85.8, ShieldGemma-27B 70.0, NemoGuard-8B 82.9, LlamaGuard3-8B 79.4, LlamaGuard4-12B 75.9, PolyGuard-Qwen-7B 87.0 [Table 2].
English response classification (avg F1): 82.0 / 83.7 / 83.9 vs WildGuard-7B 79.9, NemoGuard-8B 78.1, PolyGuard-Qwen-7B 74.0 [Table 3].
Chinese prompt (avg): 80.8 / 84.6 / 85.1 vs PolyGuard-Qwen-7B 68.4, WildGuard-7B 55.1, LlamaGuard3-8B 45.6 [Table 4].
Chinese response (avg): 84.9 / 87.3 / 87.1 vs PolyGuard-Qwen-7B 62.5, WildGuard-7B 61.5 [Table 4].
RTP-LX multilingual prompts: 8B model scores 85.0 average across 10 major languages + Others vs PolyGuard-Qwen-7B 80.9 and NemoGuard-8B 42.7 [Table 5].
PolyGuard-Response multilingual responses: 8B model scores 77.6 average vs PolyGuard-Qwen-7B 74.0 and NemoGuard-8B 69.5 [Table 6].
The 0.6B-Gen model beats LlamaGuard3-8B and LlamaGuard4-12B on every English suite shown [Tables 2–3] — concrete evidence for the 10× claim.
“Think” (reasoning-trace moderation): 8B-Gen scores 84.0 vs LlamaGuard3-8B 72.0, NemoGuard-8B 77.9, PolyGuard-Qwen-7B 81.1 [Table 3].

Why it’s interesting

The “Controversial” label is the load-bearing idea: it converts policy-inconsistency from a benchmark-vs-benchmark mess into an explicit knob downstream users can flip — a clean conceptual move that other open guards haven’t made. Pairing this with a token-level streaming head is also new — production LLM stacks are streaming-first, but every prior open guard required the full response, forcing either delayed intervention or full re-generation. Both variants ship Apache 2.0 across three sizes, which puts Qwen3Guard squarely in the Open foundation-model releases pattern of a single-family multi-variant release with multi-backend day-0 inference (HF Transformers / vLLM / SGLang). The fact that the same paper introduces a Reasoning RL application — Qwen3-4B-SafeRL trained with Qwen3Guard-Gen as the RLAIF reward signal — closes the loop: the guard is not just a downstream filter but a training-time supervisor. Contrast with Universal and Transferable Adversarial Attacks on Aligned Language Models which shows that any single hard “refusal vs comply” decision boundary is gradient-attackable; Qwen3Guard’s tri-class output and external supervisor stance is at least architecturally less brittle than baking the decision into the chat model itself, though the paper does not claim adversarial robustness and GCG-style attacks could plausibly transfer to the guard head.