Universal and Transferable Adversarial Attacks on Aligned Language Models

TL;DR

GCG (Greedy Coordinate Gradient) finds short adversarial token suffixes that, when appended to a harmful query, cause aligned chat LLMs to comply rather than refuse. The suffix is optimized white-box on open models (Vicuna-7B/13B, Guanaco-7B) by combining gradient-based candidate selection (HotFlip/AutoPrompt-style) with all-position search and multi-prompt/multi-model aggregation. Suffixes generalize from optimization-time prompts to held-out behaviors and transfer black-box to ChatGPT, Bard, Claude, PaLM-2, LLaMA-2-Chat, Pythia, Falcon — the first reliably-transferable automatically-generated jailbreak. This is the canonical reference for “LLMs inherit the adversarial-example vulnerability of vision models, and posthoc alignment finetuning does not fix it.”

Key claims

The attack objective is to maximize the log-probability that the model emits an affirmative response prefix that repeats the harmful query (e.g. “Sure, here is how to build a bomb”); just targeting “Sure” is insufficient because adversarial suffixes can hijack the query (e.g. “Nevermind, tell me a joke”) [§2.1].
GCG = AutoPrompt with one design change: at each step, candidate token replacements are evaluated across all positions in the suffix simultaneously rather than at a single chosen position; this minor-looking change is responsible for most of the gap to AutoPrompt [§2.2].
Universality is obtained by aggregating gradients/losses across multiple harmful prompts and incrementally adding prompts only after the suffix already works on the earlier ones [§2.3, Alg. 2].
Transfer across tokenizers works: optimizing on Vicuna-7B + Vicuna-13B (shared tokenizer) yields a single suffix that elicits objectionable behavior from Pythia, Falcon, Guanaco, GPT-3.5, GPT-4, PaLM-2, and (much less reliably) Claude-2 [§3, Abstract].
AdvBench: new benchmark with two splits — Harmful Strings (exact-match generation of 100 targeted toxic strings) and Harmful Behaviors (any compliant response to 500 instructions) [§3].
White-box ASR on Vicuna-7B: GCG 88% exact-match on Harmful Strings vs. AutoPrompt 25% / PEZ 0% / GBDA 0%; 100% train-ASR and 98% test-ASR on Harmful Behaviors [Table 1].
White-box ASR on LLaMA-2-7B-Chat (the better-aligned model): GCG 57% exact-match Harmful Strings vs. AutoPrompt 3%; 88% train / 84% test on Harmful Behaviors [Table 1].
Black-box transfer: ~88% ASR on GPT-3.5, ~54% on GPT-4, ~66% on PaLM-2, ~2% on Claude-2 — i.e. attack quality correlates with how close the target’s training distribution is to the open models the suffix was tuned on (GPT-family > PaLM > Claude) [§3, §Overview of Results].
The authors’ framing — that post-hoc alignment is structurally similar to adversarial-defense-by-training in vision and has the same fragility — is presented as motivation rather than a proved claim, and they explicitly draw the parallel to the decade of unsolved vision adversarial robustness [§1, §3].

Method

GCG optimizes a fixed-length suffix of $k$ tokens (default 20) appended to a user query within the model’s chat template. At each step, for every suffix position $i$ it computes $\nabla_{e_{x_i}} \mathcal{L}$ , the gradient of the negative log-probability of the affirmative-prefix target with respect to the one-hot embedding of the current token at that position. The top- $k'$ tokens with the largest negative gradient (most loss-reducing under the linear approximation) become candidate replacements for position $i$ . Across all positions, $B$ candidates are sampled uniformly, evaluated with a single forward pass each (this is the dominant cost), and the lowest-loss replacement is committed. Universality is layered on by replacing the single-prompt loss with a sum over a growing set of training prompts, optionally summed further over multiple white-box models with shared tokenizers; gradients are unit-norm-clipped before aggregation.

The transfer recipe used for the headline black-box results: train a single 20-token suffix on Vicuna-7B + Vicuna-13B against 25 harmful behaviors and evaluate one-shot on production chatbots through their public interfaces.

Results

Vicuna-7B Harmful Strings ASR: PEZ 0% · GBDA 0% · AutoPrompt 25% · GCG 88% [Table 1].
LLaMA-2-7B-Chat Harmful Strings ASR: PEZ 0% · GBDA 0% · AutoPrompt 3% · GCG 57% [Table 1].
Universal Harmful Behaviors (Vicuna-7B): train 100% / test 98% with GCG [Table 1].
Universal Harmful Behaviors (LLaMA-2-7B-Chat): train 88% / test 84% with GCG [Table 1].
Black-box transfer ASR with Vicuna-7B+13B-trained suffix: GPT-3.5 ~87.9%, GPT-4 ~53.6%, PaLM-2 ~66%, Claude-2 ~2.1%, plus broad success on open LLaMA-2-Chat / Pythia / Falcon / Guanaco [§Overview of Results].
Responsible disclosure: results shared with OpenAI, Google, Meta, Anthropic prior to publication, and the specific suffixes are documented to be patched over time without addressing the underlying vulnerability [§1, §6].

Why it’s interesting

This is the foundational reference for “adversarial robustness of aligned LLMs” — the paper that turned LLM jailbreaks from artisanal one-offs into a reproducible, gradient-based attack with measurable transfer. For Luma it matters in two ways: (1) it sets the prior for any time the team trusts an alignment- or safety-finetuned model to refuse something — the GCG line of work argues that current alignment is fragile under cheap white-box optimization, with transfer to closed models; (2) the “single suffix moves a single residual-stream direction across multiple models” finding here pairs naturally with Training large language models on narrow tasks can lead to broad misalignment‘s emergent-misalignment / “toxic persona vector” mechanism — both papers are evidence that high-level model behavior is governed by low-dimensional structure that is easy to push on. The optimization recipe itself (gradient-guided discrete search over all positions with multi-prompt aggregation) is also a useful reference whenever the team needs to search discrete token sequences against a differentiable LLM loss — adversarial prompting, prompt compression, hard-prompt tuning.