Prompt Repetition Improves Non-Reasoning LLMs

TL;DR

Doubling the user’s prompt — literally concatenating “<QUERY><QUERY>” — improves answer quality for non-reasoning LLMs (Gemini, GPT, Claude, Deepseek) at no extra generated-token cost, because the second copy’s tokens get to attend bidirectionally over the first copy under causal attention. The trick is a near-free inference-time fix for the well-known ordering sensitivity of <CONTEXT> <QUESTION> vs. <QUESTION> <CONTEXT> prompts. When reasoning mode is enabled, the effect is neutral-to-slightly-positive — reasoning models already do something equivalent by repeating parts of the user’s request internally. Reported across 7 frontier models of varying sizes.

Key claims

Causal attention makes prompt-token order load-bearing: <CONTEXT> <QUESTION> and <QUESTION> <CONTEXT> produce different predictions for the same content, because in the first form the context never gets to attend to the question and vice versa [§1, Fig. 1].
Doubling the prompt to <QUERY><QUERY> lets every prompt token attend to every other prompt token within the doubled span — the second copy’s tokens have causal access to the entire first copy, so context tokens effectively “see” the question and question tokens “see” the context [§1].
Prompt repetition improves performance on non-reasoning configurations of Gemini 2.0 Flash, Gemini 2.0 Flash Lite, GPT-4o-mini, and other popular models across the 7-model evaluation set, without increasing generated tokens or decode latency [Abstract, §2].
Cost is concentrated in the parallelizable prefill stage; output length, output format, and downstream API compatibility are unchanged, enabling drop-in deployment [§1].
When reasoning is enabled, prompt repetition is neutral to slightly positive [Fig. 4].
Motivation observation: RL-trained reasoning models often spontaneously learn to repeat (parts of) the user’s request inside their thinking trace — prompt repetition is the inference-time analogue of that learned behavior [§1].

Method

The intervention is one line of preprocessing: replace the model’s input X with X ⊕ X (string concatenation of the prompt with itself). Generation proceeds normally from the end of the doubled prompt. Under causal attention, this means every token in the second copy of the prompt can attend to every token in the first copy — so tokens that originally appeared early in the prompt (e.g. context) now have a representation that has seen tokens that originally appeared late (e.g. question), via the second copy. The extra cost is entirely in prefill — quadratic in the doubled length, but parallelizable on accelerator hardware and not on the decode critical path. Output length and format are unchanged.

Results

Reported gains across 7 popular models (Gemini 2.0 Flash, Gemini 2.0 Flash Lite, GPT-4o-mini, plus others from Anthropic and DeepSeek) in their non-reasoning configurations [§2].
Effect is largest in the regime where ordering matters most — options-first vs. question-first multiple-choice framings, where the same content under different orders gives different accuracies [Fig. 1].
Reasoning-enabled mode: neutral to slightly positive [Fig. 4].
The paper does not claim gains on reasoning models in their reasoning modes — for those, internal repetition is already learned.

Why it’s interesting

A pure-inference-time trick that costs only prefill compute and lands as a sixth (or seventh) flavor of Inference-Time Scaling, orthogonal to KV-cache compression, sub-LM scaffolds, parallel-agent orchestration, and per-step iterative refinement. For Luma’s language stack: this is a “free” knob for any single-pass LLM call where ordering of context vs. question is uncontrolled (e.g. RAG retrieval pasted before the user query) — worth A/B’ing on non-reasoning Gemini/GPT calls. For video/diffusion stacks the relevance is mostly conceptual — the analogous problem (a generator’s text encoder only seeing prompt tokens in a fixed order) is already addressed by bidirectional text encoders, so the trick mostly transfers to settings using causal LLMs as conditioners.