Recursive Language Models

TL;DR

RLMs treat the input prompt as a variable in an external REPL environment rather than tokens to be fed through the attention window. The root LM is given a Python REPL where the prompt sits as a string variable; it inspects, decomposes, and transforms that variable with code, and recursively invokes sub-LM calls on chunks without ever loading the full prompt into its own context. The instantiation (GPT-5 root + GPT-5-mini sub-calls) scales to 10M+ token inputs, dominates context-compaction, retrieval, and CodeAct baselines on four diverse long-context tasks, and on the quadratic-complexity OOLONG-Pairs benchmark lifts GPT-5 from 0.04 → 58.0 F1 at comparable cost. The framing — that long context belongs outside the network, not inside — is the contribution; everything else is the simplest implementation that lets you measure it.

Key claims

Treating the prompt as a variable in a Python REPL (rather than as direct LM input) lets a single fixed-context LM handle inputs up to two orders of magnitude beyond its window [§1, §3].
On BrowseComp-Plus (1K) at 6M–11M input tokens, RLM(GPT-5) reaches 91.33% vs 70.47% for a summary agent and 51.00% for CodeAct+BM25, while remaining within the same cost order ([Table 1]).
On OOLONG-Pairs (a quadratic-complexity aggregation task at 32K tokens) base GPT-5 scores 0.04 F1 and the summary agent scores 0.01; RLM(GPT-5) scores 58.00 and RLM without sub-calls scores 43.93 [Table 1].
Ablation: removing recursive sub-LM calls but keeping the REPL still beats base LM and baselines on most long-context settings — the REPL alone gives most of the long-context win, and sub-calls add ~10–30% on top for information-dense tasks [§3, Obs. 2].
RLM cost is comparable to base LM in median but high-variance; the median GPT-5 RLM run is cheaper than the median base GPT-5 run, with rare expensive outliers from long trajectories [§3, Obs. 4, Fig. 3].
Base LM still outperforms RLM at short input lengths — i.e. the scaffold has a non-trivial overhead for small inputs, so there is a crossover point above which RLMs win [§3, Obs. 3].
The current instantiation uses recursion depth one (sub-calls are plain LMs, not RLMs); deeper recursion is left as future work [§5].
Models without strong coding capability fail as RLMs (Qwen3-8B was unusable); the scaffold is gated on the root LM’s ability to write and reason about Python [Appendix A].

Method

The RLM takes a string prompt p and initializes a persistent Python REPL with p set as a variable. The root LM is told the length of p and is given a system prompt describing the REPL plus a single tool: code execution. The REPL exposes a sub-LM-call function so the root can chunk p and dispatch sub-tasks. The model writes code that peeks into p (e.g. regex-filtered prints), constructs sub-prompts, calls sub-LMs in parallel or sequentially, and finally emits its answer wrapped in FINAL(...) or FINAL_VAR(...). The system prompt is fixed across all tasks (one tiny extra sentence for Qwen3-Coder to discourage over-sub-calling); no per-benchmark tuning.

Evaluated configurations:

RLM — full system, sub-LM calls enabled. Root = GPT-5 (medium reasoning) or Qwen3-Coder-480B; sub-LM = GPT-5-mini or Qwen3-Coder-480B.
RLM no sub-calls — ablation; the LM has the REPL but cannot invoke sub-LMs.
Compared against: base LM, CodeAct+BM25 (ReAct + retriever; prompt fed directly), and Summary agent (iterative summarization compaction, using GPT-5-nano for compression and GPT-5 for the final answer to keep cost honest).

Benchmarks span four complexity regimes vs. input length: S-NIAH (constant), BrowseComp-Plus 1K (multi-hop QA over 6–11M tokens of documents), OOLONG/trec_coarse (linear aggregation), OOLONG-Pairs (quadratic aggregation), and LongBench-v2 CodeQA (23K–4.2M tokens). The complexity-vs-length axis is the paper’s lens for characterizing context rot: the harder a task scales with input, the earlier in token count the base LM collapses, and the bigger the RLM win.

Results

Headline numbers (RLM(GPT-5) unless noted):

BrowseComp-Plus (1K, 6–11M tokens): 91.33% vs 70.47 (summary) vs 51.00 (CodeAct) vs 0% (base, can’t fit) [Table 1].
OOLONG-Pairs (32K): 58.00 F1 vs 0.04 (base) vs 24.67 (CodeAct) vs 0.01 (summary) [Table 1].
OOLONG (131K): 56.50 vs 44.00 (base) vs 46.00 (summary) [Table 1].
CodeQA (23K–4.2M): 62.00 vs 24.00* (base, hits limits) vs 58.00 (summary) [Table 1].
Qwen3-Coder-480B variant tracks the same pattern at lower absolute numbers — RLM(Qwen3) hits 23.11 F1 on OOLONG-Pairs vs 0.06 base [Table 1].
Context-length sweep (Fig. 1): RLM(GPT-5) degradation slope vs prompt length is much shallower than base GPT-5; the two curves cross above ~10⁵ tokens on linear/quadratic tasks, after which RLM strictly dominates.
Cost: median GPT-5 RLM run is cheaper than median GPT-5 base run; ~3× cheaper than the summary agent on BrowseComp-Plus while ~20% more accurate [§3, Obs. 4].

Why it’s interesting

The paper draws an explicit analogy to out-of-core algorithms: a small fast memory (the LM context) processing arbitrary-size data by managing what gets paged in. That framing is the actual contribution — it predicts that training a model end-to-end as an RLM is the natural next step, and it cleanly separates “context rot” (a property of the network) from “context length” (a property of the scaffold). For Luma, the interesting bits are (a) the model-agnostic scaffold lifts inference behavior far more than architectural long-context tricks at the same cost, suggesting the scaffolding-vs-architecture frontier is where compute leverage lives right now, and (b) the recursion-depth=1 limitation is an explicit open question — deeper RLMs would start to look like learned hierarchical planners with REPL-as-environment.