Skip to content
Bud's Wiki

Inference-Time Scaling

TL;DR

Section titled “TL;DR”

Trading more inference compute for capability that the base model can’t deliver in a single forward pass — but the filed papers split into three operationally distinct axes: (a) scaffold-side scaling (RLM-style sub-LM calls in a REPL, where the root LM offloads work to fresh LM instances and uses the REPL as state); (b) interaction-depth scaling (MiroThinker-style multi-hundred-step tool-call trajectories in an open environment); and (c) sequence-budget scaling at fixed memory (DMS-style KV-cache compression that lets a reasoning model generate longer or more parallel chains-of-thought without raising peak memory). The three are largely orthogonal and look stackable, but no filed paper has combined them.

Key claims

Section titled “Key claims”
  • Inference-time compute can be traded for capability on tasks where the base model fails one-shot, and the gains hold across the four filed flavors: scaffold (RLM lifts GPT-5 from 0.04 → 58.0 F1 on OOLONG-Pairs, Recursive Language Models Table 1), interaction depth (MiroThinker shows monotone gains on BrowseComp/HLE/GAIA up to several hundred tool calls per task, MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §1, §3.3), sequence-budget compression (DMS adds +9.1 / +7.6 / +9.6 on AIME 24 / GPQA / LiveCodeBench at fixed memory-read budget on Qwen-R1 32B, Inference-Time Hyper-Scaling with KV Cache Compression Abstract, Fig. 1), and learned parallel-agent orchestration (K2.5 Agent Swarm adds +3.5 points over single-agent context-managed inference on BrowseComp, 74.9 → 78.4, with up to 80% wall-clock runtime reduction on wide-search scenarios, Kimi K2.5: Visual Agentic Intelligence §Agent Swarm, Benchmark table).
  • The dominant bottleneck for sequence-budget scaling in Transformer LLMs is KV-cache memory transfer, not arithmetic — for batch 256 and 8K–32K contexts, KV-cache reads exceed 90% of decode latency on Qwen-R1 1.5B and 80% on Qwen-R1 7B (Inference-Time Hyper-Scaling with KV Cache Compression Appendix G). This is why cache compression converts directly into more usable tokens at fixed runtime.
  • Sequence-budget scaling is Pareto-frontier shifting, not just a constant-factor speedup: DMS dominates training-free KV-sparsification baselines (Quest, TOVA, H2O) on both memory-reads and peak-memory axes across model sizes (Inference-Time Hyper-Scaling with KV Cache Compression §5.1, Figs. 3–4).
  • Pareto-optimal DMS configurations combine both longer chains (sequential scaling) and more chains (parallel scaling) — neither axis is sufficient alone (Inference-Time Hyper-Scaling with KV Cache Compression §5.1).
  • Scaffold-side scaling: treating the prompt as a variable in a persistent Python REPL and dispatching sub-LM calls on chunks lets a fixed-context LM operate on inputs ~two orders of magnitude beyond its window, with median RLM cost cheaper than median base LM (Recursive Language Models §3, Obs. 4).
  • Scaffold-use is a learnable skill, not free at inference: the Prime Intellect RLM ablations show that without tips the RLM scaffold underperforms the base LM on DeepDive, and on math-python it actively hurts compared to a plain Python tool — interpreted as base-model overfitting to the standard tool harness (Recursive Language Models: the paradigm of 2026 §Results across environments).
  • Interaction-depth scaling: MiroThinker pushes the open-source per-task tool-call ceiling from <100 to ~600 via a 256K context + recency-based retention, and reports monotone benchmark gains as a function of tool-call budget (MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §1, §3.3).
  • The infrastructure cost of interaction-depth scaling is non-trivial — online GRPO over multi-hundred-step trajectories requires a streaming rollout queue to handle long-tailed completion times, not just bigger batches (MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §5.3).
  • Cache-compression scaling is implementation-cheap: DMS adds no new parameters (an existing K/Q neuron is repurposed for the eviction logit), the eviction mask never needs to be materialized, and the sparsified cache is compatible with PagedAttention/FlashAttention kernels (Inference-Time Hyper-Scaling with KV Cache Compression §3.2 Performance Considerations, §3.3).
  • At sufficiently aggressive compression, cache compression can exceed dense attention on long-context tasks — DMS at CR=2 on Llama 3.2 1B Instruct beats the uncompressed model on NIAH (97.8 vs. 96.4) and VT (63.2 vs. 55.8), conjectured to fight over-squashing / lost-in-the-middle (Inference-Time Hyper-Scaling with KV Cache Compression Table 1, §5.2).
  • Learned parallel-agent orchestration is a fourth, distinct scaling axis: K2.5’s Agent Swarm trains an orchestrator policy via Parallel-Agent Reinforcement Learning (PARL) to dispatch up to 100 sub-agents executing concurrent workflows across up to 1,500 coordinated tool calls, with a Critical-Steps latency metric (longest path through the parallel execution graph) replacing total step count and structurally rewarding parallel strategies (Kimi K2.5: Visual Agentic Intelligence §Agent Swarm, §PARL). Distinct from RLM’s untrained programmatic scaffold (the orchestrator code is not learned, only the base LM is) and from MiroThinker’s sequential interaction-depth (a single agent, not a swarm).
  • Learned parallel orchestration has training-time failure modes that single-agent scaling doesn’t see: serial collapse (orchestrator defaults to single-agent execution despite parallel capacity) and spurious parallelism (orchestrator inflates parallel metrics by spawning sub-agents without meaningful task decomposition); both are defeated by separately shaped reward terms plus a staged schedule that shifts weight from parallelism early to task success late (Kimi K2.5: Visual Agentic Intelligence §PARL).
  • Generation-side, per-step iterative refinement is a fifth axis: a pretrained flow-matching video generator can be reused as a time-conditioned denoising autoencoder, with 2–3 Predict-and-Perturb iterations at each fixed noise level tt pulling the latent toward higher-density regions of the data manifold before the ODE step advances (Self-Refining Video Sampling §Methods, “Flow Matching as Denoising Autoencoder”). At ~50% additional NFEs, this beats both matched-compute NFE×2 (more discrete ODE steps with the same total NFE budget) and best-of-4 rejection sampling against a separately-trained Cosmos-Reason1-7B verifier on Wan2.2-A14B and Cosmos-Predict2.5-2B (Self-Refining Video Sampling §Results). The where-to-spend-the-NFE answer is “at the current noise level” rather than “in finer discretization” or “in a downstream verifier”.
  • Iterative-refinement scaling on CFG-trained generators has a failure mode: K≥3K \geq 3 Predict-and-Perturb iterations with CFG oversaturate static regions; the fix is uncertainty-aware masking that only refines regions where consecutive z^1\hat z_1 predictions disagree above a threshold, computed at zero extra NFE since both predictions already exist inside the loop (Self-Refining Video Sampling §Methods, “Uncertainty-aware P&P”). Parallels the orchestration-side oversaturation failure modes (serial collapse, spurious parallelism) in that aggressively scaling inference compute creates new pathologies that need targeted fixes.
  • Test-Time Training of model weights is a sixth axis: on top of a standard sliding-window Transformer, continue training on the test-time context via next-token prediction so context is compressed into weight updates rather than into a hidden state or KV cache; meta-learn the initialization end-to-end through those inner-loop updates (End-to-End Test-Time Training for Long Context Abstract, §1, Fig. 1 caption). At 3B/164B tokens, TTT-E2E matches a full-attention Transformer’s loss-vs-context-length curve while Mamba 2 and Gated DeltaNet do not, and retains RNN-style constant inference latency (2.7× faster than full attention at 128K) (End-to-End Test-Time Training for Long Context Abstract, Fig. 1). Distinct from the other five axes in that the extra inference compute lands as gradient steps on the network’s own weights, not as more forward passes (NFE×K, parallel chains, sub-LM calls) or cache surgery.

Recent contributions

Section titled “Recent contributions”
  • [2026-08-29] Lerrel Pinto — in-context learning for robots is not that hard: Pinto tweet adds another public voice on the “physical prompts as test-time compute” axis; complements S1’s matched-recipe scaling law with a low-evidence assertion of ease.
  • [2026-08-27] Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning: Q-Planning (Giridhar et al.) is a robotics-side value-guided-Best-of-N variant that closes a self-improvement loop around it: at inference, sample N candidate action chunks from the frozen BC policy and execute a single-step Q-weighted average scored by an off-policy Q-function; between iterations update only the Q-function on both successful and failed rollouts. Adds a wrinkle to the value-guided-sample-selection axis siblings to Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning QGF: not only does extra inference compute get spent on N BC draws + a Q re-score, but the scorer itself gets sharper over iterations without any change to the policy — a “the search-time-compute knob gets more useful over deployment” pattern rather than a static test-time-only recipe. On LIBERO-Object / LIBERO-Goal (both near ceiling), the effect is reported as shorter successful episodes rather than higher success rate — evidence the Q re-scorer is learning something beyond binary success at inference time.
  • [2026-08-25] S1: In-Context Learning for Robotics: S1 (Skild AI) is the second robotics filing on the “in-context sensorimotor demonstrations as inference-time compute” axis alongside GEN-1.5: Embodied Foundation Models are One-Shot Learners, but ships the first matched-recipe scaling law on this axis: a controlled 1k→100k-hour comparison of ICL vs language-conditioned prompting with identical architectures (modulo the prompt embedding), identical data, and identical compute. Two load-bearing findings: (i) language-conditioned VLA leads at 1k hours on seen tasks (53% vs 43% ICL) but ICL wins at scale — the ICL-vs-language crossover happens within the pretraining-scale sweep rather than requiring a separate axis; (ii) on unseen tasks the language baseline plateaus at 9% at 100k hours while ICL reaches 66%, and language “performance increases slowly with pre-training”. The 7× gap and the plateau argue language conditioning is a wrong-turn interface for the OOD regime, not just a weaker interface. Also reports a matched demo-budget curve on unseen tasks: interpolated crossover between 1-shot ICL and post-trained VLA at ~380 post-training episodes, with 2k episodes needed for the VLA to reach 86% and pass the 66% single-prompt ICL number. First filed scaling-law datapoint on the physical-prompt axis rather than the token / tool-call / refinement-step / weight-update axes.
  • [2026-08-21] Anirudha Majumdar — pure zero-shot task inference is the stronger test of robot foundation models: Anirudha Majumdar tweet-critique of the one-shot in-context axis for robot foundation models — argues that reported ICL gains conflate task-inference-from-scene (zero-shot) with behavior-shaping-from-demonstration, and proposes ablating the prompt entirely to isolate the two.
  • [2026-08-19] GEN-1.5: Embodied Foundation Models are One-Shot Learners: GEN-1.5 is the first filed robotics datapoint where the inference-time-compute knob is in-context sensorimotor demonstrations rather than tokens (RLM), tool calls (MiroThinker), refinement steps (Self-Refining Video Sampling), parallel agents (PARL), TTT weight updates (TTT-E2E), or KV cache surgery (DMS). A single 3–12 s demonstration in the 30 s context window buys 59% avg success on 10 tasks with zero gradient steps, and physical prompts compose — two independently recorded demos chain into one continuous behavior with intermediate bridging motions the model was never trained to produce. Prompt-substrate flexibility not previously seen in language ICL: sim-recorded prompts drive real-robot execution (zero sim data in pretraining), and human-hand demonstrations observed through the robot’s cameras drive robot-hand execution. Sharpens the concept’s Open Question about whether ICL transfers cleanly from language to embodied control with a yes at pretraining scale, and adds an unusually clean crossover regime with parameter updates — 1 gradient step on 1 min of data hits 66.5% on held-out tasks while changing weights <0.15%.
  • [2026-08-18] Equilibrium Forcing (EqF) — extending Equilibrium Matching to budget-adaptive long-horizon video generation: Equilibrium Forcing announcement (Yilun Du) — video generator adapts its inference procedure to the current sample (per-sample budget-adaptive sampling) on top of EqM’s autonomous gradient field. Tweet-only; mechanism not disclosed.
  • [2026-08-17] Training AI Scientists to Replicate Research: Faraday-27B beats Claude Opus 4.8 and GPT-5.5 running in their own harnesses at extra-high thinking effort on held-out research-replication tasks — a datapoint that a small taste-trained director + a coding-agent tool outperforms a frontier model at max thinking budget, suggesting scientific-taste post-training is a distinct scaling axis from thinking-effort inference-time compute.
  • [2026-08-11] Full-bandwidth transformer: Full-bandwidth transformer is the inverse of most entries on this page: rather than spend more inference-time tokens for extra compute, it absorbs that compute into a latent-feedback channel (top-layer hidden state fed back through a GLU into the next step’s input) and reports shorter reasoning traces at equal or better accuracy. A decoding-side compute-compression counterpoint to the emit-more-tokens paradigm — worth watching whether this survives at scale as a substitute for verbalized CoT.
  • [2026-08-06] VISTA: A Visual Harness for Reasoning in an Interactive World: VISTA is a “free reasoning + free inspection” harness for ARC-AGI-3 that spends inference compute on natural-language deliberation and read-only pixel re-examination to save environment actions (the only cost RHAE counts). Opus 5.0 reaches 100.00 mean RHAE on the 25-game public set at 56% fewer actions than first-time humans; GPT-5.6 Sol reaches 98.27. Complements Schema’s BFS-over-certified-simulator instance of the same trade (inference compute → saved environment actions), with the compute now going into free-form reasoning and lossless-memory queries rather than program synthesis and simulator search.
  • [2026-08-03] RL²-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models: RL²-VLA reports the sharpest filed evidence that inference-time-scaling in embodied control needs per-state activation, not per-run: on BridgeV2 with π₀ + oracle verifier, sample-diversification scaling e ≈ a·k^b improves action error in predicted-failure states and degrades already-accurate actions in predicted-success states. RL² therefore gates a compositional-steering primitive (composing action-expert flow velocities between a frozen base VLA and a separately-trained offline-RL policy conditioned on the base VLA’s latents) on a conformal-prediction band over a failure-probability predictor, and reports +17.3 pp OOD sim / +26.7 pp real-hardware vs the strongest Rephrase baseline. Also reports 8-rephrase × 5-sample compositional steering adds +18.6 pp over single-sample inference — sample-diversity and sample-quantity axes combining multiplicatively rather than substitutively. First filed data point that breaks the monotonicity assumption implicit in MCTS / iterative-refinement / cache-compressed-longer-chain axes on this page.
  • [2026-07-31] Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation: Explorative Modeling is the training-side dual to the inference-time axes on this page: rather than spending more compute at inference (NFE×K, parallel chains, sub-LM calls, TTT weight steps, PARL), XM spends more compute at training via K parallel candidate generations per step. End-to-end reconstructive XMs then match diffusion on control tasks with 16–256× fewer inference steps, effectively converting train-time exploration into inference-time savings — an inverse of the standard test-time-compute story. Open question this raises: does train-time K compose with test-time refinement K, or are they substitutes for the same “commit to a mode” budget?
  • [2026-07-30] How enabling two settings tripled our scores on the ARC-AGI-3 benchmark: Adds a harness-setting axis to the “how to spend inference compute” design space: two API defaults (retained reasoning across turns + summarize-on-fill compaction) triple GPT-5.6 Sol’s ARC-AGI-3 public-set RHAE from 13.3% to 38.3% at 6× fewer output tokens. Contra the usual quality/latency Pareto — here the harness-level context-management change lifts capability and reduces token spend. Complements Schema (Schema — Frontier Models with Our Harness Achieve ~99% on ARC-AGI-3 Public) at the opposite extreme of the harness-uplift spectrum on the same benchmark.
  • [2026-07-30] Introducing Inkling-Small — Thinking Machines' efficient open-weights MoE reasoning model: Inkling-Small’s variable thinking-effort curves sit above Inkling’s at every thinking budget on HLE, not just at max effort — sharpest signal that the OPD + RL improvement is a real generalization gain rather than a benchmark-hacking artifact at one operating point.
  • [2026-07-30] Local Mixtures of Experts: Essentially Free Test-Time Training via Model Merging: TTMM (Bertolissi, Hübotter, Hakimi, Krause) is a pure test-time-compute reduction relative to test-time-training: matches most of TTT’s language-modeling perplexity gain (within ~0.04 nats of TTT on Wikipedia / GitHub-Python for Llama-3.2-1B with 10 active experts) at ~100 ms overhead per prompt versus seconds for TTT gradient steps — a claimed >100× speedup. The recipe: amortize the “per-prompt fine-tune” cost at train time by clustering the corpus into many local neighborhoods, training one LoRA per cluster, then at test time merging a sparse subset of adapters into a single decoding model. Adds a distinct pole to the inference-time-scaling cluster: instead of scaling test-time compute up (thinking with tokens, MCTS, PARL) it scales test-time compute down while retaining most of the test-time-training accuracy gain — a datapoint that per-prompt specialization can be recovered without per-prompt gradient steps if the specialization is precomputed and merged on demand.
  • [2026-07-28] 1XWM: From Video to Action — 1X's video-pretrained world model as a NEO robot policy: 1X applies best-of-N to a video-generation policy for robotics: N parallel WM rollouts on ambiguous prompts (e.g. “pull tissue” occasionally producing pick-up-the-box), then execute the best. Reports monotone gains from 1 → 8 generations on real hardware, with the load-bearing move being filtering — executing bad generations hits ~0% success. Currently manual selection; VLM-evaluator automation is flagged as future work. First filed instance of TTC applied to the video-model-as-policy rather than as a reasoning trace or reward-model search.
  • [2026-07-28] Robotic Control via Embodied Chain-of-Thought Reasoning: ECoT (CoRL 2024) is the founding “reasoning-tokens-before-actions” scaling recipe for VLAs — token count jumps 50× (from 7 OpenVLA action tokens to ~350 per step) and drives a +28% absolute success gain across a 314-trial generalization suite. The paper also stakes out the two dominant mitigations for the resulting latency blow-up: synchronous reuse (keep the high-level PLAN/SUBTASK fixed for 5 steps, refreshing only low-level MOVE/GRIPPER/OBJECTS) gives +24% speed at higher success (72% vs 63% naïve — reuse smooths chattering plans), and asynchronous execution (one instance continually refreshes high-level reasoning while a second consumes the freshest chain for low-level reasoning + actions) gives +40% speed at matched success. Both are architecturally simple prompt-caching moves that compose with any downstream serving/decoding optimization.
  • [2026-07-27] τ₀-VLA: A Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation: τ₀-VLA is the first filed robotics analog of the scaffold/search inference-time-scaling axes on this page: extra compute at each high-level step is spent on world-model-guided beam search over natural-language subtasks (branching N, beam width B, depth D), with a confidence-based adaptive router deciding when to trigger it based on token-confidence statistics from the proposal model’s forward pass. Extends the RLM/MiroThinker/PARL family from LM-only contexts into hierarchical VLA control, and generalizes DriftWorld’s “WFM as inference-time planner” from action-level rollouts to language-subtask-level rollouts. Reports next-subtask accuracy climbing 50% → 74% on OOD Book Organization with more inference budget and translating to higher closed-loop task success on long-horizon real-robot episodes (up to 12 minutes) — evidence the compute-accuracy trade-off holds beyond text-only reasoning, into embodied long-horizon settings.
  • [2026-07-24] AREX: Towards a Recursively Self-Improving Agent for Deep Research: AREX’s outer self-improvement loop is an inference-time-scaling primitive: rather than searching longer in a single ReAct trajectory, spend additional compute on a constraint-wise audit of the provisional answer and re-launch targeted inner-loop rounds against the flagged unresolved constraints. The persistent scratchpad is a learned compact improvement state rather than a REPL variable (RLM) or a full context window (MiroThinker) or a Discard-All context reset (Marco-DeepResearch).
  • [2026-07-24] optimize_anything Goes omni — Composing Optimizers into Meta-Optimizer Pipelines: omni frames text-artifact optimization as inference-time compute allocation across proposer families under a matched dollar budget: parallel exploration with 1/N of budget per engine, then fresh continuation from the best beats any single-engine budget allocation by +7.8 to +18.0 pts on Frontier-CS. Contributes an “engine plateau + switch to a different engine” schedule as a new inference-time-scaling primitive, and Terrarium as an eval framework that pins task / model / dollar budget across proposers for fair comparison.
  • [2026-07-22] Han Xiao — turning frozen jina-v5-omni into an open-vocabulary image tagger via test-time scaling: Han Xiao (Jina AI) previews a talk turning frozen jina-v5-omni into an open-vocabulary, multi-label, n-gram image tagger via test-time scaling — no training, no second model, no external knowledge. A candidate seventh axis: repurposing a frozen embedding model’s use pattern at inference to solve a generative-flavored downstream task, distinct from generator-side NFE, cache surgery, sub-LM calls, TTT weight steps, learned parallel orchestration, and search. No numbers in the tweet; the mechanism is only sketched.
  • [2026-07-21] Hyra-results — Hunyuan Research Agent (results bundle): Hyra (Tencent Hunyuan Research Agent) launch results improve prior-best on 11 diverse leaderboarded tasks using an agentic outer loop, including tightening long-studied Erdős/autocorrelation constants (TTT-Discover / SimpleTES prior) and breaking 100 packing records on Erich Friedman’s Packing Center — a wide-domain agentic-search datapoint for spending inference compute against public-benchmark improvement.
  • [2026-07-20] Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline): Learning to Fold introduces Thompson-sampling bandits over inference-time hyperparameters as a distinct axis of inference-time compute for flow-matching VLAs: the same checkpoint is tuned per-garment-type over execution length, playback speed, chunk-boundary inpainting onset, CFG scale, noise temperature, and best-of-N candidate count, discovered cheaply online during rollout collection. Ties together three inference-time knobs at once — best-of-N candidate selection (via the policy’s own action-conditional success-residual Q-head, no separate reranker), CFG scale (advantage conditioning), and noise/temperature sampling — and lets a bandit search the joint space rather than picking a single scalar knob. Concrete evidence that per-condition inference-time tuning of many knobs simultaneously is a competitive lever on top of a fixed policy: the same checkpoint reached 1st place in the LeHome Challenge sim round (79.63%, 6.1-point lead over 2nd across 62 teams) partly by this tuning.
  • [2026-07-17] HDR: Hierarchical Denoising for Multi-Step Visual Reasoning: HDR proposes hierarchical coarse-to-fine sampling as a structured way to spend inference compute on visual reasoning: the model first samples a small number of long-horizon planning latents, then conditions each finer level on its parent, and finally streams leaf frames. On a six-task multi-step reasoning benchmark (maze, Hanoi, sliding puzzle, Sokoban, one-line drawing, water pouring), this hierarchical spend raises overall success 34.22 → 60.29 vs. flat streaming AR diffusion, and the hierarchy also improves robustness under reduced denoising-step budgets — a rare “spend less, still win” data point for inference-time reasoning tricks.
  • [2026-07-16] Schema — Frontier Models with Our Harness Achieve ~99% on ARC-AGI-3 Public: Schema spends inference compute on iterated program synthesis + full-history backtesting + BFS over the certified simulator, converting free planning compute into saved environment actions (M0R0 Level 4: 42 actions vs 500 human baseline; 14/25 games use 1.6–5.0× fewer actions than the human reference). Distinct from the six existing axes in that the extra compute lands as executable-program search over a joint state-representation-and-transition-rule space rather than as more forward passes, cache surgery, sub-LM calls, TTT weight steps, or parallel-agent dispatch.
  • [2026-07-16] VIA: Visual Interface Agent for Robot Control: VIA (Stanford; Hu, Sundaresan, Gao, Sadigh) reports that robot-manipulation success scales with the strength of an unmodified frontier agent (Claude Code, Codex, Fable 5) driving a browser-based 3D interface via screenshots and interface commands — Fable 5 hitting 96.7% on three LIBERO-Goal tasks and 100% on long-horizon rainbow assembly. Zero robot fine-tuning; the gains come entirely from stronger inference-time reasoning / re-planning in the frontier agent. Complements the concept’s existing test-time-compute datapoints by extending the scaling axis to embodied control through a general software-agent interface.
  • [2026-07-15] Foveated Reasoning: Stateful, Action-based Visual Focusing for Vision-Language Models: FoveateR adds a visual-token-budget variant of adaptive inference-time spend: rather than spending the full high-resolution budget uniformly, the model keeps its base tokens coarse and only injects high-res patches into the same decoding trajectory when its reasoning trace emits a foveation action. Distinct from the sequence-budget compression axis (DMS) and from external-tool loops (Gemini Agentic Vision) — the extra compute lands as additional visual tokens conditional on the ongoing reasoning, and an anti-”see-everything” reward keeps the policy from collapsing to the degenerate maximum-budget baseline.
  • [2026-07-09] Introducing Muse Spark 1.1: Muse Spark 1.1 pitches “active management” of a 1M-token context (remembering earlier actions, retrieving from earlier work, compacting critical steps) as a trained behavior — the sustained-single-agent axis of test-time compute rather than the parallel-agent contemplating-mode axis of the original Muse Spark.
  • [2026-07-08] LLM-as-a-Verifier: A General-Purpose Verification Framework: Proposes verification as a distinct scaling axis alongside pre-training, post-training, and generation-side test-time compute — orthogonal to the six generation/scaffold axes on this page. A continuous logit-expectation score readout enables three composable sub-axes at the verifier: score granularity (finer bins → better positive/negative separation), repeated evaluation (variance reduction), and criteria decomposition (complexity reduction). Reports SOTA on Terminal-Bench V2 (86.5), SWE-Bench Verified (78.2), RoboRewardBench (87.4), MedAgentBench (73.3) training-free, and improves sample efficiency of SAC (robotics) and GRPO (math) when the continuous verifier score is used as dense reward. Complements Noam Brown’s inference-compute-curve framing by putting the compute on the verifier side of the generator/verifier split.
  • [2026-07-07] Muse Video preview — MSL video generation model announcement (Alexandr Wang): Muse Image spends inference-time compute on prompt reasoning, web search, and planning (via Muse Spark) before generation — a productized instance of “spend more compute pre-generation” scaling, complementing the parallel-agents contemplating-mode pattern from the earlier Muse Spark announcement.
  • [2026-07-05] Implications of Large-Scale Test-Time Compute (Noam Brown): Noam Brown (OpenAI) frames test-time-compute scaling as an evaluation-methodology problem: scalar leaderboard scores are broken when the same model spans an ~10⁵× cost range with monotonically improving scores, some recent models are still improving past 100M tokens per query when runs are stopped for infra reasons rather than a plateau, and safety evals conducted at low inference budgets can miss capabilities that emerge under sustained higher-budget inference. Proposes performance-vs-cost/tokens/time curves in place of bar charts, treating multi-agent parallel inference as a first-class deployment axis, and RSPs / third-party benchmarks that explicitly track or bound inference-compute. First-person confirmation from a frontier lab that all six axes on this page’s Key claims matter operationally.
  • [2026-07-04] EdgeBench: Scaling Laws of Environment Learning: EdgeBench turns MiroThinker’s interaction-depth axis (axis (b) on this page) from a monotone-gain observation into a fitted scaling law: averaging 402 per-model learning curves over 12–72 h of environment interaction collapses to a log-sigmoid (S(t) = S_{\max}/(1+(t_{\mathrm{mid}}/t)^\beta)) with mean (R^2 = 0.998), derived from a frontier-percolation mean-field argument on a self-similar task graph. Applied to Sep 2025 – May 2026 model releases on 18 matched-initial-performance tasks, produces the headline claim that environment-learning speed doubles every three months — the first frontier-progress metric anchored specifically on the interaction-depth axis rather than on capability leaderboards.
  • [2026-06-19] Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering: Robotics analog of test-time scaffolding: rather than scaling the VLA at training time, treats the instruction stream as the optimization variable at test time via a learned language feedback policy plus a conformalized improvement head that abstains on OOD inputs — reports +24.7% sim / +65.0% hardware over the base VLA on seen environments.
  • [2026-05-28] Introducing Claude Opus 4.8: Claude Opus 4.8 productizes the inference-compute knob end-to-end: a user-facing effort control (low / high / extra ≡ xhigh / max) ships on claude.ai across all plans next to the model selector, formalizing the spend-more-tokens-think-more axis as a default-visible product surface. Default tier is “high”, which on coding tasks spends roughly Opus-4.7-default tokens at better performance. Also relevant: dynamic workflows in Claude Code fans out to hundreds of parallel subagents in a single session — the harness-layer analog of K2.5 PARL’s learned-orchestrator axis (Kimi K2.5: Visual Agentic Intelligence), but with no claim that the orchestrator policy is RL-trained.
  • [2026-05-27] Planning at Inference: MCTS Test-Time Scaling for Long Video Generation: Adds a search-based axis to inference-time scaling for video generators: AlphaGo-style MCTS with look-ahead rollouts and backpropagated rewards picks the next clip continuation in an autoregressive video DiT. A Multi-Tree MCTS variant handles the continuous action space. Modular (no retraining); on Cosmos-Predict2 beats Best-of-N, Greedy, and Beam on object permanence, temporal coherence, and text–video alignment, producing coherent >20s videos and reporting +18%/+47% over Sora/Kling at comparable visual fidelity. Orthogonal to per-step refinement (Self-Refining Video Sampling): refinement vs. search.
  • [2026-05-26] MIGA: Enhancing Train-Free Infinite-Frame Generation for Consistent Long Videos: MIGA — long video generation as inference-time scaling: a frozen short-clip video diffusion model is extended to infinite-frame outputs by lengthening the FIFO-Diffusion queue, adding an anomaly-triggered expanded local search (only where self-similarity drops, not on a fixed interval), and injecting already-denoised low-noise frames as cross-time conditioning. Zero training; the extra capability is paid in inference compute.
  • [2026-05-25] SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers: SEGA spends inference compute to push pretrained DiTs past their training resolution: an FFT of the current latent at each denoising step drives per-RoPE-dimension attention scaling, training-free, layered on top of YaRN/DyPE/UltraImage.
  • [2026-05-23] End-to-End Test-Time Training for Long Context: TTT-E2E — a sixth axis: spend extra compute at test time as gradient steps on the model’s own weights via next-token prediction over the context, with the train-time outer loop meta-learning the initialization end-to-end through those steps. At 3B/164B tokens, matches full-attention scaling vs. context length where Mamba 2 and Gated DeltaNet fall off, while keeping RNN-style constant inference latency (2.7× faster than full attention at 128K). Distinct taxonomically from prior TTT-as-sequence-layer work (TTT-KVB / Titans / RWKV-7), where the inner-loop loss is a KV-association proxy rather than the actual next-token loss.
  • [2026-05-22] Self-Refining Video Sampling: Adds a fifth, generation-side axis — iterated Predict-and-Perturb at each fixed noise level of a pretrained flow-matching video generator, with the trained velocity field reinterpreted as a time-conditioned DAE. At ~50% extra NFEs, beats matched-compute NFE×2 and best-of-4 rejection sampling against Cosmos-Reason1-7B on Wan2.2-A14B and Cosmos-Predict2.5-2B. Two non-trivial findings: (i) for video, the trained denoiser is a stronger manifold prior than an external 7B reasoner-as-verifier; (ii) iterative refinement on CFG-trained generators oversaturates static regions unless gated by an uncertainty mask, where uncertainty is the L1 distance between consecutive z^1\hat z_1 predictions and costs zero extra NFE to compute.
  • [2026-05-22] Kimi K2.5: Visual Agentic Intelligence: Kimi K2.5 Agent Swarm + PARL — the fourth axis on this page: a learned parallel-agent orchestrator (≤100 sub-agents × ≤1,500 tool calls) trained with shaped rewards specifically targeting two newly named failure modes (serial collapse, spurious parallelism) and a latency-oriented Critical-Steps metric (longest path through the execution graph). Reported 3–4.5× Critical-Steps savings, up to 80% wall-clock reduction, +3.5 absolute BrowseComp points over single-agent context-managed inference (74.9 → 78.4 = 60.6 single-agent → +14.3 from context mgmt → +3.5 from swarm). Production-scale datapoint distinct from RLM (programmatic, not trained) and MiroThinker (sequential, not parallel).
  • [2026-05-22] Inference-Time Hyper-Scaling with KV Cache Compression: Defines the “inference-time hyper-scaling” framing — given that KV-cache memory transfer dominates decode latency, compressing the cache lets a reasoning model generate more tokens at fixed runtime/memory. Introduces DMS, a ~1K-step retrofit that learns per-head Gumbel-sigmoid eviction with delayed execution over a sliding window, reaching 8× compression and shifting the reasoning Pareto frontier on Qwen-R1 32B by +9.1/+7.6/+9.6 on AIME 24 / GPQA / LiveCodeBench. Pareto-optimal configs use both sequential and parallel scaling.
  • [2026-05-22] MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling: Names “interaction depth” as a third scaling axis (alongside model size and context length) and demonstrates monotone improvement on BrowseComp/HLE/GAIA up to several hundred tool calls per task via a 256K context + recency-based retention + streaming-rollout GRPO.
  • [2026-05-22] Recursive Language Models: the paradigm of 2026: Implements the RLM scaffold in Prime Intellect’s verifiers with sub-LM-only tools, an answer variable with a ready flag, and llm_batch parallel dispatch — finds that the scaffold is not free at inference (base LM gets worse on math-python; needs tips on DeepDive) and frames RL’ing models end-to-end as RLMs as the next research bet.
  • [2026-05-22] Recursive Language Models: Treats the prompt as a variable in a persistent Python REPL with recursive sub-LM calls; on OOLONG-Pairs RLM(GPT-5) reaches 58.0 F1 where base GPT-5 reaches 0.04. Cleanly separates “context rot” (a property of the network) from “context length” (a property of the scaffold).

Open questions

Section titled “Open questions”
  • Do the six axes stack? KV-cache compression (DMS) + REPL scaffolding (RLM) + sequential interaction-depth (MiroThinker) + learned parallel-agent orchestration (K2.5 PARL) + per-step iterative refinement on a generator (Self-Refining Video Sampling) + test-time weight updates (TTT-E2E) target different parts of the same Pareto and look orthogonal — but no filed paper has combined any two of them. TTT-E2E is the first axis that puts the extra inference compute into the weights; whether it composes with KV-cache compression (the cache is now also irrelevant, not just compressed) or whether they’re substitutes for the same long-context budget is open.
  • Where is the crossover where each axis starts paying off? RLM has explicit base-model length crossovers (Recursive Language Models Obs. 3); DMS’s Pareto gains depend on saturating the un-compressed model’s memory budget; PARL gains depend on tasks decomposing into independent sub-tasks (K2.5 reports 3–4.5× Critical-Steps savings on wide-search tasks specifically); Self-Refining Video Sampling claims 2–3 iterations are sufficient but only shows qualitative comparisons against a fixed Wan2.2/Cosmos base; TTT-E2E’s full-attention-matching curve is reported at 3B/164B and 128K, but the crossover with full attention at smaller scales and shorter contexts is not shown. A unified evaluation protocol with a shared task suite and a “compute / memory / wall-clock at iso-accuracy” axis is missing.
  • Sequence-budget gains from DMS depend on the model already being good at long-chain reasoning at the budget the cache compression unlocks. Are the gains larger on reasoning models (Qwen-R1 *) than on non-reasoning bases? The paper reports gains primarily on R1-distilled checkpoints.
  • Interaction-depth scaling assumes monotone improvement with more tool calls, but MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling doesn’t isolate “interaction-budget” from co-scaled context length and model size. Is interaction-depth a genuinely independent axis or a re-parameterization?
  • Is “learned parallel orchestration” (PARL) actually a separate axis from “scaffold-side scaling” (RLM), or is PARL just RL-tuning the scaffold? The mechanistic difference is that PARL trains the orchestrator policy end-to-end with a reward, while RLM’s orchestrator is a fixed Python program — but at inference time both produce sub-LM dispatch graphs. No filed paper compares trained-vs-programmatic orchestrators at fixed task and compute.
  • Is the “scaffold use is a learnable skill” claim (Recursive Language Models: the paradigm of 2026) symmetric for cache compression? DMS is retrofitted via distillation already, but is there a “distill into a model that knows it has a compressed cache” effect at the policy level (e.g. preferring shorter, denser reasoning), or is it transparent to the policy?
  • All six axes require non-trivial infrastructure to actually use (PagedAttention-compatible kernels for DMS; sandboxed REPL with sub-LM dispatch for RLM; streaming rollout queue for MiroThinker; large-scale parallel-agent RL for PARL; uncertainty-masked refinement loop tied into the ODE solver for Self-Refining Video Sampling; meta-learning-through-inner-loop-TTT training pipelines for TTT-E2E). Which is cheapest to drop into a generation-side video / multimodal serving stack?
  • Does the Critical-Steps metric (Kimi K2.5: Visual Agentic Intelligence §PARL) generalize as an evaluation primitive for inference-time-scaling broadly? It’s currently used only as a training signal in PARL, but the concept of “longest path through the computation graph at iso-accuracy” applies to any setting where parallel branches can be dispatched (RLM’s llm_batch, DMS’s parallel chains).
  • Self-Refining Video Sampling beats best-of-4 rejection sampling against Cosmos-Reason1-7B as verifier (Self-Refining Video Sampling §Results, “Image-to-Video Robotics”). If a trained denoiser is a stronger manifold prior than a separately-trained VLM judge for video, when does the team’s intuition that “scale the verifier” pay off relative to “scale the generator’s own refinement loop”?
  • TTT-E2E re-opens the question of whether linear-attention sequence operators (Mamba 2, Gated DeltaNet, TTT-KVB layers) are necessary at all if a small-window Transformer + outer-loop-meta-learned TTT inner loop matches full attention scaling. Is the right interpretation “the Mamba-style state-space race is misframed”, or “TTT-E2E’s outer-loop cost is hiding what those architectures front-load into a single-pass forward”? Training-compute parity is reported at 3B/164B but the meta-learning outer-loop FLOPs are not directly compared.

Papers

Section titled “Papers”

2026-05

Section titled “2026-05”

2026-04

Section titled “2026-04”

2026-03

Section titled “2026-03”

2026-02

Section titled “2026-02”

2026-01

Section titled “2026-01”

2025-12

Section titled “2025-12”

2025-11

Section titled “2025-11”

2025-10

Section titled “2025-10”

2025-09

Section titled “2025-09”

2025-08

Section titled “2025-08”

2025-07

Section titled “2025-07”

2025-06

Section titled “2025-06”

2024-12

Section titled “2024-12”

Unknown date

Section titled “Unknown date”