Skip to content
Bud's Wiki

Inference-Time Scaling

TL;DR

Section titled “TL;DR”

Trading more inference compute for capability that the base model can’t deliver in a single forward pass — but the filed papers split into three operationally distinct axes: (a) scaffold-side scaling (RLM-style sub-LM calls in a REPL, where the root LM offloads work to fresh LM instances and uses the REPL as state); (b) interaction-depth scaling (MiroThinker-style multi-hundred-step tool-call trajectories in an open environment); and (c) sequence-budget scaling at fixed memory (DMS-style KV-cache compression that lets a reasoning model generate longer or more parallel chains-of-thought without raising peak memory). The three are largely orthogonal and look stackable, but no filed paper has combined them.

Key claims

Section titled “Key claims”
  • Inference-time compute can be traded for capability on tasks where the base model fails one-shot, and the gains hold across the four filed flavors: scaffold (RLM lifts GPT-5 from 0.04 → 58.0 F1 on OOLONG-Pairs, Recursive Language Models Table 1), interaction depth (MiroThinker shows monotone gains on BrowseComp/HLE/GAIA up to several hundred tool calls per task, MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §1, §3.3), sequence-budget compression (DMS adds +9.1 / +7.6 / +9.6 on AIME 24 / GPQA / LiveCodeBench at fixed memory-read budget on Qwen-R1 32B, Inference-Time Hyper-Scaling with KV Cache Compression Abstract, Fig. 1), and learned parallel-agent orchestration (K2.5 Agent Swarm adds +3.5 points over single-agent context-managed inference on BrowseComp, 74.9 → 78.4, with up to 80% wall-clock runtime reduction on wide-search scenarios, Kimi K2.5: Visual Agentic Intelligence §Agent Swarm, Benchmark table).
  • The dominant bottleneck for sequence-budget scaling in Transformer LLMs is KV-cache memory transfer, not arithmetic — for batch 256 and 8K–32K contexts, KV-cache reads exceed 90% of decode latency on Qwen-R1 1.5B and 80% on Qwen-R1 7B (Inference-Time Hyper-Scaling with KV Cache Compression Appendix G). This is why cache compression converts directly into more usable tokens at fixed runtime.
  • Sequence-budget scaling is Pareto-frontier shifting, not just a constant-factor speedup: DMS dominates training-free KV-sparsification baselines (Quest, TOVA, H2O) on both memory-reads and peak-memory axes across model sizes (Inference-Time Hyper-Scaling with KV Cache Compression §5.1, Figs. 3–4).
  • Pareto-optimal DMS configurations combine both longer chains (sequential scaling) and more chains (parallel scaling) — neither axis is sufficient alone (Inference-Time Hyper-Scaling with KV Cache Compression §5.1).
  • Scaffold-side scaling: treating the prompt as a variable in a persistent Python REPL and dispatching sub-LM calls on chunks lets a fixed-context LM operate on inputs ~two orders of magnitude beyond its window, with median RLM cost cheaper than median base LM (Recursive Language Models §3, Obs. 4).
  • Scaffold-use is a learnable skill, not free at inference: the Prime Intellect RLM ablations show that without tips the RLM scaffold underperforms the base LM on DeepDive, and on math-python it actively hurts compared to a plain Python tool — interpreted as base-model overfitting to the standard tool harness (Recursive Language Models: the paradigm of 2026 §Results across environments).
  • Interaction-depth scaling: MiroThinker pushes the open-source per-task tool-call ceiling from <100 to ~600 via a 256K context + recency-based retention, and reports monotone benchmark gains as a function of tool-call budget (MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §1, §3.3).
  • The infrastructure cost of interaction-depth scaling is non-trivial — online GRPO over multi-hundred-step trajectories requires a streaming rollout queue to handle long-tailed completion times, not just bigger batches (MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §5.3).
  • Cache-compression scaling is implementation-cheap: DMS adds no new parameters (an existing K/Q neuron is repurposed for the eviction logit), the eviction mask never needs to be materialized, and the sparsified cache is compatible with PagedAttention/FlashAttention kernels (Inference-Time Hyper-Scaling with KV Cache Compression §3.2 Performance Considerations, §3.3).
  • At sufficiently aggressive compression, cache compression can exceed dense attention on long-context tasks — DMS at CR=2 on Llama 3.2 1B Instruct beats the uncompressed model on NIAH (97.8 vs. 96.4) and VT (63.2 vs. 55.8), conjectured to fight over-squashing / lost-in-the-middle (Inference-Time Hyper-Scaling with KV Cache Compression Table 1, §5.2).
  • Learned parallel-agent orchestration is a fourth, distinct scaling axis: K2.5’s Agent Swarm trains an orchestrator policy via Parallel-Agent Reinforcement Learning (PARL) to dispatch up to 100 sub-agents executing concurrent workflows across up to 1,500 coordinated tool calls, with a Critical-Steps latency metric (longest path through the parallel execution graph) replacing total step count and structurally rewarding parallel strategies (Kimi K2.5: Visual Agentic Intelligence §Agent Swarm, §PARL). Distinct from RLM’s untrained programmatic scaffold (the orchestrator code is not learned, only the base LM is) and from MiroThinker’s sequential interaction-depth (a single agent, not a swarm).
  • Learned parallel orchestration has training-time failure modes that single-agent scaling doesn’t see: serial collapse (orchestrator defaults to single-agent execution despite parallel capacity) and spurious parallelism (orchestrator inflates parallel metrics by spawning sub-agents without meaningful task decomposition); both are defeated by separately shaped reward terms plus a staged schedule that shifts weight from parallelism early to task success late (Kimi K2.5: Visual Agentic Intelligence §PARL).
  • Generation-side, per-step iterative refinement is a fifth axis: a pretrained flow-matching video generator can be reused as a time-conditioned denoising autoencoder, with 2–3 Predict-and-Perturb iterations at each fixed noise level tt pulling the latent toward higher-density regions of the data manifold before the ODE step advances (Self-Refining Video Sampling §Methods, “Flow Matching as Denoising Autoencoder”). At ~50% additional NFEs, this beats both matched-compute NFE×2 (more discrete ODE steps with the same total NFE budget) and best-of-4 rejection sampling against a separately-trained Cosmos-Reason1-7B verifier on Wan2.2-A14B and Cosmos-Predict2.5-2B (Self-Refining Video Sampling §Results). The where-to-spend-the-NFE answer is “at the current noise level” rather than “in finer discretization” or “in a downstream verifier”.
  • Iterative-refinement scaling on CFG-trained generators has a failure mode: K≥3K \geq 3 Predict-and-Perturb iterations with CFG oversaturate static regions; the fix is uncertainty-aware masking that only refines regions where consecutive z^1\hat z_1 predictions disagree above a threshold, computed at zero extra NFE since both predictions already exist inside the loop (Self-Refining Video Sampling §Methods, “Uncertainty-aware P&P”). Parallels the orchestration-side oversaturation failure modes (serial collapse, spurious parallelism) in that aggressively scaling inference compute creates new pathologies that need targeted fixes.
  • Test-Time Training of model weights is a sixth axis: on top of a standard sliding-window Transformer, continue training on the test-time context via next-token prediction so context is compressed into weight updates rather than into a hidden state or KV cache; meta-learn the initialization end-to-end through those inner-loop updates (End-to-End Test-Time Training for Long Context Abstract, §1, Fig. 1 caption). At 3B/164B tokens, TTT-E2E matches a full-attention Transformer’s loss-vs-context-length curve while Mamba 2 and Gated DeltaNet do not, and retains RNN-style constant inference latency (2.7× faster than full attention at 128K) (End-to-End Test-Time Training for Long Context Abstract, Fig. 1). Distinct from the other five axes in that the extra inference compute lands as gradient steps on the network’s own weights, not as more forward passes (NFE×K, parallel chains, sub-LM calls) or cache surgery.

Recent contributions

Section titled “Recent contributions”
  • [2026-07-20] Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline): Learning to Fold introduces Thompson-sampling bandits over inference-time hyperparameters as a distinct axis of inference-time compute for flow-matching VLAs: the same checkpoint is tuned per-garment-type over execution length, playback speed, chunk-boundary inpainting onset, CFG scale, noise temperature, and best-of-N candidate count, discovered cheaply online during rollout collection. Ties together three inference-time knobs at once — best-of-N candidate selection (via the policy’s own action-conditional success-residual Q-head, no separate reranker), CFG scale (advantage conditioning), and noise/temperature sampling — and lets a bandit search the joint space rather than picking a single scalar knob. Concrete evidence that per-condition inference-time tuning of many knobs simultaneously is a competitive lever on top of a fixed policy: the same checkpoint reached 1st place in the LeHome Challenge sim round (79.63%, 6.1-point lead over 2nd across 62 teams) partly by this tuning.
  • [2026-07-17] HDR: Hierarchical Denoising for Multi-Step Visual Reasoning: HDR proposes hierarchical coarse-to-fine sampling as a structured way to spend inference compute on visual reasoning: the model first samples a small number of long-horizon planning latents, then conditions each finer level on its parent, and finally streams leaf frames. On a six-task multi-step reasoning benchmark (maze, Hanoi, sliding puzzle, Sokoban, one-line drawing, water pouring), this hierarchical spend raises overall success 34.22 → 60.29 vs. flat streaming AR diffusion, and the hierarchy also improves robustness under reduced denoising-step budgets — a rare “spend less, still win” data point for inference-time reasoning tricks.
  • [2026-07-16] Schema — Frontier Models with Our Harness Achieve ~99% on ARC-AGI-3 Public: Schema spends inference compute on iterated program synthesis + full-history backtesting + BFS over the certified simulator, converting free planning compute into saved environment actions (M0R0 Level 4: 42 actions vs 500 human baseline; 14/25 games use 1.6–5.0× fewer actions than the human reference). Distinct from the six existing axes in that the extra compute lands as executable-program search over a joint state-representation-and-transition-rule space rather than as more forward passes, cache surgery, sub-LM calls, TTT weight steps, or parallel-agent dispatch.
  • [2026-07-16] VIA: Visual Interface Agent for Robot Control: VIA (Stanford; Hu, Sundaresan, Gao, Sadigh) reports that robot-manipulation success scales with the strength of an unmodified frontier agent (Claude Code, Codex, Fable 5) driving a browser-based 3D interface via screenshots and interface commands — Fable 5 hitting 96.7% on three LIBERO-Goal tasks and 100% on long-horizon rainbow assembly. Zero robot fine-tuning; the gains come entirely from stronger inference-time reasoning / re-planning in the frontier agent. Complements the concept’s existing test-time-compute datapoints by extending the scaling axis to embodied control through a general software-agent interface.
  • [2026-07-15] Foveated Reasoning: Stateful, Action-based Visual Focusing for Vision-Language Models: FoveateR adds a visual-token-budget variant of adaptive inference-time spend: rather than spending the full high-resolution budget uniformly, the model keeps its base tokens coarse and only injects high-res patches into the same decoding trajectory when its reasoning trace emits a foveation action. Distinct from the sequence-budget compression axis (DMS) and from external-tool loops (Gemini Agentic Vision) — the extra compute lands as additional visual tokens conditional on the ongoing reasoning, and an anti-”see-everything” reward keeps the policy from collapsing to the degenerate maximum-budget baseline.
  • [2026-07-09] Introducing Muse Spark 1.1: Muse Spark 1.1 pitches “active management” of a 1M-token context (remembering earlier actions, retrieving from earlier work, compacting critical steps) as a trained behavior — the sustained-single-agent axis of test-time compute rather than the parallel-agent contemplating-mode axis of the original Muse Spark.
  • [2026-07-08] LLM-as-a-Verifier: A General-Purpose Verification Framework: Proposes verification as a distinct scaling axis alongside pre-training, post-training, and generation-side test-time compute — orthogonal to the six generation/scaffold axes on this page. A continuous logit-expectation score readout enables three composable sub-axes at the verifier: score granularity (finer bins → better positive/negative separation), repeated evaluation (variance reduction), and criteria decomposition (complexity reduction). Reports SOTA on Terminal-Bench V2 (86.5), SWE-Bench Verified (78.2), RoboRewardBench (87.4), MedAgentBench (73.3) training-free, and improves sample efficiency of SAC (robotics) and GRPO (math) when the continuous verifier score is used as dense reward. Complements Noam Brown’s inference-compute-curve framing by putting the compute on the verifier side of the generator/verifier split.
  • [2026-07-07] Muse Video preview — MSL video generation model announcement (Alexandr Wang): Muse Image spends inference-time compute on prompt reasoning, web search, and planning (via Muse Spark) before generation — a productized instance of “spend more compute pre-generation” scaling, complementing the parallel-agents contemplating-mode pattern from the earlier Muse Spark announcement.
  • [2026-07-05] Implications of Large-Scale Test-Time Compute (Noam Brown): Noam Brown (OpenAI) frames test-time-compute scaling as an evaluation-methodology problem: scalar leaderboard scores are broken when the same model spans an ~10⁵× cost range with monotonically improving scores, some recent models are still improving past 100M tokens per query when runs are stopped for infra reasons rather than a plateau, and safety evals conducted at low inference budgets can miss capabilities that emerge under sustained higher-budget inference. Proposes performance-vs-cost/tokens/time curves in place of bar charts, treating multi-agent parallel inference as a first-class deployment axis, and RSPs / third-party benchmarks that explicitly track or bound inference-compute. First-person confirmation from a frontier lab that all six axes on this page’s Key claims matter operationally.
  • [2026-07-04] EdgeBench: Scaling Laws of Environment Learning: EdgeBench turns MiroThinker’s interaction-depth axis (axis (b) on this page) from a monotone-gain observation into a fitted scaling law: averaging 402 per-model learning curves over 12–72 h of environment interaction collapses to a log-sigmoid (S(t) = S_{\max}/(1+(t_{\mathrm{mid}}/t)^\beta)) with mean (R^2 = 0.998), derived from a frontier-percolation mean-field argument on a self-similar task graph. Applied to Sep 2025 – May 2026 model releases on 18 matched-initial-performance tasks, produces the headline claim that environment-learning speed doubles every three months — the first frontier-progress metric anchored specifically on the interaction-depth axis rather than on capability leaderboards.
  • [2026-06-19] Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering: Robotics analog of test-time scaffolding: rather than scaling the VLA at training time, treats the instruction stream as the optimization variable at test time via a learned language feedback policy plus a conformalized improvement head that abstains on OOD inputs — reports +24.7% sim / +65.0% hardware over the base VLA on seen environments.
  • [2026-05-28] Introducing Claude Opus 4.8: Claude Opus 4.8 productizes the inference-compute knob end-to-end: a user-facing effort control (low / high / extra ≡ xhigh / max) ships on claude.ai across all plans next to the model selector, formalizing the spend-more-tokens-think-more axis as a default-visible product surface. Default tier is “high”, which on coding tasks spends roughly Opus-4.7-default tokens at better performance. Also relevant: dynamic workflows in Claude Code fans out to hundreds of parallel subagents in a single session — the harness-layer analog of K2.5 PARL’s learned-orchestrator axis (Kimi K2.5: Visual Agentic Intelligence), but with no claim that the orchestrator policy is RL-trained.
  • [2026-05-27] Planning at Inference: MCTS Test-Time Scaling for Long Video Generation: Adds a search-based axis to inference-time scaling for video generators: AlphaGo-style MCTS with look-ahead rollouts and backpropagated rewards picks the next clip continuation in an autoregressive video DiT. A Multi-Tree MCTS variant handles the continuous action space. Modular (no retraining); on Cosmos-Predict2 beats Best-of-N, Greedy, and Beam on object permanence, temporal coherence, and text–video alignment, producing coherent >20s videos and reporting +18%/+47% over Sora/Kling at comparable visual fidelity. Orthogonal to per-step refinement (Self-Refining Video Sampling): refinement vs. search.
  • [2026-05-26] MIGA: Enhancing Train-Free Infinite-Frame Generation for Consistent Long Videos: MIGA — long video generation as inference-time scaling: a frozen short-clip video diffusion model is extended to infinite-frame outputs by lengthening the FIFO-Diffusion queue, adding an anomaly-triggered expanded local search (only where self-similarity drops, not on a fixed interval), and injecting already-denoised low-noise frames as cross-time conditioning. Zero training; the extra capability is paid in inference compute.
  • [2026-05-25] SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers: SEGA spends inference compute to push pretrained DiTs past their training resolution: an FFT of the current latent at each denoising step drives per-RoPE-dimension attention scaling, training-free, layered on top of YaRN/DyPE/UltraImage.
  • [2026-05-23] End-to-End Test-Time Training for Long Context: TTT-E2E — a sixth axis: spend extra compute at test time as gradient steps on the model’s own weights via next-token prediction over the context, with the train-time outer loop meta-learning the initialization end-to-end through those steps. At 3B/164B tokens, matches full-attention scaling vs. context length where Mamba 2 and Gated DeltaNet fall off, while keeping RNN-style constant inference latency (2.7× faster than full attention at 128K). Distinct taxonomically from prior TTT-as-sequence-layer work (TTT-KVB / Titans / RWKV-7), where the inner-loop loss is a KV-association proxy rather than the actual next-token loss.
  • [2026-05-22] Self-Refining Video Sampling: Adds a fifth, generation-side axis — iterated Predict-and-Perturb at each fixed noise level of a pretrained flow-matching video generator, with the trained velocity field reinterpreted as a time-conditioned DAE. At ~50% extra NFEs, beats matched-compute NFE×2 and best-of-4 rejection sampling against Cosmos-Reason1-7B on Wan2.2-A14B and Cosmos-Predict2.5-2B. Two non-trivial findings: (i) for video, the trained denoiser is a stronger manifold prior than an external 7B reasoner-as-verifier; (ii) iterative refinement on CFG-trained generators oversaturates static regions unless gated by an uncertainty mask, where uncertainty is the L1 distance between consecutive z^1\hat z_1 predictions and costs zero extra NFE to compute.
  • [2026-05-22] Kimi K2.5: Visual Agentic Intelligence: Kimi K2.5 Agent Swarm + PARL — the fourth axis on this page: a learned parallel-agent orchestrator (≤100 sub-agents × ≤1,500 tool calls) trained with shaped rewards specifically targeting two newly named failure modes (serial collapse, spurious parallelism) and a latency-oriented Critical-Steps metric (longest path through the execution graph). Reported 3–4.5× Critical-Steps savings, up to 80% wall-clock reduction, +3.5 absolute BrowseComp points over single-agent context-managed inference (74.9 → 78.4 = 60.6 single-agent → +14.3 from context mgmt → +3.5 from swarm). Production-scale datapoint distinct from RLM (programmatic, not trained) and MiroThinker (sequential, not parallel).
  • [2026-05-22] Inference-Time Hyper-Scaling with KV Cache Compression: Defines the “inference-time hyper-scaling” framing — given that KV-cache memory transfer dominates decode latency, compressing the cache lets a reasoning model generate more tokens at fixed runtime/memory. Introduces DMS, a ~1K-step retrofit that learns per-head Gumbel-sigmoid eviction with delayed execution over a sliding window, reaching 8× compression and shifting the reasoning Pareto frontier on Qwen-R1 32B by +9.1/+7.6/+9.6 on AIME 24 / GPQA / LiveCodeBench. Pareto-optimal configs use both sequential and parallel scaling.
  • [2026-05-22] MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling: Names “interaction depth” as a third scaling axis (alongside model size and context length) and demonstrates monotone improvement on BrowseComp/HLE/GAIA up to several hundred tool calls per task via a 256K context + recency-based retention + streaming-rollout GRPO.
  • [2026-05-22] Recursive Language Models: the paradigm of 2026: Implements the RLM scaffold in Prime Intellect’s verifiers with sub-LM-only tools, an answer variable with a ready flag, and llm_batch parallel dispatch — finds that the scaffold is not free at inference (base LM gets worse on math-python; needs tips on DeepDive) and frames RL’ing models end-to-end as RLMs as the next research bet.
  • [2026-05-22] Recursive Language Models: Treats the prompt as a variable in a persistent Python REPL with recursive sub-LM calls; on OOLONG-Pairs RLM(GPT-5) reaches 58.0 F1 where base GPT-5 reaches 0.04. Cleanly separates “context rot” (a property of the network) from “context length” (a property of the scaffold).

Open questions

Section titled “Open questions”
  • Do the six axes stack? KV-cache compression (DMS) + REPL scaffolding (RLM) + sequential interaction-depth (MiroThinker) + learned parallel-agent orchestration (K2.5 PARL) + per-step iterative refinement on a generator (Self-Refining Video Sampling) + test-time weight updates (TTT-E2E) target different parts of the same Pareto and look orthogonal — but no filed paper has combined any two of them. TTT-E2E is the first axis that puts the extra inference compute into the weights; whether it composes with KV-cache compression (the cache is now also irrelevant, not just compressed) or whether they’re substitutes for the same long-context budget is open.
  • Where is the crossover where each axis starts paying off? RLM has explicit base-model length crossovers (Recursive Language Models Obs. 3); DMS’s Pareto gains depend on saturating the un-compressed model’s memory budget; PARL gains depend on tasks decomposing into independent sub-tasks (K2.5 reports 3–4.5× Critical-Steps savings on wide-search tasks specifically); Self-Refining Video Sampling claims 2–3 iterations are sufficient but only shows qualitative comparisons against a fixed Wan2.2/Cosmos base; TTT-E2E’s full-attention-matching curve is reported at 3B/164B and 128K, but the crossover with full attention at smaller scales and shorter contexts is not shown. A unified evaluation protocol with a shared task suite and a “compute / memory / wall-clock at iso-accuracy” axis is missing.
  • Sequence-budget gains from DMS depend on the model already being good at long-chain reasoning at the budget the cache compression unlocks. Are the gains larger on reasoning models (Qwen-R1 *) than on non-reasoning bases? The paper reports gains primarily on R1-distilled checkpoints.
  • Interaction-depth scaling assumes monotone improvement with more tool calls, but MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling doesn’t isolate “interaction-budget” from co-scaled context length and model size. Is interaction-depth a genuinely independent axis or a re-parameterization?
  • Is “learned parallel orchestration” (PARL) actually a separate axis from “scaffold-side scaling” (RLM), or is PARL just RL-tuning the scaffold? The mechanistic difference is that PARL trains the orchestrator policy end-to-end with a reward, while RLM’s orchestrator is a fixed Python program — but at inference time both produce sub-LM dispatch graphs. No filed paper compares trained-vs-programmatic orchestrators at fixed task and compute.
  • Is the “scaffold use is a learnable skill” claim (Recursive Language Models: the paradigm of 2026) symmetric for cache compression? DMS is retrofitted via distillation already, but is there a “distill into a model that knows it has a compressed cache” effect at the policy level (e.g. preferring shorter, denser reasoning), or is it transparent to the policy?
  • All six axes require non-trivial infrastructure to actually use (PagedAttention-compatible kernels for DMS; sandboxed REPL with sub-LM dispatch for RLM; streaming rollout queue for MiroThinker; large-scale parallel-agent RL for PARL; uncertainty-masked refinement loop tied into the ODE solver for Self-Refining Video Sampling; meta-learning-through-inner-loop-TTT training pipelines for TTT-E2E). Which is cheapest to drop into a generation-side video / multimodal serving stack?
  • Does the Critical-Steps metric (Kimi K2.5: Visual Agentic Intelligence §PARL) generalize as an evaluation primitive for inference-time-scaling broadly? It’s currently used only as a training signal in PARL, but the concept of “longest path through the computation graph at iso-accuracy” applies to any setting where parallel branches can be dispatched (RLM’s llm_batch, DMS’s parallel chains).
  • Self-Refining Video Sampling beats best-of-4 rejection sampling against Cosmos-Reason1-7B as verifier (Self-Refining Video Sampling §Results, “Image-to-Video Robotics”). If a trained denoiser is a stronger manifold prior than a separately-trained VLM judge for video, when does the team’s intuition that “scale the verifier” pay off relative to “scale the generator’s own refinement loop”?
  • TTT-E2E re-opens the question of whether linear-attention sequence operators (Mamba 2, Gated DeltaNet, TTT-KVB layers) are necessary at all if a small-window Transformer + outer-loop-meta-learned TTT inner loop matches full attention scaling. Is the right interpretation “the Mamba-style state-space race is misframed”, or “TTT-E2E’s outer-loop cost is hiding what those architectures front-load into a single-pass forward”? Training-compute parity is reported at 3B/164B but the meta-learning outer-loop FLOPs are not directly compared.

Papers

Section titled “Papers”

2026-05

Section titled “2026-05”

2026-04

Section titled “2026-04”

2026-03

Section titled “2026-03”

2026-02

Section titled “2026-02”

2026-01

Section titled “2026-01”

2025-12

Section titled “2025-12”

2025-11

Section titled “2025-11”

2025-10

Section titled “2025-10”

2025-09

Section titled “2025-09”

2025-08

Section titled “2025-08”

2025-07

Section titled “2025-07”

2025-06

Section titled “2025-06”

2024-12

Section titled “2024-12”

Unknown date

Section titled “Unknown date”