Test-Time Scaling Makes Overtraining Compute-Optimal

TL;DR

T2T (“Train-to-Test”) scaling laws jointly optimize model size $N$ , training tokens $D$ , and the number of inference samples $k$ under a single end-to-end compute budget that includes both training ( $C_T \propto ND$ ) and per-query inference ( $C_I \propto Nk$ ). Two complementary formulations are fit — a Chinchilla-form NLL with an added $k^{-\gamma}$ term, and a Beta-regression model of pass@ $k$ accuracy — and they agree: once you commit to a repeated-sampling deployment, the compute-optimal pretraining decision shifts dramatically into the overtrained regime, well past Chinchilla’s 20-tokens-per-parameter rule. The authors validate by pretraining 21 overtrained checkpoints from scratch and show they consistently beat the Chinchilla-optimal checkpoint, with the shift surviving SFT.

Key claims

Under Chinchilla scaling alone the compute-optimal $D^*/N^* \approx 20$ ; once an inference budget $C_I$ via repeated sampling is folded in, T2T’s optimum moves to substantially smaller, more overtrained models that get more samples at inference [§1, §4.1, Figs. 2–3].
Two independent T2T modeling approaches give the same recommendation: Approach 1 fits the negative-log pass@ $k$ as Chinchilla + a $K^{-\gamma}$ power law in samples; Approach 2 models per-question difficulty as a Beta distribution with mean/sample-size linked to Chinchilla loss via Beta regression, then composes the closed-form pass@ $k$ [§3.1, §3.2].
Inference-cost correction is multiplicative and analogous to training cost: $C_I \approx 2 N k$ per query (one forward pass per sample), so equalizing $C_I$ across model sizes gives smaller models proportionally more samples [§3.3].
Forecasts extrapolate: fits on 85 Chinchilla-style checkpoints from porian2024resolving correctly predict the relative ordering of 21 new from-scratch overtrained checkpoints; Approach 1 relative error 2.8%, Approach 2 8.4% [§4.2, Fig. 4].
The best small overtrained checkpoint at $k=10$ beats the Chinchilla-optimal checkpoint on every one of the 8 evaluated tasks — e.g. LAMBADA OpenAI 49.90% (37M) vs 27.30% (455M); Simple Reasoning 57.90% (37M) vs 18.40% (901M) [Table 1].
The shift survives post-training: after both standard FT and SFT on ARC-Easy / SciQ / OpenBookQA, the best overtrained checkpoint still beats the Chinchilla-optimal one (e.g. SFT SciQ: 66.80% at 84M vs 57.60% at 347M) — the optimal-frontier shift is somewhat subdued vs base models but does not revert to Chinchilla [§4.3, Table 2, Fig. 5].
The recovery property is preserved: setting $k=1$ recovers Chinchilla scaling exactly from Approach 1 (the $K^{-\gamma}$ term collapses into a constant absorbed by $a$ ) [§3.1].
Naive pass@ $k$ modeling that maps mean NLL → mean accuracy and then computes $1 - (1-\bar p)^k$ overestimates true mean pass@ $k$ due to the concavity of pass@ $k$ in $p$ ; Beta-regression over per-question difficulty corrects this [§3.2].
Checkpoint suite spans 5M–901M parameters and roughly 50M–120B tokens (106 checkpoints, 12 compute levels, three orders of magnitude in FLOPs) [§4.1, §4.2, Appendix C, Fig. 8].
The non-monotonic Chinchilla-frontier-under-inference-correction (smaller models with more samples sometimes beat larger ones) reproduces snell2024scaling; T2T’s frontier is both monotonic and stronger than the inference-corrected Chinchilla frontier [§4.1, Fig. 3].

Method

The setup builds Chinchilla’s loss $L(N, D) = E + A N^{-\alpha} + B D^{-\beta}$ into a joint pre-/test-time framework. Approach 1 models the negative log pass@ $k$ directly as a Chinchilla-form loss with an additive $K^{-\gamma}$ power-law term on sample count — motivated by Jensen-bounded scaling of pass@ $k$ under a Beta-difficulty assumption — yielding an inference-corrected loss that is a function of $(N, D, k)$ jointly. Approach 2 models pass@ $k$ as a true accuracy: per-question single-pass success probabilities are assumed Beta-distributed with mean $\bar p$ and sample size $\nu$ , where $\bar p$ is linked via a logit to the Chinchilla NLL prediction and $\nu$ via a log link (a standard Beta-regression parameterization); the model’s pass@ $k$ then follows from the Beta-function expression $1 - B(\nu - \bar p \nu + k, \bar p \nu)/B(\nu - \bar p \nu, \bar p \nu)$ . For each, the inference correction is plugged in via $C_I \approx 2 N k$ per query, and the joint optimization is run at fixed $C = C_T + C_I$ . Empirically they re-use the Chinchilla scaling checkpoints from porian2024resolving (85 Chinchilla-grid models on RefinedWeb, OpenLM framework, AdamW with $\eta = 3 \times 10^{-4}$ ) and add 21 from-scratch overtrained checkpoints with identical hyperparameters; evaluation is 8 tasks (4 real: LAMBADA-OAI, ARC-Easy, SciQ, OpenBookQA; 4 synthetic: knowledge recall, multi-step arithmetic, commonsense causal, spatial reasoning).

Results

The headline isoFLOP figures (Figs. 2–3) show that across 12 pretraining-FLOPs levels from $\approx 10^{18}$ to $10^{24}$ , the T2T-optimal frontier sits at dramatically smaller $N^*$ and larger $D^*/N^*$ than the Chinchilla frontier under $C_I = 8 \times 10^{10}$ FLOPs and $k=10$ ; both approaches agree (Approach 2 prescribes more aggressive overtraining than Approach 1). The extrapolation test (Fig. 4) shows that fits restricted to the Chinchilla grid still extrapolate to the overtrained corner with single-digit relative error, with Approach 1 tighter (2.8% vs 8.4%). Table 1 quantifies the empirical win on base models: at $C = 10^{21}$ FLOPs and $k=10$ , the best overtrained checkpoint beats Chinchilla-optimal on every task, often by 2–4× absolute accuracy (LAMBADA-OAI 49.90% vs 27.30%, Simple Reasoning 57.90% vs 18.40%, Spatial Reasoning 6.00% vs 1.10%). After post-training (Table 2), the gap shrinks but stays decisively on the overtrained side (SciQ SFT: 66.80% at 84M vs 57.60% at 347M; ARC-Easy SFT: 8.20% at 37M vs 3.40% at 455M). Fig. 5 reports the same conclusion in isoFLOP form post-FT/SFT: the optimal frontier shifts toward smaller, more-overtrained, more-sampled models in every case, though “somewhat subdued compared to T2T on the base models alone” — consistent with prior reports that overtrained models are harder to fine-tune (springer2025overtrained).

Why it’s interesting

This is the missing link between the team’s existing Hyperparameter scaling laws entries — which all hold the inference regime fixed and tune $N, D, \eta$ — and the Inference-Time Scaling entries, which all hold the pretrained model fixed and ask how to spend test-time compute. T2T closes the loop: if a frontier model will be deployed at pass@ $k$ with $k$ large (which is exactly the regime where Inference-Time Hyper-Scaling with KV Cache Compression‘s DMS and MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling‘s interaction-depth axis pay off), the pretraining allocation should swing hard into overtraining. It also gives a principled rationale for what model families like Llama-2-7B (290 tokens/param), Gemma-7B (857), and Gemma-2-9B (889) have been doing empirically — they were aggressively overtrained for inference-cost reasons, and T2T says that’s also the right thing to do when you plan to spend test-time compute on repeated sampling. The compute-matching framing is the same lever as Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling (sample more from a weaker model at fixed FLOPs beats sampling fewer from a stronger one); T2T essentially provides the pretraining-side prescription that matches that data-generation-side finding.