Learning to Reason in 13 Parameters

TL;DR

TinyLoRA is a re-parameterization of LoRA that scales smoothly to fewer than one parameter per layer, all the way down to a single trained scalar for an entire 7–8B model. Trained with GRPO on GSM8K, Qwen2.5-7B-Instruct hits 91% accuracy with 13 trained parameters in bf16 (26 bytes total), and ≥90% of full-FT performance can be recovered on AIME/AMC/MATH500 at ~1000× fewer parameters. Crucially, this only works under RL: matched-capacity SFT updates plateau near the base-model baseline. The paper is read here as a strong claim about what RL is actually doing to a model — installing a very low-information-rate update that SFT cannot match bit-for-bit.

Key claims

TinyLoRA generalizes LoRA below rank=1 by sharing a single low-rank factor across layers (and across position blocks within a layer), so the trained parameter count can be set arbitrarily low — down to a single scalar — independently of the model’s hidden dimension [§Abstract, §Method].
Qwen2.5-7B-Instruct trained with TinyLoRA + GRPO on GSM8K reaches 91% accuracy with 13 trained parameters (26 bytes in bf16), vs. a 76% base-model baseline [§Abstract, Fig. 1].
Training a single parameter with TinyLoRA + GRPO already yields a +4% absolute improvement on GSM8K [§Results].
TinyLoRA recovers ≥90% of full-fine-tune performance gain on AIME, AMC, and MATH500 while training ~1000× fewer parameters [§Abstract, §Results].
RL ≫ SFT in the low-parameter regime. For a 76% baseline, RL gives 91% at 13 params and 95% at 120 params, while SFT gives 83% at 13 params and 84% at 120 params — i.e. SFT updates of this size barely outperform the base model on the training set [§Results, Fig. 2].
SFT requires ~100–1000× larger updates than RL to reach equivalent accuracy across the reasoning benchmark suite [§Abstract].
bf16 is sub-optimal vs fp32 for these ultra-small updates: storing parameters in fp32 is “most performant bit-for-bit” — the precision of each trained scalar matters when there are only a handful of them [§Method].
A smooth interpolation exists from LoRA-XS at high rank → LoRA at low rank → TinyLoRA below rank=1, all evaluated on the same GSM8K + GRPO sweep [§Results].
The SFT-side mechanism is partially attributed to off-policy dynamics — training directly on (problem, answer) pairs is a form of off-policy learning, and the authors flag that dynamics could change if SFT instead trained on the model’s own outputs [§Discussion].

Method

TinyLoRA is a strict generalization of LoRA’s ΔW = B·A decomposition. Standard LoRA already collapses a d × d weight delta into d·r + r·d = 2dr trainable parameters; at r=1 this floor is 2d, which for an 8B model is still thousands of parameters per layer. TinyLoRA breaks the per-layer floor by sharing the low-rank factors across layers and across position blocks, so the total trained parameter count is decoupled from d and from layer count. At the extreme, a single scalar parameterizes the update for the entire network.

The training pipeline is standard reasoning-RL: GRPO (Shao et al., 2024) over GSM8K rollouts from Qwen2.5-7B-Instruct, with correctness-based rewards. The authors sweep parameter count from 1 → ~10⁴ and compare to (a) the base model, (b) full fine-tuning, (c) LoRA/LoRA-XS at matched parameter counts, and (d) SFT at matched parameter counts under the same TinyLoRA parameterization. On harder benchmarks (MATH, AIME, AMC) the same recipe is repeated and the parameter-vs-accuracy curve is reported.

A precision ablation finds fp32 storage of the trained scalars dominates bf16 bit-for-bit, an artifact of how few parameters there are: each scalar’s representable precision becomes a meaningful information bottleneck.

Results

GSM8K (Qwen2.5-7B-Instruct). Base: 76%. TinyLoRA + GRPO at 1 param: ~80% (+4). At 13 params: 91% (+15). At 120 params: 95%. Within 5% of full fine-tuning at 13 parameters [§Results, Fig. 1, Fig. 2].
SFT control at matched capacity. TinyLoRA + SFT at 13 params: 83%. At 120 params: 84%. The SFT curve plateaus near the base model; RL’s curve climbs steeply [§Results, Fig. 2].
Harder benchmarks (AIME / AMC / MATH500). Recovers ~90% of the full-FT performance improvement while training 1000× fewer parameters; absolute numbers not transcribed here [§Abstract, §Results].
Precision ablation. fp32-stored TinyLoRA parameters > bf16-stored at matched bit budget [§Method].
LoRA→TinyLoRA continuum. The accuracy-vs-parameters curve interpolates smoothly through LoRA-XS at higher rank into TinyLoRA below rank=1, with a less-smooth seam between LoRA-XS and standard LoRA at the low-rank crossover [§Results].

Why it’s interesting

The headline number (91% GSM8K from 13 parameters) is striking on its own, but the load-bearing claim for the wiki is the RL-vs-SFT information-density gap: at matched update capacity, GRPO yields ~15 absolute points more than SFT. That reframes the reasoning-RL story from “RL is the right objective on top of a reasoning model” to “RL is installing a much smaller, more concentrated update than SFT can produce” — which is consistent with, but tighter than, claims in the broader RLVR / GRPO literature this wiki tracks under Reasoning RL. If the result holds, it provides an actionable design lever for PEFT in the reasoning-post-training regime (the cheapest viable post-training adapter just got 1000× smaller). It also raises a concrete mechanistic question: what is RL writing into those 13 parameters that SFT can’t, given both have access to the same task and the same gradient pathway?