Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights

TL;DR

After pretraining, the Gaussian neighborhood around an LLM’s weights is densely populated with task-improving “specialist” solutions — a regime the authors call the thicket regime. This density grows monotonically with model scale (measured 0.5B → 32B on Qwen2.5), turning random weight perturbation from “needle-in-a-haystack” into a viable post-training operator. The authors instantiate this with RandOpt: sample $N$ random Gaussian perturbations, select the top- $K$ on validation, and ensemble at inference via majority vote — fully parallel, no gradients, no sequential update steps. RandOpt matches or beats PPO / GRPO / ES at equal post-training FLOPs across math (Countdown, GSM8K, MATH-500, OlyBench), code (MBPP), creative writing (ROCStories), and chemistry (USPTO) on Qwen / Llama / OLMo3 base and instruct variants in the 0.5B–8B range, with a 200×GH200 demo training Olmo-3-7B-Instruct on Countdown in 3.2 minutes to 70% accuracy.

Key claims

The “solution density” $\rho_\epsilon$ — the probability that a Gaussian perturbation around the pretrained weights improves task score by at least $\epsilon$ — increases monotonically with model scale across multiple $\epsilon$ thresholds, holding the Gaussian radius fixed [§2.1, Fig. 3].
Sampled perturbations are specialists, not generalists: the Spectral Discordance $\delta$ over 500 perturbations × 7 tasks increases monotonically with Qwen2.5 size, confirming task rankings become increasingly disjoint at scale [§2.2, Fig. 3(b), Fig. 4].
RandOpt with population $N{=}5000$ and ensemble size $K{=}50$ mostly matches or outperforms PPO, GRPO, ES, and Test-Time Majority Vote across Qwen, Llama, OLMo3 (0.5B–8B) at equal post-training FLOPs [§5.1, Fig. 6, App. Tab. D].
Ensembling is load-bearing: $K{=}1$ is substantially worse than $K{=}50$ (Fig. 1c, Fig. 11) — the population diversity is what’s being exploited, not just the top sample [§5.1].
Ensembling lifts all baselines too: 50-pass Test-Time Majority Vote on top of PPO / GRPO / ES converges them to ~similar performance, and the gap among training methods shrinks over time [§5.4, Fig. 1c, App. Tab. 4].
Wall-clock advantage at parallel compute: on 200 GH200s, RandOpt trains Olmo-3-7B-Instruct on Countdown to 70% accuracy in 3.2 minutes — sequential baselines run hundreds of steps [§5.1].
VLM transfer: on Qwen2.5-VL-3B-Instruct, perturbing only the language model while freezing the vision encoder lifts GQA accuracy from 56.6% → 69.0% (+12.4) [§5.2, Table 1].
A minimal 1D autoregressive MLP setup reproduces three regimes: needle-in-haystack (no pretraining), thicket (mixed-signal pretraining), and plateau (pretraining on a single signal type that’s also the test task) [§3, Fig. 5, App. F].
The Emergence of Thickets at Scale: RandOpt offers no improvement on GPT-2 0.1B, small gains on Qwen 0.5B, and a “rapid increase in accuracy” starting around ~1.5B parameters; RandOpt-from-scratch (no pretraining) stays near zero at all scales [§6, Fig. 8].
Gains decompose into “format thickets” and “reasoning thickets”: on GSM8K, both surface-level format corrections and genuine numerical-reasoning corrections contribute meaningfully — i.e. the thicket effect isn’t reducible to formatting fixes alone [§8, Fig. 9].
Top- $K$ distillation is cheap: on Qwen2.5-1.5B-Inst the distilled student reaches 74.9% GSM8K vs 76.4% for the $K{=}50$ ensemble and 58.8% base, at ~2% of training cost [§7, Table — distillation row].
Sandbagging is unlikely as an explanation because OLMo3-7B-Base (fully open training data, no plausible sandbagging) shows substantial RandOpt gains [§5.3, App. G].

Method

RandOpt has a training phase and an inference phase, neither of which involves gradients.

Training (random guessing and checking). Sample seeds $\{s_i\}_{i=1}^N$ and per-seed noise scales $\sigma_i$ drawn uniformly from a fixed set $S = \{0.001, 0.003, 0.01, \ldots\}$ . Each $s_i$ yields a perturbed parameter vector $\theta_i = \theta_{\text{pre}} + \sigma_i \cdot \epsilon(s_i)$ , where $\epsilon(s_i) \sim \mathcal{N}(0, I)$ . Evaluate each $\theta_i$ on a small training/validation set, keep the top- $K$ by score. All $N$ evaluations are embarrassingly parallel.

Inference (ensembling). For test input $x$ , run all $K$ kept models, aggregate via majority voting. For free-form outputs the authors use exact-match majority vote on extracted final answers (e.g. GSM8K’s ####-tagged answer).

The “training” is essentially Best-of-N in weight space rather than in output space — Best-of-N at inference is what TT-MV does. The authors argue both are operating on the same underlying density: a strong pretrained model already contains diverse experts; you can either sample them in weight space (RandOpt) or in output space (TT-MV / Best-of-N), and ensembling them composes their complementary skills.

The 1D autoregressive MLP control study (§3) makes the regime transition concrete: an MLP pretrained on a mixture of sinusoid / linear / harmonic / sigmoidal / sawtooth / square waves shows diverse, fit-improving rollouts under weight perturbation when probed with a held-out linear test signal; the same network with Xavier or Kaiming init shows no useful variation; the same network pretrained on linear signals only sits on a plateau where perturbations only hurt.

Results

LLM headline (Fig. 6, App. Tab. D): Across Qwen / Llama / OLMo3 0.5B–8B base+instruct on Countdown, GSM8K, MATH-500, OlyBench, MBPP, ROCStories, USPTO, RandOpt with $N{=}5000$ , $K{=}50$ mostly matches or beats PPO, GRPO, ES, TT-MV at equal post-training FLOPs.
VLM (Table 1): Qwen2.5-VL-3B-Instruct on GQA: 56.6 → 69.0 (+12.4), perturbing LM only, vision encoder frozen.
Distillation (Table near §7): Qwen2.5-1.5B-Inst on GSM8K: base 58.8 → distill 74.9 → RandOpt $K{=}50$ 76.4; Qwen2.5-3B-Inst: base 79.8 → distill 84.3 → RandOpt 87.1. Distillation reaches ~98% of the ensemble’s gain at ~2% of training cost.
Thickets scaling (Fig. 8): RandOpt is flat or worse than base for ≤1B params; a rapid lift starts around 1.5B, then narrows as the base catches up. RandOpt-from-scratch stays near zero at all scales tested.
Format vs reasoning decomposition (Fig. 9): On GSM8K with Qwen2.5-3B-Inst, RandOpt’s overall gain decomposes into roughly comparable contributions from format corrections and reasoning corrections, with a small regression bucket (base correct → adapted incorrect). The substantive-reasoning contribution rules out the “this is all formatting” critique.
Wall-clock (§5.1): 200×GH200 cluster trains Olmo-3-7B-Instruct on Countdown in 3.2 minutes at $K{=}50$ to 70% accuracy.
Spurious-rewards connection (§9.3): For some tasks, density is so high that most Gaussian perturbations increase accuracy, providing a partial mechanistic explanation for the prior finding that post-training on random/spurious rewards sometimes works (Shao et al., 2025).

Why it’s interesting

RandOpt re-opens the question of what GRPO actually does during post-training. If sampling-and-voting in weight space matches the leading on-policy RL recipe at equal FLOPs, then much of what filed reasoning-RL pipelines (Reasoning RL) attribute to gradient steps may be selection from a structure that pretraining has already laid down — i.e. the right framing of post-training is “find the right specialist in the thicket” rather than “build the specialist”. This is a sharper version of the “post-training as selection / reweighting” thesis that papers like Training-Free GRPO (Training-Free Group Relative Policy Optimization) have been circling: that paper kept the LLM frozen and moved the GRPO primitive into context space; this paper keeps gradients out entirely and shows the same primitive works in weight space.

The K-pass inference ensembling also slots RandOpt directly into the inference-time-scaling taxonomy (Inference-Time Scaling): it sits alongside scaffold-side scaling (RLM), interaction-depth (MiroThinker), KV-cache compression (DMS), parallel-agent orchestration (PARL), iterative refinement (Self-Refining Video Sampling), and test-time weight updates (TTT-E2E) as a seventh axis — “spend inference compute on a diverse weight-space ensemble”. Unlike the others, RandOpt’s inference cost is reducible by distillation to ~2% of the training cost (§7), making it a candidate for training-time selection that is largely free at deployment.

Practically: the smaller-weaker-yet-better finding (Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling) — sampling more from a cheaper model beats sampling fewer from a stronger one for synthetic reasoning data — has the same flavor (broad sampling > concentrated optimization) but operates in output space rather than weight space.