NaturalThoughts: Selecting and Distilling Reasoning Traces for General Reasoning Tasks

TL;DR

A data-curation study (FAIR at Meta) on what makes a reasoning-trace SFT corpus effective. The authors generate (question, R1 reasoning trace, answer) triples by running DeepSeek-R1 over questions sampled from NaturalReasoning (≈2.8M-question pool), then ablate selection strategies — random scale-up, reasoning-strategy diversity, model-disagreement difficulty, verbosity — against fixed-size 1k / 10k / 100k / 500k budgets and against existing curated sets (LIMO, S1K, OpenThoughts3). On Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct, NaturalThoughts SFT beats those baselines on general STEM reasoning (GPQA-Diamond, MMLU-Pro, SuperGPQA) and exposes a System-1/System-2 mixing recipe that improves the accuracy/length trade-off at inference.

Key claims

Simply scaling random sampling from a strong-teacher distillation corpus is a hard-to-beat baseline: performance keeps climbing through 500k examples without saturation on Llama-3.1-8B and Qwen2.5-7B [Abstract, §4].
At fixed small budgets (1k–10k), selecting examples that require more diverse reasoning strategies, or examples on which multiple candidate models disagree (a difficulty proxy), outperforms random selection; the gap narrows as the dataset grows [§4, Fig. 4].
Mixed System-1 / System-2 distillation — long-CoT traces (System-2) only on hard questions, short answers (System-1) on easy ones, split by difficulty — Pareto-beats either-pure approach and beats random mixing on accuracy vs average response length, giving the student an adaptive inference-compute allocation [§5].
NaturalThoughts-trained students outperform LIMO, S1K, and OpenThoughts3 on GPQA-Diamond, MMLU-Pro, and SuperGPQA at matched or smaller corpus sizes [Abstract, §4 Tab.].
Lagging gains on math benchmarks (AIME-style) are attributed to a source-distribution mismatch: NaturalReasoning’s prompt pool does not look like competition-math prompts, so reasoning skill transfer to those evals is weaker than to general STEM [§4 discussion].
Loss is masked on the question during SFT — training is only on the teacher’s reasoning trace and final answer — and traces are annotated with meta-reasoning strategies (self-verification, backtracking) and a verbosity score using Llama-3.1-70B-Instruct, which is what enables strategy-diversity selection [§3].

Method

The pipeline has three stages. Generation: sample questions from NaturalReasoning, prompt DeepSeek-R1 to produce (reasoning trace, answer) pairs. Annotation: Llama-3.1-70B-Instruct tags each trace with meta-reasoning strategies (self-verification, backtracking, …) and a verbosity score 0–10; questions get domain/topic tags. Selection + SFT: various selection policies (random, strategy-diversity, difficulty-by-disagreement, verbosity) draw N ∈ {1k, 10k, 100k, 500k} examples; the student (Llama-3.1-8B-Instruct or Qwen2.5-7B-Instruct) is SFT-trained via fairseq2 with AdamW and dynamic batching, loss masked on the question. The mixed-system experiment routes hard questions to long R1 traces and easy questions to short direct answers, with difficulty determined by the same disagreement proxy.

Results

Trained students beat prior curated SFT sets (LIMO, S1K, OpenThoughts3) on GPQA-Diamond, MMLU-Pro and SuperGPQA at the comparable scale [§4 Tab.]. Random scaling improves monotonically through 500k examples [§4, Fig. 3]. At 1k/10k the difficulty- and diversity-based selectors lead random by a multi-point margin, converging by 100k [§4, Fig. 4]. The System-1/System-2 mix achieves higher accuracy at shorter mean response length than either pure setting at the same total budget [§5]. Math eval gains lag the STEM-reasoning ones; the paper attributes this to source-prompt domain rather than method [§4 discussion].

Why it’s interesting

Sits squarely in the SFT-distillation lane of the post-DeepSeek-R1 reasoning literature — the same lane as OpenThoughts: Data Recipes for Reasoning Models (which the paper benchmarks against) and the earlier compute-frontier framing in Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling — but pushes the discussion from which teacher and how much data to which traces, with strategy-diversity and disagreement-difficulty as the two operational levers. It also gives the wiki its first concrete data point that random scaling is a strong baseline through 500k SFT examples, complementing Distillation Scaling Laws (which characterizes the parametric scaling) with an orthogonal data-side study. The mixed-System-1/System-2 result is a SFT-side cousin of the inference-time adaptive-compute work the wiki tracks under Inference-Time Scaling, reached without an explicit router.