OpenThoughts-Agent: Data Recipes for Agentic Models

TL;DR

OpenThoughts-Agent (OT-Agent) is a fully open SFT data-curation pipeline for broadly capable agentic models — the agentic-coding successor to OpenThoughts3 in the same group’s reasoning-data line (OpenThoughts: Data Recipes for Reasoning Models). The authors run 100+ controlled ablations across pipeline stages, assemble a 100K-example training set, and fine-tune Qwen3-32B to get 44.8% average across seven agentic benchmarks — +3.9 points over the strongest prior open-data baseline (Nemotron-Terminal-32B at 40.9%) — with strong scaling: at every training-set size in compute-controlled comparisons, OT-Agent beats alternative open datasets. The headline framing is that single-benchmark targeting (SWE-Smith, SERA, Nemotron-Terminal) leaves the cross-benchmark generalization question on the table, and the OT-Agent pipeline isolates which axes (task source diversity, sourcing, etc.) actually transfer.

Key claims

Existing open agentic data efforts (SWE-Smith, SERA, Nemotron-Terminal) each target a single benchmark, leaving the cross-task generalization question open [Abstract].
100+ controlled ablation experiments across pipeline stages systematically isolate the contribution of each stage, surfacing the importance of task sources and diversity as load-bearing factors [Abstract].
Fine-tuning Qwen3-32B on the 100K-example OT-Agent training set yields 44.8% average accuracy across seven agentic benchmarks, a +3.9 percentage-point improvement over the strongest prior open-data agentic baseline (Nemotron-Terminal-32B at 40.9%) [Abstract].
The OT-Agent data exhibits strong scaling: at every training-set size in compute-controlled comparisons, it outperforms alternative open agentic datasets [Abstract].
Full release: training sets, data pipeline, experimental data, and trained models are publicly released at the OpenThoughts-Agent GitHub repo [Abstract].

Method

The pipeline targets agentic SFT data — multi-turn trajectories with tool calls, environment observations, and verifiable end states — rather than single-shot reasoning traces. Each pipeline stage (task sourcing, source mixing, filtering, etc.) is ablated independently while the others are held at a fixed default, mirroring the methodology of the predecessor OpenThoughts3 reasoning-data study. The winning configurations are then stacked into a single recipe and scaled to 100K examples, used to SFT Qwen3-32B. Evaluation spans seven agentic benchmarks (specific suite not enumerated in the abstract); the headline claim is compute-controlled scaling — at every training-set size, the OT-Agent recipe beats alternative open datasets [Abstract]. The official code, data, and models live at github.com/open-thoughts/OpenThoughts-Agent.

Results

44.8% average across seven agentic benchmarks for OT-Agent-tuned Qwen3-32B vs 40.9% for Nemotron-Terminal-32B (+3.9 pp), the strongest prior open-data baseline [Abstract].
Strong-scaling claim: at every training-set size in compute-controlled comparisons, OT-Agent outperforms alternative open datasets [Abstract].

(Per-benchmark numbers, per-stage ablation deltas, and the seven-benchmark suite composition are not disclosed in the abstract; the linked GitHub repo holds the full experimental data.)

Why it’s interesting

OT-Agent is the cleanest filed open analog to the closed-source recipe space the agentic-SWE concept page has been tracking. It extends two lines simultaneously: (a) the OpenThoughts: Data Recipes for Reasoning Models methodology — controlled per-stage ablations, scale-up of stacked winners, base model held fixed — from reasoning SFT to agentic SFT, supplying the first apples-to-apples comparison at the agentic-data layer; (b) the On Data Engineering for Scaling LLM Terminal Capabilities / Nemotron-Terminal line on which task sources matter for terminal/agentic capability, with the difference that Nemotron-Terminal optimizes for Terminal-Bench specifically while OT-Agent optimizes for cross-benchmark transfer. The +3.9-point gap vs Nemotron-Terminal-32B at the same active parameter count is the load-bearing datapoint for the cross-task-generalization framing. The paper also fits the broader Synthetic Training Data thesis — curation gate + controlled-ablation methodology dominates returns at fixed compute — and continues the recipe-pluralism push against the “single-benchmark synthesis” pattern that Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing and TMax: A Simple Recipe for Terminal Agents both implicitly inherited. Worth contrasting with TMax: A Simple Recipe for Terminal Agents: TMax bets on RL over an open environment corpus; OT-Agent bets on SFT data curation. Both target overlapping benchmark suites, so the relative ceiling of SFT-data-curation vs RL-environment-scaling is now empirically testable.