On Data Engineering for Scaling LLM Terminal Capabilities

TL;DR

NVIDIA presents Terminal-Task-Gen, a two-stage synthetic-data pipeline (dataset adaptation + skill-based synthesis) that produces Terminal-Corpus (~366K trajectories) and trains Nemotron-Terminal, a family of Qwen3-initialized agents (8B/14B/32B) for Terminal-Bench 2.0. The 32B variant reaches 27.4% on Terminal-Bench 2.0, outperforming the 480B Qwen3-Coder (23.9%) and the 120B GPT-OSS-high (18.7%), while the 14B already beats GPT-OSS-120B at one-eighth the size. The contribution is data-side: explicit ablations on filtering, curriculum mixing, long-context training, and source-mix scaling — the training-time interventions the closed terminal-agent labs have not disclosed.

Key claims

Nemotron-Terminal models, fine-tuned from Qwen3 on Terminal-Corpus, jump Terminal-Bench 2.0 accuracy from 2.5%→13.0% (8B), 4.0%→20.2% (14B), and 3.4%→27.4% (32B); the 32B variant outperforms Qwen3-Coder 480B (23.9%) at ~15× fewer parameters [Table 3].
The data pipeline is intentionally split coarse-to-fine: dataset adaptation (math 163K + code 35K + SWE 32K prompts wrapped in Terminal-Bench scaffold, no LLM in the loop) supplies broad coverage; skill-based synthesis over 9 domains × a primitive-skill taxonomy supplies targeted novel scenarios with controllable difficulty [§4.1, §4.2].
Each generation strategy alone is weaker than the union: adapters-only reaches 9.66% TB2.0, synthetic-tasks-only reaches 12.4%, and skill-based generation (12.4%) dominates seed-based (6.18%) at similar sample counts [Table 6].
Within the adapter family, individual splits underperform the union — Math alone 5.39%, Code 6.29%, SWE 7.02%; all combined 9.66% — confirming source diversity beats any single source [Table 6].
Pre-built domain-specific Docker images (9 shared base images instead of per-task Dockerfiles) are the engineering choice that makes large-scale synthesis tractable: eliminates Dockerfile validation overhead, reduces resource footprint, and decouples environment from task generation [§4.2.3].
Solution leakage is explicitly suppressed: task prompts shown to the agent never reveal algorithm/code, and reference solutions (when present in seed data) are used only for deriving ground-truth test expectations [§4.2.3 Solution Isolation].
Synthetic data unlocks categories where Qwen3 baselines scored 0% across all three sizes: Data Querying 0%→60%, Model Training 0%→50%, Debugging 0%→33%, Data Processing 5%→50%, Security 2.5%→27.5% (32B variant) [Table 4].
Teacher choice is DeepSeek-V3.2, justified by its strong Terminal-Bench 2.0 score (38.2%) and adapter-format benchmark performance (AIME 93.33, LiveCodeBench v6 67.20, SWE-bench Verified 52.40) [§4.3, Tables 2 & 3].
Filtering by trajectory completeness (“complete-only” — discard incomplete teacher rollouts) underperforms keeping all trajectories on the union (8.09% vs 9.66%) and on the math/SWE splits individually, suggesting verbose-trajectory penalty is non-monotone with completeness filtering [Table 7].
Decontamination is by 14-gram overlap with Terminal-Bench 2.0 test prompts; additional quality filters remove identity leaks and Chinese-character contamination [§4.4].

Method

Terminal-Task-Gen has two complementary stages. Dataset adaptation maps existing high-quality math (163K from OpenMathReasoning via Nemotron-Cascade), code (35K from OpenCodeReasoning), and SWE (32K from SWE-Bench-Train + SWE-reBench + SWE-Smith + SWE-Fixer-Train) prompts into the Terminus 2 system prompt template with dataset-specific instruction suffixes — no LLM is needed for this transformation. Skill-based synthesis defines 9 domains (data processing/querying/science, debugging, dependency management, file ops, scientific computing, security, software engineering) and for each a taxonomy of primitive skills across algorithmic, systems, data-processing, mathematical, testing, and web/security families; an LLM (DeepSeek-V3.2) recombines 3–5 primitives per task into novel scenarios with synthesized pytest test cases. Both stages emit (instruction, supplementary files, pytest suite, domain Docker image) tuples. Trajectories are generated by Terminus 2 inside Harbor-orchestrated containers (extended to Singularity for HPC clusters), then SFT-trained on Qwen3-8B/14B/32B with veRL (LR 2e-5, weight decay 1e-4, 2 epochs, 32K seq length, batch size 128, AdamW, cosine schedule with 10% warmup).

Results

Terminal-Bench 2.0: 13.0 (8B) / 20.2 (14B) / 27.4 (32B). Nemotron-Terminal-32B beats Qwen3-Coder-480B (23.9), DeepSeek-V3.2 685B is the open-source ceiling at 38.2 [Table 3].
Category breakdown (32B): Software Engineering 31.7 (vs Qwen3 5.0), System Administration 31.1 (vs 6.7), Debugging 33.3 (vs 0.0), Data Processing 50.0 (vs 5.0), Data Querying 60.0 (vs 0.0), Model Training 50.0 (vs 0.0) [Table 4].
Per-source ablation: Adapters-only 9.66 / Synthetic-only 12.4; individual contributions Math 5.39, Code 6.29, SWE 7.02, Seed-based 6.18, Skill-based 12.4 [Table 6].
Filtering ablation: complete-trajectory-only filtering actively hurts on the union (8.09 vs 9.66 with no filter), inverting the expected sign [Table 7].
Compute footprint: 8B/14B trained on 4 nodes × 8 GPUs (32 GPUs) with sequence parallelism 2; 32B on 16 nodes (128 GPUs) — modest by frontier standards.

Why it’s interesting

This is the most explicit recipe disclosure to date for the “synthetic-data for terminal agents” axis that the Introducing GPT-5.3-Codex release describes as already-frontier behind closed weights. It complements Qwen3-Coder-Next Technical Report — which discloses an SWE-Bench-Pro-targeted recipe with PR mining + bug injection + cross-scaffold training — by attacking the Terminal-Bench axis specifically with a skill-taxonomy generator rather than a PR miner. It also sharpens Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing‘s thesis: the closed-loop multi-agent synthesizer InfTool reached 70.9% on BFCL with no human labels; Terminal-Task-Gen reaches near-frontier Terminal-Bench with a simpler single-pass pipeline (no multi-agent coordination, no Dockerfile generation per task), and explicitly argues against the multi-agent direction on cost-scaling grounds. The 9-Docker-image fixed-pool trick is also the kind of engineering decision the wiki has not previously seen made load-bearing.