TMax: A Simple Recipe for Terminal Agents

TL;DR

TMax (Allen Institute for AI + UW, June 2026) is an open recipe for training terminal agents — LLMs that drive a shell, read stdout/stderr, and iterate — combining a 14.6K-environment RL dataset (TMax-15k) with a plain GRPO-derived training recipe. The dataset is generated by hierarchically sampling Gemini-3-Pro across nine structured axes (domain, skills, personas, task complexity, command complexity, etc.) with graded verifiers and per-task Docker images; soft filtering during RL replaces the expensive multi-rollout validation that prior pipelines use. The recipe (DPPO + token-level loss + async rollouts + FP32 LM head) trains Qwen3.5-2B/4B/9B/27B; TMax-9B hits 27.2% on Terminal Bench 2.0 (strongest open <10B under official settings, beats prior 32B agents, approaches Claude Haiku 4.5 at 29.8%), and TMax-27B hits 42.7%, approaching 1T-parameter Kimi K2.5 (43.2%). Gains transfer across harnesses (+9–15 pts on OpenHands/mini-swe-agent/Terminus-2), to SWE-Bench Verified (+9.5), and to AIME (+17.8) under a terminal-agent scaffold.

Key claims

TMax-15k is the largest open terminal RL dataset that releases full environment data (2.5× the next-largest), generated by hierarchical sampling across 9 structured axes — domain + skill seeded from Nemotron-Terminal’s taxonomy, plus 6 new orthogonal axes for diversity and difficulty [§2.1].
TMax-15k is the hardest terminal dataset measured: Pass@1 42%, Pass@8 53% (lowest of any compared dataset including SWE-smith at 72%, Endless Terminals at 95%); 13-gram contamination check against Terminal Bench finds 0% overlap [§2.2, Table].
Domain and skill balance dominate prior datasets: 0.998 domain balance and 0.732 skill balance (fraction-of-uniform score), vs prior datasets concentrating 34–95% of mass on one domain (SWE-smith is 95% software engineering) [§2.2, Table].
Soft filtering via RL itself eliminates the multi-rollout validation step that prior pipelines use: GRPO drops all-same-reward groups automatically, all-zero filtering rate is <8 samples per batch, and the only inference-time guarantee needed is that the Docker image builds [§2.1].
Multi-modal fixtures (PNG, audio, video, stripped binary, vendored package) ship with task files and the text-only policy interacts via terminal tooling (OCR, audio transcription, ffmpeg) [§2.1].
Graded verifiers (metric thresholds e.g. accuracy ≥ 0.95, adversarial corpora, fuzz equivalence vs oracle, multi-protocol service checks) replace exact string-equality and provide continuous hillclimbing signal while increasing turn length [§2.1].
Training recipe is plain GRPO with four targeted stability fixes: DPPO (binary-TV mask on vLLM↔training logprob disagreement), token-level loss (DAPO-style), fully async rollouts with zero-stddev group filtering + active sampling, and FP32 LM head — the last matters most for hybrid Qwen3.5 [§3.1].
TMax-15k beats every other tested RL dataset on the same Qwen3.5-9B base: TB Lite 57.2 / TB 2.1 28.8, vs the next-best Endless Terminals at 52.6 / 25.5 — a near-13-point TB 2.1 lift over base [§3.2, Table].
TMax-15k consistently drives more steps per training episode than other datasets and the model learns to think more over training (assistant tokens per turn climb steadily for both reasoning and tool-calling) [§3.2].
TMax-9B (Qwen3.5-9B + RL) reaches 27.2% on Terminal Bench 2.0 — strongest open-weights model under 10B under official Terminal Bench settings, beats prior 32B terminal agents, approaches Claude Haiku 4.5 (29.8%) [§3.3].
TMax-27B (Qwen3.6-27B + RL) reaches 42.7% on Terminal Bench 2.0 (from 39.6%), approaching 1T Kimi K2.5 (43.2%) and 230B MiniMax M2.7 (45.1%) [§3.3].
Transfer to non-terminal benchmarks: TMax-9B improves SWE-Bench Verified 44.0 → 53.5 (+9.5) and terminal-agent AIME’24/25 73.3 → 91.1 (+17.8) — evidence that RL on terminal data teaches genuine new skills, not benchmark-fitting [§3.4, Table].
Harness transfer: TMax-9B improves by 9–15 points across every harness tested, including ones never seen in training — OpenHands 36.0→46.9, mini-swe-agent 44.1→55.3, Terminus-2 36.4→45.3 [§3.4, Table].
SFT before RL helps weaker bases (Qwen3-8B: TB Lite 7.3 → 11.5 with TMax SFT → 16.4 with large SFT) but hurts strong post-trained bases (Qwen3.5-9B: TB Lite 41.9 → 35.5 with TMax SFT → 31.3 with large SFT) — heavily post-trained models have more to lose from imitation [§4.1, Table].
Terminal-agent RL is hard to stabilize past 200–300 steps; numeric mismatch between training and inference is the main culprit, exacerbated by the hybrid Qwen3.5 architecture; FP32 LM head + DPPO + larger group size collectively limit collapse [§4.2].

Method

The TMax pipeline has two pieces. Data generation composes each task by hierarchically sampling Gemini-3-Pro across nine axes: domain and skill (seeded from Nemotron-Terminal’s taxonomy), plus six orthogonal axes targeting diversity (personas: 6–18 per domain, multi-modal fixtures) and difficulty (task complexity from a few commands to 30–60-command workflows; command complexity from bash-only to bash + code + system services). Difficulty is calibrated by uniform sampling across buckets, and verifiers are graded (metric thresholds, adversarial corpora, fuzz equivalence, multi-protocol checks) rather than exact-string. Scalability comes from skipping multi-rollout validation entirely — RL’s natural drop of all-same-reward groups acts as a free soft filter, and only the Docker build needs to succeed.

Training is GRPO with DPPO (masks tokens where vLLM and training logprobs disagree, using a binary TV-divergence approximation), token-level loss (DAPO), fully async rollouts with zero-stddev filtering, active sampling, and an FP32 LM head. Infrastructure extends open-instruct with vLLM rollouts and Podman/Apptainer sandboxes; a typical run uses 2 training + 6 inference H100 nodes for 2–3 days. Key hyperparameters: 500 steps, group size 32, 8 prompts per batch, 65,536-token max context, 64 max tool calls per episode. Interleaved thinking is preserved across turns. The agent harness is mini-swe-agent with a persistent shell (chosen over Terminus-2 for small-model robustness).

Results

Headline. TMax-9B (Qwen3.5-9B + TMax-15k RL) reaches 27.2% on Terminal Bench 2.0 — strongest open-weights model <10B under official settings, beats prior 32B terminal agents, approaches Claude Haiku 4.5 (29.8%). TMax-27B (Qwen3.6-27B + RL) reaches 42.7% (from 39.6%), approaching 1T-parameter Kimi K2.5 (43.2%). Smaller TMax-2B and TMax-4B reach 2.9% and 18.9% (from 2.3% and 16.6%).

Dataset comparison on Qwen3.5-9B (mean over 3 runs): TMax-15k 57.2 / 28.8 (TB Lite / TB 2.1), Endless Terminals 52.6 / 25.5, OpenThinker-Agent 53.0 / 25.1, CLI-Gym 50.7 / 25.1, TermiGen 49.4 / 25.1, SWE-Smith 47.2 / 21.0, TerminalTraj 45.8 / 18.0, baseline 41.9 / 16.1.

Generalization. SWE-Bench Verified 44.0 → 53.5 (+9.5); AIME’24/25 (terminal-agent harness) 73.3 → 91.1 (+17.8). Across harnesses on TB Lite: mini-swe-agent + persistent shell 41.9 → 57.2; OpenHands 36.0 → 46.9; mini-swe-agent 44.1 → 55.3; Terminus-2 36.4 → 45.3.

SFT ablation. For Qwen3.5-9B (strong base), SFT hurts: 41.9 → 35.5 (TMax SFT) → 31.3 (large mixture) on TB Lite. For Qwen3-8B (older base), SFT helps: 7.3 → 11.5 → 16.4. The paper attributes the gap to the strong base already being heavily post-trained.

Why it’s interesting

TMax is the cleanest open recipe so far for the RL Environment Platforms cluster’s “verifier-cheap, action-expensive” pattern applied to terminal agents — explicitly skipping the multi-rollout validation that SETA: Scaling Environments for Terminal Agents (env repo) uses and replacing it with the RL filter itself, and reaching scores that beat 32B agents at 9B. It’s also the first open terminal RL drop where the dataset is measurably harder (Pass@1 42%, Pass@8 53%) than prior released sets like SETA: Scaling Environments for Terminal Agents (env repo) (~400 tasks) and On Data Engineering for Scaling LLM Terminal Capabilities (Nemotron-Terminal). The finding that SFT hurts heavily post-trained Qwen3.5 contradicts the standard “SFT → RL” prescription used by Qwen3-Coder-Next Technical Report and similar agentic-SWE recipes, and is a useful caution for the Agentic Software Engineering cluster as base models keep accumulating post-training. The harness-transfer result (+9–15 pts across unseen harnesses) is a stronger generalization signal than most filed agentic-SWE papers report, since Qwen3-Coder-Next Technical Report §3.1.2 specifically flags within-vs-cross-scaffold transfer as weak.