FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale

TL;DR

FrontierSmith is a synthesizer that mutates abundant closed-ended competitive-programming problems into open-ended optimization tasks with continuous reward signals, then filters by idea divergence and packages each task as a runnable training environment. Training Qwen3.5-9B and 27B with GRPO on 200 FrontierSmith problems matches or beats training on human-curated open-ended data (FrontierCS, ALE-bench): 9B FrontierCS Avg@5 jumps 1.80 → 10.62, 27B 7.70 → 19.82. The point is that the dominant data primitive for long-horizon coding RL — open-ended tasks with continuous scoring — can be cheaply generated from the existing closed-ended coding corpus, rather than hand-authored.

Key claims

The bottleneck for long-horizon optimization agents is open-ended training data: classic RLVR sources >100K competitive-programming problems, but FrontierCS + ALE-bench + KernelBench + MLS-Bench together total only hundreds of open-ended tasks [§The Data Bottleneck].
The FrontierSmith mutation primitive operates on a problem tuple (objective O, input constraints C_I, output constraints C_O) along three axes: change O to an optimization objective, restrict C_O to make exact solutions infeasible at scale, or generalize C_I to make a tractable problem hard [§From Closed to Open].
Idea divergence — semantic (LLM-judge comparison of solver strategies) + behavioral (score-vector comparison across generated tests) — is the filter that distinguishes genuine open-ended tasks from problems that are open-ended in wording but dominated by one strategy [§Filtering for Real Diversity].
Each surviving task is packaged in the FrontierCS judge sandbox as a (test-case generator, normalized [0,1] verifier) pair. Scoring uses baseline normalization: a trivial baseline produces value x, submission produces y, reward is the normalized improvement; crashes/timeouts/invalid outputs receive 0 [§Building Training Environments].
Qwen3.5-9B FrontierCS Avg@5 improves from 1.80 (base) to 10.62 after GRPO on 200 FrontierSmith problems; ALE-bench from 327.22 to 633.58. This is close to the 11.17 from training on human-curated FrontierCS data and beats the FrontierCS-trained 558.49 on ALE-bench [§Experiments].
Qwen3.5-27B FrontierCS Avg@5 improves 7.70 → 19.82 with FrontierSmith; ALE-bench 352.52 → 661.64. At this scale FrontierSmith beats the human-curated FrontierCS training baseline on both benchmarks [§Experiments].
The gain is from the closed→open transformation, not generic coding RL: training directly on the HardTests closed-ended seed corpus (the same source) reaches only 5.38 FrontierCS / 397.18 ALE-bench at 9B — far below FrontierSmith’s 10.62 / 633.58 [§Experiments].
The gain is not from RL dynamics alone: random-reward training on FrontierCS reaches 3.04 FrontierCS / 376.82 ALE-bench [§Experiments].
Dropping the idea-divergence filter drops FrontierCS from 10.62 → 8.57 and ALE-bench from 633.6 → 564.4 — the filter, not just the mutation, is load-bearing [§Experiments].
FrontierSmith tasks elicit genuinely long-horizon agent behavior: Claude SDK with Sonnet 4.6 reaches 113 turns and 6.3M tokens on FrontierSmith, comparable to ALE-bench and well above HardTests / FrontierCS in this experiment [§Experiments].
The release ships with a 178-task long-horizon agent leaderboard on Harbor (Kimi K2.6 at 46.9 with 67.2 avg steps, Claude Code Opus 4.7 at 43.0 with 77.2 avg steps and ~251K tokens) and FrontierCS leaderboards on 172 algorithmic + 68 (other) problems, where Gemini 3.0 Pro tops both at Score@1 33.12 / 46.55 [landing page leaderboards].

Method

FrontierSmith starts from a pool of closed-ended seeds (LeetCode/competitive-programming style; experiments use HardTests). For each seed, an LLM proposes mutations along the three problem-tuple axes. An LLM judge then samples solutions to each candidate from multiple solver agents and estimates whether those solutions express meaningfully different algorithmic ideas. Candidates that survive the semantic-divergence pass get a test generator and a baseline-normalized verifier built; a second behavioral filter compares score vectors of the sampled solutions across the generated tests and drops candidates where solutions collapse to narrow score profiles. Surviving tasks are pushed into the FrontierCS judge sandbox.

Training is GRPO on Qwen3.5-9B / 27B over 200 synthesized problems. The reward is the continuous normalized improvement over the trivial baseline (analogous max/min form depending on the objective), so the policy sees graded signal rather than binary pass/fail.

Worked example given in the post: minimum spanning tree (clean greedy) → add a degree constraint requiring every vertex to have degree at most D → at D=2 this reduces to TSP, and at realistic scales exact optimality is no longer practical, so solution quality becomes continuous [§From Closed to Open].

Results

Headline training results (GRPO on 200 synthesized problems, evaluated as Avg@5 on FrontierCS-172 algorithmic + ALE-bench-lite):

Qwen3.5-9B: base 1.80 / 327.22 → FrontierSmith 10.62 / 633.58. Human-curated FrontierCS training reaches 11.17 / 558.49 — FrontierSmith ties on in-domain, wins on ALE-bench.
Qwen3.5-27B: base 7.70 / 352.52 → FrontierSmith 19.82 / 661.64. Beats human-curated FrontierCS training on both benchmarks at this scale.
HardTests closed-ended baseline (9B): 5.38 / 397.18 — confirms the closed→open transformation is the source of gain.
Random reward (9B): 3.04 / 376.82 — confirms reward shape matters.
No-filter ablation: 10.62 → 8.57 on FrontierCS, 633.6 → 564.4 on ALE-bench [§Experiments].

Long-horizon behavior (Claude SDK / Sonnet 4.6): FrontierSmith elicits 113 turns / 6.3M tokens, matching the ALE-bench long-horizon regime; HardTests stays short-horizon.

The Harbor 178-task preview leaderboard reports Kimi K2.6 at 46.9 (67.2 avg steps, 70.6 avg tools, 155.6K tokens) and Claude Code Opus 4.7 at 43.0 (77.2 avg steps, 42.2 avg tools, 251K tokens) — similar capability headline, different failure modes (the post highlights tool-use volume vs token spend as the divergence axis).

Why it’s interesting

This is the cleanest filed instance of the “closed-ended → open-ended task mutation” data primitive — adjacent to but distinct from the multi-agent synthesizer loops on Synthetic Training Data (cf. Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing for tool-use, Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning for video CoT). Where those papers generate trajectories or annotations on top of an existing task distribution, FrontierSmith generates the tasks themselves by transforming a different distribution (closed-ended → open-ended), and isolates the transformation as the source of gain with both a HardTests-direct and a random-reward control.

It also extends RL Environment Platforms: SETA, Toolathlon-GYM, and OpenReward standardize the env↔trainer protocol, but FrontierCS + FrontierSmith is the first filed instance where the task synthesizer itself is the contribution and the environment shell is treated as commodity infrastructure (the existing FrontierCS sandbox). It also adds an evaluation axis the env-platforms cluster doesn’t yet cover: a public leaderboard with continuous scoring over open-ended optimization tasks, vs the binary/pass-fail flavor of most agentic-SWE benchmarks. The “idea divergence” filter is a more substantive answer to the “what makes a synthesized task worth training on” question than the verifier-shortcut detection in Qwen3-Coder-Next Technical Report (which gates on whether the verifier itself is functional, not on whether the task has solution-space breadth).