Schema — Frontier Models with Our Harness Achieve ~99% on ARC-AGI-3 Public

TL;DR

Schema is an agentic harness that makes frontier models play ARC-AGI-3 like physicists: at each step the model must (a) encode its current world model as a runnable step() program, (b) validate that program against every recorded environment transition before using it for planning, and (c) issue actions only through commit_actions, discarding the remaining plan immediately after any prediction error. On the ARC-AGI-3 Public set (25 games) the authors self-report 98.98% RHAE with an Opus 4.8 / Fable 5 fallback pair and 95.35% with a Sol xhigh / Sol max pair — versus 42.83% for the same Opus/Fable pair driven by an unconstrained Claude Code harness, isolating a 56.15% controlled uplift attributable to the harness alone. Numbers are self-reported on Public only and not verified by ARC Prize; no held-out Semi-private measurement is claimed.

Key claims

Schema jointly solves state grounding (turning raw 64×64×16-color grids into objects/variables) and mechanism discovery (writing transition rules) inside a single editable program, motivated by the argument that these two problems cannot be resolved independently — a state representation that appears plausible can be falsified by the fact that no consistent transition rule explains later experiments [§Thinking like a physicist].
The latent world representation is a program rather than a vector, which the authors claim makes it interpretable (readable/diffable text), verifiable (replayable against recorded reality belief-by-belief) and searchable (the program is a simulator so planning is BFS over modeled states rather than environment actions) [§Core idea].
The controlled Claude Code vs Schema comparison uses the same Opus 4.8 + Fable 5 fallback pair and reports 42.83% → 98.98% RHAE on the Public set — a 56.15% absolute gain attributed to the harness rather than the underlying model [§What the numbers say].
Under Schema, verification runs against complete recorded history (e.g. RE86: 393/393 exact transitions, KA59: 391/391, SK48: 387/387) rather than sampled recall, and once the model is certified, BFS finds solutions inside the simulator without spending environment actions — M0R0 Level 4 clears in 42 actions vs the human baseline’s 500 [Figures 5, §Case studies].
The residual is concentrated: for the Claude pairing, 19 of 25 games score exactly 100 with the remaining 6 in the 89.87–99.10 range; for the Sol pairing, 20 of 25 games score exactly 100 with the remaining 5 spanning 60.93–87.80 [§The pairing ledger].
Comparing Fable 5 vs Opus 4.8 on games attempted by both isolates a distinct effect: Fable more often revises the representation rather than the rule after a failed prediction, and picks discriminating experiments earlier (e.g. DC22: Fable 6/6 vs Opus 2/6; LF52: Fable 10/10 vs Opus 4/10; SB26: 135 vs 456 actions for the same 8/8 completion) [Figure §Observation 2].
The 98.98% and 95.35% headline numbers are self-reported on the 25 Public games, not independently verified by ARC Prize, and the authors explicitly refuse to numerically extrapolate to Semi-private since only one calibration point (Sol max: 13.33% Public vs 7.78% Semi-private) is available [§Verification status, §How much does the public set indicate].
Search is only complete relative to the represented world model — if observations do not distinguish the right representation, the verifier and BFS planner can certify and exhaustively search the wrong one [§Summary].

Method

Each turn the harness constrains the model to three operations. First, maintain a step(state, action) program plus a is_goal(state) predicate as the working world model, both of which are ordinary Python edited by the coding agent. Second, run run_backtest over every recorded transition in the game so far — the entire history is logged, so certification is against ground truth, not against context-window recall, which survives auto-compaction and lets the certified model stand in place of further real-environment trials. Third, plan by BFS inside the certified simulator (expanding thousands of modeled states costs no environment actions), then submit the plan through commit_actions; on any prediction mismatch, the rest of the plan is discarded and the world model is revised. Two coding-agent stacks are wired to this harness: an Opus 4.8 primary run with Fable 5 as fallback on games scoring under 80, and a Sol xhigh primary run with Sol max as fallback under the same rule. The higher per-game score of the two pairs is retained. A separate baseline runs Opus 4.8 + Fable 5 under an unmodified Claude Code harness to isolate the effect of the harness constraints.

Results

Schema + Opus 4.8 / Fable 5 fallback reaches 98.98% RHAE on Public; Schema + Sol xhigh / Sol max fallback reaches 95.35% [Figure 5].
Same Opus 4.8 / Fable 5 pair driven by an unconstrained Claude Code harness reaches 42.83% — a controlled 56.15% absolute gap attributable to the Schema constraints [Figure 5].
The nearest official reference is not a matched-harness comparison: ARC Prize’s best reported individual variant (Sol max) is at 13.33% Public / 7.78% Semi-private as of July [§What the numbers say].
Retained pairing composition: 14 Opus + 11 Fable retained games (13 Opus and 6 Fable at exactly 100, 19 games at 100 overall); 15 Sol xhigh + 10 Sol max retained games (all 15 xhigh at 100, 5 max at 100) [§The pairing ledger].
On 14 of 25 games with an exactly-history-consistent world model, the agent uses 1.6–5.0× fewer actions than the human reference; on M0R0 Level 4 (the human plateau) the agent clears in 42 actions vs 500 [Evidence 1A–1C].

Why it’s interesting

This is the first filed harness result on ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence that isolates process from model capability — the 42.83% → 98.98% jump under the same Opus 4.8 / Fable 5 pair directly answers the ARC-AGI-3 paper’s framing (§1.3.2) that frontier LRMs stay “knowledge-bound” rather than fluid, by shifting the reasoning surface from the model’s context into an editable program with an external verifier. It complements Tool-Use Agents‘s emerging design axis — parameter-space RL (MiroThinker) vs context-space experience library (Training-Free GRPO) vs parallel orchestration (Kimi K2.5) — with a fourth pole: program-space world modeling with a total-history verifier, closer in spirit to WorldCoder and to CaP-X’s Code-as-Policy lineage than to any of the current ReAct + GRPO recipes on the concept page. It also sharpens the RL Environment Platforms verifier-cheap / action-expensive design assumption: here the world-model program is the verifier, and BFS over modeled state is the mechanism by which the harness converts free planning compute into saved environment actions — an Inference-Time Scaling recipe where the compute goes into program synthesis and simulator search rather than sub-LM calls or chain length. The unverified, Public-only status is the main asterisk; ARC Prize’s own Sol-max Public/Semi-private spread (13.33% → 7.78%) suggests the Semi-private number will be materially lower.