SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness

TL;DR

SoL-Pi is an RSI-style auto-research pipeline pointed at the harness of a long-running coding agent rather than at model weights or task-specific prompts. The outer loop proposes, tests, and keeps harness modifications across a broad slate of environments; four mechanisms survive selection — action execution, context compaction, observation handling, and delegated reading — and compose into the shipped SoL-Pi harness. On the 51-task EdgeBench evaluation, SoL-Pi matches the reference Pi harness in task performance across GPT-5.6 Sol and Opus 5 while cutting recorded token traffic by 44.7–49.0% and API cost by roughly one third, translating to estimated hourly savings of $8.75–$13.50 vs native Codex and Claude Code harnesses and $4.36–$5.71 vs Pi. It is the “recursive self-improvement of the harness” instantiation of the loop that AIDE² applies to whole coding agents and that AutoDesign applies to design harnesses.

Key claims

Casts token efficiency as the binding constraint for scaling recursive self-improvement of coding agents once work expands from isolated code completions into long trajectories of reasoning, tool use, and feedback [Abstract].
Applies an RSI-inspired outer loop at the harness layer, scaling auto-research rollouts across increasingly numerous and diverse environments rather than optimizing the model or task-specific prompts [Abstract].
Reports that reusable harness improvements transfer beyond their development setting when the outer loop is run at sufficient environment diversity, framing the result as automated harness discovery reaching “production-level outcomes” [Abstract].
Four mechanisms survive the outer-loop selection and compose into SoL-Pi: action execution, context compaction, observation handling, and delegated reading [Abstract].
On the 51-task EdgeBench evaluation, SoL-Pi matches the reference Pi harness in task performance across both GPT-5.6 Sol and Opus 5 backbones [Abstract].
Recorded token traffic drops by 44.7–49.0% and API cost by roughly one third vs Pi at matched performance on EdgeBench [Abstract].
Estimated hourly savings for a long-running agent: $8.75–$13.50 vs native Codex and Claude Code harnesses, and $4.36–$5.71 vs Pi [Abstract].

Method

SoL-Pi treats the coding-agent harness — the scaffolding that manages context window use, tool invocation, delegation, and observation handling — as the object of an RSI-style outer loop, holding the underlying model fixed. The loop proposes candidate modifications, runs them as rollouts across a diverse slate of environments (harness-rollout tasks), and keeps only modifications whose reusable improvements survive across the slate rather than overfitting to a single development setting. The abstract frames this as scaling the auto-research loop — the payoff comes from running rollouts across many environments so that only harness changes with broad transfer are retained.

Four mechanisms survive selection and are composed into the shipped SoL-Pi harness: (1) action execution — how the harness dispatches and runs tool calls; (2) context compaction — how it compresses history so long trajectories fit in a bounded context window without losing task-relevant state; (3) observation handling — how it processes verbose tool outputs (file dumps, test logs, terminal traces) before they enter the model’s context; and (4) delegated reading — how the agent offloads reading of large artifacts to sub-agents or specialized processors rather than paying to stream them through its own context. The abstract does not name the outer-loop proposer or the specific selection criterion; concept-page-level detail about the loop’s structure will need the full PDF.

Results

EdgeBench performance parity: SoL-Pi matches the reference Pi harness on the 51-task EdgeBench evaluation across GPT-5.6 Sol and Opus 5 model backbones [Abstract].
Token savings: recorded token traffic is reduced by 44.7–49.0% vs Pi at matched performance [Abstract].
Cost savings: API cost drops by roughly one third vs Pi at matched performance [Abstract].
Hourly savings estimates: $8.75–$13.50/hour vs native Codex and Claude Code harnesses; $4.36–$5.71/hour vs Pi [Abstract].

Why it’s interesting

SoL-Pi lands on the AI-for-AI Research page at a granularity that no filed system currently occupies cleanly: it’s harness-only recursive self-improvement — the outer loop optimizes only the scaffolding, keeping the model, task prompts, and inner-agent logic fixed — and its headline claim is not capability but cost. AIDE²: First Evidence of Recursive Self-Improvement rewrites the whole inner coding agent’s Python source over 100 unattended steps and reports capability gains on MLE-Bench/ALE-Bench/WeatherBench 2; AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design optimizes a design harness (paper→poster) with a meta-harness loop and reports quality gains that transfer across seven (agent, model) pairs. SoL-Pi is the token-cost counterpart: same recursive-self-improvement recipe, one level down, with the payoff measured in dollars per hour of long-running agent time rather than in benchmark points. That makes it a direct instantiation of the “harness carries portable capability” observation AutoDesign reports at the quality axis, and a concrete empirical answer to the LLM Inference Efficiency question of where the token budget of a modern coding agent actually goes — the four surviving mechanisms (action execution, context compaction, observation handling, delegated reading) are a data-driven decomposition of that budget rather than a hand-designed one.