AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design

TL;DR

AutoDesign frames long-horizon multimodal-to-media generation (concretely, academic paper → conference poster) as a model + harness system, and optimizes the harness rather than the model weights. An inner code-agent loop iteratively edits the current poster artifact; an outer meta-harness optimizer rewrites the harness itself using rollout feedback so it accumulates reusable experience. On PosterBench (100 papers, five disciplines), AutoDesign hits 78.32 — 7.45 points above closed-source Claude Design — and integrating the learned DesignHarness into seven independent code-agent × model configurations lifts the average from 54.99 to 67.39 (+12.4 pts). A single autonomous run makes 253 tool calls / 11 editing turns in ~40 minutes for under $3.

Key claims

Casts multimodal-to-media condensation (paper → poster) as a long-horizon agentic process centered on a model-harness system, and argues existing paradigms leave the harness static — motivating a meta-harness optimizer that mutates the harness based on rollout feedback and drives recursive self-improvement [Abstract].
The framework has two nested loops: an inner code-agent that edits the current poster artifact against the paper input, and an outer meta-harness optimizer that rewrites the harness itself from rollout traces, aligned with human design priors [Abstract].
On the PosterBench Main Track (100 papers spanning five disciplines) AutoDesign scores 78.32, surpassing the closed-source commercial system Claude Design by 7.45 points [Abstract].
Across seven controlled (code-agent, model) configurations, integrating the learned DesignHarness consistently improves output quality, raising the average PosterBench Score from 54.99 to 67.39 — a +12.4% relative lift — evidence that the harness itself carries transferable capability independent of the underlying model [Abstract].
A fully autonomous end-to-end run executes 253 tool calls and 11 editing turns within 40 minutes at under $3 per poster, reaching average conference-poster quality in human evaluation [Abstract].
Introduces PosterBench — a 100-paper Main Track spanning five disciplines and PosterBench-mini, a shared 10-paper subset for controlled evaluation — as the concrete instantiation of the long-horizon-design measurement problem [Abstract].
A system-blind human study reports AutoDesign as the highest human preference among the evaluated systems [Abstract].

Method

AutoDesign treats the design pipeline as a model + harness system where the harness is the object of optimization. The inner loop is a code agent that reads a paper and iteratively edits a poster artifact (tool calls plus editing turns). The outer loop — a meta-harness optimizer — inspects rollout feedback (successes, failures, quality signals) and mutates the harness itself, which then serves as the environment for future inner-loop runs. The training corpus is aligned with human design priors and accumulated empirically over rollouts; the DesignHarness that falls out of the outer loop is the reusable artifact, portable across (code-agent, model) pairs.

To evaluate, the authors introduce PosterBench: a 100-paper Main Track across five disciplines for open-ended scoring, and PosterBench-mini (a shared 10-paper subset) for controlled ablations. The benchmark is used both as the training signal for the meta-harness and as the reported evaluation.

Results

PosterBench Main Track headline: AutoDesign 78.32 vs Claude Design 71 (approx from +7.45 gap) [Abstract] — the top result among evaluated systems and the first filed paper-to-poster number above the closed-source commercial reference.
Harness transferability: dropping the learned DesignHarness into seven independent (code-agent, model) configurations lifts the mean PosterBench score from 54.99 → 67.39 (+12.4 pts), with per-config improvements of 5.0–19.6 PosterBench points [Abstract, tweet summary].
Autonomy budget: 253 tool calls and 11 editing turns in ~40 minutes for under $3 per poster [Abstract].
Human study: system-blind human evaluation ranks AutoDesign first for preference [Abstract].

Why it’s interesting

AutoDesign is a new granularity on the AI-for-AI Research page: it’s the first filed instance where the object of optimization is a harness for a creative-professional deliverable (paper→poster), sitting between AIDE² editing an ML agent’s Python source (AIDE²: First Evidence of Recursive Self-Improvement) and Sol/ENPIRE optimizing deployment stacks or robot policies. The transferability result — one learned harness lifts seven independent (code-agent, model) pairs by 5–20 points — directly triangulates the concept page’s open question about “next-round improvement from re-injecting prior-round outputs” and the Databricks finding that harness choice moves per-task cost by >2× at fixed model+quality (Benchmarking Coding Agents on Databricks' Multi-Million Line Codebase). It also extends optimize_anything Goes omni — Composing Optimizers into Meta-Optimizer Pipelines‘s “Meta-Harness” proposer (which the omni composed variant used on Frontier-CS) to a domain where the deliverable is a rendered visual artifact rather than program source, and it beats a closed-source commercial reference (Claude Design), a rare comparison on this concept page.