Training AI Scientists to Replicate Research

TL;DR

Inherent Labs introduces Replica, a scalable RL task space where an agent must reproduce a specific figure from a research paper under fixed time and compute budgets without seeing the target plot, and uses it to post-train Faraday, a 27B “AI Scientist” that directs a coding-agent tool (GPT-5.5 Codex) via long-horizon RL. Faraday surpasses Claude Opus 4.8 and GPT-5.5 in their own harnesses at maximum thinking effort on held-out replication tasks, despite directing a tool model orders of magnitude larger than itself. The reward-signal contribution is a per-task rubric-based LLM judge — validated against a human study — with multi-sample aggregation and turn-level credit assignment for training stability. The framing is that replication is a stepping-stone curriculum for open-ended innovation, because recovering the “99% perspiration” absent from a paper requires the same hypothesis-driven exploration as original research.

Key claims

Replica is a task space of 310 paper-replication tasks drawn from 100 ML and AI-for-science papers spanning NLP, materials science, and weather forecasting, with each task requiring reproduction of a specific figure under fixed time and compute without access to the original plot [§Abstract, §From replication to innovation].
Faraday (27B) trained with long-horizon RL on Replica surpasses Claude Opus 4.8 (in Claude Code, thinking effort “extra high”) and GPT-5.5 (in Codex, thinking effort “extra high”) on held-out replication tasks, and produces more faithful replications in every paper-category slice of the suite [§Abstract, §From replication to innovation].
The reward is an LLM-as-judge with per-task rubrics; a human study validates that rubric-based judging captures expert “research taste” with greater consistency and lower noise than an LLM-baseline judge [§RL for non-verifiable domains].
Training stability comes from two additional judge modifications on top of per-task rubrics: multi-sample aggregation and turn-level credit assignment [§RL for non-verifiable domains].
Faraday employs GPT-5.5 Codex as a tool at inference time, directing a model orders of magnitude larger than its own 27B parameters and still improving replication performance over the larger model used directly [§Scalable scientific oversight].
Faraday generalizes to a stronger tool at test time: trained with GPT-5.4-mini as its coding tool, it transfers to directing GPT-5.5 Codex at evaluation without retraining [§Scalable scientific oversight].
Faraday has no test-time reward and requires no hand-coded evolutionary harness, unlike prior AI-Scientist systems; the authors frame this as the model having learned to value discoveries “intrinsically” through training [§Scalable scientific oversight].
Qualitative rollout analysis reports that Faraday adopts a “more scientifically-principled approach” than the baselines [§Abstract].
The authors position replication as the base of a curriculum for open-ended innovation: additional features beyond single plots can be masked, resource constraints tightened or relaxed, and papers can be entirely imagined — all pointing the same Faraday model at less-specified tasks [§From replication to innovation].

Method

Replica formulates each task as a paper-plus-figure-target pair: the agent receives the paper with the target figure masked, a time budget, and a compute budget, and must produce a plot that a rubric-based judge scores against the (hidden) ground-truth figure. Faraday is a 27B model post-trained with long-horizon RL on this task space, where each rollout is a multi-turn trajectory in which Faraday invokes a coding-agent tool (GPT-5.5 Codex or GPT-5.4-mini during training) to execute experiments and produce the candidate figure. The reward-signal design is the load-bearing engineering contribution: rather than a fixed metric or a single LLM-judge scalar, per-task rubrics decompose “did the replication succeed” into task-specific criteria, and the judge score is stabilized by (a) aggregating multiple judge samples and (b) attributing credit at the turn level rather than only the trajectory outcome. A separate human study confirms the rubric-judge agrees with expert judgment on replication quality.

At inference, Faraday plays the role of scientific-taste layer directing a much larger coding-agent tool; the paper reports a clean generalization result where Faraday trained against GPT-5.4-mini directs GPT-5.5 Codex at test-time and improves over that tool used standalone in its own harness (Codex, extra-high thinking).

Results

Faraday-27B beats Claude Opus 4.8 (in Claude Code, extra-high thinking) and GPT-5.5 (in Codex, extra-high thinking) on held-out Replica tasks, with more faithful replications in every paper-category slice [§Abstract, §From replication to innovation].
Faraday improves particularly on recent research — papers published after the base model’s pretraining cutoff — showing that the learned skills transfer beyond memorized content [§From replication to innovation].
Per-task rubric judges show greater consistency and lower noise than a plain-LLM-judge baseline in the human-alignment study [§RL for non-verifiable domains].
Test-time tool-generalization: trained with GPT-5.4-mini, Faraday improves outputs of the stronger GPT-5.5 Codex without retraining [§Scalable scientific oversight].
(No numerical benchmark table is disclosed in the abstract or blog post; concrete per-task-category deltas are presumably in the paper body.)

Why it’s interesting

Faraday is the second filed AI-for-AI-research system after AIDE²: First Evidence of Recursive Self-Improvement (AIDE²) to demonstrate that a smaller, taste-trained model directing a larger tool beats the larger tool used directly — but it’s the first to do so at 27B parameters with a real production coding-agent tool (GPT-5.5 Codex), and the first to demonstrate test-time tool generalization (train with mini, direct the big one at eval). Together with AIDE²’s recursive self-improvement result, Faraday sharpens the concept-page pattern that scientific taste is a learnable layer separate from underlying model scale — the coding agent supplies raw capability, the scientist model supplies the compositional judgment.

The rubric-based judge with multi-sample aggregation + turn-level credit assignment is a concrete recipe for training stability in non-verifiable reward domains, which sits next to Enhancing Rubric-based RL via Self-Distillation (rubric-based RL via self-distillation) and ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents (rubric-benchmark for deep-research agents) — the wiki now has three filed instances of rubric-as-training-signal, all pointing at the same conclusion that per-task decomposition beats scalar-judge for research-agent training. Contrasts with prior AI-Scientist systems like AlphaGo Moment for Model Architecture Discovery (ASI-ARCH) and FARS: Fully Automated Research System (FARS) which use hand-coded harnesses and either novelty-check gates or evolutionary loops — Faraday removes both.

Framing replication as the base of an innovation curriculum echoes the ResearchClawBench and Delphi framings (ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery, Delphi: Scaling Laws That Extrapolate 300× Past the Fit) where re-discovery → new-discovery is the natural evaluation spectrum for research agents; Faraday operationalizes it as an RL curriculum rather than just an evaluation axis.