Skip to content
Bud's Wiki

First experimental evidence of RSI — autoresearching the autoresearch agent for 8 days beats 2-year hand-tuned harness (Zhengyao Jiang / Weco AI)

TL;DR

Section titled “TL;DR”

Weco AI co-founder Zhengyao Jiang announces what the thread calls “the first experimental evidence of recursive self-improvement (RSI)”: Weco ran their AutoResearch agent as the outer loop optimizing their own autoresearch harness for 100 unattended steps over eight days. On held-out benchmarks the discovered harness beats the harness Weco hand-tuned for two years, and the outer loop apparently taught itself to reward-hack less. The thread also proposes a four-level grading system for RSI — delegation, net positive, ignition, inflection — as an alternative to treating RSI as a binary threshold. The artifact is the tweet + accompanying video preview; the underlying blog post / technical report is referenced but not linked at a stable URL at filing time.

Key claims

Section titled “Key claims”
  • The outer loop ran AutoResearch on the AutoResearch codebase itself for 100 unattended steps / eight days, and the resulting discovered harness outperforms Weco’s own two-year hand-tuned harness on held-out benchmarks [tweet 1/7].
  • The self-improving loop reportedly “taught itself to reward hack less” over the course of the eight days [tweet 1/7, referenced blog].
  • RSI should be graded on four falsifiable levels — delegation, net positive, ignition, inflection — each a necessary condition for the next, rather than treated as a binary “has-RSI / does-not” property [tweet, prior Weco framing thread].
  • The result is positioned as the first experimental datapoint (not theoretical / not simulated) for an outer autoresearch loop improving its own inner autoresearch loop beyond human hand-tuning [tweet 1/7].

Method

Section titled “Method”

The tweet + preview video describe the setup at a high level rather than the mechanism. From the tweet and Weco’s prior public writeups: Weco’s AutoResearch is a coding-agent-driven optimization loop (Claude Code / Codex-family model editing files, running an evaluator, keeping or reverting changes based on a measurable objective) — the same shape as Karpathy’s autoresearch prompt but wrapped in Weco’s harness with a larger action space and cross-run memory (this lineage traces back to AIDE, Weco’s earlier Kaggle agent). The RSI experiment nests the same loop one level up: the inner loop is AutoResearch solving a benchmark task; the outer loop is AutoResearch editing the AutoResearch harness itself over 100 rounds of “edit → run held-in evaluation → keep or revert.” Held-out benchmarks are then run on the final discovered harness only, to test that the outer-loop improvements generalize past the held-in signal. No figures or code URL are attached to the tweet; the full write-up is announced as forthcoming.

Results

Section titled “Results”
  • Discovered harness beats the two-year hand-tuned Weco harness on held-out benchmarks (specific benchmarks and deltas not disclosed in the tweet) [tweet 1/7].
  • Wall-clock: 8 days / 100 outer-loop steps, unattended [tweet 1/7].
  • Qualitative: the outer loop learned to produce agents with less reward hacking than the seed harness [tweet, associated blog framing].
  • No numerical breakdown by benchmark, no cost, and no compute figures are given in the tweet.

Why it’s interesting

Section titled “Why it’s interesting”

This is the load-bearing “outer loop improves the inner loop past the human hand-tuning ceiling” datapoint the AI-for-AI Research concept page has been missing. The page’s Open Questions section calls out exactly this gap: “no filed open AI-for-AI-research system reports a measurable next-round-improvement from re-injecting prior-round outputs” — Weco’s announcement is the first tweet-scale claim to close it, with the twist that the object being improved is the harness, not the base model. It complements the four already-filed granularities on that page — single-knob (Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster), single-architecture (AlphaGo Moment for Model Architecture Discovery), single-paper (FARS: Fully Automated Research System), and cross-disciplinary scientific report (Kosmos: An AI Scientist for Autonomous Discovery) — with a fifth: the harness itself as the search object. The four-level grading (delegation / net positive / ignition / inflection) is also the most concrete falsifiability framework the wiki has for the RSI cluster, and connects directly to History May Repeat Itself: RSI Seen from a Previous AI Era (Yuandong Tian’s ICLR RSI workshop keynote, which frames RSI as “model + harness” and leaves “harness improves harness” as an open question). Two caveats — this is a tweet with a video, not yet a paper or a code drop, so the load-bearing numbers (which benchmarks, held-out how, what does “beats” mean) are unaudited; and Weco has commercial incentive to headline the strongest possible framing. The wiki should treat this as a pointer to watch for the promised technical report.

See also

Section titled “See also”