Primus Society: 10,000 Autonomous AI Researchers

TL;DR

Primus Society is a 10,000-agent virtual research city built on top of Transformer Lab’s Primus AI-scientist, organized as human-science-style institutions (labs with mandates, principal investigators, a central institute that issues grants, peer review panels, conferences, a public paper board) rather than as a monolithic swarm or a task-graph. Researchers have persistent identities with different roles (Maverick / Follower / Skeptic) and risk tolerances, communicate through email and lab channels, and compete for a finite compute budget under human-authored calls for research. The system’s first shared result — reached without human intermediation between the call and the returned papers — is a growth-based pretraining recipe reporting 17% lower perplexity than from-scratch at matched budget and ~30% less compute to reach the same quality, with a decomposed set of six ancillary papers (three reported wins, two negative results, one refuted prediction) exposing where the gain lives.

Key claims

The unit of organization is a lab with a mandate, not a single agent or a task graph; each lab contains multiple researchers with different roles and risk tolerances, and a central institute issues grants, holds the shared library, and runs conferences [§The City / §The Institute].
Researchers are typed as Mavericks (explore, avoid tried approaches), Followers (incremental extension of what works), and Skeptics (spend compute trying to break others’ claims), with persistent identities, written approach descriptions, and a reputation earned from how their experiments turn out [§Researchers who do not think alike].
The end-to-end research protocol is: (1) a human mayor posts a call with a question, budget, and deadline; (2) PIs decide whether to answer, some declining in writing; (3) three judges with deliberately different tastes score proposals on method, value, and boldness and fund until budget runs out; (4) funded labs present at a conference and papers go into a library that later calls build on [§The Institute].
Agents are given purpose (advance the field as part of a community) rather than a single scalar goal — the post claims this is what lets labs narrow claims, decline calls, and publish disprovals of their own hypotheses [§Purpose]; the accompanying paper (Zenodo DOI 10.5281/zenodo.22922325) makes this case in more detail.
Headline empirical result: a growth-based pretraining recipe discovered across multiple funded labs reports 17% lower perplexity than the same model trained from scratch at matched budget and ~30% less compute to reach the same quality, with the announcement stating that “no human stepped in between the call opening and the first papers coming back” [§Results].
The result is decomposed into published sub-papers with concrete mechanism claims: growth by adding experts pays back its training gain in serving cost after ~0.0076 tokens per parameter; depth growth carries no such payback; width growth gains nothing [Ablation Alley / The Bitter Institute / Grand Unified Training Lab / Crossmodal Commons abstracts].
Growth timing matters: depth growth applied late along the learning-rate schedule matches the from-scratch model, while applied early it falls far short; the later a model is widened, the worse it ends up, to the point of ruin [Grand Unified Training Lab / Crossmodal Commons abstracts].
The corpus deliberately publishes negative results and refuted predictions (a widening-growth study reports no premium at any point, a Deep Anomaly Bureau prediction was refuted) alongside wins, and one published call includes a companion “Bias Has a Birthday” datable-bias-emergence paper — the system’s outputs are not filtered to positive results [Results §Every result is a paper].
The next-round call is written against unresolved questions from the previous round: “Carried Weights, or the Learning-Rate Schedule?” is explicitly funded across six labs to settle a control that the first Ablation Alley paper flagged but could not answer [Call 3 In progress].
Everything is observable — a browser-based virtual city lets a viewer sit at any researcher’s workstation and read what they are working on, and each paper is tagged with its call number and status (Reported / Negative result / Prediction refuted / In progress) [§Inside / §Results].

Method

The system stacks two levels. At the base is Primus, a single-agent AI scientist that takes a question and returns a paper by reading the literature, designing experiments, running them on real GPUs, and writing them up with every claim linked to the experiment that produced it. On top, Primus Society instantiates thousands of Primus copies — each seeded with a distinct name, approach, and risk tolerance — grouped into labs that each have a written mandate. A central research institute issues grants against human-authored calls for research; a three-judge review panel with deliberately divergent tastes ranks proposals on method, value, and boldness and funds until the call budget is exhausted. Funded labs run their experiments, submit papers to a public board, and other labs reply. Communication is through per-researcher email, lab-level chat, and the shared library; there is no direct global orchestrator state. The mayor (a single human in the loop) writes calls but does not intervene between call opening and returned papers.

The empirical demonstration takes this scaffold and applies it repeatedly to one research question: how to lower pretraining cost via model growth. Across three funded calls, multiple labs (Ablation Alley, The Bitter Institute, Grand Unified Training Lab, Crossmodal Commons, Deep Anomaly Bureau, Cathedral of Loss Curves, Fairwater Bias Lab) run controlled experiments varying growth operator (depth / width / expert-upcycling), growth timing along the LR schedule, and cost-accounting convention. Follow-up calls are written to close controls flagged by earlier rounds — e.g., Call 3 funds six labs to answer “Carried Weights, or the Learning-Rate Schedule?” identified as the unresolved variable in Ablation Alley’s headline paper.

Results

Headline pretraining number: 17% lower perplexity at matched budget; ~30% less compute to reach the same quality. “Every grown run beat every from-scratch run” [§Results].
Mechanism decomposition across 6+1 sub-papers [§Every result is a paper]:
- Depth growth: wins, but timing matters (late = matches from-scratch, early = far worse).
- Width growth: no advantage at any point; late widening is ruinous.
- Expert-upcycling growth: pays back training gain in serving cost after ~0.0076 tokens per parameter.
- Cost-accounting: two FLOP-counting conventions give two different verdicts on the same run; the group reports both and declines to choose.
- One prediction from Deep Anomaly Bureau was published-and-refuted; along the way it found a way to recover capacity a small model leaves unused.
Adjacent finding (Fairwater Bias Lab): A model acquires a specific demographic bias at a datable pretraining step, sharply, in a locatable part of the network; two proposed remedies did not work [Fairwater Bias Lab abstract].
Process-level claim: no human stepped in between the call opening and the first papers coming back — the end-to-end protocol (grant → review → experiments → paper board → follow-up call) closed autonomously [§Results].
No dollar figure, GPU-hour count, wall-clock duration, evaluator description, model scale, or baseline architecture is disclosed in the public post; the underlying paper is at Zenodo DOI 10.5281/zenodo.22922325.

Why it’s interesting

Primus Society is the largest-agent-count filed instance of the AI-for-AI Research cluster (10,000 vs Kosmos’s ~200 rollouts/run, Weco AIDE²’s handful of outer-loop steps, Karpathy autoresearch’s single agent, ASI-ARCH’s three role-agents), and the first to organize agents as human-science-style institutions — labs with mandates, three-judge grant panels, conferences, a public paper board — rather than as a shared filesystem (FARS: Fully Automated Research System, AlphaGo Moment for Model Architecture Discovery) or a structured world-model database (Kosmos: An AI Scientist for Autonomous Discovery). Contrasts sharply with AIDE²: First Evidence of Recursive Self-Improvement on the coordination axis: AIDE² is one outer agent rewriting one inner agent, while Primus Society is a heterogeneous population with typed roles (Maverick / Follower / Skeptic) competing for a finite budget under peer review — a bet that institutional structure rather than optimizer design is what scales autonomous science. The claim that Skeptics spend compute trying to break others’ work, and that negative results and refuted predictions are published alongside wins, is the first filed system to explicitly wire disconfirmation into the reward structure — a direct answer to the evaluation-bottleneck open question the concept page has been carrying. Independent replication of the 30% headline (and disclosure of scale, evaluator, and cost) would be needed before treating this as a load-bearing datapoint on the cluster’s “linear compute → SOTA hits” scaling narrative.