ACT-2 Preview: Generalizing Reliability

TL;DR

ACT-2 is Sunday Robotics’ second-generation home-robot foundation model, presented via a preview post that reports 99.1% zero-shot success on autonomous laundry folding across unseen homes, 9 garment types, and 785 attempts — the first Sunday result staged against a declared “Solve” standard (declared scope + declared adaptation cost). The central claim is a generalization gap result: pretraining on high-quality sensorized human data collected via Sunday’s proprietary “Memo” capture hardware shrinks the in-domain vs out-of-domain post-training gap from 82 percentage points (0% pretrain) to 0 (100% pretrain), enabling reliable local hill-climbing from small amounts of in-house data. They also claim that at full pretraining scale, a single fine-tuning example can teach the model a new folding technique that generalizes to held-out garments. If robust, this reframes robot post-training closer to the LLM in-context / few-shot regime that GPT-3-era work legitimized.

Key claims

Scaling pretraining data closes the in-domain vs out-of-domain post-training generalization gap monotonically: 0% pretrain → 82 pp gap, 12% → 10 pp, 25% → 8 pp, 50% → 4 pp, 100% → 0 pp, with in-domain SR pinned at 100% from 12% pretrain onwards [Table “Gap Between In- vs Out-of-Domain”].
Pretraining data quality/composition matters at matched volume: at 12.5%, 25%, and 50% subsampling, “high-quality subsampling” beats “uniform subsampling” by 31.8 / 34.3 / 28.2 pp of success rate at matched validation loss deltas, and validation loss correlates with downstream success — making val loss a practical proxy for expensive real-robot evals [Table “Success Rate vs. Pretraining Scale”].
At full pretraining scale, four independent SFT copies of ACT-2, each given one demonstration of a different folding technique, all successfully executed those techniques on a held-out garment unseen during SFT — presented as the first end-to-end model result of one-shot behavior acquisition that generalizes to unseen environments [§“One demonstration is enough to teach a new behavior”].
Overall zero-shot success on autonomous laundry folding: 99.1% ± 0.3% SE across 785 attempts on 9 garment types under a declared scope (unseen rooms, varied surfaces, varied lighting, robot on left/right/foot of bed) and declared adaptation cost (zero per home, fixed checkpoint) [§“Results”].
Success rate holds across bed sheet color (99.2 light / 98.0 dark / 99.6 colored), starting configuration (98.8 pile on bed / 100 basket on bed / 99.5 basket on ground), and robot position (98.7 / 100 / 98.5); lowest by garment is blouses at 94.7% [Table “Success Across Environments” and “Success Across Garments”].
Fold quality: mean 4.72/5 stars across 778 completed folds under a fixed pre-registered rubric, 73.8% receiving 5★, 98.3% ≥ 4★; garment-mean quality range 4.61 (blouses) to 5.00 (sleeveless tops) [Table “Quality Remains High Across Garment Types”].
Median completion time 2 min 13 sec, mean 2 min 19 sec, from garment retrieval to stack placement including autonomous retries; per-garment medians range 1 min 14 sec (shorts) to 2 min 44 sec (thin long-sleeved tops) [Table “Completion Time of Successful Folds”].
The post proposes a “Solve” evaluation standard for robotics: reliable performance across a declared scope at a stated adaptation cost, arguing that raw success percentages are non-comparable without both — a direct pushback on demo-driven progress reporting [§“A standard for measuring robotics progress”].

Method

ACT-2 is described at recipe level rather than at architectural detail. The base model is pretrained on a “high-quality, high-diversity, sensorized human dataset” collected via Sunday’s proprietary hardware (Memo), curation system, and processing pipeline — i.e. the substrate is human data captured with an instrumented rig rather than robot teleoperation, similar in spirit to the UMI-scale approach of Xiaomi-Robotics-1 (XR-1) — Scaling VLA Foundation Models with 100K Hours of Embodiment-Free UMI Pre-training but with sensorization beyond RGB. Post-training uses supervised fine-tuning on curated in-house “Memos” (fleet-side captures), and the paper’s central experimental design is to sweep pretraining data volume (0% to 100%) while holding the post-training procedure fixed, then measure in-domain (post-training environments) vs out-of-domain (held-out environments, objects, configurations) success. The claimed emergent capabilities — one-shot new-behavior acquisition, edge-case recovery from garments on the floor or in crumpled states, disturbance robustness during human intervention, and whole-body manipulation exploiting Memo’s mobile base and height adjustment — are demonstrated qualitatively rather than benchmarked. The 785-attempt laundry evaluation used a purpose-built grading tool with two-round annotation (initial grade + independent review, differences resolved by review lead), a rubric fixed before evaluation, and annotator training on reference folds — the most explicit protocol description Sunday has published.

Results

Generalization gap collapses to 0 at full pretraining scale (see Key claims); at 0% pretrain the model overfits (96% ID / 14% OOD) but is essentially unusable OOD.
Overall 99.1% success across 785 zero-shot attempts on 9 garment types, 98.3% of folds ≥4★, mean quality 4.72/5.
Blouses are the hardest garment (94.7% success, 4.61 mean quality); Sunday attributes this to their lightweight highly deformable construction providing fewer stable geometric cues for grasping and alignment.
Median 2 min 13 sec per fold end-to-end.
One-shot behavior acquisition on held-out garments confirmed on 4 independent techniques (no numeric success rate given for the one-shot experiment itself, only qualitative “all four successfully executed”).

Why it’s interesting

Two things stand out. First, if the generalization-gap curve replicates, it means the “how much data does a new task need” question in home robotics is downstream of pretraining scale in a very specific way — the same LLM-era GPT-1 → GPT-3 curve, but for sensorimotor policies. This is the same shape Xiaomi-Robotics-1 (XR-1) — Scaling VLA Foundation Models with 100K Hours of Embodiment-Free UMI Pre-training argues for with UMI hours and that EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data argues for with egocentric video, but Sunday’s variant uses sensorized human capture and reports the sharpest one-shot claim yet. Second, the “Solve” standard is an unusually explicit response to the demo-driven reporting problem that recurs in the wiki’s VLA papers (see VLA Models open questions) — declaring scope + adaptation cost up-front is the kind of protocol change that would make RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies-style comparisons more meaningful, and contrasts with the demo-heavy launches on this page (e.g. RoboTTT — 8k-timestep robot policy via test-time training (Jim Fan / NVIDIA GEAR) RoboTTT, LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World LEGS). Caveats: this is a preview blog, not a paper — no architecture details, no comparison against a common baseline like π0.5 on the same evaluation, and the one-shot generalization claim is demonstrated on one task family (folding techniques) with n=4.