Low-data inductive-bias lessons don't translate to the high-data regime — embracing chaos produced emergent policy capabilities (Nishant Desai / Sunday Robotics)

ListenEmbracing Chaos: When Clean Data Becomes the Enemy8:45

Show notes

A first-person retrospective from Nishant Desai of Sunday Robotics: after a year running both recipes, the team reports that last year's carefully-curated, inductive-bias-heavy robot policy was competent but brittle — and that deliberately letting the dataset span the real-world state distribution produced capabilities they never predicted. We unpack why low-data lessons don't transfer to the high-data regime, where this fits alongside the broader "diversify and scale beats filter and curate" thesis, and what it means for anyone training generative models. No numbers, no artifact — but a rare candid reversal from a team that had committed to the opposite bet. **Sources** - Low-data inductive-bias lessons don't translate to the high-data regime — embracing chaos produced emergent policy capabilities (Nishant Desai / Sunday Robotics)

Subscribe to the channel: https://podcast.lumalabs.link/27a25f51ea81d41a8253f46c6133a35e/feed.xml

TL;DR

Nishant Desai (ML, Sunday Robotics) posts a first-person retrospective: last year’s robot policy relied on inductive biases and strict data-collection procedures, producing a capable but brittle policy; this year the team “embraced chaos” and let the dataset span the real-world state distribution, and unpredicted capabilities emerged. No numbers, no artifact — a one-paragraph observation that generalizes the year-over-year lesson their team learned. Relevant because it’s another lab’s first-hand version of the “clean data is the enemy” thesis, from someone shipping VLAs in the real world rather than on a benchmark.

Key claims

Lessons learned in low-data, overfit regimes do not transfer to the high-data regime; the failure mode of last year’s policy was brittleness despite competence [tweet body].
Last year’s recipe leaned on inductive biases + strict data-collection procedures to make a small dataset train something useful; the tradeoff was robustness [tweet body].
This year’s recipe deliberately let the dataset “span the real-world state distribution” rather than curate it, and unpredicted capabilities emerged from the change [tweet body].

Method

Not applicable — the tweet is an observation, not a paper. There is no method write-up, no ablation, no benchmark result. It’s a public reflection on a private recipe change; the corresponding artifact (Sunday Robotics’ current-generation policy) is not linked.

Results

Not applicable — no quantitative claims. The only outcome asserted is qualitative: the emergent capabilities “we would never have predicted.”

Why it’s interesting

This is the third independent data point on the wiki for the “diversify + scale beats filter + curate” position in robot policy training, but the first from a first-person operator perspective rather than a benchmark result or a scaling-law paper. Spirit-v1.5: Clean Data Is the Enemy of Great Robot Foundation Models framed it editorially as “clean data is the enemy of great robot foundation models” and shipped weights on top of that thesis; A Bitter Lesson for Data Filtering made the general pretraining-side argument that at high compute, aggressive filtering loses to scale. Desai’s tweet is a small-team, real-deployment version — the interesting bit is not the claim itself (which is now a wiki-tracked pattern) but that a team that had committed to the inductive-bias / strict-collection recipe publicly reports the reversal after a year of running both.

It also cuts against the sensorimotor-primacy counter-position (Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level) in a specific way: the emergent capabilities came from broadening the data distribution, not from adding tactile / contact / force channels — implicitly a bet that the missing generalization was in coverage of state-space rather than in modality of sensing.