The Bitter Lesson

TL;DR

Sutton’s 2019 essay distills 70 years of AI into one claim: general methods that leverage computation win over methods that bake in human knowledge of the problem domain. The historical pattern — chess, Go, speech recognition, computer vision — is the same: researchers encode what they think they know; search and learning at scale eventually displace that encoding; the resulting success is “bitter” because it feels like a loss for the human-centric approach. The two methods that scale arbitrarily with compute are search and learning; the two things that should not be built in are our own ideas about how minds represent the world (space, objects, agents, symmetries). The piece is the canonical reference for the “stop encoding priors, let compute find them” position that recurs across the wiki.

Key claims

The dominant long-run driver of AI progress is Moore’s-law-style exponentially falling cost of computation; general compute-leveraging methods beat human-knowledge-leveraging methods “by a large margin” over timescales longer than a research project [Opening].
The human-knowledge approach and the compute-leveraging approach tend to run counter in practice even when they need not, because time spent on one is not spent on the other and because human-knowledge methods complicate systems in ways that make them less able to absorb more compute [Opening].
Computer chess (Kasparov 1997, deep search + special hardware) is the archetype: the search-based approach was dismissed by human-knowledge chess researchers as “brute force” and “not how people play,” then won anyway [Chess §].
Computer Go repeated the pattern 20 years later: enormous initial effort went into leveraging human Go knowledge or special game structure; search applied effectively at scale (plus self-play value-function learning) made that effort irrelevant or counterproductive [Go §].
1970s DARPA speech recognition — HMM-based statistical methods beat methods based on knowledge of words, phonemes, and vocal-tract structure. Deep learning in speech is the latest step in the same direction: even less human knowledge, even more computation, even larger training sets [Speech §].
Computer vision followed the same arc: SIFT, generalized cylinders, edge-based pipelines are discarded; modern deep nets use only convolution and certain invariances [Vision §].
Four-point summary of the lesson: (1) AI researchers try to build knowledge into agents; (2) this helps in the short term and is personally satisfying; (3) it plateaus and inhibits further progress in the long run; (4) the breakthrough eventually comes from an opposing scaling-compute-based approach via search and learning [Closing §].
The two general-purpose methods that scale arbitrarily with increased computation are search and learning; everything else should be built on top of these [Closing §].
The second meta-point: the contents of minds are “tremendously, irredeemably complex” — do not try to find simple ways to think about space, objects, multiple agents, or symmetries; these are arbitrary features of the outside world. Build in only the meta-methods that can discover and capture that complexity, not the discoveries themselves [Closing §].

Method

A ~1,100-word position essay, no equations, no experiments. The method is historical case analysis across four fields (chess, Go, speech, vision) plus two normative generalizations. No figures.

Results

Not applicable — this is a position piece. Its “result” is a predictive claim that has become one of the most-cited heuristics in frontier ML: that pretraining-scale compute + general architectures will displace handcrafted inductive biases wherever the regime allows it.

Why it’s interesting

This is the canonical reference behind a position the wiki already tracks under multiple names:

The Bitter Lesson is the normative frame behind The flavor of the bitter lesson for computer vision‘s argument that explicit 3D intermediates (NeRF, SDFs, SE(3) camera poses, segmentation, optical flow) are the current generation’s “human-knowledge” artifacts to be dissolved by generative-rollout video world models — the same essay anchors a seventh WFM framing on World Foundation Models.
It is the implicit baseline for The 4-bitter Lesson: Balancing Stability and Performance in NVFP4 RL (titled explicitly against it: “The 4-bitter Lesson”) and the stability-vs-compute debate on NVFP4 RL.
It frames the data-side scaling-vs-filtering debate indexed by A Bitter Lesson for Data Filtering (“A Bitter Lesson for Data Filtering”) — the generalization of Sutton’s claim from architecture to data curation.
Tongyi DeepResearch’s “Native ReAct” design (Tongyi DeepResearch: A New Era of Open-Source AI Researchers) explicitly cites the Bitter Lesson as motivation for stripping prompt engineering out of agentic rollouts.
Nishant Desai’s “low-data inductive-bias lessons don’t translate to the high-data regime” result (Low-data inductive-bias lessons don't translate to the high-data regime — embracing chaos produced emergent policy capabilities (Nishant Desai / Sunday Robotics)) is an empirical instance of claim 3 (short-term wins plateau at scale) on the VLA substrate.

Filing it as a wiki page lets future citations resolve against a single canonical The Bitter Lesson slug rather than scattered external links.