Roy Xie — modular capability hill-climbing with weight merging as a cheap alternative to MOPD

TL;DR

Roy Xie observes that frontier labs are converging on a modular post-training pattern — start from one base model, run per-domain capability climbs in parallel, then merge all specialists at the end — and names DeepSeek-V4 and Kimi K3 as recent examples. The thread then pushes back on multi-teacher on-policy distillation (MOPD) as the default merger: a careful revisit of published OPD-vs-SFT comparisons finds that clean-data, well-tuned SFT often matches OPD, and when teacher and student share the same base, direct weight merging (“a quick merge”) is a strictly cheaper first step. Framing: start cheap and simple, only reach for MOPD if the simpler mergers don’t deliver.

Key claims

Frontier post-training is becoming “more and more modular” — same base, parallel domain climbs, merge at end — with DeepSeek-V4 and Kimi K3 cited as recent instances of the pattern.
Published OPD-vs-SFT comparisons may be undertuned on the SFT side; with careful tuning and clean data, SFT often performs similarly to OPD, calling the net value of MOPD into question for some capability-integration settings.
When teacher and student share the same base, direct weight merging is a natural and cheap alternative to distillation — “a quick merge” is the first thing to try before paying the compute of MOPD.
Recommended default: start simple and cheap (SFT, merging); only adopt expensive MOPD-style recipes if the simple mergers fail to deliver the targeted gain.

Method

A Twitter thread by Roy Xie synthesizing observations from recent open frontier releases plus a controlled revisit of published OPD-SFT comparisons. The thread presents four figures (not individually captioned in extracted text) illustrating the modular pattern and the OPD-vs-SFT comparison. No new models or benchmarks are introduced; the contribution is a framing + recipe-ordering recommendation grounded in reinterpreting published work.

Results

No new numerical results. The thread’s load-bearing claims are qualitative: (1) the modular pattern is empirically visible across recent frontier open releases (DeepSeek-V4, Kimi K3); (2) in the author’s revisit, well-tuned SFT matches OPD on the relevant comparisons; (3) weight-merging is a viable cheap substitute when the shared-base condition holds.

Why it’s interesting

Direct challenge to the “MOPD-as-default” story that has been building across Reasoning RL — MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training (the canonical MOPD paper), MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement (MiMo-V2.6’s MOPD² post-RL stage), Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation (Nemotron-Cascade 2’s MOPD stage), and On-Policy Distillation (Thinking Machines’ OPD write-up). Xie’s SFT-matches-OPD finding sits alongside The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation‘s controlled compatibility trichotomy for when MOPD actively interferes rather than integrates — both arguments converge on “MOPD is not free, and its wins depend on undertuned baselines or specific thinking-pattern-overlap conditions.” The weight-merging recommendation complements the souping literature (Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performance) and the DeMix-as-data-mixing-proxy move in Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-training — three independent arguments that linear weight combinations do more work than the OPD-forward frame suggests.