MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement

TL;DR

MiMo-V2.6 is Xiaomi’s follow-up to the MiMo-V2-Flash / V2.5-Pro line, presented as an omni-modal MoE family (Pro at 1.02T total / 42B active, Flash at 310B/15B) whose central claim is that scaled agentic RL — not architecture or pretraining — is the load-bearing capability lever. The RL run spends $2.6M /$ 0.9M respectively across three explicit scaling axes: training compute (1,568-prompt batches, 25K rollouts/step, 2.7–3.7B tokens/step, up to 1M context, fully asynchronous), environment/harness diversity (code, general workflow, visual coding, cybersecurity, mixed harnesses), and grader compute (12.7% of budget devoted to groupwise agentic grading beyond binary tests). Reported benchmarks rise monotonically through RL — e.g. Pro from 58.4 → 72.6 on DeepSWE — and the team open-sources the training dynamics, RL environments, RL framework, and a 9B distilled student (MiMo-V2.6-Distill-Qwen-9B) as a reproducibility substrate for agentic RL research.

Key claims

Hybrid-SWA backbone with sparse MoE without shared experts, interleaving N SWA layers (window=128) then one Global-Attention layer per hybrid block; the very first block uses GA + dense FFN to stabilize early representation learning [§2.1, Table 1].
Pro: 70 layers (60 SWA / 10 GA), hidden 6144, 384 experts (8 active), 42B/1.02T; Flash: 48 layers (39 SWA / 9 GA), hidden 4096, 256 experts (8 active), 15B/310B; both with 192/128 QK/V head dims and 128 SW size [Table 1].
Speculative decoder is a 5-layer SWA drafter based on the DFlash block-diffusion design (Chen et al., 2026), predicting 7 tokens per forward pass conditioned on backbone hidden features + clean anchor, verified in parallel by the backbone [§2.4].
MiMo-ViT replaces fixed non-overlapping windows with sink-augmented SWA and alternates row-major / column-major serialization; periodic GA layers aggregate global context. Trained from scratch on 4T+ image tokens using only cross-entropy multimodal understanding loss — no contrastive objective, with the LLM kept trainable to provide semantically meaningful gradients [§2.2].
Audio pipeline is two-stage: a 20-layer RVQ Audio Tokenizer trained on 20M hours (speech + music + audio) producing 20 discrete tokens/frame at 25Hz, followed by an audio patch encoder that groups 4 frames into a patch with bidirectional intra-patch self-attention, reducing rate to 6.25Hz [§2.3].
Pre-training is two-stage (text-only → omni-modal joint training); Flash sees 48T tokens (26T text + 22T omni) and Pro sees 30T (27T + 3T); context extended from 32K → 256K partway through [§3.1].
Mid-training switches the optimizer from AdamW to a Muon variant (Muown, adds explicit row-norm control on top of Muon) for hidden weight matrices, keeping AdamW for embeddings / LM head / MoE router; motivated by observed AdamW efficiency loss beyond the critical batch size and Muon’s better data efficiency at large batch [§3.2].
Adam→Muon switch mid-run is normally destabilizing (per Qu et al. 2026) but MiMo-V2.6 reports no loss spikes throughout mid-training, with MXFP4 quantization-aware training enabled at the same time [§3.2].
Mid-training also serves as a long-context extension stage (256K → 1M in the final phase) and an agent-centric data mixture stage (coding, general, visual, research trajectories + repository-level code + image/video/audio) [§3.2].
RL compute is scaled to $2.6M (Pro) /$ 0.9M (Flash); each step uses a batch of 1,568 prompts × G=16 rollouts = 25K sequences ≈ 2.7–3.7B tokens (110K–150K tokens/sequence average) at up to 1M context [§4.1].
Cost breakdown for Pro: Training 43.5%, Rollout 43.8%, Grader 12.7% (Flash similar); rollout parallelizes over sequences, training shards over data-parallel ranks, HBM sized to keep rollout arithmetic-intensive [Fig. 3].
Long-tailed rollout + grading handled by partial rollout (Kimi Team 2025): unfinished sequences are interrupted at batch collection and resumed the next phase; the cost is re-prefill (KV cache rebuild after each policy update), and batch size + partial rollout are jointly tuned so re-prefill is amortized [§4.1].
Train–inference consistency is enforced by R3 (Ma et al. 2025) plus replay of top-p sampling candidate sets (Liu et al. 2025a; Microsoft AI 2026) to align sampling geometry between the training and inference engines [§4.1].
Dynamic sampling filters all-pass / all-fail groups to keep effective gradients; a Sample Mixer stabilizes per-task sample composition in the batch across tasks with different rollout durations and pass rates [§4.1, §6.3].
RL objective is a prompt-mean-aggregated group-relative form with token-level mask $M_{i,t}$ and importance-sampling ratio $r_{i,t}$ : $L(\theta) = -E[\frac{1}{\sum |o_i|} \sum_i \sum_t r_{i,t} M_{i,t} A_i \log \pi_\theta(o_{i,t} | q, o_{i,<t})]$ over prompts drawn from the union of task datasets [Eq. 1].
RL environments span four domains — Code Agent, General Professional Workflows, Visual Artifacts, Cybersecurity — with reward-hacking mitigation designed in from the start via execution checks, rollout-based audits, and adversarial testing [§4.2, §4.2.6].
Coding tasks are synthesized through five complementary pathways (GitHub PR + issue mining, in-house “vibe coding” employee requests, specification-driven, source-code-driven via CodeMidas [Ye et al. 2026], long-horizon multi-step engineering), plus licensed data + public sources (SWE-Bench-Verified/Multilingual, PrimeIntellect) [§4.2.1, Fig. 4].
Supervision-accuracy pipeline for code tasks: (a) revise/remove tests enforcing behavior absent from the spec, (b) run four rollouts per task and have an auditing agent inspect them in a shared workspace to review specification–test alignment [§4.2.1].
Robustness pipeline for code tasks: run F2P / P2P test cycles both before and after applying the reference patch to detect environment flakiness [Fig. 4].
Groupwise agentic grading (§4.3) is the third scaling axis: instead of binary same-reward-for-all-passing solutions, compare within-group solutions to distinguish quality; converted to learning signal via GRS (Groupwise Reward Synthesis) and GAR (Groupwise Advantage Redistribution) [§4.3, §1].
Router freezing is used as a stabilizer for scaled RL — the MoE router is frozen during RL to prevent catastrophic drift; combined with a multi-layer defense against reward hacking [§1, §5.4].
You Only RL Once is the deliberate framing of §5 — a single very long RL run, not multiple short RL rounds — enabled by the infrastructure described in §6 [§5].
Infrastructure decouples control plane from data plane to move massive routing-tagged multi-modal trajectories at scale, uses a pool of harnesses for high-concurrency multi-framework rollout, and unifies trajectory representation across agentic frameworks [§6, §6.1–6.2].
Post-RL capability broadening uses MOPD² — Multi-Domain On-Policy Distillation, second-generation variant of the MOPD recipe from MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training — to reintegrate any domain capabilities compressed by the You-Only-RL-Once run [§5.6].
Open-source deliverables: MiMo-V2.6-Distill-Qwen-9B (lightweight student), curated multi-domain task environments with verifiers, end-to-end RL training framework, and a composable mini-harness for flexible agentic interactions [§7].
Benchmark trajectories: MiMo-V2.6-Pro DeepSWE 58.4→72.6, MiMo-V2.6-Flash 48.7→65.7; monotone rise across DeepSWE v1.1, SWE-Bench Pro, MiMo Code Bench, AutomationBench v1.0.6, MiMo Visual Coding, and MiMo Cyber Bench through RL [Fig. 1, §4.1].
Ablation on MOE router freezing: framed as necessary for stable RL at this scale, with the freeze applied to prevent RL-time router drift while allowing expert weights to update [§5.4].

Method

MiMo-V2.6 is built around three coordinated design decisions that reinforce each other. Architecture: a hybrid-SWA sparse-MoE Transformer where five (Flash) or six (Pro) SWA blocks alternate with one GA block per hybrid group, keeping the receptive field local enough for cheap long-context serving while preserving global information flow. The audio and vision encoders share the SWA design language (windowed attention with periodic GA aggregation) so all three modalities live in the same compute regime; audio is quantized to 20-token RVQ frames at 25Hz then compressed to 6.25Hz by the patch encoder, image tokens flow through MiMo-ViT with row/column-alternating SWA + periodic GA. A 5-layer DFlash-style SWA drafter provides 7-token speculative decode.

Pretraining + mid-training: two-stage pretraining (text → omni) at 32K then 256K context, followed by a mid-training phase that (a) extends context to 1M, (b) trains on an agent-centric data mixture spanning agentic trajectories + code + image + video + audio, (c) switches the hidden-weight optimizer from AdamW to Muown — a Muon variant with row-norm control — with embeddings/LM head/router kept on AdamW, and (d) enables MXFP4 QAT to prepare the model for low-precision serving. The report emphasizes that this optimizer swap was executed mid-training on an already-pretrained model without loss spikes, contra the prior warning from Qu et al. 2026.

Scaled RL: three explicit scaling axes trained through a fully asynchronous stack. Compute scaling: 1,568-prompt batches × G=16 = 25K sequences per step, 2.7–3.7B tokens per step, up to 1M context, partial-rollout to amortize the long tail of rollout completion times, R3 + top-p replay to keep training and inference engines consistent, dynamic sampling to drop all-pass/all-fail groups. Environment scaling: synthesized code tasks from five pathways plus general-workflow, visual-coding, and cybersecurity domains, with an auditing agent + rollout-based supervision-alignment check + F2P/P2P flakiness detection built into the task synthesis pipeline. Grader scaling: 12.7% of RL compute is spent on groupwise agentic grading rather than binary test outcomes, with GRS + GAR converting within-group quality differences into fine-grained per-token learning signal.

The infrastructure to support this at scale (§6) decouples control and data planes so trajectories carrying MoE routing + multimodal payloads move independently of orchestration signals, pools multiple agent-harness frameworks so batches can span heterogeneous rollout stacks, and uses a Sample Mixer to keep per-task sample composition stable across tasks with wildly different rollout durations and pass rates. Post-RL, a MOPD² stage reintegrates domain capabilities compressed by the You-Only-RL-Once run.

Results

DeepSWE v1.1 (avg@3): Pro 58.4 → 72.6, Flash 48.7 → 65.7 — the anchor scaling-law-shaped curve of the paper [Fig. 1, Fig. 3].
Across all six benchmark tasks reported in Fig. 1 (DeepSWE v1.1, SWE-Bench Pro, MiMo Code Bench, AutomationBench v1.0.6, MiMo Visual Coding, MiMo Cyber Bench), both Pro and Flash show monotone improvement through RL — no single-domain overfitting reported [Fig. 1].
Cost breakdown (Pro): Training 43.5% / Rollout 43.8% / Grader 12.7%; Flash: 40.9% / 44.9% / 14.2% [Fig. 3 right].
Speed target: MiMo-V2-Flash (the predecessor) hit 150 tokens/s decoding via SGLang with a 5:1 SWA-to-GA ratio and 128-token windows (Xiaomi MiMo-V2-Flash — 309B/15B MoE with 5:1 SWA hybrid attention, 73.4 SWE-Bench Verified); MiMo-V2.6 keeps the same architectural family but the tech report focuses on RL-time compute rather than serving throughput, so no direct new decode-speed number is given.
Distilled student MiMo-V2.6-Distill-Qwen-9B reports consistent RL gains across diverse tasks + agent harnesses, positioned as a reproducible reference RL run for the community rather than a headline number [§7].

Why it’s interesting

MiMo-V2.6 is the canonical long-form technical writeup behind the MiMo line that the wiki has been tracking through announcement pages (Xiaomi MiMo-V2-Flash — 309B/15B MoE with 5:1 SWA hybrid attention, 73.4 SWE-Bench Verified, Fuli Luo on MiMo-V2-Flash engineering — Hybrid SWA, MTP, MOPD post-training, MiMo-V2-Pro (a.k.a. Hunter Alpha) — Artificial Analysis model card, MiMo-V2.5-Pro-UltraSpeed: Pushing 1T-Parameter Model Generation Speed to 1000 TPS) and their MOPD companion paper (MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training). Filing it lets several open threads on this wiki close: (a) the “$2.6M RL run, 25K rollouts/step, 12.7% compute on graders” numbers give a concrete counterpoint to the Training frontier knowledge work agents: A 397B RL training guide with SkyRL framing that “algorithm choices moved 3.9 pts, post-training moved 10–12 pts” — MiMo-V2.6 pushes past 14 points on DeepSWE with a single very long RL run and explicitly names grader compute as its own scaling axis; (b) the MOPD² stage grounds MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training as a load-bearing post-RL step in a frontier open MoE, not just Xiaomi’s small-model recipe; (c) the router-freezing choice is the sharpest filed statement on RL stability of an ultra-sparse MoE — Mitigate Silent Expert Death In Ultra-Sparse MoE argued expert death is the killer, MiMo-V2.6 argues router drift under RL is, and freezes it entirely.

The paper also lands in the middle of two live wiki debates. Against History May Repeat Itself: RSI Seen from a Previous AI Era (Tian’s “is RL the right RSI primitive?”) it comes down hard on the RL side — the abstract explicitly frames scaled RL as “a concrete path towards recursive self-improvement”, and the You-Only-RL-Once framing plus $2.6M budget is a maximalist position against Training-Free GRPO or program-search alternatives. Against Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe (Rethinking OPD’s failure taxonomy) it uses MOPD² as the capability-broadening tool after RL, doubling down on same-origin distillation as the reintegration mechanism rather than post-RL merge or task-arithmetic.

Complements Kimi K3 Technical Report — Open Frontier Intelligence (which uses multi-teacher OPD to consolidate specialists after RL) and NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 (SFT → async-GRPO → RLHF → MOPD) — three frontier-scale open reports now converge on the async RL + MOPD two-step, but MiMo-V2.6 is the only one to publish the cost breakdown and open-source the full RL environment + framework + distilled student. Also gives the wiki its first explicit named recipe (Muown) for switching optimizers mid-run without a loss spike, orthogonal to the FP16-vs-BF16-vs-NVFP4 RL-stability debate on Training stability at scale.