MAI-Thinking-1: Building a Hill-Climbing Machine

TL;DR

MAI-Thinking-1 is Microsoft AI’s first frontier reasoning model — a 35B-active / 1T-total MoE pretrained from scratch on 30T tokens of in-house, distillation-free human data on 8K GB200 GPUs, then RL-climbed across three domain specialists (STEM, agentic coding, helpfulness/safety) that are consolidated into one model. The technical report frames model development as a “hill-climbing machine”: a scaling-ladder ablation protocol, NLL eval suite, and an RL recipe + infrastructure tuned for sustained log-linear improvement over thousands of steps. Headline numbers are 52.8% on SWE-Bench Pro, 97.0% on AIME 2025, 94.5% on AIME 2026, and 87.7% on LiveCodeBench v6, with broad parity to Claude Sonnet 4.6. Three explicit design principles — capabilities should be learned not inherited (no third-party distillation), simplicity is sustainable, scientific rigor avoids shortcuts — make this report read as much as a methodology paper for industrial-scale training as a model release.

Key claims

Architecture is a dense-MoE-interleaved decoder Transformer with 5:1 local:global attention, 78 layers, hidden dim 6656, 8/512 LatentMoE per MoE layer; the global attention layers use NoPE rather than RoPE [§2.1, Fig. 2, Table 1].
The interleaved layout (high-sparsity MoE + dense FFN every other layer) matches medium-sparsity MoE-every-layer on FLOPs-based EG (≈1.03) but wins on wall-clock EG_Time (≈0.82 for the every-layer variant means it is slower at the same loss), so the interleaved design is preferred under their MFU [Table 2, §2.2.3].
Sparsity scales cleanly inside the family: increasing top-8/{256, 512, 1024} produces consistent FLOPs efficiency gains (1.09 → 1.18 → 1.20 average, with code at 1.20–1.39) [Fig. 3].
LatentMoE compresses tokens 2× before all-to-all dispatch and expands back after combine; routing decisions are made on the original (uncompressed) representation, each token routes to 8 of 512 experts with softmax gating [§2.1, “Feed-forward and MoE”].
Global-batch load balancing matters more than the loss form: GShard-style loss with cross-DP-and-microbatch aggregation performs similarly to loss-free balancing as long as global aggregation is used [§2.1].
The MoE implementation is fully dropless with variable-message-size all-to-all and bounded-memory guarantees, after the authors observed that token-dropping conclusions can differ from dropless even at low drop rates [§2.1].
Ablation methodology is built around a scaling ladder (L12–L78, fixed aspect ratio D = L·256/3) at fixed TPP (100–200 near Chinchilla, 500–1000 for overtrained main runs) and efficiency gain EG = baseline-cost-to-reach-candidate-loss / candidate-cost, with both EG_FLOPs and EG_Time variants [§2.2.1, §2.2.2, Table 1].
Evaluation during training uses ~40 NLL benchmarks across 5 categories with a weighted target 0.5·Code + 0.175·STEM + 0.175·Math + 0.1·General + 0.05·Multilingual reflecting heavy reasoning-and-code prioritization [§2.3, Eq. 3].
Pre-training data is built in-house from publicly available + licensed human sources (web HTML, web PDFs, books/journals, public GitHub, news, multilingual, domain-specific) with no LM-generated synthetic data during pre-training, and ML-benchmark decontamination [§1, §2.4, Appendix A].
Pre-training is 30T tokens followed by 3.55T tokens of mid-training that explicitly upweights STEM/math/coding; max context after mid-training reaches 256K [§2.6, Appendix B].
RL climb starts from a base model with no prior exposure to reasoning traces and produces three domain specialists (STEM reasoning, agentic coding/tool-use, helpfulness/safety), then consolidates capabilities into a single model [§3, §3.5].
Both STEM and agentic RL climbs sustain log-linear performance growth over many thousands of steps on AIME 2025, the hard LiveCodeBench v6 (post-Jan-25) subset, and SWE-Bench Verified [Fig. 1, §3.2, §3.3].
Headline benchmark scores: 52.8% SWE-Bench Pro, 97.0% AIME 2025, 94.5% AIME 2026, 87.7% LiveCodeBench v6; competitive with Sonnet 4.6 across STEM, agentic coding, knowledge, IF, long context, safety, health, tool calling [§1, §4].
Helpfulness and safety are trained jointly inside the RL climb against internal safety benchmarks, with continuous internal + independent red-teaming before release [§3.4, §5].

Method

The pre-training architecture uses periodic attention from Gemma 3 (5 local + 1 global per cycle, local window 512, RoPE base 10k local, NoPE global), GQA with 8 KV heads and head dim 128, QK-norm, tied input/output embeddings, no biases, RMSNorm at both block input and output before the residual, and the o200k_base tokenizer (200,019 vocab). Feed-forward layers alternate between dense SwiGLU and a LatentMoE block: a shared down-projection compresses 2× into a latent space, the all-to-all dispatch and expert computation (3× expansion inside each expert) happen there, then a shared up-projection returns to full dimension and the all-to-all combine runs in latent space too. Routing is computed on the original (pre-down-projection) hidden state, 8 of 512 experts via softmax gating, fully dropless, GShard-style aux loss with full global-batch aggregation. Pre-training runs at 8K GB200 GPUs on Microsoft’s Azure cluster with an in-house distributed stack (Sec. 2.8, “YOLO”) that uses FlashAttention-4 and Ulysses-style context parallelism.

The ablation protocol fixes architecture aspect ratio (D = L·256/3, query heads = L rounded up to 16, dense FFN expansion 2×, LatentMoE compression 2× / expert expansion 3×) across the L12 → L78 ladder, runs each candidate at fixed tokens-per-parameter (TPP), fits an NLL-vs-cost power law L = A·C^-α + E on the baseline, and reports EG as the cost-ratio at matched loss for both FLOPs and time. The RL climb runs three domain specialists from the same base, then merges them; the report describes a robust RL recipe with self-distillation and infrastructure improvements as the source of sustained log-linear gains over thousands of steps. Long-context extension to 256K is done in mid-training via progressive context-length scaling (Appendix B).

Results

52.8% SWE-Bench Pro, 97.0% AIME 2025, 94.5% AIME 2026, 87.7% LiveCodeBench v6 [§1, §4].
Competitive with Claude Sonnet 4.6 across STEM, agentic coding, knowledge, IF, long context, safety, health, tool calling [§1, §4.1].
Interleaved dense+MoE matches MoE-every-layer at EG_FLOPs ≈ 1.03 but the every-layer variants come in at EG_Time ≈ 0.69–0.84, i.e. the every-layer variants are 16–31% slower at matched loss [Table 2].
EG_FLOPs improves steadily with sparsity at top-8: 1.09 (256 experts) → 1.18 (512) → 1.20 (1024), averaged across categories; code shows the strongest gains (1.20–1.39) [Fig. 3].
Three sustained log-linear RL curves: AIME 25 and LiveCodeBench-v6-hard on the STEM specialist, SWE-Bench Verified on the agentic specialist, all rising steadily over many thousands of RL steps [Fig. 1].
Long context: 256K after mid-training; performance details in Appendix B.

Why it’s interesting

This is the highest-profile current example of an industrial-scale, no-third-party-distillation training program publishing its full methodology — and the “hill-climbing machine” framing is exactly the operational stance the wiki tracks under AI-for-AI Research (treating model development as a system-level optimization loop with scaling-ladder ablations + EG metrics). The LatentMoE + alternating dense/MoE + NoPE-global + 5:1 local attention recipe is the same architectural cocktail crystallizing across recent open MoEs — most directly Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning, which introduces LatentMoE and explicitly justifies the latent compression on systems grounds, and the routing/balancing design overlaps with Quantile Balancing: A Hyperparameter-Free MoE Load Balancing Method. The “no LM-generated synthetic data during pre-training” stance contrasts sharply with the Synthetic Training Data consensus that rephrasing-by-small-LM is now the default — Microsoft’s claim that “capabilities should be learned, not inherited” is the wiki’s first frontier-scale data point on the opposite side of that argument. The sustained log-linear RL curves over thousands of steps echo the agentic-RL recipes in Scalable Training of Mixture-of-Experts Models with Megatron Core, On Data Engineering for Scaling LLM Terminal Capabilities, and GLM-4.5: Agentic Foundation Model from Z.ai, but with the explicit claim of starting RL from scratch with no prior reasoning-trace exposure.