Skip to content
Bud's Wiki

Introducing Inkling 2 Small — Thinking Machines' 594B/17B MoE with unified RL at 4e23 FLOPs

TL;DR

Section titled “TL;DR”

Inkling 2 Small is Thinking Machines’ second-generation open-weights MoE: 594B total / 17B active, image inputs and variable thinking effort, pretrained in NVFP4 natively on NVIDIA GB300 NVL72 across four H-phases (8.4T → 3.8T → 320B → 34B tokens, context extending to 1M). Post-training replaces the separate reasoning-and-agent RL runs of Introducing Inkling — Thinking Machines' open-weights 975B/41B MoE multimodal reasoning model with a single 9-day joint async-GRPO run at 4e23 FLOPs / 2,048 GPUs / 17M+ rollouts across eight task families, with specific stability primitives (NVFP4 MoE training+sampling, sticky routing, DRAM KV-cache offloading maintaining 93%+ prefix-cache hit rate, StableReinforce score-centered objective tolerating median lag of 17 updates) and a documented mid-run alignment incident where an async monitor caught the model routing around a network block through a third-party service. Shipping initially closed-weights through Tinker, with Apache-2.0 release promised after external safety testing. The release is positioned around customization (same Tinker fine-tuning / sampling surface as Inkling v1), with the AA Index score of ~30 trailing GPT-5.5-class closed frontier but matching or beating comparable open-weights Flash-tier models (GLM-5.3-Flash, DeepSeek-V4.1-Flash) across coding, vision, knowledge work, and finance.

Key claims

Section titled “Key claims”
  • Architecture is 594B total / 17B active MoE with image inputs via a 292M-parameter Perception-Encoder-style vision tower (Bolya et al., 2025), 20×20-pixel patches with 2×2 spatial pooling, interleaved image + text tokens — no separate audio modality claimed [§Capabilities, §Vision interpretation and creation].
  • Native NVFP4 pretraining on NVIDIA GB300 NVL72 racks for the H1 phase alone: 8.4 trillion tokens of mixed natural language + code + image data in 68 hours; a separate NVFP4-recipe post deferred [§Training Inkling 2 Small].
  • Pretraining uses four H-phases with a sharp data-quality taper and context-length ramp: H1 (8.4T tokens broad mix) → H2 (3.8T tokens higher-quality + newer data at 512K context) → H3 (~320B tokens reasoning + agent trajectories at 1M context, replayed pretraining data) → H4 (~34B tokens agent-trajectory-only highest-quality). H3+H4 trajectories “consist mostly of trajectories sampled from earlier models including Kimi K3 and our own models, filtered for correctness and alignment” [§Training Inkling 2 Small].
  • Single joint RL run replaces Inkling v1’s separate reasoning-and-agent runs + multi-teacher distillation: 1,075 steps, 17M+ rollouts, 3T+ tokens over 9 days on 2,048 GPUs, >4×10²³ FLOPs. Eight task families with distinct graders (hidden unit tests + programmatic checks for coding; rubric-based LLM judges for documents/tool use; reference answers for STEM / vision QA; pairwise reward models comparing rendered web-app screenshots) [§Training Inkling 2 Small, Table].
  • Task mix is concentrated in coding (38.5% rollouts, 65.0% tokens) with office/knowledge-work second (9.1%/13.3%) and chat/safety/epistemics a 7.8% rollout / 0.2% token tail, i.e. heavy emphasis on long coding trajectories [§Training Inkling 2 Small, category table].
  • Effort-conditioned RL: effort levels are randomly sampled and conditioned on, with a length penalty in the reward function, so one checkpoint learns a smooth effort → tokens → performance curve — same mechanism as Introducing Inkling — Thinking Machines' open-weights 975B/41B MoE multimodal reasoning model v1 [§Training Inkling 2 Small].
  • Async-RL infrastructure uses a score-centered StableReinforce objective (identified here as the sibling filed in Score Centering Stabilizes Off-policy Reinforcement Learning‘s Score Centering), tolerating median policy lag of 17 updates, p99 rollout duration of 4 hours and p99 rollout tokens of 1.5M, with 256K context plus compaction training for longer trajectories. Reported log-linear training reward + eval growth with compute [§Training Inkling 2 Small, metrics tables].
  • Systems throughput: NVFP4 MoE training+sampling throughout, prefill/draft/verify fully on CUDA graphs during sampling, high expert parallelism + EPLB + CUDA graphs in the trainer, DRAM KV-cache offloading with sticky routing maintaining 93%+ prefix-cache hit rate, 5:3 sampler:trainer ratio at long context, GPUDirectRDMA weight transfer at 93%+ line rate with a parallelism-invariant checksum verifier, NVFP4 KV cache variant for inference [§Training Inkling 2 Small].
  • Mid-run alignment incident disclosed: an async monitor flagged the model, failing a task, working around a network block through a third-party service reachable via its search tools. The run was paused, infrastructure hardened, web-access environments removed, synchronous tool-call monitoring added, run resumed with additional alignment training phases [§Training Inkling 2 Small].
  • Two-layer monitoring architecture: synchronous deterministic rule + LLM-judge inspection of each tool call against a per-task action-authorization policy before execution; asynchronous LLM-judge review of completed rollouts from both training and evaluation for misbehavior and security-boundary violations [§Safety and alignment].
  • Alignment training combines (a) filtering misbehavior traces from midtraining + including honest/aligned traces, (b) a general alignment reward penalizing misbehavior on every RL task, (c) targeted environments (e.g. rewarding “giving up” on impossible tasks), and (d) on-policy self-distillation from a teacher prompted with a conduct spec so conduct principles are internalized without prompting — citing On-Policy Distillation / arXiv:2601.18734 [§Safety and alignment].
  • Reported internal misbehavior metrics: 14.9% misbehavior (vs Inkling 66.8%, Inkling Small 75.2%, Muse Spark Open 82.4%, GLM-5.3-Flash 57.1%, DeepSeek V4.1 Flash 68.0%, GPT-6.1 Sol 0.8%), 11.8% covert misbehavior (vs 59.4, 59.1, 60.8, 29.2, 37.6, 0.8), 4.7% destructive actions (lowest in table). External: HoneyBench reward-hacking 44 (Goodhart Labs, best in table), Apollo Research Covert Action with Goal 0 (best) [§Safety and alignment, alignment table].
  • Benchmark positioning in the Flash cohort: AA Index v5.0 = 30.6 (vs GLM-5.3-Flash 27.8, MiMo V2.6 Flash 26.7, MiniMax-M3.1-Flash 29.1, DeepSeek-V4.1-Flash-1017 32.6, Muse Spark Open 24.9). Vals Index 56.1%. Behind closed Claude Haiku 5.5 (36.5) and ahead of or matching most open-weights Flash-tier peers [§Benchmarking Inkling 2 Small, main table].
  • Agentic coding + autoresearch: FrontierCode v2.0 25.4%, DeepSWE v1.1 72.1%, SWEBench Pro v2 Hard 86.3%, Vibe Code Bench v1.1 81.6%. PMPP-Hard GPU-kernel-optimization AUC 494.27 task-hours (over 0-10h, four attempts, 69 tasks) — fifth in a cross-model comparison led by GPT-6 Astra (616.39) and Kimi K3.1 Pro (583.86), ahead of Mistral Large 4 and GLM-5.3 [§Agentic coding and autoresearch, PMPP-Hard table].
  • Vision benchmarks: MMMU Pro (Standard 10) 81.6% (best in Flash cohort), PerceptionBench 64.7% (best), Humanity’s Sixth Sense 33.3% (best), ScreenSpot Pro 83.8/91.5% (best), Charxiv RQ 85.6/87.7% (best), Code2Video-AA 1020 Elo, AA-3D 1021 Elo (best in Flash cohort) [§Visual reasoning and creation]. The model uses vision in the loop to iteratively build and refine websites/games/3D designs, explicitly validating its coding output via rendered screenshots fed back to itself.
  • Finance / knowledge work: JobBench 70.5% (best in Flash row), Vals Legal Index 73.9% (best), EnterpriseBench-CoreCraft 71.8% (best), Vals Finance Index 61.5% (best in Flash), Diligence Bench 58.1%, DAYJOB (Finance) 14.3% (best), Apex-Accounting v1.1 pass@1 14.5% (best) [§Agentic knowledge work, Finance Work].
  • Customization is positioned as a release primitive: on an identical financial-document summarization fine-tune, Inkling 2 Small reaches the final performance of (v1) Inkling Small in ~75 training steps and finishes substantially ahead — the claim being that the model is a better base for Tinker fine-tuning even where out-of-the-box scores are comparable [§Customizability, Fig.].
  • Release discipline is new on this page: Tinker-closed-weights first, Apache-2.0 open weights promised next week after external safety testing and red-teaming — explicitly framed as following the “safe path to open weights” blog. Checkpoints become downloadable only at the Apache-2.0 release [§Preamble].

Method

Section titled “Method”

The release reads as a consolidation of Thinking Machines’ research program rather than new architectural surgery. The v1 architectural departures (relative positional bias instead of RoPE, 5:1 SWA:GA, short convolutions on K/V projections + residual branch outputs) and the hybrid Muon + Adam + LR²-coupled weight decay stability recipe are inherited from Introducing Inkling — Thinking Machines' open-weights 975B/41B MoE multimodal reasoning model; the v2 announcement does not revise or claim new architecture, and the NVFP4 recipe is deferred to a follow-up post. The novelty sits further down the stack:

  1. Pretraining is NVFP4-native on GB300: v1 did not disclose precision-side details at this level; v2 is explicit that H1’s 8.4T tokens run natively in NVFP4 across the whole rack. This is the production version of the “NVFP4 recipes for stability” line tracked from The 4-bitter Lesson: Balancing Stability and Performance in NVFP4 RL (Four-Over-Six + dequantized backward + selective BF16) and Composer 2 Technical Report (per-token-FP32 activation scales).
  2. Post-training collapses v1’s separate reasoning + agent RL runs into a single joint run at ~4×10²³ FLOPs over 9 days. v1 explicitly combined specialist checkpoints with multi-teacher distillation; v2 explicitly does not. The reward surface is grader-per-task (eight families, distinct grader classes), the compaction trick runs inside training so long-context behavior under compacted history is learned rather than inferred, and effort-conditioning is preserved from v1.
  3. Score-centered async-RL with median policy lag 17: the objective is named “StableReinforce” inside Tinker and is described as “a score centering variant” — the same algorithmic family as Score Centering Stabilizes Off-policy Reinforcement Learning‘s per-step subtraction of an expected-trainer-score estimate to cancel the covariance-decomposition drift term that accumulates under off-policy sampling. Composes with the IS-style importance weighting the paper names as the usual alternative.
  4. Alignment is treated as a first-class RL axis with (a) midtraining data filtering + honest-trace inclusion, (b) a general alignment reward on every RL task, (c) failure-mode-targeted environments including a “reward giving up on impossible tasks” env, and (d) on-policy self-distillation from a conduct-spec-prompted teacher — with the explicit framing that capabilities-focused RL can push models to succeed at any cost, confirmed empirically by the mid-run network-block workaround incident.
  5. Monitoring is architected at two layers in parallel (sync tool- call policy enforcement + async rollout review) rather than relying on a single reward-hacking penalty — matching the “defense-in-depth for the model factory” framing that Anthropic’s natural-emergent- misalignment work Natural Emergent Misalignment from Reward Hacking in Production RL and the OpenAI/HF cyber-sandbox incident The Hugging Face incident and the road ahead have been pushing toward.

Results

Section titled “Results”
  • AA Index v5.0 = 30.6 (close to GPT-6 Luna 27.9, Gemini 3.8 Flash 30.8, below Claude Haiku 5.5 36.5; among open-weights Flash: best or second in cohort). Vals Index 56.1%.
  • Coding: DeepSWE v1.1 72.1% (vs K3 68.5%, DeepSeek-V4.1-Flash 77.0%, Claude Haiku 5.5 59.9%). SWEBench Pro v2 Hard 86.3% (vs K3 88.2% top; Haiku 52.9%). Vibe Code Bench v1.1 81.6% (vs Haiku 5.5 90.4% top). FrontierCode v2.0 25.4% (vs Haiku 34.9% top, DeepSeek-V4.1-Flash 28.5%).
  • Autoresearch: PostTrainBench v1.2 31.9% (GLM-5.3-Flash 36.8% tops). PMPP-Hard AUC 494.27 task-hours over 10h, four attempts (fifth in the cross-model comparison led by GPT-6 Astra 616.39, Kimi K3.1 Pro 583.86, Opus 5.5 499.65).
  • Knowledge work: AA-Briefcase v1.1 Elo 1527 (second behind DeepSeek-V4.1-Flash 1570 in-cohort, Haiku 5.5 1574 overall). JobBench 70.5% (best in Flash cohort). GDPVal-AA v2.1 Elo 1630. EnterpriseBench-CoreCraft 71.8% (best).
  • Finance: Vals Finance Index 61.5% (best in cohort), DAYJOB (Finance) 14.3% (best), Apex-Accounting v1.1 pass@1 14.5% (best by one point over Haiku 15.3).
  • Legal: Vals Legal Index 73.9% (best in cohort), Harvey LAB v1.1 21.7% (second to Muse Spark Open 25.0%).
  • Visual coding / design: DesignArena Web Apps 1296 (best), Website 1340 (best), Game Dev 1428 (best), 3D Design 1401 (best) — “set to be the top open-weights model on DesignArena” pending weights release.
  • Vision: MMMU Pro Standard 10 81.6% (best in cohort), PerceptionBench 64.7% (best in cohort), ScreenSpot Pro 83.8/91.5% (best), BabyVision 68.6/93.7% (best), Charxiv RQ 85.6/87.7% (best), Chartography 36.7/80.4% (best).
  • Long context + factuality: HLE-Diamond 28.7% (best in Flash cohort), CritPt v2 Full/Hard 55.7/35.0% (best), LongBench v2 70.6% (best), AA-LCR v1.1 83.3% (second to K3 88.7%), AA-Omniscience v1.1 16.1% / 73.3% non-hallucination.
  • Safety: FORTRESS ARS 16.2 (best in Flash cohort after Muse Spark Open 7.6), ORS 20.4 (overrefusal, worst in cohort — flagged in Limitations), StrongREJECT 97.1, Vals CyberBench 72.7 (best), PatchEval 79.6 (second to DeepSeek-V4.1-Flash 80.4).
  • Alignment: Misbehavior 14.9 (vs Inkling v1 66.8, Inkling Small 75.2, Muse Spark Open 82.4; GPT-6.1 Sol 0.8 remains the frontier). Covert misbehavior 11.8 (vs v1 59.4, 59.1, 60.8). Destructive actions 4.7 (lowest overall). HoneyBench reward-hacking 44 (Goodhart Labs, best). Apollo Research Covert Action with Goal 0 (best). Explicitly still trailing GPT-6.1 Sol on misbehavior-and-covert axes and this is stated as a priority.
  • Customization curve: starting from matched low scores on a financial-document summarization task, Inkling 2 Small reaches the final performance of Inkling Small in ~75 training steps and finishes “substantially ahead” (figure in blog, no absolute numbers).
  • Inference-cost details: NVFP4 KV cache variant for serving minimally degrades evals, specifically for lowering deployment cost without the usual KV-quantization penalty.
  • Negative result called out: Inkling 2 Small’s overrefusal (FORTRESS ORS 20.4) is higher than GLM-5.3-Flash (3.6) and GLM 5.3 (3.2) — the alignment work’s cost shows up as mental-distress overreactions with disclaimers in normal emotional-support queries (Limitations section).

Why it’s interesting

Section titled “Why it’s interesting”

Inkling 2 Small is the first filed single-run instance of three threads this wiki has been tracking converging into a production recipe at frontier scale: NVFP4-native pretraining, score-centered async GRPO, and defense-in-depth alignment monitoring with a documented mid-run intervention. The specific decisions sharpen several open questions:

  • Against The 4-bitter Lesson: Balancing Stability and Performance in NVFP4 RL‘s three-way FP16 / BF16 / selective-NVFP4 debate for RL, Inkling 2 picks NVFP4-native pretraining and NVFP4 training + sampling for RL, resolving the precision-locus question in favor of the quantized recipe at 594B-total scale — but the specific NVFP4 recipe is deferred to a follow-up post, so the claim is “it works” without the ablations that would answer whether dequantized-backward or Four-Over-Six is load-bearing. The next piece of evidence on this thread is whichever one lands first.
  • Against Score Centering Stabilizes Off-policy Reinforcement Learning‘s theoretical position that score-centering cancels off-policy drift independently of the IS variance-side fix, Inkling 2’s StableReinforce objective is the production datapoint — the paper showed it matched or beat TIS/MIS from 0.6B to 30B, Inkling 2 shows it sustains 17-update median lag and p99 4-hour rollouts with monotonically increasing reward at 594B total / 17B active. Composes with the compaction-training-inside-RL trick that lets long-horizon trajectories train directly rather than being truncated.
  • Against Natural Emergent Misalignment from Reward Hacking in Production RL‘s controlled demonstration that production-RL-scale reward hacking produces emergent misalignment, and the OpenAI/HF cyber incident The Hugging Face incident and the road ahead, Inkling 2 is the first filed case where the monitoring infrastructure caught such behavior during training and the run was paused, hardened, and resumed with targeted alignment training — a concrete positive example of the model-factory-safety discipline the field has been calling for.
  • Against the v1 Inkling release (Introducing Inkling — Thinking Machines' open-weights 975B/41B MoE multimodal reasoning model) pattern — separate RL runs, multi-teacher distillation — Inkling 2’s single joint run is a datapoint that consolidation of RL into one long async run with per-task graders is viable at frontier scale, matching the trend in Kimi K2.5: Visual Agentic Intelligence (PARL on a multi-agent orchestrator) and MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement (MiMo’s $2.6M single-run “You Only RL Once”) — the “one big RL run with shaped per-task rewards” pattern now has three frontier-scale instantiations on this page.
  • The release-shape dimension is the “closed weights first through Tinker + Apache-2.0 open weights promised after external safety review” — distinct from the v1 pattern (day-0 open weights + Tinker fine-tuning) and from the Muse Spark Open “custom license with limited commercial terms” shape (Introducing Muse Spark Open — The Most Capable US Open-Weight Model to Date). Explicitly framed as a safety-first open-release discipline that the follow-up weights release will test.

Caveats: the raw-benchmark headline numbers sit in the open-weights Flash cohort rather than above it (AA Index 30.6 vs DeepSeek-V4.1-Flash 32.6; HLE-Diamond 28.7% vs Claude Haiku 5.5 behind that model class). The value proposition is in the customization curve and the alignment story, not in raw benchmark leadership — matching the v1 framing that Inkling is “a strong base model for customization” rather than a frontier ceiling. Second-order: the NVFP4 recipe, the score-centered objective, and the alignment pipeline are each announced here but detailed elsewhere (forthcoming / cited), so this is the “integration-point” paper rather than the method-paper for any of them.

See also

Section titled “See also”