Ming-flash-omni 2.0

TL;DR

Ming-flash-omni 2.0 is Ant Group’s open-source (MIT-licensed) any-to-any omni-MLLM, released as a HuggingFace + ModelScope drop on 2026-02-11. It is built on the Ling-2.0 sparse Mixture-of-Experts backbone (100B total / 6B active per token) and adds three things over the October 2025 Preview: an end-to-end unified acoustic synthesis pipeline (speech + audio + music in one channel) using continuous autoregression coupled with a Diffusion Transformer (DiT) head with zero-shot voice cloning and emotion/timbre/ambience control; a “vision-to-knowledge” cognition path; and a native multi-task architecture that unifies segmentation, generation, and editing for high-dynamic image manipulation. Inputs: image/text/video/audio. Outputs: image/text/audio. The closest published tech report is arXiv:2510.24821, which covers the Preview predecessor, not 2.0; the 2.0 release is announced via the HF card and a WeChat post.

Key claims

100B-total / 6B-active sparse-MoE backbone built on Ling-2.0; positioned as a generational advance over Ming-flash-omni Preview and as a new open-source SOTA among omni-MLLMs [HF card §Introduction].
Unified end-to-end acoustic generation pipeline integrates Speech, Audio, and Music in a single channel — via continuous autoregression + DiT head — enabling zero-shot voice cloning and controllable attributes (emotion, timbre, ambient atmosphere) [HF card §Immersive and Controllable Unified Acoustic Synthesis].
“Vision-to-knowledge” synthesis: high-resolution visual capture combined with a vast knowledge graph for expert-level identification (plants, animals, regional cuisine, landmarks, artifacts including era / form / craftsmanship) [HF card §Expert-level Multimodal Cognition].
Native multi-task architecture unifies segmentation, generation, and editing with spatiotemporal semantic decoupling, with stated strengths in atmospheric reconstruction, seamless scene composition, and context-aware object removal while maintaining texture coherence and spatial depth [HF card §High-Dynamic Controllable Image Generation and Manipulation].
Released under MIT license; ~238 GB of safetensors weights sharded over 42 files; sub-modules include vision, audio, talker, transformer, vae, byt5, mlp, scheduler, connector [HF tree listing].
Code path: BailingMM2NativeForConditionalGeneration in modeling_bailingmm2.py, plus an AutoProcessor that handles image/video/audio via a process_vision_info helper and a Whisper-encoder code path for audio (audio_kwargs={"use_whisper_encoder": True}) [HF card §Example Usage].

Method

The model is a sparse-MoE any-to-any MLLM. The omni stack extends Ling-Flash-2.0 (the sparse-MoE LLM backbone) with modality-specific encoders for image, video, and audio, fused through a connector / MLP projection into the LLM token stream, and a “talker” module for audio output. Routing is modality-level (extending Ming-lite-omni’s modality-router pattern) so that the small per-token active-parameter budget (6B) is allocated across modalities without modality collisions. The 2.0 release introduces a continuous-autoregression-plus-DiT-head decoder for unified acoustic generation: the AR backbone emits a continuous latent stream and a DiT head decodes that into Speech / Audio / Music inside one channel, enabling cross-domain audio control (emotion, timbre, atmosphere) and zero-shot voice cloning. For image generation/editing/segmentation the model uses a native multi-task generation head built on the same backbone, with a VAE submodule shipped alongside the LM safetensors.

The 2.0 release ships as a sharded HF repo (42 safetensors files, ~238 GB total) plus a reference notebook (cookbook.ipynb) in the inclusionAI/Ming GitHub repo. The example pipeline runs in bf16 with FlashAttention-2 and split_model device-mapping across multiple GPUs; expected load time is ~10 minutes per the README, and the H20-GPU code path pins nvidia-cublas-cu12==12.4.5.8.

Results

The release does not include numerical benchmarks in the HF card; the closest published numbers come from the Preview predecessor (arXiv:2510.24821), which reports SOTA on contextual ASR across 12 ContextASR benchmarks, leading dialect-aware ASR across 15 Chinese dialects, and GenEval 0.90 on image generation. The 2.0 announcement frames itself qualitatively as a “new SOTA open-source omni-MLLM” with improvements in (a) visual encyclopedic knowledge, (b) immersive speech/audio synthesis with zero-shot voice cloning, and (c) high-dynamic image generation and manipulation. Numerical 2.0-vs-Preview comparisons are not in the model card; the WeChat announcement post (linked from the card) is the only first-party source for headline numbers at filing time and was not fetched here. As a downstream signal: 3,022 downloads in the first ~week, 4 spaces (cafe3310/ling-open-studio, cafe3310/ming-uniaudio-demo, cafe3310/ming-uniaudio-demo-en, imspsycho/ling-open-studio), and the model card cites both the Ming-Omni paper (arXiv:2506.09344) and the Ming-Flash-Omni preprint (arXiv:2510.24821).

Why it’s interesting

For Luma’s audio work, the headline is the continuous-AR + DiT-head decoder for a unified speech/audio/music channel — this is a concrete open-weights datapoint for the “treat acoustic generation as a continuous-latent AR problem with a diffusion head” recipe, with zero-shot voice cloning and attribute control as the productized output. It is the first MIT-licensed any-to-any omni model at the 100B / 6B-active scale and continues the pattern (LTX-2, Qwen3-TTS, HunyuanImage 3.0, Kimi K2.5) of frontier-grade open releases shipping with the full inference surface — see Open foundation-model releases. The model also unifies segmentation + generation + editing inside a single native multi-task head, which is a direct evolution of the Generative-Segmentation-as-Editing paradigm introduced by the Preview.