Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understanding

TL;DR

Lumina-DiMOO is an open-source 8B omni-modal foundation model built entirely on discrete diffusion — text, images, and image-conditioned tasks all go through a single masked-token denoising backbone with no AR factorization and no hybrid AR+diffusion split. It is initialized from LLaDA-SFT, supports text-to-image, image editing, subject-driven generation, inpainting, and image understanding under one architecture and arbitrary resolutions, and reaches GenEval 0.88 / DPG 86.04 / MMMU 58.6 — outperforming MMaDA (same architectural family) by ~25 points on GenEval and beating BAGEL, Janus-Pro, and TokenFlow-XL on the unified-model table reported on the project page. Block-wise semi-autoregressive parallel prediction + a custom cache claim up to 32× T2I speedup over AR baselines, making it a concrete datapoint for “discrete diffusion as a unifier” against the dominant AR (Janus-Pro / Show-o) and AR+diffusion (BAGEL / OneFlow) UMM paradigms.

Key claims

• A single fully-discrete-diffusion backbone handles text-to-image, image-to-image (editing, subject-driven, inpainting), and image understanding without architecture switches or per-task heads [project page Overview].
• Achieves SOTA on unified-model leaderboards at 8B params: GenEval overall 0.88 (vs Janus-Pro-7B 0.80, BAGEL-14B 0.82, GPT-4o 0.84, MMaDA-8B 0.63) [project page GenEval table].
• DPG overall 86.04 (vs Janus-Pro-7B 84.19, BAGEL-14B 85.07, GPT-4o 85.15, MMaDA-8B 69.97) — best among filed unified models on this benchmark [project page DPG table].
• Understanding-side parity at 8B: POPE 87.4, MME-P 1534.2, MMB 84.5, SEED 83.1, MMMU 58.6 — outperforming MMaDA (similar discrete diffusion architecture) by ~28 MMMU points and matching/beating Janus-Pro-7B on MMB and SEED [project page Understanding table].
• Block-wise semi-autoregressive parallel prediction plus a bespoke caching method (ML-Cache + Region-Adaptive Sampling) yields up to 32× T2I acceleration over AR baselines and an additional ~2× from the cache alone [GitHub README; emergentmind summary].
• Initialized from LLaDA-SFT; an ablation against from-scratch training quantifies the gain from inheriting text capability within the discrete-diffusion framework [arxiv html §7].
• Self-GRPO RL post-training is used as a final stage — applied jointly to generation and understanding signals — and is cited as one of the two main drivers (alongside the staged pretraining data recipe) for the >25-point GenEval gap vs MMaDA [liner.com review].

Method

A single Transformer denoiser trained as a masked discrete diffusion model over a joint tokenization covering text tokens, image VQ tokens, and task / modality boundary tokens. Inputs and outputs from any modality enter and exit through the same vocabulary, so a turn like “describe this image” and “generate an image of a dog” use the same forward pass — only the unmasking schedule and the input mask pattern differ. The model is initialized from LLaDA-SFT (an 8B masked-diffusion text LM) to inherit text capability, then continued-pretrained in stages on text+image data, with a Self-GRPO RL stage at the end.

At inference, decoding is block-wise semi-autoregressive: the model emits one block of tokens at a time via parallel masked-token denoising within the block, then moves on to the next block conditioned on completed blocks. This is what enables the 32× T2I throughput claim — within a block, all tokens are predicted in parallel; across blocks the schedule is sequential but coarse. A custom caching scheme (ML-Cache) reuses intermediate activations across denoising steps for an additional ~2× speedup, and Region-Adaptive Sampling allocates more denoising steps to high-uncertainty image regions.

Results

Generation (GenEval, 8B unified): overall 0.88, beating Janus-Pro 0.80, BAGEL-14B 0.82, GPT-4o 0.84, MMaDA-8B 0.63, and dedicated-generation models like FLUX.1-Dev 0.66 and SD3-Medium 0.74. Position score 0.85 is the standout — 6 points above Janus-Pro’s 0.79 and 10 above GPT-4o’s 0.75 [project page Table 1].

Generation (DPG, 8B unified): overall 86.04 — beats BAGEL-14B 85.07 and GPT-4o 85.15, with Relation at 94.31 (best on the table) [project page Table 2].

Understanding (8B unified): POPE 87.4 (tied with Janus-Pro), MMB 84.5 (best on the unified table, only BAGEL-14B’s 85.0 is higher at much larger scale), SEED 83.1 (best on table), MMMU 58.6 (best on table, beats BAGEL-14B 55.3) [project page Table 3].

Inference: claims up to 32× T2I throughput over AR baselines via block-wise semi-AR parallel prediction; an additional 2× from the bespoke caching method. Supports arbitrary resolutions, which the project page lists as a concrete advantage over fixed-resolution unified models like MMaDA and Janus-Pro.

Open release: full code + checkpoints under Alpha-VLLM/Lumina-DiMOO on GitHub; diffusers + ComfyUI support added on 2025-10-21; VLMEvalKit evaluation harness on 2025-11-27. Follow-up papers HT-GRPO (2026-05) and dMLLM-TTS (CVPR 2026) and a “test-time scaling for diffusion MLLMs” piece (2025-12) extend the model line.

Why it’s interesting

Lumina-DiMOO is the strongest filed datapoint to date for fully-discrete diffusion as a unifier across modalities — every other 8B-class unified model on the wiki either uses AR over a tokenized vocabulary (Reconstruction Alignment Improves Unified Multimodal Models, Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generation) or hybridizes AR with a diffusion image head (OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flows, Omni: Context Unrolling in Omni Models (ByteDance Seed)). It also sharpens Diffusion Language Models beyond text-only: where Mercury and I-DLM chase throughput on text and APS (Test-Time Anchoring for Discrete Diffusion Posterior Sampling) uses discrete dLLMs for posterior sampling on images at test time, Lumina-DiMOO trains a single discrete-diffusion model end-to-end on both modalities and reports both >SOTA unified-model scores and 32× T2I speedup — a direct rebuttal to the “discrete diffusion doesn’t scale to image generation” prior. The MMaDA → Lumina-DiMOO delta (25 GenEval points at matched architecture family) suggests the bottleneck for prior discrete-diffusion UMMs was data and post-training, not the diffusion choice itself.

See also

• Diffusion Language Models — Lumina-DiMOO extends the dLLM concept from text-only throughput races into a unified text+image foundation model, with the discrete-diffusion objective as the unifier
• Unified Multimodal Models — sits in the End-to-End cell of the UniG2U taxonomy (UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?) as a from-scratch (vs MLLM+DiT-bolt-on) UMM trained jointly across modalities
• The Design Space of Tri-Modal Masked Diffusion Models — Apple’s tri-modal (text+image+audio) MDM is the closest filed architectural cousin; Lumina-DiMOO is text+image only but ships the higher-quality production model
• Reconstruction Alignment Improves Unified Multimodal Models — AR-paradigm contrast: RecA improves unified models post-hoc via reconstruction alignment, while Lumina-DiMOO bakes the unification into the discrete-diffusion objective from the start
• Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generation — another sparse unified architecture for perception+generation at similar scale; AR sparse MoE vs Lumina-DiMOO’s dense discrete diffusion
• NEO-unify: Building Native Multimodal Unified Models End to End — NEO-unify removes the VAE entirely in an MoT architecture; Lumina-DiMOO keeps a VQ tokenizer but removes the AR factorization — orthogonal cuts at the same problem