Show-o2: Improved Native Unified Multimodal Models

TL;DR

Show-o2 is the successor to Show-o, a native unified multimodal model that handles text, image, and video understanding + generation in one stack. It operates in a 3D causal VAE latent (Wan2.1’s) and constructs a dual-path unified visual representation that fuses high-level SigLIP-style semantic features with low-level VAE features via spatial-(temporal) fusion. A pretrained Qwen2.5 LLM is fitted with two heads — a language head trained with next-token prediction and a flow head (DiT-style adaLN-Zero transformer layers) trained with flow matching — and run with an omni-attention mask (causal between blocks, full attention inside visual blocks). A two-stage recipe (flow-head-only pretrain → full-model SFT) avoids needing a giant text corpus and preserves language ability while adding visual generation. The 1.5B and 7B variants beat similarly-sized UMMs across MME / GQA / MMMU / DPG-Bench and competitive on GenEval, trained on just 66M image-text pairs.

Key claims

A dual-path visual representation that concatenates SigLIP-distilled semantic features and a raw 2D-patch projector of the VAE latent (both extracted from the same 3D causal VAE space) supports both understanding and generation in a single sequence [§3.1, Eq. 3].
The semantic layers are SigLIP-so400m-patch14-384 weights distilled to operate on noised VAE latents — distillation reaches ~0.9 cosine similarity with the original SigLIP on the 66M training set, with noise applied at probability 0.3 in the last 20K iterations [§4.1].
Omni-attention (causal across blocks, full attention within unified visual representations) is what lets the same LLM serve next-token prediction on text and flow matching on images/videos without separate models [§3.1].
A two-stage training recipe — Stage-1 trains only the projector, spatial-temporal fusion module, and flow head on 66M image-text pairs; Stage-2 fine-tunes the full model on 9M understanding + 16M filtered generation data — preserves the base LLM’s language knowledge without a massive text corpus [§3.2, Table 2].
Scaling from 1.5B to 7B is done by reusing the Stage-1 flow head and inserting a lightweight MLP transformation to bridge hidden sizes; the 7B Stage-1 alignment-only phase is 3K iterations with 2K warm-up [§3.2, §4.1].
The 7B Show-o2 reaches MME-p 1620.5, MMMU-val 48.9, MMStar 56.6, AI2D 78.6, beating Janus-Pro-7B and TokenFlow-XL-14B across most metrics with similar or smaller data [Table 3].
On GenEval the 1.5B model scores 0.73 overall and the 7B scores 0.76 — competitive with Janus-Pro-7B (0.80) using ~half the data (66M vs 144M image-text pairs) [Table 4].
On DPG-Bench the 7B reaches overall 86.14, the best filed number in Table 5 vs SD3-Medium, Janus-Pro-7B, and Mogao-7B [Table 5].
Video generation: the 2B variant outperforms Show-1 (6B), Emu3 (8B), and VILA-U (7B) on VBench total score with competitive numbers vs CogVideoX-5B and Step-Video-T2V-30B [Table 6].
Mixed-modality interleaved generation: during fine-tuning, noise is applied to all images in an interleaved sequence with probability 0.3 (otherwise only the later ones), letting the model emit a [BOI] token to autonomously initiate image generation mid-sequence and continue the story [§4.4].

Method

Inputs are tokenized text or VAE-encoded image/video latents (Wan2.1’s 3D causal VAE with 8× spatial / 4× temporal compression). The visual latents go through a dual-path extractor: (a) SigLIP-distilled semantic layers (operating on possibly-noised latents) and (b) a 2D patch-embedding projector capturing low-level information; the two streams are concatenated along the feature dimension, RMSNorm’d, and run through two MLPs to form the unified visual representation $u$ . A time-step embedding is prepended for generative modeling.

The sequence — text embeddings interleaved with visual blocks — feeds a pretrained Qwen2.5 LLM under omni-attention (causal between blocks, full inside visual blocks). Two heads emit predictions: the language head does next-token prediction on text positions; a DiT-style flow head (transformer layers with adaLN-Zero time modulation) predicts the flow-matching velocity on visual positions. Training combines NTP loss and flow-matching loss [Eq. 4].

Stage-1 pretrains only the new modules (projector, spatial-temporal fusion, flow head) on 66M image-text pairs at 432² resolution for 150K + 40K iterations; Stage-2 unfreezes the full model and SFTs on 9M understanding + 16M filtered generation data for 35K iterations. Scaling up to 7B reuses the trained flow head with a hidden-size-bridging MLP. Higher-resolution training (512², 1024²) and a TextAtlas subset are added later for text-rendering quality.

Results

Multimodal understanding [Table 3]: Show-o2-1.5B reaches MME-p 1450.9, MMMU-val 37.1, AI2D 69.0 — best among ~1.5B UMMs (beats Janus-Pro-1.5B’s 36.3 MMMU). Show-o2-7B reaches MME-p 1620.5, GQA 63.1, MM-Bench 79.3, MMMU-val 48.9, MMStar 56.6, AI2D 78.6 — beats Janus-Pro-7B (62.0 / 79.2 / 41.0), Mogao-7B, and even the 14B TokenFlow-XL on MMMU and AI2D.

GenEval [Table 4]: 1.5B = 0.73 overall (Single 0.99, Two 0.86, Counting 0.55, Colors 0.86, Position 0.46, Color Attr 0.63), 7B = 0.76 overall (Single 1.00, Two 0.87, Position 0.52). BAGEL-14B (0.88) and Mogao-7B (0.89) remain ahead but were trained on 1.6B and unspecified-but-larger datasets respectively.

DPG-Bench [Table 5]: 7B = 86.14 overall (Global 89.00, Entity 91.78, Attribute 89.96, Relation 91.81, Other 91.64) — the best filed number.

Video generation [Table 6, VBench]: 2B variant achieves the reported total comparable to Gen-2, Pika-1.0, and VideoCrafter-2.0, with substantially fewer parameters than the 6B–30B unified competitors (Show-1, Emu3, Step-Video-T2V).

Why it’s interesting

Show-o is one of the canonical AR+Diffusion UMMs cited as the v1 reference point on the wiki’s Unified Multimodal Models concept page; this is the v2, and it changes the recipe in two ways worth noting. First, it picks a 3D causal VAE (Wan2.1) as the substrate, which is what makes images and videos share an encoder — a path also taken by OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning (which uses HunyuanVideo’s MMDiT) and DuoGen: Towards General Purpose Interleaved Multimodal Generation (which treats a video DiT as the image-generation head). Show-o2 is the most direct point of comparison on whether a “video VAE + flow head” is enough vs DuoGen’s “freeze a pretrained video DiT entirely”. Second, its dual-path representation parallels but is distinct from a Dual-stream diffusion transformer — the duality is in the encoder side (semantic + low-level features fused before the LLM), not in two separate transformer towers for text and image. Contrast with NEO-unify: Building Native Multimodal Unified Models End to End (NEO-unify), which goes the other direction and drops both the vision encoder and the VAE — Show-o2 is evidence that retaining both at the cost of a distillation step is competitive with much larger data budgets.