Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE

TL;DR

Mamoda2.5 (ByteDance’s Moderation LLM team) is a unified AR-Diffusion model that combines multimodal understanding and image+video generation/editing in a single stack: an autoregressive front-end conditions a DiT-MoE backbone (128 experts, Top-8 routing; 25B total / 3B active) that denoises a VAE latent. The team reports top-tier results on VBench 2.0 and a new high on OpenVE-Bench video editing, matching Kling O1 in their evaluation. A joint few-step distillation + RL post-training step compresses the 30-step editing model to 4 steps, claiming up to 95.9× faster video editing inference and a 98% success rate in their internal advertising video-editing deployment.

Key claims

Unified AR-Diffusion stack: an AR module produces conditioning features, an MoE-based Diffusion Transformer (DiT-MoE) iteratively denoises latents conditioned on those features and the text/visual inputs, and a VAE maps to/from pixel space — one architecture spanning understanding, image generation, video generation, and video editing [§1, Architecture].
DiT-MoE design uses 128 experts with Top-8 routing for a 25B-parameter model that activates only 3B parameters per token, reducing training cost while scaling capacity [Abstract].
Reports top-tier VBench 2.0 generation scores and a new SOTA on OpenVE-Bench video editing quality, matching Kling O1 in the authors’ eval [Abstract, Fig. 1].
A joint few-step distillation + RL framework compresses a 30-step editing model to 4 steps, claiming up to 95.9× video-editing inference speedup vs open-source baselines [Abstract].
Deployed for content moderation and creative restoration in advertising, with a reported 98% success rate in their internal video-editing scenario [Abstract].

Method

The architecture has three components: (1) an autoregressive module that consumes interleaved text/visual context and emits conditional representations; (2) an MoE-based diffusion backbone (DiT-MoE) that iteratively denoises VAE latents conditioned on the AR outputs and text/visual features, using 128 experts with Top-8 routing for a 25B-total / 3B-active model; (3) a VAE encoder/decoder bridging pixels and latents. The same architecture handles understanding, image generation, video generation, and video editing — the AR module provides the unified conditioning channel and the MoE diffusion does the heavy lifting on the generation side.

For deployment-grade inference cost, the team applies a joint few-step distillation + reinforcement learning recipe (specific algorithm not in the abstract) that compresses the 30-step editing teacher into a 4-step student. The paper frames this as joint rather than the more common “distill first, RL the student later” pipeline.

Results

VBench 2.0: reported as top-tier among evaluated models. OpenVE-Bench: reported as a new record in video-editing quality, matching Kling O1 (a top proprietary model). Inference speedup: up to 95.9× faster video editing vs open-source baselines after the 30→4 step distillation+RL. Production: 98% success rate in ByteDance’s internal advertising video editing scenario. Headline numbers come from the abstract and Fig. 1; per-axis VBench 2.0 breakdowns and OpenVE-Bench numbers are in the body but were not retrievable at filing time.

Why it’s interesting

Mamoda2.5 sits in the AR+Diffusion branch of the UMM taxonomy (alongside BAGEL, OpenUni, BLIP-3-o, MetaQuery, DuoGen — see Unified Multimodal Models) but pushes the diffusion side to a fine-grained MoE: 128 experts / Top-8 routing inside the DiT itself, not just in the LLM. This is a different architectural bet from LightBagel: A Light-weighted, Double Fusion Framework for Unified Multimodal Understanding and Generation (light dual-fusion) and from the encoder-free MoT path of NEO-unify: Building Native Multimodal Unified Models End to End — same problem (unifying understanding + generation), explicitly different sparsity story. Pairing it with a joint distillation+RL step also connects to Distribution Matching Distillation Meets Reinforcement Learning, which is the closest filed paper articulating why combining distillation with RL during few-step compression matters; Mamoda2.5 is an applied data point in that direction for a video editor.