Paris 2.0: A Decentralized Diffusion Model for Video Generation

TL;DR

Paris 2.0 extends the Paris: A Decentralized Trained Open-Weight Diffusion Model Decentralized Diffusion Model (DDM) recipe from images to text-to-video — three 11B Flux MM-DiT expert diffusion models are trained in complete isolation (no gradient sync, no parameter sharing, no activation exchange), then a lightweight router picks experts at each denoising step. In a low-resolution T2V study at matched total compute, the decentralized pool cuts FVD from 561.04 → 279.01 (~2× improvement) over a monolithic baseline trained on the same data, and lifts CLIP text-video similarity and aesthetic score. Multi-stage checkpoints are released at 256×256 and 768×768 with MIT license, decoded through Tencent’s HunyuanVideo VAE; the release is the first decentralized-trained open-weights video model.

Key claims

Three 11B Flux MM-DiT experts are trained without gradient synchronization, parameter sharing, or activation exchange, then composed via a lightweight transformer router selecting top-K experts at each denoising step [Architecture Details table; Inference Pipeline].
At matched total compute on the same training data, the decentralized pool cuts FVD from 561.04 (monolithic baseline) to 279.01, a ~2× improvement [Results table].
CLIP text-video similarity improves from 0.2032 ± 0.0011 (monolithic) to 0.2178 ± 0.0012, and aesthetic score from 3.7950 ± 0.0077 to 3.9036 ± 0.0082, both at p-level standard errors that don’t overlap [Results table].
Generated samples report higher mean motion (0.712 ± 0.057 vs 0.555 ± 0.043 px/frame), which the paper flags as descriptive (no preferred direction) rather than a quality claim [Results table].
The pipeline decodes routed-velocity latents into pixels via Tencent’s HunyuanVideo VAE (4× temporal, 8× spatial), with T5 + CLIP ViT-L/14 text conditioning — all three external components are unbundled and fetched from upstream under their own licenses [Required External Components, Architecture Details table].
Two-stage training delivers 256×256 latents (32×32 per frame) at stage 2 and 768×768 (96×96 per frame) at stage 3, both checkpoints released per expert in both unwrapped single-file and sharded formats [Architecture Details table, What This Repository Contains].
The release supports both text-to-video and image-to-video generation; access is gated behind manual review by Bagel Labs, with research/evaluation use granted under MIT and commercial use requiring written agreement [Access page; License].
Paris 2.0 explicitly closes the open problem flagged by Paris 1.0: temporally coherent video generation under fully decentralized training [arXiv abstract].

Method

Three independent 11B Flux MM-DiT expert diffusion models are trained on disjoint partitions of the training data, with each expert seeing only its partition’s data, optimizing its own flow-matching objective, and never exchanging gradients, parameters, or activations with any other expert during training. After training, a lightweight transformer router is trained to score the current noisy latent and select the top-K experts at each step of the reverse diffusion process; the routed velocity prediction is then decoded into pixel-space video frames through the frozen HunyuanVideo VAE (4× temporal compression, 8× spatial). Training proceeds in two stages: stage 2 at 256×256 video resolution (32×32 latent per frame) and stage 3 at 768×768 (96×96 latent per frame). Each expert ships both unwrapped single-file (master.safetensors) and sharded (model/) checkpoints for compatibility with different inference frameworks, and the inference loop wires T5 + CLIP ViT-L/14 text conditioning into the routed MM-DiT velocity field before HunyuanVAE decoding.

Results

FVD: 279.01 (Paris 2.0 DDM) vs 561.04 (monolithic baseline) — ~2× distance reduction at matched total compute [Results table].
CLIP text-video similarity: 0.2178 ± 0.0012 vs 0.2032 ± 0.0011 [Results table].
Aesthetic score: 3.9036 ± 0.0082 vs 3.7950 ± 0.0077 [Results table].
Motion (px/frame): 0.712 ± 0.057 vs 0.555 ± 0.043; the paper labels this descriptive only [Results table].
All comparisons are against a monolithic model trained on the same data under a matched total compute budget; the headline study is the low-resolution text-to-video regime [arXiv abstract; Results].

Why it’s interesting

Paris 2.0 is the cleanest extension of Paris: A Decentralized Trained Open-Weight Diffusion Model‘s zero-communication training recipe to a new modality, and the first filed paper that demonstrates fully decentralized training works for temporally coherent video — not just images. The architectural choice is also significant: where Paris 1.0 used 8 DiT-XL/2 experts (605M each, 4.84B total) at image scale, Paris 2.0 jumps to 3 Flux MM-DiT experts at 11B each (33B total in the pool) with the HunyuanVideo VAE as the pixel decoder. That puts it alongside VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo and Scaling and Optimizing Frontier Model Training (Fireworks AI) in the late-2025/early-2026 Distributed training parallelism landscape, but at the opposite endpoint of the synchronization axis — VeOmni and Fireworks reduce comm overhead via Async-Ulysses and 4D parallelism but still synchronize; Paris 2.0 does not synchronize at all.

The router-over-independently-trained-experts pattern is also a distinct point in MoE Routing Design from joint-training MoE: standard MoE (LongCat-Flash, MiMo-V2-Flash, Qwen3-Next) trains routers and experts together with auxiliary load-balancing losses; Paris 2.0 trains experts in complete isolation and the router post-hoc. As a video-diffusion analog of that pattern at 11B-per-expert scale, this is the first filed system to claim it scales — and the FVD halving against a same-data, same-compute monolithic baseline is the empirical evidence cited.