Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation

TL;DR

Kandinsky Lab (AI-Forever / Sberbank) releases a three-lineup open-weights family for high-resolution image and 10-second video synthesis: Image Lite (6B params, T2I + editing), Video Lite (2B, T2V + I2V), and Video Pro (19B, T2V + I2V). All use a latent flow-matching DiT backbone with Qwen2.5-VL + CLIP text encoders and the HunyuanVideo 3D VAE; the heavy investment is on the release package — 19 checkpoints across pretrain / SFT / CFG-distilled / diffusion-distilled / no-CFG variants, 5s and 10s lengths, two resolutions (HD/SD), plus camera-control LoRAs, ComfyUI nodes, day-0 Diffusers support, and multi-attention-engine inference (SDPA / FA2 / FA3 / SageAttention / NABLA sparse for 10s). Kandinsky 5.0 Video Pro is ranked #1 open-source on LMArena’s text-to-video leaderboard as of December 2025.

Key claims

Three-lineup family covering image and video at two parameter scales: Image Lite (6B, T2I + editing), Video Lite (2B, T2V + I2V), Video Pro (19B, T2V + I2V), all generating up to 10-second video [§Abstract, §1].
Common backbone: latent flow-matching DiT with cross-attention to Qwen2.5-VL + CLIP text embeddings, HunyuanVideo 3D VAE for video latents [§Pipeline (Video Pro)].
Multi-stage training pipeline: large-scale pretrain → self-supervised fine-tuning (SFT) → CFG distillation → diffusion distillation (16-step variant) → RL-based post-training [§Abstract].
Lite Video distill checkpoints reach ~35s latency for a 5s clip on a single H100 at 16 NFE (vs. 139s SFT at 100 NFE) — ~4× speedup [Model Zoo table, Lite Video].
10-second video generation uses NABLA sparse attention rather than dense attention to stay tractable on a single H100 [§Optimized Inference / Attention engine].
Lite (2B) outperforms larger Wan 2.1 14B and Wan 2.2 A14B in side-by-side human evaluation on Movie Gen benchmark prompts [Side-by-Side evaluation, Lite Video].
Pro (19B) is positioned in side-by-side human evals against Veo 3, Veo 3 Fast, and Wan 2.2 A14B (T2V and I2V) [Side-by-Side evaluation, Video Pro].
Release ships 19 checkpoints + ComfyUI nodes + day-0 Diffusers integration + camera-control LoRAs (separate Lite and Pro LoRA collections) + LoRA training repo (kandinsky-5-lora-train) [Project Updates timeline].
Inference is engineered to fit on consumer-tier VRAM: full pipeline runs at 24 GB with offloading; Qwen text encoder ships an NF4 (bitsandbytes) quantization option [§Optimized Inference].
Russian-language and Russian-cultural prompts are an explicit data-curation axis — Image Lite and Lite Video both claim best open-source Russian concept understanding [§Kandinsky 5.0 Image Lite, §Kandinsky 5.0 Video Lite].
Kandinsky 5.0 Video Pro is ranked Top-1 open-source on LMArena’s text-to-video leaderboard as of 2025-12-12 [Project Updates timeline].

Method

The three lineups share a latent flow-matching diffusion-transformer backbone. Text conditioning is dual: Qwen2.5-VL (a multimodal LLM, used here as a heavyweight text encoder; an NF4-quantized variant is supported for low-VRAM inference) plus CLIP for shorter-range alignment. For video, the visual latent space is provided by the HunyuanVideo 3D VAE rather than a Kandinsky-trained tokenizer — a deliberate cost-saving choice that lets the team focus compute on the DiT. The DiT is the only trained generative module and conditions on text via cross-attention.

The training pipeline is the part most worth studying. The team explicitly factors quality enhancement into four sequential post-pretraining stages: SFT (self-supervised on a curated quality-filtered subset), CFG distillation (folds classifier-free guidance into the model so a single forward pass replaces the 2× cost), diffusion distillation (compresses to 16 steps), and RL-based post-training. The 8-variant Lite Video model zoo is the cross-product of {pretrain, SFT, no-CFG, distill-16-steps} × {5s, 10s} — every stage’s checkpoint is shipped.

For 10-second generation on a single H100, dense attention is replaced with the NABLA sparse-attention algorithm. The release also exposes a --attention_engine switch covering SDPA, FA2, FA3, and SageAttention; this is shipped as an explicit knob, not a compile-time choice. MagCache (an approximate step-caching method) is supported on SFT and no-CFG checkpoints for further latency reduction.

Results

Lite Video 2B SFT 5s: 139s on H100; distilled 5s: 35s (~4× speedup) [Model Zoo, Lite Video].
Lite Video 2B SFT 10s: 224s on H100; distilled 10s: 61s [Model Zoo, Lite Video].
Pro Video 19B SFT 5s SD: 560s on H100; HD: 1241s [Model Zoo, Video Pro].
Pro Video 19B SFT 10s SD: 1158s on H100 [Model Zoo, Video Pro].
Image Lite 6B T2I: 13s for a single 1280×768 image at 100 NFE on H100 [Model Zoo, Image Lite].
Side-by-side human evaluation on Movie Gen Bench expanded prompts: Lite Video beats Sora, Wan 2.1 14B, Wan 2.2 A14B, Wan 2.2 5B, and Wan 2.1 1.3B (full SBS plots) [Side-by-Side evaluation, Lite Video].
LMArena T2V leaderboard, 2025-12-12: Pro Video #1 among open-source models [Project Updates].
Distillation quality cost: distill-16-step Lite is compared head-to-head against SFT-100-step Lite in dedicated SBS plots at both 5s and 10s [Distill Side-by-Side, Lite Video].

Why it’s interesting

The headline shape of the release matches the recent open-weights playbook (NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry, LTX-2: Efficient Joint Audio-Visual Foundation Model, HunyuanImage 3.0 Technical Report) — ship enough variants, quantizations, and inference backends that adoption is friction-free. What’s distinctive here is the combination of (a) a complete image+video family from one team rather than a single modality, (b) the 8-variant Lite zoo deliberately exposing every post-pretrain stage as a separately-downloadable checkpoint, and (c) explicit Russian-language data curation — a localization axis that most of the other releases ignore. The 2B Lite beating 14B Wan in human evals is a notable counter-datapoint to the parameter-count-equals-quality assumption that papers like Seedance 1.0: Exploring the Boundaries of Video Generation Models and Waver: Wave Your Way to Lifelike Video Generation implicitly carry. The NABLA-for-10s decision also makes this concrete evidence for the kind of dense→sparse attention swap Radial Attention: O(n log n) Sparse Attention with Energy Decay for Long Video Generation and Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention argue is necessary at minute-scale.