Waver: Wave Your Way to Lifelike Video Generation

TL;DR

Waver is ByteDance Foundation Vision’s 12B-parameter rectified-flow DiT for unified text-to-video, image-to-video, and text-to-image generation at native 720p with a cascade refiner to 1080p, durations of 5–10 s, and an explicit goal of ranking with the proprietary frontier. It uses a Hybrid Stream architecture — the first M=16 layers are MMDiT-style Dual Stream, the next N=40 layers are Single Stream — over Wan-VAE latents with a dual flan-t5-xxl + Qwen2.5-32B-Instruct text encoder pair, and unifies T2I/T2V/I2V via a per-frame binary task mask concatenated to the input channel. Trained on >200M curated video clips across a 256p→1024p T2I curriculum, 192p→480p→720p T+I2V curriculum (sigma-shift schedule per stage), and a 1080p Refine+SFT phase with window-attention super-resolution. Reports Top-3 placement on the Artificial Analysis T2V and I2V leaderboards as of 2025-07-30, outperforming Kling 2.0 and Wan 2.1 on internal benchmarks and matching/exceeding Veo 3 on motion and visual quality.

Key claims

Hybrid Stream — 16 Dual-Stream blocks (per-modality params, joint self-attention) followed by 40 Single-Stream blocks (shared params on concatenated tokens) — converges faster than either pure design at the 12B scale [§2.1, Fig. 5].
Task unification across T2I / T2V / I2V is achieved purely via input-channel concatenation of a noisy video latent (16ch), a conditional-frame latent (16ch, zero-filled for non-conditioning frames), and a per-frame binary task mask (4ch), giving a 36-channel input dimension with no head-level changes [§2.1, Tab. 1].
Hybrid positional encoding = 3D RoPE (temporal/H/W relative position) plus a factorized learnable absolute position embedding summed across the three axes; the absolute component is reported to accelerate convergence on top of the relative component [§2.1].
Cascade Refiner takes a 720p generation, adds pixel degradation (downsample to 360p and upscale back to 1080p) + latent degradation (additive noise on the VAE latent at random weight) and learns the inverse map via flow matching, reducing 1080p inference cost by ~40% vs direct 1080p generation [§2.2, §4.1].
The Refiner uses window attention (alternating 2×2×1 spatial and 1×1×2 temporal windows) for middle layers while retaining full attention in the first and last eight layers, balancing efficiency and quality [§2.2].
Dual text encoders (flan-t5-xxl + Qwen2.5-32B-Instruct) demonstrably improve T2I prompt following over the single-encoder flan-t5-xxl baseline [§2, Fig. 4].
Training pipeline filters ~200M video clips through hierarchical stages (192p pre-training keeps the broadest semantics; 480p adds an 8-D MLLM quality model; 720p expands to a 12-D quality model; 720p CT adds source/semantic balancing; SFT adds synthetic and manually-selected complex-motion data) [§3.5, Tab. 2].
Quality model is a VideoLLaMA3 SFT trained on >1M manually-labeled clips across 13 low-quality dimensions; the model’s “high quality” predictions show 78% manual-validation accuracy [§3.2].
Caption model is Qwen2.5-VL co-trained on caption + sub-action-timestamp tasks with DPO/DPOP for long-form stability; downstream effect is fine-grained motion adherence [§3.3].
Training uses lognorm(0.5, 1) timestep sampling for T2I and mode(1.29) for T2V/I2V; the mode sampling is reported to increase motion amplitude in generated videos [§4.1, github README §3].
Inference uses APG (Adaptive Projected Guidance) extended to video — decomposing the CFG update into parallel + orthogonal components with normalization threshold 27 and guidance scale 8 — to reduce artifacts and oversaturation [github README §5, citing arXiv:2410.02416].
Representation alignment via Qwen2.5-VL video features, applied REPA-style during pre-training, accelerates convergence of the video DiT [§4.2].
On internal Waver-bench 1.0 (304 prompts) Waver outperforms Kling 2.0 and Wan 2.1 on motion quality, visual quality, and prompt following; on the Hermes Motion Testset (96 sports-heavy prompts) it improves over Veo 3, Kling 2.0, and Wan 2.1 on motion quality and prompt following [§1, §5].
Public Artificial Analysis Arena: Top 3 in both T2V and I2V tracks as of 2025-07-30 10:00 GMT+8 [§1].

Method

Mechanically, Waver is a 12B rectified-flow DiT over Wan2.1-VAE latents. The Task-Unified DiT takes a 36-channel input — primary noisy video latent (16ch) ⊕ conditional-frame latent (16ch, zero outside conditioning frames) ⊕ binary task mask (4ch) — through 16 Dual-Stream blocks (separate text/video AdaLN/QKV/MLP, joint self-attention) and 40 Single-Stream blocks (shared params over concatenated tokens), with 24 heads × 128 head-dim. Text is conditioned via a dual encoder pair (flan-t5-xxl + Qwen2.5-32B-Instruct) and positional information is supplied by 3D RoPE + factorized learnable absolute embeddings summed across temporal/H/W.

Training is a multi-stage curriculum: T2I-only at 256p→512p→1024p, then T2V at 192p (12 fps→16 fps), then joint T2V+I2V at 480p (16 fps, 20% probability of I2V conditioning) and 720p (24 fps), with a Continue Training and SFT pass at 720p. The 1080p Refiner is trained from the 720p CT checkpoint via window-attention super-resolution with pixel + latent degradation. Sigma-shift in the rectified-flow noise schedule increases with resolution (1.0 / 2.0 / 3.0 for T2I; 1.5 / 2.5 / 3.0 / 4.5 for T+V tasks across resolutions). T2V uses mode(1.29) timestep sampling vs lognorm(0.5, 1) for T2I to push more probability into mid-noise regions associated with motion synthesis. Representation alignment via Qwen2.5-VL video features is applied during pre-training to accelerate convergence.

Results

Top 3 on Artificial Analysis T2V and I2V leaderboards (as of 2025-07-30) [§1].
Internal Waver-bench 1.0 (304 prompts spanning sports, daily activities, landscapes, animals, machinery, surreal scenes, animations): wins on motion / visual / prompt-following vs Kling 2.0 and Wan 2.1; wins on motion + visual vs Veo 3, slightly behind Veo 3 on prompt following [§1, §5].
Hermes Motion Testset (96 prompts, 32 sports types): improvements over Veo 3, Kling 2.0, Wan 2.1 on motion quality and prompt following [§1, §5].
Cascade Refiner reduces 1080p inference time by ~40% (720p→1080p) and ~60% (480p→1080p) vs direct 1080p generation [github README §2].
Trained on >200M video clips total across the multi-stage curriculum [§1, §3].

Why it’s interesting

Waver is a fourth independent open data-point for the Hybrid Dual→Single-Stream recipe that the wiki’s Dual-stream diffusion transformer page has been tracking — and the first one at the 12B video-foundation scale (vs LTX-2/MOVA/SkyReels-V4, which are all joint-audio-video). The fact that ByteDance Foundation Vision converged on the same M-then-N layout that SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model adopted for audio-video, while also explicitly framing it as a convergence-acceleration trick rather than a parameter-efficiency one, sharpens the page’s “hybrid layout converges faster than either pure approach” claim. The release also resolves a slice of the open question on Seedance 2.0: Advancing Video Generation for World Complexity: Waver predates Seedance 2.0 by ~6 months from the same group and confirms ByteDance Seed is shipping the dual-stream-with-hybrid-collapse recipe as its house style, not a one-off. Separately, the cascade-refiner-with-flow-matching-and-latent-degradation recipe is a useful counterpoint to FastVideo: Create a 5s 1080p Video in 4.5s on a Single GPU‘s distillation-only approach to fast 1080p — Waver buys 40-60% of the same inference-time savings without distillation, by paying for it in a second model with window attention.