FastVideo FastH3 V1 — 4-Step Sparse Distilled MiniMax H3 for 14x Blackwell Speedup

TL;DR

FastH3 Preview v1 is a 4-step sparse-distilled post-train of MiniMax H3 (33B dense single-stream Omni-Transformer for joint text-to-video-and-audio) by the Hao AI Lab FastVideo team, released Aug 27 2026 with open weights + LoRA + inference code. Method is distillation-first (not architectural change): the recommended checkpoint (4-step VSA / Data-Free) trains from prompts without target videos, requires FastVideo’s Video Sparse Attention (VSA-H3) kernel backend, and drops end-to-end 15s / 768p generation on 1× B200 from ~678.7 s to under 13 s (14× headline speedup, sub-realtime on 8× B200). Training used 1k+ B200 hours on multi-shot audio-visual-synced input distribution. Native support for variable resolution, aspect ratio, and duration in a single checkpoint (T2VA only for Preview v1; FL2VA and Ref2VA in development). Positioned as the FastWan/FastWan-QAD lineage extended to the H3 frontier, and is the direct baseline that openvdn’s VideoDeltaNet (VideoDeltaNet on MiniMax H3 — Near-Lossless Hybrid Attention for Video Models, filed same day) claims to beat on quality with a hybrid-attention retrofit instead of distillation.

Key claims

FastH3 Preview v1 generates 15s / 768p video with joint audio in under 13 s on 8×B200 (sub-realtime); up to 14× speedup vs Base H3 dense on a single Blackwell GPU [§Speed].
The recommended checkpoint is 4-step VSA / Data-Free — data-free means trained from prompts without target videos [§FastH3 Preview v1].
Native support for variable resolution, aspect ratio, and duration (square, portrait, landscape, ultrawide 768p; height/width must be multiples of 32) in a single checkpoint [§FastH3 Preview v1].
The recommended checkpoint requires FastVideo’s Video Sparse Attention (VSA-H3) backend + kernel to achieve reported speed and quality; dense attention is not a drop-in substitute [§FastH3 Preview v1].
Training used 1k+ B200 training hours on real-world multi-shot, visual-audio-synced input distribution [§Quality].
Distillation reuses H3-Base’s text encoder, video VAE, audio VAE, tokenizers, and schedulers unchanged; only the base transformer is distilled [§FastH3 Preview v1].
Preview v1 ships full weights + pre-extracted LoRA for the recommended VSA / Data-Free checkpoint, plus dense and synthetic-data ablation checkpoints [§FastH3 Preview v1, §Openness].
Preview v1 currently supports T2VA only — FL2VA (first/last-frame conditioning) and Ref2VA (reference transformer path) checkpoints are in development [§FastH3 Preview v1].
Roadmap: image-ref (FL2VA) and full omni-ref (Ref2VA) in the coming weeks; motion and quality improvements; NVFP4 and GPU memory reduction; RTX / DGX Sparks / Apple MLX targeting; new runs using NVIDIA FastGen’s Parallel Decoding Distillation (PDD) [§What’s Next].
End-to-end 1× B200 latency at 1344×768 / 24 FPS with audio (median of 3 timed requests after warmup): Base H3 Dense FA4 = 132.5 s (5s), 377.4 s (10s), 678.7 s (15s); FastH3 achieves 14× speedup at 15s [§Performance, table].
FastH3 is positioned as continuing the “FastWan sparse distillation” + “FastWan-QAD” lineage — same pattern of releasing checkpoints paired with FastVideo training + inference stacks [§Why open H3 matters].

Method

Post-training pipeline distills MiniMax H3’s base transformer via a 4-step data-free sparse distillation recipe. The distillation reuses H3-Base’s frozen text encoder, video VAE, audio VAE, tokenizers, and schedulers — only the DiT is modified. VSA-H3 (Video Sparse Attention) is the inference-time attention kernel that the distilled weights are calibrated for. The recommended checkpoint is trained without target videos (“Data-Free”), using prompts only.

Ablation checkpoints (VSA-Synthetic-Step1300, VSA-Synthetic-Step1900, Dense-DataFree) are shipped alongside for comparison, each with matching pre-extracted LoRAs. All Preview v1 checkpoints are 4-step.

Serving is via FastVideo’s optimized inference stack. Performance measurements at 1344×768 / 24 FPS with generated stereo audio use the median of 3 timed requests after one warmup, excluding model load / compile time; end-to-end time includes prompt encode, denoise, decode, audio, muxing, and file output.

Collaboration: Nuva Lab; NVIDIA FastGen (Julius Berner, Chao Liu, Arash Vahdat); NVIDIA Enterprise Products (Pengcheng Li, Cliff Woolley).

Results

Base H3 Dense FA4 end-to-end on 1× B200: 132.5 s (5s video), 377.4 s (10s), 678.7 s (15s). Base H3 on 4× B200: 40.6 s (5s), 108.7 s (10s), 193.1 s (15s).

FastH3 headline: up to 14× speedup over Base H3 on 1× B200; sub-realtime (<13s for 15s video) on 8× B200.

Fifteen qualitative reference videos with generated stereo audio and detailed prompts are provided (multi-shot Kazakh dance, dragon flight, contemporary theater, storybook animation, three-designer motion-graphics, etc.). No quantitative video-generation benchmarks (VBench, MOVA Arena, JavisBench, Verse-Bench) are reported. No numeric quality delta vs Base H3.

Why it’s interesting

First open post-training accelerator on the H3 frontier open joint-AV base. MiniMax H3 (MiniMax H3 — Open-Weights Omni-Reference Video-Audio Generation Model Card) was the first frontier open-weights joint text-to-video-and-audio model at 33B dense; FastH3 is the first filed community post-train, arriving less than a month after the H3 open release. Extends the FastWan → FastWan-QAD lineage the FastVideo team has been building on Alibaba’s Wan family to a new base model and modality (joint AV rather than V-only).
Distillation-first counterpart to VideoDeltaNet on the same base model. VideoDeltaNet on MiniMax H3 — Near-Lossless Hybrid Attention for Video Models (filed together in the same Slack post) takes the opposite retrofit route — architecture change (hybrid attention with Video Delta Attention) with more conservative 8-step distillation — and explicitly claims better quality than FastH3 on the same H3 base. This makes FastH3 the first quality-baseline datapoint against which architecture-first retrofits of open joint-AV models are measured. Both papers land within one week of each other on the same frontier base model, defining the retrofit design axis (distillation-only vs distillation+architecture) for open joint-AV.
Data-free 4-step VSA distillation is more aggressive than the Diffusion Distillation canon has documented so far for joint-AV. The wiki’s page lists step-distillation for Wan / Krea / Qwen-Image but no 4-step joint-audio-video with sparse-attention constraint prior to this. Whether the audio branch survives the distillation with quality intact is not quantitatively answered by the release — this is the open question flagged in the Joint audio-video generation cluster about how much of the audio-visual sync a 4-step student can preserve.
Ships with the FastVideo training stack itself, not just checkpoints. Distinct from most video-DiT accelerator releases — the training recipe + inference stack are the deliverable, matching the Diffusion serving optimization frontier trend of framework-level rather than per-deployment recipe consolidation.