FastVideo: Create a 5s 1080p Video in 4.5s on a Single GPU

TL;DR

The Hao AI Lab FastVideo team reports a production-grade text-image-to-audio-video (TI2AV) inference stack that turns the open-source LTX-2.3 model into a single-GPU 1080p real-time generator: ~4.55 s end-to-end latency for a 5 s, 1088×1920, 24 FPS clip with audio on a single NVIDIA B200, claimed as a 3.9× speedup over the next-fastest option and the first sub-5-second 1080p T2AV pipeline. The recipe is system-level rather than algorithmic — NVFP4-quantized linear layers, SM100/SM103-tuned attention kernels, end-to-end graph and kernel fusion across the prompt-encode / latent-prep / denoise / decode pipeline, plus optimized ffmpeg for the surrounding I/O. A public demo is served on half of a GB200 NVL72 (36×GPUs with Rust middleware), each GPU acting as a load-balanced replica.

Key claims

End-to-end latency: ~4.55 s for 5 s × 1088×1920 × 24 FPS TI2AV (LTX-2.3 base) on a single B200 GPU [§“FastVideo’s New Real-Time Inference Stack”; §“Outlook and Conclusion”].
Reported 3.9× end-to-end speedup relative to the next-fastest 1080p option [§“FastVideo’s New Real-Time Inference Stack”].
Prior real-time video work tops out at 480p or 720p and typically lacks audio; 1080p is identified as the regime where attention cost, memory pressure, and pipeline overhead all become bottlenecks simultaneously [§3].
For comparison, Google’s Veo-3 Fast is reported by the post at ~55 s for an 8 s clip — used as the motivating “too slow for ideation” baseline rather than a head-to-head benchmark [§2].
Attention kernels are specifically optimized for Blackwell SM100/SM103 (B200/B300) to handle 3D spatiotemporal attention at full-HD token counts [§“Fast attention for video generation”].
NVFP4-quantized linear layers in the DiT are used to exploit Blackwell’s native FP4 tensor cores while preserving output quality; framed as essential for fully utilizing B200/B300 throughput [§“Aggressive low-precision execution with NVFP4”].
Kernel fusion is applied across all major stages — prompt encoding, latent preparation, denoising, decoding — on the basis that real latency is dominated by everything around the denoising loop, not the loop alone [§“End-to-End graph and kernel optimization”].
IPC overhead and the surrounding frame/audio I/O pipeline were identified via FastVideo’s profiling tooling as a non-trivial latency contributor, and addressed with a CPU-specific optimized ffmpeg build [§“System-Level efficiency beyond the model”].
The post argues that running 1080p generation on a single GPU eliminates the need for sequence parallelism or multi-GPU serving strategies, which they treat as standard for fast video inference today [§“Outlook and Conclusion”].
Demo serving: half a GB200 NVL72 (36 GPUs) with one serving replica per GPU and Rust middleware for load balancing [§“Serving Optimizations”].
NVIDIA Dynamo recently added FastVideo as a supported diffusion backend [§“Acknowledgement”].

Method

The pipeline takes LTX-2.3 — Lightricks’ open-source TI2AV model in the LTX-2 family (LTX-2: Efficient Joint Audio-Visual Foundation Model) — and applies a four-layer optimization stack tuned for Blackwell-class GPUs.

(1) Attention kernels are rewritten for SM100/SM103 to handle 3D spatiotemporal attention at 1080p latent token counts, which the post identifies as the dominant FLOP consumer in modern video DiTs. The team references their earlier sparse-attention work (arxiv 2505.13389) as part of the FastVideo lineage adopted by frontier labs.

(2) NVFP4-quantized linear layers in the DiT trunk exploit Blackwell’s native FP4 tensor-core support; the post claims output quality is preserved while throughput rises dramatically vs BF16.

(3) Graph- and kernel-level fusion is applied end-to-end across prompt encoding, latent preparation, denoising, and decoding — treating the entire inference path as a single systems problem rather than just optimizing the denoise loop.

(4) System I/O around the model — IPC, frame and audio processing, storage — is profiled with FastVideo’s instrumentation and reduced via a CPU-optimized ffmpeg build. The post explicitly avoids algorithmic step reduction (distillation, few-step) as the lever; the speedup story is purely system-level.

For serving, half a GB200 NVL72 (36 GPUs) acts as 36 single-GPU replicas behind a Rust middleware load balancer rather than as a single distributed-serving cluster.

Results

5 s × 1088×1920 × 24 FPS TI2AV clip with audio: ~4.55 s end-to-end on a single B200 [§“FastVideo’s New Real-Time Inference Stack”].
Single-GPU footprint — no sequence parallelism or multi-GPU sharding required for 1080p [§“Outlook and Conclusion”].
3.9× reduction in end-to-end latency vs the next-fastest published 1080p option [§“FastVideo’s New Real-Time Inference Stack”]; specific competitor not named in the post.
No quantitative quality comparison against base LTX-2.3 is reported in the post itself; the claim is “preserving output quality” qualitatively.

Why it’s interesting

This is the first wiki entry to push a 1080p T2AV system into sub-5-second end-to-end latency on a single GPU, and it does so by leaving the underlying model — LTX-2.3 — algorithmically untouched. That cleanly factors the “real-time video” problem into two orthogonal levers: algorithmic (distillation, few-step, AR rewiring) and systems (kernels, precision, fusion, I/O). Helios: Real Real-Time Long Video Generation Model takes the algorithmic route — Adversarial Hierarchical Distillation, Pyramid UniPC, Multi-Term Memory Patchification to compress a 14B model to 19.5 FPS on an H100 — and reports an 80% larger model running on weaker hardware. FastVideo takes the systems route on Blackwell and gets a fundamentally different operating point: full-HD with audio, 5 s clips, no distillation. The two are stackable in principle, and the comparison is interesting precisely because neither paper combines both. The companion data point on the systems lever for image generation is Run FLUX.1-dev three times faster (Torch compile + max-autotune + fused QKV + channels-last → 3× FLUX speedup); FastVideo is the video-with-audio analog and the most aggressive single-GPU 1080p result the wiki has seen.