Skip to content
Bud's Wiki

TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times

TL;DR

Section titled “TL;DR”

TurboDiffusion is a video-diffusion acceleration framework from Tsinghua/Shengshu/UC-Berkeley that stacks four orthogonal levers — SageAttention2++ low-bit attention, fine-tunable Sparse-Linear Attention (SLA), rCM step distillation, and W8A8 INT8 quantization of Linear layers — on a single pretrained Wan2.1/Wan2.2 video model and reports 97-199× end-to-end diffusion speedup on a single RTX 5090, reducing a Wan2.2-I2V-A14B-720P 5-second generation from 4549s to 38s and a Wan2.1-T2V-14B-720P generation from 4767s to 24s. Quality is reported as comparable to the unaccelerated baseline on the prompt set shown; the report is technique-stacking with reference implementations rather than a new algorithm.

Key claims

Section titled “Key claims”
  • TurboDiffusion achieves 97× speedup on Wan2.1-T2V-1.3B-480P (184s → 1.9s), 120× on Wan2.2-I2V-A14B-720P (4549s → 38s), 170× on Wan2.1-T2V-14B-480P (1676s → 9.9s), and 199× on Wan2.1-T2V-14B-720P (4767s → 24s), all on a single RTX 5090 [Fig. 3].
  • The waterfall decomposition on Wan2.1-T2V-14B-720P attributes the 199× to: CPU offload (baseline 4767s → 3182s OOM-recovery), +W8A8 & FusedNorm (3182s → 2783s, 1.14×), +rCM step distillation (2783s → 84s, 33.3×), +SageSLA fused attention (84s → 24s, 3.45×) [Fig. 4].
  • Attention is accelerated via SageAttention2++ — block-wise quantized attention with INT8/INT4 tensor cores [§1.1, ref 3].
  • Sparse-Linear Attention (SLA) is fine-tuned in place of full attention with a Top-K ratio of 0.1 (≈90% attention sparsity), and is composed with SageAttention into a single CUDA implementation called SageSLA [§1.1, §1.3, §2.1].
  • Step distillation uses rCM (large-scale score-regularized continuous-time consistency from Zheng et al.) reducing 100 sampling steps to 3-4 at inference; rCM is merged with the SLA-finetuned weights through parameter averaging so attention-level acceleration carries through [§1.1, §1.2, §1.3].
  • W8A8 Linear-layer quantization uses INT8 weights and activations at 128×128 block-wise granularity, compressing the model by ~half and exploiting INT8 Tensor Cores; LayerNorm and RMSNorm are reimplemented in Triton/CUDA [§1.1, §1.3].
  • Compared to FastVideo at default settings (3 sampling steps, 0.8 attention sparsity), TurboDiffusion is faster — Wan2.1-T2V-14B-720P: 24s vs FastVideo 72.6s; Wan2.1-T2V-14B-480P: 9.9s vs 26.3s; Wan2.1-T2V-1.3B-480P: 1.9s vs 5.3s [§2.2, Figs. 12-29].
  • Recommended deployment knobs: Top-K ∈ [0.1, 0.15] for attention sparsity and 4 sampling steps “to consistently achieve the best video quality” [§2.1].
  • Speedup is reported on RTX 5090 specifically and “substantially” but smaller on RTX 4090 and H100; latencies exclude text encoding and VAE decoding stages [§2.1, §2.2].
  • For Wan2.2-I2V-A14B-720P the measured 120× is below the 199× of Wan2.1-T2V-14B-720P because of switching overhead between the high-noise and low-noise expert; the authors claim theoretical speedup is identical [Fig. 3 caption].

Method

Section titled “Method”

The training recipe takes a pretrained video diffusion model and runs two finetunes in parallel: (a) replace full attention with SLA and finetune the model to adapt to sparsity, and (b) distill the same pretrained teacher into a few-step student via rCM. The parameter updates from both branches are then merged into a single set of weights, so the deployed model inherits both 90%-sparse attention and few-step sampling without a sequential stage that would compound the distillation gap.

At inference, attention is served by SageSLA — the SLA pattern fused into a SageAttention2++-style block-quantized CUDA kernel. The number of denoising steps is collapsed from 100 to 3-4. Linear-layer weights and activations are dynamically INT8-quantized at 128×128 block granularity, executed on INT8 Tensor Cores, and norm layers are replaced with hand-written Triton/CUDA kernels. The waterfall in Fig. 4 shows the gross-margin attribution: rCM is doing most of the arithmetic-reduction (33.3×), SageSLA is doing most of the per-step acceleration on top of it (3.45×), and W8A8+FusedNorm is the small constant-factor cleanup (1.14×).

Results

Section titled “Results”

End-to-end diffusion latency on a single RTX 5090, original vs TurboDiffusion vs FastVideo, for 5-second video [Fig. 3, Figs. 12-29]:

  • Wan2.1-T2V-1.3B-480P: 184s → 1.9s (TurboDiffusion); FastVideo 5.3s.
  • Wan2.1-T2V-14B-480P: 1676s → 9.9s; FastVideo 26.3s.
  • Wan2.1-T2V-14B-720P: 4767s → 24s; FastVideo 72.6s.
  • Wan2.2-I2V-A14B-720P: 4549s → 38s; FastVideo n/a (no accelerated variant published).

The Wan2.1-T2V-14B-720P waterfall attributes the 199× to: rCM 33.3× (step distillation, 2783s → 84s), SageSLA 3.45× (84s → 24s), W8A8 + FusedNorm 1.14× (3182s → 2783s), with CPU offload providing the remaining baseline-recovery [Fig. 4].

Quality is shown as qualitative side-by-side video grids across roughly a dozen prompts per model [Figs. 5-29]. No quantitative metric is reported — no VBench, no human preference rate, no distillation-fidelity score. The text describes outputs as “maintaining comparable video quality” and “demonstrating clear superiority to FastVideo” [§2.2, Abstract].

Why it’s interesting

Section titled “Why it’s interesting”

This is the maximal stack of the diffusion-acceleration frontier on the wiki applied to one production-scale video model. The waterfall is a useful artifact in its own right: it puts numbers on something the wiki’s Diffusion serving optimization cluster has been discussing structurally — step distillation buys an order of magnitude, IO-aware sparse attention buys another factor of a few, and W8A8 / norm fusion is small but free. It composes pieces that have been filed individually elsewhere: Attn-QAT: 4-Bit Attention With Quantization-Aware Training is the closest training-time sibling on the same Wan2.1-14B base (low-precision attention with fidelity preservation), FastWan: Generating a 5-Second Video in 5 Seconds via Sparse Distillation is the most direct serving-side comparator (also Wan-class, also sparse distillation, but trades 5s wallclock at 1080p for a TI2AV goal on B200), and Streaming Autoregressive Video Generation via Diagonal Distillation is the most direct distillation-side comparator (also Wan2.1, also ≤4 NFE, but going after AR streaming rather than full-clip generation). Contrasts with Neodragon: Mobile Video Generation using Diffusion Transformer, which is the same recipe pattern (text-encoder distill + decoder swap + MMDiT pruning + step distill) but retargeted to a phone NPU instead of an RTX 5090. The report is light on quantitative quality evaluation — no VBench, no human study, no per-component quality ablation — which is the natural follow-up question for anyone considering deploying this rather than reading it.

See also

Section titled “See also”