Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification

TL;DR

Sol-Attn is a training-free dynamic sparse-attention kernel for video diffusion transformers that unifies routing, sparse computation, and approximate correction in a single online-softmax pass. Instead of materializing a proxy-score map and then top-K or top-p selecting blocks (the SpargeAttention / XAttention / SVG2 pattern), Sol-Attn thresholds block proxy scores on-the-fly during the online-softmax sweep, letting the per-query budget flex with content while avoiding the cost of building the proxy map at all. Unselected blocks are not discarded — their proxy scores are reused as an approximate correction in the same pass. End-to-end on B200 the kernel reports 2.02× (Wan 2.1-14B), 2.12× (HunyuanVideo-13B), and up to 2.4× (LTX 2.3) over dense FlashAttention-3; composed inside Sol-Engine (with kernel fusion and caching) the numbers rise to 3.48× and 5.08×.

Key claims

Existing training-free dynamic sparse-attention methods for video DiTs face two structural problems: rigid, unpredictable, and costly routing (fixed top-K yields imbalanced quality; top-p over a full proxy map yields dynamic but imbalanced compute), and lossy keep-or-drop sparsification that discards unselected blocks entirely, degrading accuracy under aggressive sparsity [Abstract].
Sol-Attn’s core mechanism is on-the-fly block thresholding with proxy-score reuse: proxy scores are compared against a threshold during the online softmax sweep itself, selecting critical blocks without materializing the proxy map [Abstract].
The same online-softmax pass reuses proxy scores of unselected blocks as an approximate correction rather than dropping them, tightening the accuracy–efficiency trade-off relative to keep-or-drop schemes [Abstract].
End-to-end vs dense FlashAttention-3: 2.02× on Wan 2.1-14B, 2.12× on HunyuanVideo-13B, up to 2.4× on LTX 2.3 [tweet, headline].
Composed with kernel fusion and caching inside Sol-Engine: 3.48× on Wan 2.1-14B and 5.08× on HunyuanVideo-13B [tweet, headline].
The paper reports averaged 2.1× (video generation) and 2.3× (video editing) end-to-end speedups while preserving visual quality across image and video benchmarks [Abstract].
The B200 kernel is still under further optimization; the code lives on the sol-engine branch of NVlabs/Sana alongside the Sol-Engine framework [tweet].

Method

Sol-Attn keeps the outer online-softmax structure of FlashAttention (tile queries in SMEM, stream KV tiles, maintain running max and denominator) but replaces the “materialize proxy map → top-K/top-p → gather selected blocks → recompute exact softmax” pattern used by SpargeAttention, XAttention, and SVG2 with a fused decision-and-compute step. For each KV block, a proxy score (typically a cheap low-precision or centroid-based estimate of the block’s contribution to the current query tile’s softmax) is compared against a threshold; blocks that pass compute exact attention and update the running statistics, while blocks that fail do not: their proxy score is folded back into the denominator as an approximate correction so the softmax normalization does not drift. The threshold — not a fixed K or p — is the user-facing knob, which converts the per-query budget from an imposed constraint into a data-dependent outcome. Because proxy scores are never assembled into a full N×N (or block-N × block-N) map, the overhead that dominates the “cheap dynamic routing” claim of prior methods disappears; because unselected blocks are corrected rather than dropped, aggressive sparsity settings degrade more gracefully. The kernel targets B200 and pairs with the Sol-Engine integrator agent as its “sparse attention” skill lever.

Results

Wan 2.1-14B: 2.02× end-to-end vs dense FlashAttention-3; 3.48× inside Sol-Engine [tweet].
HunyuanVideo-13B: 2.12× end-to-end vs dense FlashAttention-3; 5.08× inside Sol-Engine [tweet].
LTX 2.3: up to 2.4× end-to-end vs dense FlashAttention-3 [tweet].
Aggregate: 2.1× (video generation) / 2.3× (video editing) end-to-end speedup while preserving visual quality [Abstract].

Note: quantitative visual-quality metrics (PSNR / VBench / human preference) are not disclosed in the abstract or launch tweet available at filing; the paper’s Tables presumably carry them but were not visible from the abs page.

Why it’s interesting

Sol-Attn is the sparse-attention lever of Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation made explicit — same authors (NVIDIA + MIT HAN Lab: Haopeng Li, Yitong Li, Junsong Chen, Enze Xie, Song Han et al.), same target hardware (B200), same integration point (NVlabs/Sana sol-engine branch, cf. SANA — Efficient High-Resolution Image and Video Generation family (NVlabs/Sana, sol-engine branch)). It occupies a specific position on the Diffusion serving optimization frontier: where Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation gets its speedup by reshuffling memory (k-means permutation to recover contiguous blocks) and Radial Attention: O(n log n) Sparse Attention with Energy Decay for Long Video Generation gets its speedup by fixing the mask up-front (static energy-decay radial pattern), Sol-Attn’s contribution is never materializing the proxy map at all — the routing decision is folded into the same online-softmax pass as the compute. That is a third design point on the IO-Aware Kernel Design template: FlashAttention’s contribution was softmax-over-blocks with running statistics; SVG2 added tile-shape variability as a first-class concern; Sol-Attn adds proxy-score reuse-in-place. The headline 2.02–2.4× is in the same ballpark as SVG2’s H100 numbers (2.30× on HunyuanVideo, 1.58× on Wan 2.1) but on a different attention baseline (FA3, not FA2) and a different GPU generation (B200, not H100). The 5.08× inside Sol-Engine matches the Sol-Engine paper’s story that sparse attention is one lever among five and needs the framework to compound.