Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation

TL;DR

Sol Video Inference Engine is a training-free, agent-native framework for accelerating video diffusion inference. It organizes five acceleration levers — cross-step caching, sparse attention, token pruning, quantization, and kernel fusion — into a per-deployment tuning problem, then solves that problem with parallel skill agents (one per lever) whose local candidates are composed by an integrator agent under human quality feedback. On three architecturally distinct video DiTs (Cosmos3-Super 64B MoT, LTX-2.3 22B two-stage, SANA-Video 2B linear-attention), the same framework produces >2× end-to-end speedup on B200 while maintaining VBench quality — with dramatically less hand engineering than the per-deployment recipes that dominate this cluster.

Key claims

Video diffusion acceleration is an instance-specific tuning problem: the best recipe for one (model, hardware, serving configuration) triple often fails to transfer to another, because backbone architecture, attention concentration patterns, numerical sensitivity, spatial/temporal resolution, and hardware memory/precision characteristics all shift the dominant bottleneck [§3.2].
The framework organizes five widely applicable techniques — cross-step caching, sparse attention, token pruning, quantization, and kernel fusion — into a full stack, and treats each as a local search space rather than a fixed recipe [§4.2].
The workflow is three stages: parallel skill agents optimize each technique locally (independent search per lever), an integrator agent composes local candidates into a global stack, and a human validator provides visual-quality feedback because PSNR is not aligned with perceptual quality (Figure 6) [§4.1].
Local-then-global composition is necessary, not just convenient: the final end-to-end speedup is not the product of per-lever local gains, because many techniques are lossy and directly stacking locally optimal choices accumulates error to an unacceptable output [§4.1, “Agent as integrator”].
Cache selection is treated as instance-specific rather than as a single generalizable method — TeaCache, EasyCache, and TaylorSeer are all in the search space, with the cache agent tuning skip-step schedule, cached feature/residual choice, compensation rule, warmup, and max cached-step constraint [§4.2.1].
Sparse attention selection is layer-wise: the agent decides which layers can be sparsified safely, which backend (PISA, SpargeAttention, SVG, SVG2) fits the current model, and what sparsity pattern to use — layer selection is part of the tunable [§4.2.2].
Quantization is treated as selective per-layer local tuning rather than uniform low-bit conversion — layer sensitivity, tensor shapes, weight/activation bitwidths, and timestep-dependent scaling all enter the search [§4.2.4].
Kernel fusion is deliberately scheduled as a late-stage optimization on the partially accelerated stack, because the bottleneck operators shift after cache/sparsity/pruning are applied — typical fusion candidates are GEMM+GELU, GEMM+residual, and RoPE+norm [§4.2.5].
On B200 (Cosmos3-Super with sequence-parallel 4-GPU, LTX-2.3 and SANA-Video on 1 GPU), the full stack achieves 2.27× / 2.38× / 2.77× E2E speedup vs SGLang baselines (99.6 s → 43.9 s / 97.8 s → 41.0 s / 29.4 s → 10.6 s) [Table 1].
The framework generalizes across radically different backbones: 64B Mixture-of-Transformers (Cosmos3-Super), 22B two-stage generate-then-refine (LTX-2.3 with res_2s second-order sampler that breaks Euler-schedule cache heuristics), and 2B linear-attention (SANA-Video) — the same technique stack applies but concrete implementations differ per target [§5].
On Cosmos3-Super, the diffusion-cache agent selected a TeaCache-style policy that reduced latency 99.6 s → 52.4 s, with additional gains from sparse attention, quantization, and kernel fusion stacked on top (waterfall in Figure 4) [§5.2].
PSNR is not sufficient as an integration signal — visual outputs with shifted details can register large PSNR distances while remaining human-acceptable, and outputs with blur, temporal jitter, or degraded motion coherence register small PSNR distances while being unacceptable (Figure 6) — the human validator loop is load-bearing [§4.1].

Method

Sol Video Inference Engine treats video-DiT acceleration as an optimization problem indexed by three axes: model, hardware, and serving configuration. Given a concrete deployment target, five technique families are each equipped with a skill agent that searches its local hyperparameter space in parallel with the others (cache: policy + skip schedule + compensation; sparse attention: layer selection + backend + pattern; token pruning: criterion + ratio

layer schedule + reconstruction; quantization: per-layer precision
scaling; kernel fusion: which operator sequences to fuse, custom vs compiler-generated). The skill agents produce locally optimized candidates. An integrator agent then composes them via global search — accounting for lossy-technique interactions and implementation compatibility (e.g. quantization × kernel fusion constraints). A human validator inspects each integrated candidate on a fixed prompt set and returns pass/fail on visual quality; that feedback tunes how aggressive the next optimization round can be. The framework is evaluated on B200 with an SGLang serving stack, with Cosmos3-Super using 4-GPU sequence parallelism and LTX-2.3 / SANA-Video running single-GPU.

Results

Cosmos3-Super (64B, 4×B200 SP): 99.6 s (SGLang) → 43.9 s Sol Engine, 2.27× [Table 1]. Cache alone: 99.6 s → 52.4 s. Remaining ~1.2× from sparse attention + quantization + fusion.
LTX-2.3 (22B, 1×B200): 97.8 s → 41.0 s, 2.38× [Table 1]. Notable because the res_2s second-order sampler breaks Euler-schedule cache heuristics, so per-model cache tuning is essential.
SANA-Video (2B, 1×B200): 29.4 s → 10.6 s, 2.77× [Table 1]. Linear-attention backbone; sparse-attention gains vanish, so relative contribution shifts to quantization + fusion.
Quality is reported as “near-lossless VBench quality” preserved across all three deployments [§5.2, Abstract]; per-lever quality ablations are not tabulated in the paper body available at ingest.

Why it’s interesting

This is the first entry in Diffusion serving optimization that frames the recipe search itself as the deliverable rather than a concrete recipe. The standing challenge on that concept page has been that per-deployment tuning does not scale — Run FLUX.1-dev three times faster is one recipe on FLUX/H100, FastVideo: Create a 5s 1080p Video in 4.5s on a Single GPU is another on LTX-2/B200, TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times is another on Wan/RTX 5090, Neodragon: Mobile Video Generation using Diffusion Transformer is another on Pyramidal-Flow/Hexagon NPU — and none transfer. Sol Engine’s response is to make agents do the per-instance tuning: the same five-lever framework, differently instantiated per (model, hardware, config). It also crosses into AI-for-AI Research: alongside Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster (single-knob model training) and AlphaGo Moment for Model Architecture Discovery (architecture search), it stakes out the systems-optimization granularity — autoresearching not the model or the training recipe but the deployment stack. The 2.27–2.77× headline is modest next to TurboDiffusion’s 97–199× (which stacks step distillation + sparse attention + quantization at training time on Wan), but the comparison isn’t apples-to-apples: Sol Engine is training-free and per-model rather than pre-baked into merged weights.