PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference

TL;DR

PackForcing is an autoregressive long-video diffusion framework built on Wan2.1-T2V-1.3B that bounds the KV cache to a constant size by partitioning history into three functional regions — sink tokens (full resolution, never evicted), compressed mid tokens (16× spatiotemporal compression via a dual-branch HR/LR module with dynamic top-k selection), and recent tokens (full resolution). A learned dual-branch compressor (progressive 3D conv + pixel-space re-encoding) replaces the FIFO eviction used by prior streaming methods, and an incremental temporal-only RoPE adjustment fixes positional discontinuities when old compressed blocks are dropped. Trained on 5-second clips, it generates coherent 2-minute videos at 16 FPS on a single H200, hitting VBench Dynamic Degree 56.25 (60s) / 54.12 (120s) — the best in class — and is the only baseline whose CLIP-T drop across 60s is under 1.2 points.

Key claims

The autoregressive long-video bottleneck is a forced trade-off between error accumulation (which needs more context) and unbounded KV-cache growth (which forces context to be less); prior fixes — history truncation, sliding windows, attention sinks — all sacrifice long-range coherence [§1].
A three-partition KV cache (sink + compressed mid + recent) bounds per-layer attention context to a constant N_sink + k·N_comp + N_recent tokens regardless of video length T [§3.2, Eq. 7].
Sink tokens — the first two generation blocks kept at full resolution and never evicted — anchor global semantics; they consume only ~0.5% of the token budget over a 2-minute video [§3.2].
A dual-branch compression module (HR: cascaded strided 3D convolutions in latent space; LR: VAE decode → 4× temporal / 2× spatial pooling → VAE re-encode → patch embed) achieves 16× spatiotemporal token reduction with information that single-pathway aggressive pooling destroys [§3.3, Eq. 8, Fig. 2b].
Dynamic Context Selection retrieves only the top-k most-relevant compressed mid blocks per generation step via query-key affinity; unselected tokens remain archived (non-destructive soft-routing), yielding +0.8 subject consistency and +0.12 CLIP over strict FIFO eviction [§3.5, Table 5].
Affinity scoring is computed once per block (at the first denoising step) and on a sub-sampled set of queries and half the heads, holding selection overhead to <1% [§3.5].
An incremental temporal-only RoPE adjustment fixes the position gap that appears when evicted mid blocks leave the sink at temporal index 0..t₀ while the earliest surviving mid token sits at t₀ + Δ; multiplicative RoPE composition lets the fix run once per eviction across all layers at <2% inference cost [§3.4, Eq. 10, Table 9 (appendix)].
A dual-resolution shifting pipeline computes the full-resolution KV and a low-resolution backup concurrently during chunk generation, so when a chunk ages out of “recent” it slides into “mid” without a separate compression pass — hiding compression latency [§3.4].
An empirical attention-pattern study justifies the design: attention demand persists across the full generated history (invalidating FIFO) and highly-attended tokens are sparsely / dynamically distributed (Jaccard distance 0.75 between consecutive selection steps) [§3.6, Fig. 3].
Trained on a 20-latent-frame (~5s) temporal window, PackForcing achieves up to 24× temporal extrapolation, generating coherent 2-minute / 1920-frame videos at 16 FPS on a single H200 [Abstract, §1].
On VBench at 60s, PackForcing leads on Dynamic Degree 56.25 (vs. Deep Forcing 53.67, CausVid 48.43) and Overall Consistency 26.07 (vs. LongLive 25.73, Rolling Forcing 25.73), at competitive Subject Consistency 90.49 and Background Consistency 93.46 [Table 1].
On VBench at 120s, performance degradation is marginal (Dyn. Deg. -2.13, Subj. Cons. +2.35, Aesthetic Quality -0.58 vs. its own 60s numbers) — Self-Forcing drops Subject Consistency by 6.01 and Aesthetic Quality by 5.47 over the same range [Table 1, §4.2].
PackForcing maintains the highest CLIP-T alignment throughout 60s with only a 1.14-point drop (34.04 → 32.90); Self-Forcing drops 6.77 points (33.89 → 27.12), CausVid 1.86, Rolling Forcing 1.60 [Table 2].

Method

The system runs on a Wan2.1-T2V-1.3B flow-matching backbone, generating 16-FPS video in causal blocks of 3 latent frames. KV history is stored in three buffers: sink (the first N_sink=2 blocks, full-resolution KV, never modified); mid (older history, compressed by the dual-branch module to 1/16 the token count, with dynamic top-k retrieval); recent (the most recent N_recent blocks at full resolution, plus the current block). At each new block, attention concatenates [sink ; top-k of mid ; recent ; current], a constant-size window. The compressed-mid capacity is strictly bounded; when it fills, the oldest blocks are evicted and a multiplicative temporal-only RoPE rotation is applied to the sink keys to close the resulting position gap without recomputing the cache.

The dual-branch compressor runs both branches in parallel: the HR branch applies a cascade of strided 3D convolutions with SiLU activations directly on the VAE latent (temporal first, then three stages of spatial), preserving fine structure; the LR branch decodes the latent back to pixels, applies 3D average pooling (4× temporal, 2× spatial), re-encodes through the frozen VAE, and patch-embeds, preserving perceptual layout. The two outputs sit in the same dimensional space and are fused by element-wise addition. The pipeline is “dual-resolution shifting”: both branches run during chunk generation, so when a recent block ages out, its compressed twin is already ready to slide into the mid buffer. Training uses the Self-Forcing causal-student setup (ODE-trajectory-aligned init, asymmetric DMD with a 1:5 generator/fake-score update ratio, AdamW 5e-7 / 5e-6), batch size 8, 3000 iterations on 20-latent-frame (~5s) windows. Inference uses 4 distilled denoising steps, CFG=8.0, timestep shift 8.0.

Results

VBench-Long, 60s generation (Wan2.1-T2V-1.3B base, 16 FPS, 128 MovieGen prompts): Dynamic Degree 56.25 / Motion Smoothness 98.29 / Overall Consistency 26.07 / Imaging Quality 69.36 / Aesthetic Quality 62.56 / Subject Consistency 90.49 / Background Consistency 93.46. Deep Forcing: 53.67 / 98.56 / 21.75 / 67.75 / 58.88 / 92.55 / 93.80 [Table 1].
VBench-Long, 120s generation: Dynamic Degree 54.12 / Overall Consistency 26.05 / Aesthetic Quality 61.98 / Subject Consistency 92.84. Self-Forcing collapses to Subj. Cons. 74.40 / Aest. Qual. 51.68 over the same horizon [Table 1].
CLIP-T trajectory at 10s intervals (0–60s): 34.04 / 33.99 / 33.70 / 33.37 / 33.24 / 32.90 (Overall 33.54). The only method whose drop stays under 1.2 points; LongLive (33.46 overall) and Rolling Forcing (32.95) are close, Self-Forcing (30.71) is not [Table 2].
Bounded resources: KV cache held to ~3 GB (vs. ~80 GB unbounded full attention for 2-minute video at 16 FPS), 30 transformer layers. 2-minute videos run on a single H200 [Abstract, §1].
Ablations (Table 5): Dynamic Context Selection over strict FIFO eviction gives +0.8 Subject Consistency and +0.12 CLIP. Incremental RoPE adjustment adds <2% inference cost (Table 9, appendix).
Temporal extrapolation: trained on 5s clips, generates 2-minute videos — a 24× extrapolation factor — operating effectively either zero-shot or after distillation [Abstract].

Why it’s interesting

PackForcing slots cleanly between the two existing operating points on the Context Length / Quality Trade-off in Video Generation Pareto: Pretraining Frame Preservation in Autoregressive Video Memory Compression (PFP) keeps context cheap with a pretrained random-frame-preservation encoder, Context Forcing: Consistent Autoregressive Video Generation with Long Context (Context Forcing) keeps context useful with a long-context teacher + surprisal-driven cache. PackForcing combines the partitioned KV cache idea (functionally identical to Context Forcing’s Sink + Slow + Fast architecture) with explicit learned compression of the middle partition (functionally similar to PFP, but trained jointly with the AR student rather than as a separate pretraining objective). The dual-branch HR+LR compressor and the dynamic top-k retrieval are the two genuinely new elements. The headline finding — that short-video supervision is sufficient for high-quality long-video synthesis when the cache is partitioned and compressed correctly — strengthens the position that the long-horizon problem is more about memory management than about training horizon, and is consistent with Context Forcing’s claim that a small effective context (~20s) is enough when the cache is structured. The incremental RoPE adjustment is reusable independent of the rest of the system, and complements Context Forcing’s Bounded Positional Encoding as a second cheap RoPE-side trick worth lifting.