Context Length / Quality Trade-off in Video Generation

TL;DR

A fundamental trade-off in autoregressive video DiTs: longer context window → better identity / scene / motion consistency across the rollout, but quadratic attention cost and noisier conditioning. VAE channel bottlenecks, learned patchifier compression, sparse / FIFO / surprisal-driven KV caches, learned memory encoders, bounded positional encoding, and training-time degradation perturbations all sit on different points of this Pareto. The filed papers explore four operating regimes — explicit compression (PFP), functional cache partitioning (Context Forcing), curriculum-based consistency injection (LongVie 2), and partitioned-cache-with-learned-mid-compression (PackForcing).

Key claims

• The context-length / quality trade-off in autoregressive video DiTs can be turned into a white-box learning problem by training an encoder φ(H) to compress history H and measuring frame-retrieval quality at arbitrary indices Ω (Pretraining Frame Preservation in Autoregressive Video Memory Compression §3.1).
• A pretrained encoder dropping memory through the VAE’s 16-channel bottleneck is suboptimal — bypassing the bottleneck via a residual branch that writes directly into DiT-inner channel widths (3072/5120) gives PSNR 17.41 vs. 12.93 for FramePack-equivalent large-patchifier compression at the same 4×4×2 rate (Pretraining Frame Preservation in Autoregressive Video Memory Compression §3.1, Table 1).
• Random-index training is necessary for the compression encoder — fixed-position retrieval (e.g. only ending frames) causes the encoder to “cheat” by spending its tokens on a few frames and ignoring the rest (Pretraining Frame Preservation in Autoregressive Video Memory Compression §3.1).
• The KV cache can be partitioned into three functional regions — Attention Sink (first tokens), Slow Memory (long-term, surprisal-promoted), Fast Memory (FIFO local) — and this partitioning is necessary, not just helpful: a training-free ablation that applies the cache structure on top of a non-distilled student degrades, showing that cache and training procedure both matter (Context Forcing: Consistent Autoregressive Video Generation with Long Context §3.3, Table 3).
• An alternative three-partition cache (Sink + Compressed Mid + Recent) keeps the mid partition with learned compression rather than dropping it, achieving 16× spatiotemporal token reduction via a dual-branch (HR 3D-conv + LR pixel-space-re-encode) module fused by element-wise addition (PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference §3.2, §3.3, Eq. 8).
• A fully recurrent alternative collapses the entire prior context to a single fixed-size memory vector built by attention over prior segments’ hidden states, with the memory cross-attended at every Transformer block during denoising — context cost is constant in video length rather than partitioned/compressed (MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation §3, Fig. 2A).
• Bounded Positional Encoding — constraining temporal RoPE indices to a fixed range regardless of generation step — is essential: removing it drops VBench Total from 82.45 → 73.52 (Context Forcing: Consistent Autoregressive Video Generation with Long Context §3.3, Eq. 8, Table 3).
  • Underlying mechanism (new): RoPE-based attention loses both locality bias and token-relevance consistency as context length grows, with failure probability approaching 0.5 — a content-independent, provable limit that all of the RoPE-side workarounds in this concept are implicitly working around (RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably).
  • Background: RoPE itself rotates Q/K feature pairs by position-proportional angles whose frequencies are set by a base hyperparameter; the relative-offset property of the resulting dot product is what every workaround in this concept is preserving or restoring (Rotary Positional Embeddings Explained | Transformer).
  • Constructive alternative: Re-expressing RoPE in polar coordinates shows the dot product’s position-of-maximum-response depends on key/query content (the what-where interaction term); PoPE removes that term by giving each element a position-only phase and a softplus-activated magnitude, yielding 10× zero-shot length extrapolation on PG-19 (1024 → 10240) without YaRN-style interpolation (Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings §2-3, Fig. 2).
• An incremental temporal-only RoPE adjustment — applied once per eviction event, exploiting RoPE’s multiplicative composition — fixes the position gap left in the sink when old mid blocks are evicted, at <2% inference cost (PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference §3.4, Eq. 10).
• An alternative length-extrapolation mechanism — relative positional encoding inside the memory-attention layer between memory keys and current-segment queries — allows training on ≤5s clips while inferring at >120s (MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation §3).
• Surprisal-driven slow-memory promotion (cosine distance between successive key vectors > τ) beats uniform interval-1 / interval-2 sampling on VBench Total (82.45 vs. 80.82 / 81.11) (Context Forcing: Consistent Autoregressive Video Generation with Long Context Table 3).
• Dynamic top-k retrieval over the compressed mid-buffer (query-key affinity, computed once per block on sub-sampled queries / half the heads) beats strict FIFO eviction by +0.8 Subject Consistency and +0.12 CLIP, at <1% overhead (PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference §3.5, Table 5).
• Effective context can exceed 20s during inference vs. 1.5–9.2s for the prior streaming-tuning SOTA (Infinity-RoPE 1.5s, LongLive 3.0s, Rolling Forcing 6.0s, FramePack-F1 9.2s) (Context Forcing: Consistent Autoregressive Video Generation with Long Context §1, Table 2 caption).
• Short-video supervision is sufficient for long-video synthesis when the cache is partitioned and compressed correctly — trained on 5s clips, PackForcing extrapolates 24× to coherent 2-minute generation (PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference Abstract, §1); MALT reports the same recipe (≤5s training → >120s inference) with a recurrent memory vector instead of a partitioned cache (MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation §Detailed Description, Fig. 6).
• Combining two memory encoders at different compression rates (e.g. 4×4×2 for the dominant pathway plus a 2×2×4 that prioritizes temporal compression) preserves fine spatial detail like text on signs at the cost of more context — a knob on the Pareto, not a free lunch (Pretraining Frame Preservation in Autoregressive Video Memory Compression §4.5, Fig. 10).
• Cross-clip drift is the dominant failure mode at the multi-minute regime — LongVie 2’s “history-context guidance” stage is specifically the alignment mechanism that targets it after controllability and degradation-robustness are addressed (LongVie 2: Multimodal Controllable Ultra-Long Video World Model project page).
• Headline operating points span a wide range — PFP: 20s history at ~5k context on a 12GB consumer GPU (Pretraining Frame Preservation in Autoregressive Video Memory Compression Fig. 1); Context Forcing: 60s at 17 FPS, effective context >20s (Context Forcing: Consistent Autoregressive Video Generation with Long Context Table 2); PackForcing: 2-minute generation at 16 FPS on a single H200, KV cache bounded to ~3 GB (PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference Abstract); LongVie 2: 5-minute continuous generation (LongVie 2: Multimodal Controllable Ultra-Long Video World Model project page); MALT: ≥1024 frames at 8 FPS (~128s) on W.A.L.T-style DiTs (MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation §Detailed Description).
• RoPE base hyperparameter trades distinguishing positions against distinguishing tokens — the now-standard practice of increasing the RoPE base for long-context models helps the token side at a provable cost on the position side (RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably).
• On an Indirect Indexing diagnostic (find a character at a specified offset from a specified source character in a random string), RoPE Transformers reach 11% final-token accuracy while PoPE Transformers reach 95% — a clean demonstration that content/position entanglement in RoPE actively blocks pointer-arithmetic-style position lookup, not just a theoretical concern (Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings Table 1).

Recent contributions

• [2026-09-01] Atlas: A World Model for Spatial Intelligence: Atlas turns the reconstruction-vs-imagination trade-off into a runtime dial via spatial context: it accepts 1 to 100+ reference images placed at explicit 3D positions and behavior shifts monotonically from imagination toward faithful reconstruction as inputs grow — “the more it sees, the less it imagines” — supporting 1-minute 1440p output at the long end.
• [2026-08-28] OpenDM — DM0.5: An Open-World Foundation Model for General-Purpose Embodied Intelligence: DM0.5 (Dexmal) supplies a robotics-domain datapoint on the “just make the KV window bigger” side of this axis — up to ~60 seconds of historical visual context in a Gemma3-4B VLM + 680M flow-matching Action Expert VLA, used both for long-horizon coherence and for a specific in-episode learning-from-a-single-human-demonstration experiment where the human’s start-of-episode demonstration and the robot’s subsequent execution phase must both fit in the same context window. Contrasts with the memory-augmentation answer (MEM: Multi-Scale Embodied Memory for Vision Language Action Models MEM’s dual-timescale memory) and the memory-into-weights answer (RoboTTT — 8k-timestep robot policy via test-time training (Jim Fan / NVIDIA GEAR) RoboTTT / WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time WAM-TTT) — three different substrates for the same operating goal, never benchmarked head-to-head at matched horizons.
• [2026-08-27] Gemini Omni 1.1 Flash lets you build with more control: Gemini Omni 1.1 Flash surfaces the tradeoff as a product-side chunked API: 10 seconds of prior clip context per extension call (up from 1s in the launch model), extendable in 10s increments to a 40s cumulative cap — a much shorter horizon than open long-AR-video recipes (LongVie 2 at 5 min, PackForcing at 2 min) at unspecified fidelity trade-off; no benchmark tables, so unclear whether the cap is a quality floor or a product decision.
• [2026-08-21] 4DAnyone: Create Anyone in 4D from a Casual Monocular Video: Reference Context Packing (RCP) is a novel operating point: instead of evicting or sinking old tokens, it packs a growing reference set into a fixed-length mixed-resolution context to keep reference-attention O(1) while preserving cross-group appearance guidance — applied to camera-controlled video DiTs generating dense target views for 4DGS lifting.
• [2026-08-14] Qwen3.8 open weights release — 27B dense multimodal + 2.4T-A95B MoE (Alibaba Qwen): Qwen3.8-27B lists 262K native context extendable to 1M via YaRN — Alibaba continues to publish YaRN as the extrapolation path in launch materials, without noting any independent long-context quality benchmarks in the announcement.
• [2026-08-10] MEM: Multi-Scale Embodied Memory for Vision Language Action Models: MEM’s video encoder factorizes attention into spatial + causal temporal (every 4th layer), reducing joint T×N attention cost from O((TN)²) to O(TN² + T²N) while dropping past-timestep tokens after the encoder — the VLA backbone sees the same token count as a single-image VLA, so temporal information is compressed into the current-frame tokens rather than expanding the downstream context length.
• [2026-08-08] MiniWorld: Democratizing the Training of Video World Models from Scratch: MiniWorld combines a rolling KV cache with pipelined asynchronous denoising to keep per-step compute bounded during streaming AR video generation from a from-scratch-trained block-causal DiT.
• [2026-07-31] Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers: Chimera reports zero-shot temporal extrapolation from 5-second training clips to 30-second video generation — 6× duration extension with only 6.5% FID degradation in the last five seconds and no length-specific fine-tuning. Attributed to the KDA + MLA hybrid attention’s fixed-state long-context behavior operating on a shared parameter path with no positional embeddings; adds a new operating point where the length-extrapolation mechanism is architectural (no RoPE to break, linear-attention state carries history) rather than cache-partitioning or memory-encoder-based.
• [2026-07-31] Towards Video World Models: Xun Huang position essay frames the wiki’s forgetting-drifting problem as “persistence” — a soft constraint for video WMs that is architecturally tightened by the real-time constraint: unlike LLMs where users tolerate growing inference time with context, video WMs face a hard latency ceiling and cannot slow down over time. Names three complementary axes tracked on this page and elsewhere — aggressive compression of temporally-distant frames (FAR, FramePack; “needle in a haystack” failure mode named), integration with persistent 3D conditions (GEN3C, WVD, SPMem; brittle in dynamic scenes), and linear-RNN long-range recurrence (Po et al. 2025, ConvLSTM ancestry).
• [2026-07-29] Wonder: Video World Model Done Better: Wonder adds a new operating point: sparse-attention selection over full-fidelity history KV caches (no partition, no compression, no latent warping) for constant-latency retrieval at arbitrary context length; reports 16 FPS at 0.5s constant latency up to one minute of rollout with return-to-previously-observed regions preserved.
• [2026-07-27] Kimi K3 Technical Report — Open Frontier Intelligence: Kimi K3 tech report — 1M-token context deployed via the KDA/Gated-MLA 3:1 hybrid with NoPE on all MLA layers (position-sensitivity carried by KDA), removing RoPE-frequency retuning / YaRN when extending context.
• [2026-07-24] SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation: Compiled DiT forward pass is 3.2× faster than a matched full-softmax baseline at 720p/60s, with the gap expanding at longer durations — a controlled hybrid-vs-softmax runtime scan at video length that matches the LLM-side “throughput advantage grows with context” pattern from Jet-Nemotron and Kimi Linear.
• [2026-07-21] AlayaWorld: Interactive Long-Horizon World Modeling: AlayaWorld realises constant per-chunk compute at in-principle unbounded horizon via a bounded rolling prefix — 1 fixed sink frame + a 6-latent-frame compressed temporal memory + a capped 10-frame spatial-memory render + 1 I2V nearby frame — attended jointly with the noised target. The sink is pinned at RoPE position 0 and held fixed across all chunks as a global identity anchor (during training drawn as a remote frame ≥8 latents away from the target so the model cannot extrapolate the next chunk from it).
• [2026-07-17] HDR: Hierarchical Denoising for Multi-Step Visual Reasoning: HDR’s Sparse Hierarchical Attention Pattern gives each token access to local + parent + first-frame context only, so effective context grows logarithmically via a tree hop rather than linearly with a long window — a distinct point on the trade-off curve from KV-partitioning (Context Forcing) or memory-compression (PFP) approaches. Enables coarse plans to reach any leaf in at most one vertical hop while attention cost stays sparse.
• [2026-07-16] Kimi K3 — Open Frontier Intelligence (2.8T MoE with KDA + AttnRes): Kimi K3 offers 1M-token native context and reports BrowseComp = 90.4 at 1M tokens with no context management, vs the Claude-card 300K compaction baseline — an explicit test of whether frontier agentic quality survives full-length context on a hybrid-linear-attention backbone.
• [2026-07-15] RoboTTT — 8k-timestep robot policy via test-time training (Jim Fan / NVIDIA GEAR): RoboTTT (Jim Fan / NVIDIA GEAR launch tweet) extends the concept’s Pareto to robotics: instead of a KV-cache partition or a recurrent memory vector cross-attended into a DiT, the entire history is compressed into a small TTT core’s weights via one SGD step per sensor reading, at constant per-step inference cost. The reported Context Scaling Curve is the strongest “no saturation” datapoint yet from the robotics side — 128 → 8K timesteps of pretraining hill-climbs monotonically, with 8K beating 1K by 62% closed-loop; Fan claims 1M-timestep robot context is reachable. Launch-tweet only; no benchmark numbers.
• [2026-07-09] LingBot-World 2.0 / LingBot-World-Infinity — Infinite Worlds with Versatile Interactions: LingBot-World 2.0 claims unbounded interaction horizon at 720p / 60 fps via causal pretraining plus attention-sink + local-window inference (CLI: --sink_size 6 --local_attn_size 18) — a new open-source operating point on the horizon vs. quality Pareto that pushes past v1’s ~10-minute ceiling; no quantitative long-horizon benchmark numbers disclosed in the README at filing time.
• [2026-06-09] Latent Spatial Memory for Video World Models: Mirage compresses long-horizon scene context via a 3D-indexed cache of diffusion latent tokens rather than via in-network KV partitioning, learned bottlenecks, or RGB-space point clouds. The depth-guided back-projection is content-adaptive geometric compression, but on the latent manifold the generator already uses — avoiding both the quadratic-attention cost of long context windows and the encode/decode information loss of SPMem-style RGB caches.
• [2026-06-08] Scaling Video Training with Parallelism: The systems-side enabler for the long-video-DiT recipes filed in this concept. Argues that long-video training requires within-sample (sequence/context) parallelism, not just across-sample DP — and that for AR teacher forcing the SP unit must be a temporal chunk holding matched clean-history + noisy-target latents (Balanced SP), not a slice of the concatenated clean/noisy sequence. Pairs with LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation (whose training stack uses this), and with LongVILA’s MM-SP for the understanding side.
• [2026-06-06] Direct Video-Action Models — Causal Video Models Are Data-Efficient Robot Policy Learners: Rhoda AI’s DVA makes “hundreds of frames of video context, natively” the load-bearing capability for robotics — explicitly contrasted with the few-frame context of typical VLAs. The Context Amortization training strategy is what makes the long-context regime trainable: every position in a long sequence supervises a noisy-future prediction off noise-free history, so the long-context conditioning the inference mask requires is also what’s seen densely at training. Validated downstream by a shell-game memory probe (object tracked across multiple swaps) and an end-to-end returns-processing deployment that resolves visually ambiguous current frames purely from video history.
• [2026-06-02] Tilde Research — Wall Attention: length generalization via diagonal forget gates in softmax attention: Tilde’s Wall Attention announcement — claims 4k-training → 200k+-token generalization with no continued pretraining, by replacing RoPE with per-channel diagonal forget gates inside softmax attention. If reproducible, this is a much wider extrapolation ratio than the FoX / Gated-Attention baselines it’s pitched against.
• [2026-05-27] Planning at Inference: MCTS Test-Time Scaling for Long Video Generation: Inference-time MCTS over candidate continuations as a way to push long-horizon consistency without enlarging the context window: the search tree externalizes the consistency burden onto a verifier-scored rollout rather than into more attention context.
• [2026-05-26] MIGA: Enhancing Train-Free Infinite-Frame Generation for Consistent Long Videos: MIGA — first train-free operating point on the long-AR-video Pareto: a sampler wrapper over FIFO-Diffusion that uses a zigzag noise schedule (Two-stage Training-inference Alignment) to narrow the noise span per forward pass, plus Dual Consistency Enhancement (anomaly-triggered self-reflection at the queue tail + long-range frame guidance from the queue head). Sits parallel to PFP / Context Forcing / PackForcing / LongVie 2 / MALT, all of which require training; useful baseline for separating “training helps” from “scheduling + cross-time attention help”.
• [2026-05-25] SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers: SEGA addresses the spatial analogue of RoPE long-context degradation: dynamically rescales RoPE components by FFT-derived per-band spectral energy to recover structure and detail at out-of-training-distribution image resolutions (4096² from FLUX / Qwen).
• [2026-05-25] Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings: PoPE — Polar Coordinate Positional Embeddings. Re-expresses RoPE in polar form to localise the what-where confound to a single interaction term in the dot product, then removes it by giving each key/query element a softplus magnitude and a position-only phase $p\,\omega_i$, with an optional per-head learnable phase bias. Solves an Indirect Indexing diagnostic at 95% vs RoPE’s 11%, lowers perplexity on MIDI / human genome / OpenWebText at 124M–774M, and extrapolates from 1024 to 10240 tokens on PG-19 without interpolation while RoPE diverges. Constructive counterpart to RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably‘s lower-bound result. No vision experiments — leaves open whether the axial RoPE schedules used in image/video DiTs admit the same drop-in re-parameterisation.
• [2026-05-24] Rotary Positional Embeddings Explained | Transformer: Reference explainer covering the RoPE construction itself (2D rotations on Q/K feature pairs, frequencies set by base θ, relative-offset property of the post-rotation dot product, axial extension to 2D/3D inputs) and a from-scratch PyTorch implementation. Background for the RoPE-side claims and workarounds in this concept; not itself a new operating point.
• [2026-05-24] RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably: Provides the missing theoretical foundation — proves that RoPE-based attention loses both locality bias and token-relevance consistency as context grows (failure probability → 0.5), and that the RoPE-base hyperparameter cannot simultaneously preserve position- and token-distinguishability. Recasts every RoPE-side patch already in this concept (Bounded Positional Encoding, incremental temporal-only RoPE adjustment, relative-PE-inside-memory-attention) as necessary workarounds for a content-independent lower bound. The author’s accompanying takeaway — that agentic / context-folding approaches may be more effective than scaling raw context — links this concept to the recursive-LM / context-folding cluster on the LM side.
• [2026-05-24] MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation: MALT Diffusion — adds a fifth operating point: a single recurrent memory vector of fixed size, built by attention over prior segments’ hidden states, cross-attended at every diffusion-Transformer block, with relative positional encoding inside memory attention as the length-extrapolation mechanism and noise augmentation on the memory vector at training as the on-manifold regulariser. Training cost is independent of video length (stop-grad through prior segments). Conceptually upstream of PFP (explicit compression), Context Forcing (partitioned cache), and PackForcing (partitioned cache + mid compression) — all three can be read as evolutions of MALT’s memory-vector idea in different directions.
• [2026-05-23] PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference: PackForcing — a three-partition KV cache (Sink + Compressed Mid + Recent) where the mid partition is compressed 16× by a learned dual-branch module (HR 3D-conv + LR pixel-space-re-encode), then accessed via dynamic top-k retrieval; an incremental temporal-only RoPE adjustment closes position gaps from mid-block eviction. Trained on 5s clips, extrapolates 24× to coherent 2-minute generation at 16 FPS on a single H200, with VBench Dynamic Degree 56.25 / Overall Consistency 26.07 at 60s and CLIP-T drop under 1.2 points over the full 60s. A fourth operating point on the Pareto: combines Context Forcing’s partitioning idea with explicit learned mid-buffer compression.
• [2026-05-22] LongVie 2: Multimodal Controllable Ultra-Long Video World Model: Adds a third operating point — multi-minute (5-min) continuous generation via a multi-stage training curriculum (multimodal guidance → degradation-aware training → history-context guidance) rather than explicit cache or memory-encoder design.
• [2026-05-22] Context Forcing: Consistent Autoregressive Video Generation with Long Context: Functional KV-cache partitioning (Attention Sink / Slow Memory / Fast Memory), surprisal-driven consolidation, and Bounded Positional Encoding. Names the “forgetting–drifting dilemma” as the underlying trade-off vocabulary.
• [2026-05-22] Pretraining Frame Preservation in Autoregressive Video Memory Compression: First filed datapoint that isolates the compression sub-problem with a “preserve any random frame perfectly” pretraining objective; recommends 4×4×2 latent compression as the sweet spot.

Open questions

• Can the five approaches be composed? Memory-encoder compression (PFP) + functional cache partitioning + surprisal slow-memory (Context Forcing) + learned mid-buffer compression with top-k retrieval (PackForcing) + curriculum-based consistency (LongVie 2) + recurrent memory vector (MALT) target different parts of the same Pareto and look stackable, but no filed paper has tested combinations. PackForcing and Context Forcing in particular share the partitioning idea but differ on whether to select or compress the long-term region — a combined approach (compress then select) is the obvious next experiment. MALT is the fully-recurrent extreme — does adding a partitioned/compressed cache on top of MALT’s memory vector help?
• Where does the surprisal-promotion threshold τ come from? Context Forcing reports the choice is robust but doesn’t characterize its dependence on content type (dense-shot-shift vs. single-long-shot); PFP claims that dense-shot-shift hides error accumulation entirely.
• PackForcing’s dual-branch HR+LR compressor, PFP’s bottleneck-bypassing residual encoder, and MALT’s recurrent-attention memory encoder are three different recipes for the same compression sub-problem — none has been compared head-to-head on the same backbone or benchmark.
• Should compression rates be content-adaptive? PFP shows that combining 4×4×2 + 2×2×4 helps preserve text — a static per-region compression mix — but no filed paper has tried adaptive compression where the rate is chosen per-frame or per-region. PackForcing’s top-k retrieval is temporally adaptive but not spatially adaptive.
• The Pareto evaluation protocols don’t agree: PFP uses user-study ELO + Cloth/Identity/Instance consistency on internal benchmarks; Context Forcing uses VBench + windowed DINO/CLIP-F/CLIP-T on MovieGenBench prompts; PackForcing uses VBench-Long with 128 MovieGen prompts + CLIP-T at 10s intervals; LongVie 2 introduces LongVGenBench; MALT reports Kinetics-600 long-video prediction. A shared benchmark for the long-AR-video Pareto is missing.
• All four “Wan-era” filed papers are Wan-family. How much of the reported tradeoffs are an artifact of Wan2.1/Wan2.2’s specific VAE channel width, RoPE schedule, and patchifier rates? MALT’s results are on W.A.L.T (a different DiT-family) and would be the cleanest cross-backbone replication target.
• Bounded Positional Encoding (Context Forcing), incremental RoPE adjustment (PackForcing), and relative-PE-inside-memory-attention (MALT) all address a RoPE-side problem in long-horizon AR rollouts but with different mechanisms. Are they substitutes or complements? RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably gives the lower-bound framing but doesn’t compare workarounds directly.
• If RoPE has a content-independent failure floor (RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably), how much of the long-context win in the video-DiT setting is from the workarounds vs. from non-RoPE inductive biases (e.g. spatial attention windows, compressed memory)? The video setting may be more forgiving than the text setting because spatial structure provides redundant positional signal.
• Does PoPE (Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings) work on axial / 2D / 3D positional schedules used in image and video DiTs? PoPE’s text-side extrapolation result (1024 → 10240 PG-19, zero-shot) is the strongest length-generalization datapoint among RoPE alternatives so far, but the paper runs no vision experiments and the standard image/video extension of RoPE applies separate frequency banks per spatial axis — it isn’t obvious that PoPE’s per-element softplus-magnitude / position-only-phase construction transfers cleanly.
• Is PoPE composable with the cache-side workarounds in this concept (Bounded PE, incremental temporal RoPE adjustment, relative-PE-inside-memory-attention)? Both directions plausibly help — PoPE removes the underlying confound; the cache-side patches restrict the regime — but they touch overlapping parts of the model and no filed paper has tried the combination.

Papers

2026-05

• LongVie 2: Multimodal Controllable Ultra-Long Video World Model — LongVie 2: Multimodal Controllable Ultra-Long Video World Model (published 2026-05-22)
• LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation — LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation (published 2026-05-18)
• DeltaToken: 100× Token-Efficient Video Representation for World Models — DeltaToken: 100× Token-Efficient Video Representation for World Models (published 2026-05-16)
• RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably — RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably (published 2026-05-15)
• SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer — SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer (published 2026-05-14)

2026-04

• Lyra 2.0: Explorable Generative 3D Worlds — Lyra 2.0: Explorable Generative 3D Worlds (published 2026-04-15)
• VideoFlexTok: Flexible-Length Coarse-to-Fine Video Tokenization — VideoFlexTok: Flexible-Length Coarse-to-Fine Video Tokenization (published 2026-04-14)

2026-03

• PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference — PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference (published 2026-03-26)
• Helios: Real Real-Time Long Video Generation Model — Helios: Real Real-Time Long Video Generation Model (published 2026-03-04)

2026-02

• Context Forcing: Consistent Autoregressive Video Generation with Long Context — Context Forcing: Consistent Autoregressive Video Generation with Long Context (published 2026-02-05)
• Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization — Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization (published 2026-02-03)

2026-01

• Advancing Open-source World Models (LingBot-World) — Advancing Open-source World Models (LingBot-World) (published 2026-01-29)
• Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention — Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention (published 2026-01-01)

2025-12

• Pretraining Frame Preservation in Autoregressive Video Memory Compression — Pretraining Frame Preservation in Autoregressive Video Memory Compression (published 2025-12-29)
• Yume-1.5: A Text-Controlled Interactive World Generation Model — Yume-1.5: A Text-Controlled Interactive World Generation Model (published 2025-12-22)
• End-to-End Training for Autoregressive Video Diffusion via Self-Resampling — End-to-End Training for Autoregressive Video Diffusion via Self-Resampling (published 2025-12-17)
• HY-World 1.5 (WorldPlay): A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency — HY-World 1.5 (WorldPlay): A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency (published 2025-12-17)
• Flowception: Temporally Expansive Flow Matching for Video Generation — Flowception: Temporally Expansive Flow Matching for Video Generation (published 2025-12-12)
• RELIC: Interactive Video World Models with Long-Horizon Memory — RELIC: Interactive Video World Models with Long-Horizon Memory (published 2025-12-01)

2025-11

• Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout — Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout (published 2025-11-25)
• Infinite Context Length with Global but Constant Attention Memory — Infinite Context Length with Global but Constant Attention Memory (published 2025-11-24)
• Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation — Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation (published 2025-11-19)
• PAN: A World Model for General, Interactable, and Long-Horizon World Simulation — PAN: A World Model for General, Interactable, and Long-Horizon World Simulation (published 2025-11-12)
• Cambrian-S: Towards Spatial Supersensing in Video — Cambrian-S: Towards Spatial Supersensing in Video (published 2025-11-06)

2025-10

• Generative View Stitching — Generative View Stitching (published 2025-10-28)
• LongCat-Video Technical Report — LongCat-Video Technical Report (published 2025-10-25)
• Krea Realtime 14B: Real-Time, Long-Form AI Video Generation — Krea Realtime 14B: Real-Time, Long-Form AI Video Generation (published 2025-10-15)
• Self-Forcing++: Towards Minute-Scale High-Quality Video Generation — Self-Forcing++: Towards Minute-Scale High-Quality Video Generation (published 2025-10-02)

2025-09

• Radamés Ajna — testing nano-banana with recursive editing as long video gen — Radamés Ajna — testing nano-banana with recursive editing as long video gen (published 2025-09-30)
• SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer — SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer (published 2025-09-29)
• Rolling Forcing: Autoregressive Long Video Diffusion in Real Time — Rolling Forcing: Autoregressive Long Video Diffusion in Real Time (published 2025-09-29)
• LongLive: Real-time Interactive Long Video Generation — LongLive: Real-time Interactive Long Video Generation (published 2025-09-26)
• Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesis — Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesis (published 2025-09-11)
• Qwen3-Next-80B-A3B — hybrid Gated DeltaNet + Gated Attention, ultra-sparse 512-expert MoE, MTP (Alibaba Qwen announcement) — Qwen3-Next-80B-A3B — hybrid Gated DeltaNet + Gated Attention, ultra-sparse 512-expert MoE, MTP (Alibaba Qwen announcement) (published 2025-09-11)
• Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings — Decoupling the “What” and “Where” With Polar Coordinate Positional Embeddings (published 2025-09-05)

2025-08

• InfinityHuman: Towards Long-Term Audio-Driven Human Animation — InfinityHuman: Towards Long-Term Audio-Driven Human Animation (published 2025-08-27)
• DeepSeek-V3.1-Base — long-context-extended V3 base with UE8M0 FP8 microscaling — DeepSeek-V3.1-Base — long-context-extended V3 base with UE8M0 FP8 microscaling (published 2025-08-21)
• FastWan: Generating a 5-Second Video in 5 Seconds via Sparse Distillation — FastWan: Generating a 5-Second Video in 5 Seconds via Sparse Distillation (published 2025-08-04)

2025-06

• Radial Attention: O(n log n) Sparse Attention with Energy Decay for Long Video Generation — Radial Attention: O(n log n) Sparse Attention with Energy Decay for Long Video Generation (published 2025-06-24)
• Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition — Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition (published 2025-06-20)
• VideoMAR: Autoregressive Video Generation with Continuous Tokens — VideoMAR: Autoregressive Video Generation with Continuous Tokens (published 2025-06-17)
• Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation — Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation (published 2025-06-11)
• Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion — Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion (published 2025-06-09)
• Video World Models with Long-term Spatial Memory — Video World Models with Long-term Spatial Memory (published 2025-06-05)
• Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation — Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation (published 2025-06-04)
• Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval — Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval (published 2025-06-03)

2025-05

• Test-Time Training Done Right (LaCT) — Test-Time Training Done Right (LaCT) (published 2025-05-29)

2025-02

• MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation — MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation (published 2025-02-18)

Unknown date

• Rotary Positional Embeddings Explained | Transformer — Rotary Positional Embeddings Explained | Transformer (published unknown)