Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout

TL;DR

∞-RoPE is a training-free, inference-time framework that converts existing short-horizon Self-Forcing video DiTs (here Wan2.1-T2V-1.3B distilled to 5s rollouts) into infinite-horizon, action-controllable, multi-cut generators by reparameterizing temporal 3D-RoPE and KV-cache management — no retraining, no long-video supervision, no architectural change. Three interconnected components: Block-Relativistic RoPE (moves temporal indices into a local reference frame anchored at the cache window so generation can advance past the base model’s 1024-frame RoPE horizon), KV Flush (clears cache to only the global sink + last latent frame on prompt change for instant action responsiveness at constant memory), and RoPE Cut (controlled discontinuous jumps in temporal RoPE for cinematic scene cuts inside a single rollout). At 240s, ∞-RoPE beats Rolling-Forcing on VBench Overall (0.8309 vs 0.8017) while running at the same 17 FPS throughput as Self-Forcing/Rolling-Forcing.

Key claims

Autoregressive video DiTs trained under Self-Forcing on only 5s clips already possess the capacity for highly dynamic infinite-horizon generation — what’s needed is relativistic inference-time adaptation, not more training [§1, Fig. 2].
Block-Relativistic RoPE redefines temporal coordinates within a moving local reference frame anchored at the cache onset f₀, with earlier cached blocks rotated backward in RoPE space to preserve relative temporal geometry, eliminating fixed absolute temporal positions [§4.1, Eq. 2].
When the cache exceeds the teacher’s native horizon f_limit, earliest cached blocks are semanticized — collapsed to a shared minimum index — preserving their content as a temporally agnostic context prior rather than dropping them [§4.1, Eq. 3].
KV Flush retains only two anchors on prompt change — the global sink latent (stabilizes attention normalization) and the last generated latent (preserves local temporal continuity) — providing instant prompt responsiveness with constant memory and zero recompute, vs. LongLive’s KV-Recache which rebuilds the entire cache via cross-attention with the new prompt [§4.2, Fig. 4].
RoPE Cut offsets the current block’s temporal RoPE index by Δ inside the rollout, producing scene cuts with identity preservation through the cut because the relativistic coordinate system itself shifts rather than indexing into an absolute position [§4.3, Eq. 4].
At 240s, ∞-RoPE beats all autoregressive baselines on VBench Overall (0.8309 vs Rolling-Forcing 0.8017, CausVid 0.7758, Self-Forcing 0.6850) while matching their 17 FPS throughput; gains widen as duration increases (5s: 0.8377 / 60s: 0.8298 / 120s: 0.8236 / 240s: 0.8309 vs Rolling-Forcing 0.8332 / 0.8146 / 0.8162 / 0.8017) [Tables in §4.2, §5].
Attention-map analysis shows Block-Relativistic RoPE preserves a sharp diagonal band (local recency) plus a persistent attention-sink column far beyond the 3D-RoPE horizon — i.e. the relativistic re-anchoring prevents attention collapse [§4.4, Fig. 6(a)].
With KV Flush active, attention from new frames to intermediate past frames is suppressed and redirected onto the sink + last pre-flush frames — mechanistic explanation for the instant prompt-following [§4.4, Fig. 6(b)].
With RoPE Cut active, the attention map splits into two nearly disjoint diagonal blocks (the new segment attends only to itself + the sink), severing temporal context cleanly — mechanistic explanation for the scene-cut effect [§4.4, Fig. 6(c)].
∞-RoPE is plug-and-play on top of Self-Forcing variants — Wan2.1-T2V-1.3B distilled by Self-Forcing is the demonstrated base, all results use KV cache size 6, f₀=21, CFG=3.0, timestep shift 5.0 [§5].

Method

The base model is Wan2.1-T2V-1.3B distilled into a 4-step causal generator via Self-Forcing, which natively produces 5s / 21-latent-frame clips at 16 FPS. Autoregressive self-rollout proceeds in blocks of 3 latent frames; the KV cache holds a fixed-size window of past blocks. The Wan 3D-RoPE has a hard length cap of 1024 along each axis, but the teacher was only trained on 21-frame horizons — exceeding either limit degrades attention.

Block-Relativistic RoPE changes how temporal RoPE indices are assigned. Instead of absolute global indices, the current generation block at the cache front gets index t = f_limit - 1, and all earlier blocks in the cache are rotated backward into [f_limit - L_cache, f_limit - 1], where L_cache is the resident cache size. As generation advances, the entire local frame slides forward while staying anchored at the teacher’s maximum index — the cache contents are temporally re-anchored, not their absolute positions. When the effective cache exceeds the teacher’s native 21-frame horizon (the “unbounded” regime), the earliest blocks are semanticized — their indices collapse to a single shared minimum — preserving them as content-only context anchors. The authors motivate this with a cognitive-neuroscience analogy: episodic memory consolidating into semantic memory over time.

KV Flush is invoked at prompt-change events. The entire cache is discarded except (i) the global sink latent (the first generated frame, which the attention-sink literature shows is the attention-normalization anchor) and (ii) the most recent generated latent frame (preserving local motion continuity). New action prompts then condition directly on these two anchors. This is the constant-memory, zero-recompute alternative to LongLive’s KV-Recache, which has to cross-attend the new prompt against every cached token to rebuild the cache.

RoPE Cut is invoked at scene-cut events. Instead of incrementing the temporal RoPE index sequentially, the current block’s index is offset by Δ. Combined with Block-Relativistic RoPE’s moving reference frame, this re-anchors subsequent generation as if the jumped segment lived at past indices — the model never falls off the RoPE horizon because the coordinate system itself shifts.

All three operators are inference-time only and require no fine-tuning of the underlying Self-Forcing model.

Results

VBench evaluation on >100 videos generated from MovieGenBench prompts at four durations (5s, 60s, 120s, 240s):

240s VBench Overall: ∞-RoPE 0.8309, Rolling-Forcing 0.8017, CausVid 0.7758, SkyReels-V2 0.7147, Self-Forcing 0.6850, NOVA 0.6662 [§5, 240s table].
120s VBench Overall: ∞-RoPE 0.8236, Rolling-Forcing 0.8162, CausVid 0.7799, Self-Forcing 0.7271 [§5, 120s table].
60s VBench Overall: ∞-RoPE 0.8298, Rolling-Forcing 0.8146, CausVid 0.7940, Self-Forcing 0.7715, SkyReels-V2 0.7768 [§4.2 table].
5s VBench Overall: ∞-RoPE 0.8377, vs Self-Forcing 0.8398, Wan2.1 0.8319, LTX-Video 0.8439 — the relativistic reformulation costs essentially nothing at 5s where it has no work to do [§4.2 table].
Subject consistency at 60s: ∞-RoPE 0.9444, Rolling-Forcing 0.9409, Self-Forcing 0.8360 [§4.2 table].
Dynamic Degree at 240s: ∞-RoPE 0.64, Rolling-Forcing 0.40, CausVid 0.36, Self-Forcing 0.24 — ∞-RoPE produces more motion while maintaining identity, not the usual trade-off [§5, 240s table].
Throughput: 17.01 FPS (matched to Self-Forcing, CausVid, Rolling-Forcing) — no throughput cost from the relativistic reformulation [§4.2 table].
User study against LongLive, SkyReels-V2, Self-Forcing on action-controlled generation reports ∞-RoPE wins on Text Alignment, Subject Consistency, Motion Smoothness, and Video Quality [§5.2].

Why it’s interesting

This is a fifth distinct operating point on the long-horizon autoregressive-video Pareto catalogued by Context Length / Quality Trade-off in Video Generation, orthogonal to the four already filed there. PFP (Pretraining Frame Preservation in Autoregressive Video Memory Compression), Context Forcing (Context Forcing: Consistent Autoregressive Video Generation with Long Context), PackForcing (PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference), and MALT Diffusion (MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation) all train something — a compression encoder, a long-context teacher, a learned mid-buffer, a recurrent memory vector. ∞-RoPE trains nothing: it’s a pure inference-time reparameterization of an existing Self-Forcing checkpoint. The relativistic frame is conceptually adjacent to Context Forcing’s Bounded Positional Encoding (which the same Wan2.1 base benefits from, contributing ~9 VBench Total points in their ablation) and PackForcing’s incremental temporal-only RoPE adjustment, but pushes the idea further: the temporal coordinate system is always a moving local frame, not just bounded.

It also names and operationalizes the attention-sink mechanism that StreamingLLM and the Discovery of Attention Sinks characterized for streaming LLMs — KV Flush is essentially a video-DiT version of “keep the first few tokens + recent tokens, evict the middle” but applied per-prompt-change rather than per-step. The Self-Forcing++ paper (Self-Forcing++: Towards Minute-Scale High-Quality Video Generation) is the direct training-time counterpart targeting minute-scale rollouts via long DMD; ∞-RoPE achieves longer rollouts (240s) at the same cost as the original Self-Forcing checkpoint. Note that Context Forcing’s effective-context table specifically lists “Infinity-RoPE 1.5s” as a baseline they outperform — but that was scored on Context Forcing’s own forgetting-drift metric, not VBench Overall at 240s, and ∞-RoPE is best read as targeting the positional-encoding failure mode that RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably gives a provable lower bound on.

The Slack note (“seems potentially quite relevant for ai humans / mach”) is on point: KV Flush’s instant prompt-responsiveness is precisely the property a real-time interactive avatar system needs — switch action prompts mid-rollout without recomputing the cache or losing identity through the transition.