Helios: Real Real-Time Long Video Generation Model

TL;DR

Helios is a 14B autoregressive diffusion video model from PKU-YuanGroup + ByteDance that reaches 19.5 FPS end-to-end on a single H100 GPU and generates minute-scale T2V/I2V/V2V — claimed to be faster than several 1.3B distilled baselines (Self-Forcing++, LongLive, Rolling Forcing) while matching Wan2.2-14B quality. It refuses three “standard” toolkits — anti-drifting heuristics (self-forcing, error-banks, keyframe sampling), inference acceleration tricks (KV-cache, causal masking, sparse/linear attention, quantization), and training parallelism/sharding — replacing them with: (1) Unified History Injection that turns a bidirectional pretrained model into an AR generator without causal masking; (2) Easy Anti-Drifting (Relative RoPE + First-Frame Anchor + Frame-Aware Corrupt) targeting three canonical drift modes (position/color/restoration shift); (3) Deep Compression Flow — multi-term memory patchification + pyramid UniPC sampling + Adversarial Hierarchical Distillation reducing 50 → 3 steps. The paper also releases HeliosBench (240 prompts × 4 duration regimes up to 1440 frames).

Key claims

14B Helios reaches 19.5 FPS on a single H100 for end-to-end inference — faster than Krea-RealTime-14B at 6.7 FPS on the same hardware and competitive with 1.3B distilled baselines [Abstract, §1].
Causal masking (CausVid lineage) “fundamentally changes the inference regime of bidirectional pre-trained models and may limit the achievable quality”; Helios’s Unified History Injection preserves bidirectional inference by separating a clean historical context from the noisy context to be denoised [§1, §3.1.1].
Guidance Attention modulates historical keys with head-wise amplification tokens and skips cross-attention to the history (text semantics are already baked in), strengthening history’s influence on future-frame generation without re-injecting redundant text supervision [§3.1.2, Eq. 1–2].
Three canonical drift modes are named and addressed individually: Position Shift (RoPE indices beyond training horizon) → Relative RoPE constrains temporal indices regardless of target length; Color Shift (saturation/RGB statistics destabilize past a transition point) → First-Frame Anchor always retains the first frame as a global visual anchor; Restoration Shift (blur/noise accumulation from imperfect history) → Frame-Aware Corrupt independently perturbs each historical latent during training with exposure / noise / down-up-sample / clean (per-frame Bernoulli sampling) [§3.2, Fig. 5–6].
Multi-Term Memory Patchification partitions history into short/mid/long-term windows with progressively higher 3-D Conv compression rates (e.g. 1×1×1 → 2×2×2 → 4×4×4); the total token count is kept constant regardless of target video length [§3.3.1, Eq. 3–4, Fig. 7].
Pyramid Unified Predictor Corrector is a multi-scale UniPC sampler — early steps integrate the ODE at low spatial resolution (global structure / color / layout), later steps at full resolution (edges / textures); transitions use nearest-neighbor upsample + noise-covariance correction (PyramidFlow-style); the state buffer is reset at each stage transition because cached UniPC predictions have incompatible shapes across scales [§3.3.2, Eq. 5–9, Fig. 8].
Adversarial Hierarchical Distillation uses only the autoregressive Helios-Base as the teacher (no real-data discard) and generates only a single section per training step — claimed to deliver long-video anti-drifting performance comparable to long self-rollout strategies (Self-Forcing++, Rolling Forcing) without their cost; supports 50 → 3 sampling steps [§3.4.2].
The distillation pipeline closes ODE-distillation init via Staged Backward Simulation across pyramid stages, schedules timestep sampling with a Beta distribution whose parameters follow cosine decay (Dynamic Re-noise) to focus on high-noise early in training, then mixes in an adversarial GAN objective with multi-granularity discriminator branches across DiT layers + approximate R1 regularizer (APT-style) for teacher-independent supervision [§3.4.2, Eq. 10–16].
Three released checkpoints share architecture but differ in sampling regime: Helios-Base (v-prediction + standard CFG + custom HeliosScheduler), Helios-Mid (v-prediction + CFG-Zero* + HeliosScheduler, intermediate), Helios-Distilled (x₀-prediction + HeliosDMDScheduler, “Best Efficiency”) — group offloading shrinks VRAM to ~6 GB on Helios-Distilled [README §“Model Download”].
HeliosBench releases 240 prompts spanning four duration regimes: very short (81 frames), short (240), medium (720), long (1440) [§1].
Infrastructure: Helios fits up to four 14B models within 80 GB GPU memory and trains without parallelism or sharding frameworks, enabling image-diffusion-scale batch sizes [Abstract, §1].

Method

Helios converts a bidirectional pretrained video diffusion model (Wan-family) into an AR continuation generator by concatenating a clean historical context H ∈ ℝ^{B×C×T_h×H×W} with a noisy target context N ∈ ℝ^{B×C×T_n×H×W} and denoising only N — the timestep of H is hard-fixed at 0 throughout sampling. Task switching is implicit: H all zero → T2V, last frame nonzero → I2V, otherwise V2V. Inside the self-attention layers, history keys are amplified head-wise by learned tokens (Guidance Attention) and cross-attention is restricted to N to avoid redundant text re-injection.

The anti-drifting machinery is three independent training-side fixes: Relative RoPE rebases temporal indices into a fixed range regardless of target length (and breaks RoPE×multi-head periodicity that drives repetitive motion, citing LoL); First-Frame Anchor always keeps frame 0 in H during both training and inference; Frame-Aware Corrupt independently perturbs each historical frame in training with one of {exposure / additive noise / downsample-upsample / clean} per a Bernoulli schedule, mimicking realistic history drift.

Efficiency comes from two compressions and a distillation. Multi-Term Memory Patchification splits H into three temporal windows with progressively higher 3D-Conv compression ratios (e.g. 1×1×1, 2×2×2, 4×4×4) — the patch budget is fixed regardless of how many seconds of history are retained. Pyramid UniPC is a coarse-to-fine multi-resolution UniPC sampler with per-stage timestep boundaries, NN-upsampled state transitions, and stage-reset prediction buffers. Adversarial Hierarchical Distillation is a teacher-forced DMD variant: Helios-Base is the only teacher; staged backward simulation iterates intermediate-state predictions across pyramid stages; Beta-cosine timestep schedules focus early training on global structure and later on fine detail; multi-granularity discriminators distributed across DiT layers add a real-data GAN signal on top of the distribution-matching loss, with the discriminator seeing random crops to bound memory.

Results

End-to-end speed: 19.5 FPS on a single H100 GPU for a 14B model; community-tested up to 20.89 FPS on better hardware; ~10 FPS on a single Ascend NPU [Abstract, README “Latest News”].
Speedup vs prior 14B SOTA: Krea-RealTime-14B reaches 6.7 FPS on H100 (and 11 FPS only on B200) and suffers severe drifting per the Helios paper [§2.2].
Claimed 128× speedup for a 14B model while maintaining comparable quality vs the dense-Wan2.2 baseline [§1].
Memory footprint: training fits up to four 14B models in 80 GB GPU memory; group offloading at inference reduces Helios-Distilled to ~6 GB VRAM [§1, README §“Group Offloading”].
Sampling cost: 50 → 3 NFE via Adversarial Hierarchical Distillation; Helios-Distilled documented configurations use pyramid_num_inference_steps_list = [2, 2, 2] (6 total) with guidance_scale = 1.0 and amplified first chunk [§3.4, README §“Inference”].
Generation cadence: 33 frames per chunk; example duration table maps 1449-frame requests to ~60 s of video at 24 FPS or 90 s at 16 FPS [README §“Inference”].
HeliosBench: 240 prompts × {81, 240, 720, 1440}-frame regimes — quantitative comparisons against Self-Forcing++ / LongLive / Rolling Forcing / Krea-RealTime are in the (truncated) experiments section of the technical report and on the project page.
Day-0 ecosystem support: Diffusers, SGLang-Diffusion, vLLM-Omni, Ascend-NPU, Cache-DiT all merged Helios pipelines within ~24h of release [README §“Latest News” 2026-03-04 / 2026-03-06].

Why it’s interesting

Helios is the first paper on the wiki to argue — concretely and at 14B scale — that the entire causal-masking + Self-Forcing-rollout + ODE/DMD-distillation stack that Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation and Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation are trying to fix may be the wrong recipe to begin with: keep the bidirectional model, concatenate clean history + noisy future, denoise only the future. The bet pays off in raw FPS (19.5 vs Krea-RealTime-14B’s 6.7) and removes the train–inference rollout-length dependence that Context Forcing: Consistent Autoregressive Video Generation with Long Context frames as the core forgetting-vs-drifting dilemma. Easy Anti-Drifting (Relative RoPE + First-Frame Anchor + Frame-Aware Corrupt) is a clean, additive answer to the same drift modes that Context Forcing’s Bounded Positional Encoding + Slow/Fast Memory cache and FlowAct-R1’s Memory Refinement attack from very different angles — three distinct architectural answers to one phenomenon, now all on the wiki. Multi-Term Memory Patchification belongs in the same family as Pretraining Frame Preservation in Autoregressive Video Memory Compression (PFP) — both move history into a learned compressor with content-adaptive rate — but Helios skips the pretraining stage and bakes compression directly into the AR fine-tune. Finally, the “no parallelism / no sharding” claim and the 80GB-fits-four-14B-models figure are striking enough that they should be cross-checked against veScale-FSDP: Flexible and High-Performance FSDP at Scale‘s opposing thesis that you actually do want a flexible FSDP framework at 14B+.