MirageLSD: The First Live-Stream Diffusion AI Video Model

TL;DR

MirageLSD is Decart’s Live-Stream Diffusion video model — a frame-by-frame autoregressive video generator that turns a live camera/screen/game feed into a stylized or transformed world stream with under 40 ms per-frame latency on Hopper GPUs at 24 fps, with no fixed clip length. The recipe combines Diffusion Forcing (per-frame denoising) with history augmentation — fine-tuning on corrupted history frames so the model learns to anticipate and correct its own drift — plus Hopper-targeted Mega Kernels, architecture-aware pruning, and Shortcut Distillation to hit the latency budget. It is the second model from Decart after Oasis (the “AI Minecraft” world model) and is the first filed datapoint in the wiki of a truly interactive (sub-frame latency, infinite duration) productized video diffusion system.

Key claims

MirageLSD is positioned as the first system to do infinite, frame-by-frame real-time video diffusion with continuous prompting/editing as the video is being generated, in contrast to prior real-time systems that generate in chunks and therefore incur unavoidable batching latency [§Overview].
Per-frame latency under 40 ms on Hopper GPUs, 24 fps output, no time cap on stream length [§Performance].
“Responsiveness” — the worst-case latency from a new user prompt to the prompt taking effect — is claimed to be ≥16× lower than prior autoregressive video models [§Responsiveness].
Infinite-length generation is achieved by combining two ingredients: (i) Diffusion Forcing (per-frame denoising in the autoregressive rollout, citing Chen et al.) and (ii) history augmentation, where during training the model is fine-tuned on history frames with corruption injected, teaching it to anticipate and correct input artifacts and resist drift [§Method].
The latency budget is reached via custom Hopper Mega Kernels + architecture-aware pruning + Shortcut Distillation; the post explicitly mentions writing GPU assembly to hit the target [§Performance, §Engineering].
Limitations the team flags: short-term memory only (no long-range identity / layout consistency mechanism), and only text-guided global control — no structured spatial control (keypoints, scene annotations) [§Limitations].

Method

LSD is described as a frame-by-frame autoregressive diffusion model: each output frame is denoised conditioned on the input frame, the user prompt, and a finite window of prior generated frames. There are two training innovations on top of the diffusion-forcing baseline. First, the forcing schedule denoises each frame independently rather than denoising a full clip jointly, eliminating the chunk-batching latency tax. Second, history augmentation — during fine-tuning, the conditioning history frames are deliberately corrupted (the post doesn’t fully specify the corruption distribution), so the rollout-time distribution shift between clean teacher-forced history and the model’s own drifting outputs is closed during training rather than at inference. Inference latency is then driven down by Mega Kernels (custom fused kernels on Hopper), architecture-aware pruning (the post is light on detail), and Shortcut Distillation (cited as the few-step distillation lever) to a sub-40 ms/frame budget at 24 fps. The post is a product-launch blog, not a paper — there are no architecture-level diagrams (FLOPs, depth, width), no quantitative benchmarks against prior video generators, and no ablations of the listed ingredients.

Results

The results section of the post is qualitative — a demo video showing live restyling of a webcam stream and game footage, plus a public Mirage web demo (and announced iOS/Android apps). The only quantitative claim against prior work is the 16× responsiveness margin over prior autoregressive video models; the comparison set is not enumerated. Subsequent third-party coverage indicates MirageLSD is later commercialized as an API on Crusoe Cloud and integrated with agent frameworks (Vision Agents + Gemini 3) for voice-driven live restyling. The headline numbers — <40 ms/frame, 24 fps, infinite length — are the only first-party performance specs given.

Why it’s interesting

The wiki has a thick lineage of AR-video drift-fighting recipes (Autoregressive Video Generation §Key claims) — Self-Forcing, Context Forcing, FlowAct-R1, PFP, MALT, Causal Forcing / Causal Forcing++ — and MirageLSD is the productized version of the same idea family: Diffusion Forcing + a noise-augmented-history training trick that closes the train-test rollout-distribution gap. It complements MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation, Context Forcing: Consistent Autoregressive Video Generation with Long Context and Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation by being the first filed evidence that this entire technique class can be pushed to frame-by-frame sub-40 ms latency under heavy distillation + kernel co-design rather than chunkwise generation. It contrasts with FlowAct-R1: Towards Interactive Humanoid Video Generation (chunkwise 3-NFE / 25 fps, structured 4-slot memory bank) — MirageLSD claims a finer-grained per-frame denoiser and trades structured memory for a short context window. As a product, it also slots into World Foundation Models as the second commercial Decart WFM after Oasis, complementing the closed-flagship pattern of Project Genie: Experimenting with infinite, interactive worlds and Introducing Runway Labs — but unlike Genie 3 or Runway’s GWM-1, MirageLSD is specialized for video-to-video transformation, not free-form world rollout.