Into the Dreamverse: Vibe Directing in FastVideo

TL;DR

Dreamverse is a prototype interface from the Hao AI Lab FastVideo team that operationalizes vibe directing — a chat-driven workflow where users steer video generation through many small natural-language revisions instead of one long prompt. It is built directly on top of the team’s sub-5-second 1080p inference stack (FastVideo: Create a 5s 1080p Video in 4.5s on a Single GPU) and produces 30-second scenes as chained 5-second 1080p clips while a chat window stays open for live re-direction. The system is paired with K2-V2 (a 360-open foundation model from the Institute for Foundation Models) for the language side, and is positioned as the UI counterpart to the FastVideo systems story: when generation gets faster than viewing, the ideation loop closes and inference “feels like directing your own scenes” rather than waiting on prompts.

Key claims

Real-time AI video generation enables a new interaction paradigm — vibe directing — defined by analogy with vibe coding: many small natural-language revisions to steer a scene, not one large prompt [§TL;DR; §“What is vibe directing”].
The interface generates 30-second scenes as a chain of 5-second 1080p clips, each produced in ~4.55 s on a single GPU, with a chat window kept open for ongoing user direction [§“Dreamverse”; §“Powered by K2-V2”].
A faster feedback loop is claimed to improve creative outcomes independently of base-model quality: “Better creative work comes from a faster feedback loop, not just a better model” [§“Why vibe directing matters”].
The language side of Dreamverse runs on K2-V2, described as a 360-open foundation model from IFM with general-purpose language capabilities, long-context understanding, and tool-augmented workflows [§“Powered by K2-V2”].
Acknowledged stack collaborators: NVIDIA (compute / Dynamo backend integration), Institute for Foundation Models, MBZUAI, and Lightricks (LTX-2 model open-sourced upstream of FastVideo) [§“Acknowledgement”].
Dreamverse code is not yet pushed to the public FastVideo branch as of the post’s publication [§Note at end].

Method

Dreamverse is a UI + orchestration layer rather than a new model. The underlying generation engine is FastVideo’s real-time inference stack — NVFP4-quantized linear layers, SM100/SM103-tuned 3D spatiotemporal attention kernels, end-to-end graph/kernel fusion across prompt-encode → latent-prep → denoise → decode, and an optimized ffmpeg I/O path — which delivers a 5 s × 1088×1920 × 24 FPS clip with audio in ~4.55 s on a single B200 GPU (FastVideo: Create a 5s 1080p Video in 4.5s on a Single GPU). On top of that, Dreamverse:

Chains successive 5-second clips into a 30-second scene, treating the running scene as the conditioning context for each subsequent generation (an interactive-loop variant of streaming AR video — same regime as Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation, Helios: Real Real-Time Long Video Generation Model, and Xmax X1 — Real-Time Interactive Video Model (Product Announcement)).
Exposes a chat window that runs K2-V2 as the directing-language interface — users issue commands (“keep the subject, change the camera angle”, “slow the camera”, “continue the scene”) in natural language rather than rewriting the full prompt.
Serves the prototype as a public demo at dreamverse.fastvideo.org.

The post explicitly contrasts this with the dominant single-shot, one-big-prompt workflow of Sora-class systems, framing the latter’s 1–2 minute per-clip latency as the binding constraint that prevents iterative direction.

Results

No quantitative results, ablations, or user studies are reported. The post is a prototype + thesis announcement.
Latency claim is inherited from the underlying FastVideo 1080p stack — ~4.55 s for a 5 s 1088×1920 24 FPS clip on a single B200 — and not re-measured for the Dreamverse loop itself.
30-second-scene-from-5s-chains is asserted as a product capability; no consistency, drift, or subject-identity numbers are reported.
K2-V2’s contribution to the system (latency budget, prompt rewriting quality, vs. an alternative LLM router) is not quantified.

Why it’s interesting

Dreamverse pins down what an inference-stack win (sub-5-second 1080p) actually buys at the interface layer — and the answer the post gives is “a new interaction paradigm,” not just “the same UX but faster.” That makes it the explicit UX counterpart to FastVideo: Create a 5s 1080p Video in 4.5s on a Single GPU, which only delivered the systems story. It also sits next to Xmax X1 — Real-Time Interactive Video Model (Product Announcement) (Xmax X1’s real-time interactive video model), Helios: Real Real-Time Long Video Generation Model (Helios’s algorithmic real-time long-video), and the Causal Forcing lineage (Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation) as another data point in the same “what changes when video generation gets fast enough to be interactive?” thesis — but Dreamverse is the only one of those that ships a chat-driven directing UI as the headline artifact rather than the underlying model. The “vibe directing as the analog of vibe coding” framing is rhetorically sharp but undermeasured: there is no user-study evidence that the iterative workflow actually produces better videos than the single-prompt workflow at matched total time.