Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training

TL;DR

Prism is a project page for a joint video-audio generation system whose stated contribution is “Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training” — i.e. a sparse-attention recipe used during training (not just inference) to let a dual-modality video+audio DiT scale to native 2K resolution. The landing page carries one demonstration clip (a dialogue scene with speech, ambient sound effects, and camera-stable medium-shot composition) but no technical report, no architecture details, no benchmarks, no code link, and no explicit affiliation at the time of filing. First-author Shuyuan Tu (francis-rings) is the author line behind StableAnimator (CVPR 2025), StableAvatar, FlashPortrait, and MotionFollower, all of which involve Tencent as a listed affiliation on at least one prior repo. The Slack post framed it as “tencent shadowdropped sota oss videogen, allegedly” — treat both the Tencent attribution and the SOTA claim as unverified until a paper or weights drop.

Key claims

The system is positioned as native 2K joint video+audio generation, not an upsampled or refined output [project page title].
The acceleration mechanism is dynamic sparse attention applied during training, not training-free inference-time sparsification [project page title].
The single demo clip on the landing page shows synchronized speech + ambient audio + continuous camera-stable framing [demo video narration].

Method

Not disclosed on the project page as of filing. The page is a static landing with one embedded demo video and no README, no arxiv link, no technical report, no architecture diagram, and no training-recipe disclosure. The only mechanical signal is in the title: the authors frame their contribution as dynamic sparse attention used during training of a native-2K joint audio-video model — in contrast to the more common pattern of training a dense base at ≤1K and either (a) applying training-free sparse attention at inference (SVG2, Radial Attention, VSA2 lineage) or (b) tacking on a bidirectional 2K refiner post-hoc (DreamX-Creator 1.0’s “Autoregressive 1-Step 2K Refinement”). Whether Prism’s recipe is a trainable-sparse-attention variant along the lines of NSA / VSA / Sparse Forcing, and whether the dual-modality coupling is dual-stream-with-bidirectional-cross-attention (LTX-2 / MOVA / SkyReels-V4) or single-stream Omni-Transformer (daVinci-MagiHuman / MiniMax H3), is not stated.

Results

No quantitative benchmarks are reported on the project page. The single demo clip shows a plausible medium-shot dialogue with speech and ambient store sounds, but there is no VBench / JavisBench / MOVA Arena / Verse-Bench number, no comparison to open-weight baselines (LTX-2, MOVA, SkyReels-V4, MiniMax H3, HappyHorse 1.0), and no FPS / wall-clock / parameter-count disclosure. The “SOTA” framing in the Slack note is Dario’s reading of a product-level demo, not an authors’ claim.

Why it’s interesting

Prism would be the first native-2K entry in the open joint audio-video cluster if the Tencent attribution and open-source release both land. Every other open joint-A+V model in the wiki trains at ≤1080p and either upscales (SkyReels-V4’s two-pass low-res-full + high-res-keyframes; see SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model) or refines (DreamX-Creator’s bidirectional few-step 2K refiner stage). Pushing the native training resolution up means the sparse-attention mechanism has to work during training at long sequence lengths, which is a strictly harder problem than the training-free sparse-attention lineage (Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation, Radial Attention: O(n log n) Sparse Attention with Energy Decay for Long Video Generation) can address — and it sits in the trainable-sparse-attention family tracked by VSA / FastWan / Sparse Forcing on the video-only side (FastWan: Generating a 5-Second Video in 5 Seconds via Sparse Distillation, Improving Video Sparse Attention with Fine-grained Router and Sparse Rebasing). Prism’s claim, if validated, would collapse those two layers (native-2K joint training + sparse-attention efficiency) into one recipe. Until a paper, weights, or at minimum a README lands, the entry is a bookmark — the demo is credible but every mechanical claim is in the title alone.