MiniMax H3 — Open-Weights Omni-Reference Video-Audio Generation Model Card

TL;DR

Hugging Face model card for MiniMax H3 — the actual open-weights drop promised at the July 31 launch tweet MiniMax H3 — Omni-Reference open-weights video generation model. H3 is an omni-modal generative system that jointly emits video and 32 kHz stereo audio up to 15 s at 768p (locally) or 2K (via the H3-Regenerate-2K API pass), from free-form text + image + video + audio references. The generator is a 33B-parameter dense, single-stream Omni-Transformer over packed multimodal tokens — with modality-specific AdaLN branches accounting for ~13B of those parameters — using Qwen3-VL-32B as the frozen H3-Encoder, a f16t4d24 H3-VisualVAE, and a stereo H3-AudioVAE at 40 Hz. Two task-specific CFG-distilled checkpoints ship at BF16 (FL2VA and Ref2VA), with day-0 support for vLLM, diffusers, ComfyUI, and SGLang. The H3-Context-IR prompt-refinement stage and the H3-Regenerate-2K 2K-upscaling stage are not open-sourced — they remain hosted API components. Native sparse attention is trained in but withheld from the initial open release.

Key claims

H3 generates joint video + stereo 32 kHz audio at up to 15 s, 24 FPS, with a default short-side of 768 px, extending to 2K via the H3-Regenerate-2K in-context upscaling module [System Overview, table].
The complete H3 system is decomposed into three stages: H3-Context-IR (multimodal prompt parser producing a “Context Intermediate Representation”), H3-Base (768p video+audio generator), and H3-Regenerate-2K (in-context 2K regeneration). Only H3-Base is open-sourced in the initial release; H3-Context-IR and H3-Regenerate-2K remain hosted API components [System Overview, §H3-Context-IR, §H3-Regenerate-2K].
H3-Base is a dense single-stream Omni-Transformer — not a dual-stream video+audio DiT — over a packed multimodal token sequence with modality-specific input/output layers and modality-specific AdaLN, but no modality-specific attention or FFN structure [§H3-Omni-Transformer].
Total H3-Omni-Transformer parameters: 33B dense, of which ~13B live in AdaLN branches. AdaLN modulation outputs can be precomputed and cached, so those ~13B parameters need not be loaded at inference-only deployment [§H3-Omni-Transformer].
Text encoder is the full pretrained Qwen3-VL-32B, with the H3-Encoder tapping its 50th-layer hidden states as conditioning for the Omni-Transformer [§H3-Encoder].
Visual latents come from H3-VisualVAE (f16t4d24: 16× spatial, 4× temporal, 24 latent channels) followed by a 1×2×2 (t,h,w) patchifier before the transformer, giving effective 32× spatial and 4× temporal downsampling. Audio latents come from H3-AudioVAE, which independently encodes each of the two stereo channels of 32 kHz audio to a 40 Hz token stream and re-combines them at decode [§H3-VisualVAE, §H3-AudioVAE].
Positional encoding uses a three-dimensional MM-RoPE over (t, h, w) — the temporal axis unifies video-frame and audio-token positions in a common temporal frame [§H3-Omni-Transformer].
Native sparse attention is introduced in the final stage of training to reduce long-sequence cost, but the initial open-source release ships full-attention inference only; the sparse-attention implementation is deferred to a later update [§H3-Omni-Transformer].
Two task-specific checkpoints ship, both CFG-distilled, both BF16: H3-Base-FL2VA (text ± first/last frame → video+audio) and H3-Base-Ref2VA (multimodal reference-to-video+audio with ≤9 images / ≤3 video clips of 2–15 s / ≤3 audio clips of 2–15 s / ≤12 total files) [table “Model Variants and Input Specifications”, table “Local Deployment of H3-Base”].
Ref2VA supports mixed-modality references where audio references must be paired with image or video references and cannot be the sole input [table “Model Variants and Input Specifications”].
Serving support is packaged day-0 across vLLM, diffusers, ComfyUI (with vendor-supplied R2V and T2V templates), and SGLang; the model card gives copy-paste sglang serve invocations with --num-gpus 4 --ulysses-degree 4 --performance-mode speed and per-variant --model-variant {fl2va,ref2va} flags [§Local Deployment of H3-Base, §Sglang Deployment].
2K quality via H3-Regenerate-2K is architecturally an in-context regeneration rather than a dedicated super-resolution head: the 768p latent + original multimodal context are fed back through the H3 base to re-render at 2K, framed as an instance of task generalization within a single model [§H3-Regenerate-2K].
Reproducibility artifacts on the model card: for each of T2VA, FL2VA, Ref2VA, and Full-2K T2VA / I2VA / Ref2VA, the card ships the exact request JSON (with the integrated_multimodal_description / overall_soundscape / non_diegetic_music prompt-structure convention), a runnable script, and reference MP4 outputs [§Reproducible 768p cases, §Full 2K-Workflow].
The Ref2VA case in the documentation demonstrates that H3 supports a specific “video editing + audio reference + audio reuse” edit pattern: given a source video and a spoken-voice audio reference, H3 can animate the source subject’s face to speak new dialogue while retaining background music from the original audio track and copying the timbre from the reference [§case-Ref2VA request JSON].
The public prompt-structure convention exposes three separately-authored fields — integrated_multimodal_description, overall_soundscape, and non_diegetic_music — as the operational contract between H3-Context-IR and H3-Base [§case-T2VA request JSON].
The distributed checkpoint is a single self-contained HF-style repo containing processor/, tokenizer/, text_encoder/, transformer/, visual_vae/, and audio_vae/ subdirectories per variant [§Local Deployment of H3-Base].

Method

H3-Base encodes each modality with a modality-specific tower (H3-Encoder for text — a Qwen3-VL-32B pass tapped at layer 50; H3-Encoder + H3-VisualVAE for images and video; H3-AudioVAE for audio), then packs all latents into a single multimodal sequence with modality-specific input/output heads and a shared H3-Omni-Transformer trunk. RoPE is three-dimensional over (t, h, w), giving a common temporal index across video and audio latents at their respective rates (video after 4× temporal downsampling and 1×2×2 patchification; audio at 40 Hz per channel).

The Omni-Transformer trunk is 33B dense with no modality-specific attention or FFN branches, but modality-specific AdaLN modulation on the order of ~13B parameters. The two open-sourced checkpoints (FL2VA and Ref2VA) are CFG-distilled BF16 weights. Sparse attention is trained in but not shipped: initial open release runs full attention.

At inference, the complete “official” workflow chains H3-Context-IR (hosted API) → local H3-Base 768p → H3-Regenerate-2K (hosted API), with the model card documenting the exact SGLang endpoint config + MiniMax API calls to reproduce this pipeline. A local-only workflow is also documented: H3-Base 768p from a Prompting-Guidance-format prompt, without 2K refinement.

Results

The model card ships no benchmark numbers — no VBench, no MOVA Arena / JavisBench / Verse-Bench entries, no head-to-head against LTX-2 / SkyReels-V4 / MOVA / Ovi. Validation instead comes from reproducible reference cases: for each of T2VA, FL2VA, Ref2VA (768p) and Full-2K T2VA / I2VA / Ref2VA (2K), the card provides the exact request JSON, a runnable script, and both a “local-deployment” MP4 and a “direct-API” MP4 for side-by-side comparison. This is the disclosure format — the release is a working-artifact drop rather than a numeric leaderboard claim.

Sizing benchmarks that do appear: 33B-total dense Omni-Transformer (~20B active at inference-only if AdaLN caching is applied). 768p / 24 FPS / up to 15 s per generation. Stereo 32 kHz audio at 40 Hz latent per channel. Ref2VA input cap: 12 files across ≤9 images + ≤3 video clips (2–15 s each) + ≤3 audio clips (2–15 s each, must be paired with a visual reference).

Why it’s interesting

First single-stream open joint-T2AV model at scale. The Joint audio-video generation cluster’s TL;DR names dual-stream DiT with bidirectional cross-attention as the consensus recipe — LTX-2 (LTX-2: Efficient Joint Audio-Visual Foundation Model), MOVA (MOVA: Towards Scalable and Synchronized Video-Audio Generation), SkyReels-V4 (SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model) are all dual-stream. H3 breaks that with a dense single-stream 33B transformer over packed multimodal tokens at the frontier open-weights tier, joining daVinci-MagiHuman (Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model (daVinci-MagiHuman)) — currently the only other single-stream open joint-AV filed paper — as the second data point for the single-stream family, but at a much larger scale and with public 2K reference outputs. Direct answer to the concept’s open question about “capacity allocation across modalities”: H3 puts no modality-specific attention or FFN capacity at all, only modality-specific AdaLN.
Extends the Open foundation-model releases pattern from LLMs to open video at the frontier. MiniMax’s prior open-weights track was LLM/agent only — M2 / M2.5 / M2.7 / M3 (MiniMax M3 announcement — first open-weights model combining coding, agentic, and natively-multimodal frontier capabilities, MiniMax-M2: a Mini model built for Max coding & agentic workflows, MiniMax-M2.5, MiniMax M2.7: Early Echoes of Self-Evolution). H3 is the first video entry and it ships with the standard multi-variant + multi-backend + reproducible-cases packaging (BF16 FL2VA + Ref2VA, vLLM + diffusers + ComfyUI + SGLang, per-task request scripts and reference MP4s). Notable *anti-*packaging move: the H3-Context-IR prompt-refiner and the H3-Regenerate-2K 2K pass are not open-sourced, so the API is the only route to the marketed 2K quality — a hosted-API-around-open-base pattern that the cluster hasn’t yet named.
Concrete counterpoint to Kling 2.6 opacity. The Joint audio-video generation page currently flags Kling 2.6 (Kling Video 2.6 — first Kling AI model with native audio (announcement)) as a closed native-audio launch with zero technical disclosure. H3 provides the opposite datapoint from another frontier lab: full architecture, exact parameter counts (33B / ~13B AdaLN / Qwen3-VL-32B encoder / f16t4d24 VAE / 40 Hz audio), variant sizing, sparse-attention roadmap, and reproducible reference outputs — all as day-zero open disclosure.
Re-resolves the earlier “no HF link at launch” flag on MiniMax H3 — Omni-Reference open-weights video generation model. The launch tweet’s community note about the missing HF link was captured on the tweet page as an open item; the HF model card is the direct answer, one day after the tweet.