Wan-Alpha / Video Generation with Stable Transparency via Shiftable RGB-A Distribution Learner

TL;DR

Wan-Alpha is a text-to-RGBA-video generator built on top of frozen Wan2.1-T2V-14B: it embeds the alpha channel into the existing RGB latent space via a small VAE-side feature-merge block + two LoRA-adapted decoders, then trains a single DoRA module on the Wan DiT with rectified-flow loss. By keeping the alpha distribution disentangled from the RGB distribution in the latent (“shiftable RGB-A”), it preserves the base model’s visual prior, runs in 4 steps without CFG via LightX2V, and reports ~15× faster generation than TransPixeler at higher resolution and frame count. The CVPR 2026 highlight paper claims SOTA on transparency rendering, semi-transparent objects, glowing effects, and hair-strand detail.

Key claims

An RGB-A VAE that encodes the alpha channel into the same latent shape as the base RGB VAE lets a frozen pretrained DiT generate RGBA video without re-training the backbone — only two decoder LoRAs (rank 128) for RGB and alpha are needed, plus one DoRA (rank 32) on the DiT [§2.1, §3.1].
The VAE encoder is duplicated for the alpha branch and fused via a causal-3D feature-merge block built from Wan-VAE’s residual + attention layers; KL penalty is dropped because the model isn’t trained from scratch and arbitrary-variance latents are acceptable [§2.1, Eq. 6].
To prevent RGB-background-color/transparency confusion, the RGB input is hard-rendered with a random color from {black, blue, green, cyan, red, magenta, yellow, white} before encoding [§2.1, Eq. 5].
Training supervises three outputs — pure alpha, soft-rendered (alpha-blended over random color), and hard-rendered (alpha thresholded to 1) — each with an L1 + VGG-perceptual + Sobel-edge composite loss [§2.1, Eqs. 9-15].
VAE is trained on a hybrid corpus of 10 image-matting + 3 video-matting datasets (AIM-500, AM-2K, P3M-10K, DVM, VideoMatte240K, etc.) plus internet videos, totaling 77k training clips after a 95/5 split [§2.1].
The diffusion transformer is fine-tuned with DoRA rather than LoRA on only 437 carefully curated and manually re-captioned samples (301 matting videos + 20 matting images + 116 internet special-effects videos); DoRA gave better semantic alignment than LoRA [§2.2].
Captions are produced in two stages — Qwen2.5-VL-72B generates short+long captions, then humans refine — and all captions are in Chinese; the multilingual Wan base still generates from English prompts [§2.2].
Inference loads only the two VAE decoder LoRAs and the DiT DoRA on top of stock Wan; weights can be merged into the base, so the runtime adds zero extra compute over baseline Wan [§2.2].
Combined with LightX2V acceleration, the model generates a 720×1280 81-frame 16 FPS RGBA clip in 128 seconds in 4 sampling steps without CFG, vs. 32 minutes for the open TransPixeler at lower res and frame rate — a ~15× wall-clock speedup [§3.2].
Limitations: occasional unintended background elements and inaccurate alpha masks with holes/missing regions [§3.4, Figs. 19-20].

Method

The architecture (Fig. 2) splits an input RGBA video into a 3-channel RGB stream and a 3-channel duplicated-alpha stream. The RGB stream is first hard-rendered over a random background color (Eq. 5) to break the encoder’s tendency to confuse “RGB happens to be black” with “alpha is zero”. Both streams pass through the frozen Wan-VAE encoder, after which a learned 3D-causal feature-merge block (Fig. 3) — residual blocks + attention, but with causal temporal convolutions for cheap long-video encoding — predicts a single joint latent matching the original Wan-VAE latent shape (Eq. 6). Decoding uses the same frozen Wan-VAE decoder twice: once with an RGB-LoRA producing the RGB video, once with an alpha-LoRA producing the alpha video (Eq. 7). Training supervises three rendered targets — α, soft-render (Eq. 8 with α∈[0,1]), and hard-render (α thresholded) — each with the L1 + VGG-perceptual + Sobel-edge composite loss (Eqs. 9-15). VAE trains 60k iterations, batch size 2.

For text-to-video (Fig. 4) the base is frozen Wan with umT5 text encoding. Only a rank-32 DoRA is trained on the DiT, using the rectified-flow velocity-matching loss (Eqs. 16-18) on the curated 437-sample dataset. DiT trains 1.5k iterations, batch size 8. At inference, decoder LoRAs and DiT DoRA are merged into the base, so deployment is identical to vanilla Wan — and LightX2V’s 4-step CFG-free sampling drops to 128 s per 81-frame 720×1280 clip.

Results

Quantitative-light, comparison-heavy paper: head-to-head against TransPixeler (CVPR 2025 SOTA), both open (CogVideoX-5B based) and closed (Adobe Firefly).

Speed: 128 s vs. 32 min on the same hardware for one RGBA video, at higher resolution (720×1280 vs. TransPixeler’s lower) and double the frame count (81 vs. 49) and double the FPS (16 vs. 8) — ~15× faster [§3.2].
Quality (qualitative, Figs. 10-16): claimed wins on motion consistency, alpha edge sharpness, semi-transparent rendering (smoke, glow, crystal, force-field hexagons), and fine details (hair strands, fur).
Compute footprint at inference: zero extra parameters over baseline Wan after merging LoRA/DoRA [§2.2].
Coverage: handles photographic-style portraits, animals, semi-transparent objects (bubbles, smoke), particle/glow effects, and Chinese-prompt → English-prompt zero-shot transfer.

Why it’s interesting

This is the first wiki entry for the single-layer-with-alpha video branch of Layered Image/Video Decomposition — every other paper in that cluster targets image-domain RGBA generation (ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation ART, LayerDiffuse) or training-free omnimatte-style video decomposition (OmnimatteZero: Fast Training-free Omnimatte with Pre-trained Video Diffusion Models). Wan-Alpha is what the joint-generation route (LASAGNA-style, Controllable Layered Image Generation for Real-World Editing (LASAGNA)) looks like when ported to a frozen large video DiT: don’t retrain the backbone, encode α into the existing latent, supervise with rendered composites, ship as a LoRA/DoRA bundle. The 4-step LightX2V acceleration also lands it in the same neighborhood as the Diffusion Distillation line — the model is a useful data point that the LoRA-on-frozen-backbone recipe composes cleanly with consistency-style few-step distillation, since the alpha branch inherits both. The bigger open question this paper sharpens is whether RGBA-as-latent-channel-shift (Wan-Alpha) or RGBA-as-region-crop-token (ART) is the right tokenization for multi-layer video — Wan-Alpha solves the BG+single-FG case efficiently but says nothing about K≥3.