Qwen-Image-Layered: Towards Inherent Editability via Layer Decomposition

TL;DR

Qwen-Image-Layered is an end-to-end diffusion model that decomposes a single RGB image into multiple semantically disentangled RGBA layers, in a single forward pass with a variable number of layers (trained up to 20). Built on Qwen-Image, the model is adapted via three ingredients: an RGBA-VAE that shares a latent space between RGB and RGBA, a VLD-MMDiT block that uses a Layer3D RoPE (x, y, layer index) to discriminate between any number of output layers, and a 3-stage curriculum that walks the base model from text-to-RGB → text-to-RGBA → text-to-multi-RGBA → image-to-multi-RGBA. The decomposed layers enable consistency-preserving edits — recolor / replace / delete / move / resize one layer while every other pixel is exactly untouched — and the same backbone can be run as a text-to-multilayer generator (Qwen-Image-Layered-T2L). On Crello it beats LayerD by ≥10 points of Alpha soft IoU at every layer-merge tolerance.

Key claims

A single end-to-end MMDiT can decompose an image into a variable number of RGBA layers in one pass; on Crello, Qwen-Image-Layered-I2L hits RGB L1 0.0594 / Alpha soft IoU 0.8705 at zero layer-merge tolerance, vs LayerD’s 0.0709 / 0.7520 [Table 1].
An RGBA-VAE that encodes both RGB and RGBA into one shared latent (4-channel, kernel-init trick that copies pretrained RGB weights and zeros the new α weights/bias) eliminates the input/output distribution gap that separate-VAE prior work suffered from [§3.1].
Layer3D RoPE — splitting position channels into (x, y, layer-index) with conditioning images assigned layer = -1 — is what lets a single MMDiT block discriminate between any number of layers; ablating it collapses the model to undifferentiated layers (RGB L1 0.2809 / Alpha soft IoU 0.3725 at tolerance 0) [Table 2, §3.2].
Multi-Modal attention concatenating (text condition, conditioning image latent, target layer latents) suffices to model both intra-layer and inter-layer interactions without the explicit inter-/intra-layer attention scaffolding used by LayerDiff/DreamLayer [§3.2, Fig. 3].
A 3-stage curriculum (text-to-RGB → text-to-RGBA → text-to-multi-RGBA → image-to-multi-RGBA, 500K + 400K + 400K steps) is necessary: removing multi-stage training drops Alpha soft IoU from 0.8705 to 0.6504 at tolerance 0 [Table 2, §3.3].
The same model serves as a text-to-multilayer generator (Qwen-Image-Layered-T2L): predicting the final composite and its transparent layers jointly (ART-style) propagates information between composite and parts, and qualitatively beats ART on semantic coherence (e.g. ART misses bats / cats that the T2L variant renders) [§4.4.3, Fig. 7].
Combining Qwen-Image-T2I + Qwen-Image-Layered-I2L (decompose what T2I generates) outperforms the direct T2L route on aesthetics and text alignment — the I2L decomposer can ride on top of any strong text-to-image base [§4.4.3].
The RGBA-VAE itself sets a new bar for transparent-image reconstruction: PSNR 38.82 / SSIM 0.9802 / rFID 5.31 / LPIPS 0.0123 on AIM-500, beating LayerDiffuse-SDXL, AlphaVAE-SDXL, and AlphaVAE-FLUX on all four metrics [Table 3].
Real-world PSD files, filtered + spatially-non-overlapping-layer-merged + auto-captioned with Qwen2.5-VL, are a viable replacement for synthetic data (MULAN) and simple-design data (Crello) — both of which lack complex layouts and semi-transparent layers [§4.1, Fig. 4(a)].
Decomposition supports recursive application: any output layer can itself be re-decomposed, enabling “infinite” depth refinement (model card showcase).

Method

The model decomposes an input RGB image $I$ into $K$ RGBA layers $\{(C_i, \alpha_i)\}$ that alpha-blend back to the input. Three modifications to Qwen-Image make this work:

RGBA-VAE. The first encoder conv and last decoder conv are extended from 3 to 4 channels. Pretrained RGB weights are copied into channels 1-3; the new α weights/bias are initialized so the alpha output starts at 1 (opaque), keeping RGB reconstruction intact at init. Trained on both RGB (α=1) and RGBA data with the original Qwen-Image VAE losses (reconstruction + perceptual + KL regularization). The crucial property: RGB inputs and RGBA outputs now live in one shared latent space, encoded per layer with no cross-layer compression.

VLD-MMDiT. Each block uses Multi-Modal attention that concatenates three sequences — text condition tokens, the conditioning RGB latent (image-to-layers task), and the target multi-layer latent — and lets the attention model both within-layer and across-layer relations directly. Positional encoding is Layer3D RoPE: the channel dimension is split into (x, y, layer-index), echoing Qwen-Image’s MSRoPE but adding a layer axis. The conditioning RGB image is tagged with layer-index = -1, separating it from the positive layer indices used by output layers (and from indices used by other task heads like text-to-multilayer generation).

Multi-stage training. Stage 1 swaps in the RGBA-VAE and trains jointly on text-to-RGB and text-to-RGBA (single-image, gets the model used to transparency). Stage 2 adds text-to-multi-RGBA — the model now predicts the composite and the layers jointly, à la ART, learning to use the new layer dimension. Stage 3 adds the conditioning image input and trains the image-to-multi-RGBA decomposition task. 500K / 400K / 400K Adam steps at LR 1e-5. Max layers = 20.

For the I2L head, flow-matching (rectified flow) is used: noise + a logit-normal-sampled timestep produce the intermediate state $z_t$ , and the model regresses the velocity. The conditioning image latent participates in attention as an extra (layer = -1) sequence rather than via cross-attention injection.

Results

Crello decomposition (Table 1). vs LayerD at layer-merge tolerance 0 / 1 / 2 / 3 / 4 / 5:
- RGB L1: 0.0594 / 0.0490 / 0.0393 / 0.0377 / 0.0364 / 0.0363 (Qwen-Image-Layered-I2L) vs 0.0709 / 0.0541 / 0.0457 / 0.0419 / 0.0403 / 0.0396 (LayerD).
- Alpha soft IoU: 0.8705 / 0.8863 / 0.9105 / 0.9121 / 0.9156 / 0.9160 vs 0.7520 / 0.8111 / 0.8435 / 0.8564 / 0.8622 / 0.8650.
- Two segmentation+Hi-SAM baselines (VLM Base + Hi-SAM; YOLO Base + Hi-SAM) trail both, confirming that end-to-end joint decomposition beats segment-then-inpaint pipelines both on color fidelity (RGB L1) and especially alpha quality (~12-point Alpha IoU gap at zero merge tolerance).
Ablation (Table 2). Removing Layer3D RoPE collapses Alpha soft IoU to 0.3725; further removing RGBA-VAE drops to 0.5844; removing multi-stage training drops to 0.6504. Each component contributes 5-50 IoU points.
RGBA reconstruction (Table 3, AIM-500). PSNR / SSIM / rFID / LPIPS = 38.82 / 0.9802 / 5.31 / 0.0123 vs AlphaVAE-FLUX 36.94 / 0.9737 / 11.79 / 0.0283 — i.e. ~2-point PSNR and >2× lower rFID over the previous best.
Editing UX (Fig. 6). Compared with Qwen-Image-Edit-2509, the layered approach (decompose → manual layer edit → recomposite) handles layout / resize / reposition instructions that Qwen-Image-Edit-2509 fails on, and avoids the pixel-level shifts that global editors introduce in nominally-unchanged regions.
Multilayer synthesis (Fig. 7). Qwen-Image-Layered-T2L produces semantically coherent multilayer compositions where ART drops entities (bats, cat); chaining Qwen-Image-T2I → Qwen-Image-Layered-I2L further improves aesthetics and prompt alignment.

Why it’s interesting

This is the natural successor to the layered-decomposition lineage already on the wiki: it inherits the gray-background-encoded-RGBA + 3D-RoPE-over-(x,y,layer) + jointly-predict-composite-and-layers playbook from ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation (which is the strongest joint-generation prior baseline here), but flips ART’s generation-first orientation into a decomposition-first one — with a clean single-forward-pass variable-layer head where prior decomposers (Generative Image Layer Decomposition with Visual Effects (LayerDecomp), OmnimatteZero: Fast Training-free Omnimatte with Pre-trained Video Diffusion Models) either split BG / FG only or recurse. It also directly supersedes Qwen-Image-Layered-Control: text-guided layer extraction — same Qwen-Image base, same goal — replacing that community work’s “swap output head + LoRA-finetune on a graphics dataset” retrofit with a from-scratch multi-stage training scheme that explicitly trains a new RGBA-VAE rather than reusing the RGB one. Whether the principled recipe also fixes that retrofit’s “photographic complex lighting” failure mode is a live empirical question the paper doesn’t directly evaluate; the PSD training set is design-heavy.