EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing

TL;DR

EffectErase is a CVPR 2026 framework that jointly trains video object removal and insertion on a paired dataset (VOR) of ~60k triplet videos — half synthetic (Blender renderings of 3D scenes + objects + animations with designed camera trajectories) and half real captures (with Ken Burns camera motion) — each triplet containing a clip with the object, a clip without it, and the object mask. The model is a DiT with an Adaptor that fuses the removal/insertion pair latents with noise, conditioned on a Task-Aware Region Guidance prompt; cross-attention maps from all DiT blocks are max-pooled into a single affected-region heatmap supervised by an explicit Effect Consistency loss (L_EC) that forces both directions of the task to attend to the same physical region. The headline capability is removing or inserting an object together with all its visual effects — shadows, lighting changes, reflections, and scene deformations (e.g. removing a boat also removes its wake).

Key claims

The dataset (VOR) is a 60k-pair hybrid of (a) Blender-synthesized scenes with public 3D assets, designed natural object and camera trajectories, and (b) real-world captures across diverse scenes/object categories, with Ken Burns simulating camera motion; SAM2 segments and human annotators clean masks [§Dataset Pipeline].
Training the same model on a paired removal+insertion task (rather than removal alone) gives the model a self-consistent signal about which pixels belong to the object plus its effects, because both directions must agree on the affected region [§Method].
Cross-attention in each DiT block uses fused image features as Query and a Task-Aware Region Guidance prompt embedding as Key/Value, so the attention map per block surfaces which regions are tied to the (task, object) — these are aggregated across blocks via max pooling into a maximal-activation map [§Method].
The Effect Consistency loss L_EC supervises the aggregated attention map so removal and insertion converge to the same affected-area mask — i.e. the model is explicitly trained to attribute shadows / reflections / deformations to the object [§Method].
A single trained checkpoint switches between removal and insertion at inference by changing inputs, with no separate head [§Method].
Side-by-side comparisons on WILD_ENV scenes show stronger background completion and effect removal than ObjectClear, OmniPaint, ProPainter, DiffuEraser, VACE, Minmax-Remover, and ROSE [Demo gallery].

Method

EffectErase is built around a DiT trained with two paired latent streams. During training, a “before” video (object present) and an “after” video (object removed) are both VAE-encoded; an Adaptor module fuses these two latent streams with the noise input that the DiT denoises. Conditioning is supplied via a Task-Aware Region Guidance prompt embedding (a text/role embedding that distinguishes “remove” from “insert” and identifies the target object). Inside each DiT block, the fused image tokens act as Query and the prompt embedding acts as Key/Value, so each block produces a (token × prompt) cross-attention map. These maps are aggregated across all blocks; max pooling yields a single maximal-activation heatmap over the spatial-temporal token grid. The Effect Consistency loss L_EC compares this heatmap to a target region (object + its effects) and is the mechanism that ties removal and insertion to the same attended area, regardless of which direction the model is being trained for at the step. At inference, the user supplies one video + an object mask, plus the task flag (remove or insert), and the same model runs in either direction.

The training data (VOR) is a 60k-clip hybrid. The synthetic half is rendered in Blender from 3D environments, objects, and animations sourced from public collections, with designed natural object trajectories and camera trajectories. The real half is captured across diverse environments and object categories using physical cameras, then post-processed with a Ken Burns effect to simulate camera motion. SAM2 produces initial segmentations; human annotators clean and refine. Each VOR sample is a triplet: (clip with object, clip without object, object mask).

Results

The project page surfaces qualitative comparisons only — quantitative numbers are presumably in the CVPR paper PDF. On the WILD_ENV001_00024 and WILD_ENV003_00088 demo scenes, EffectErase is compared against:

Object-aware removers: ObjectClear, OmniPaint.
Video inpainters: ProPainter, DiffuEraser, VACE.
Recent effect-aware removers: Minmax-Remover, ROSE.

In each split-screen comparison the prior methods leave residual shadows, reflections, or wake artifacts where EffectErase produces a clean background. No FID / PSNR / VBench numbers are stated on the page.

Why it’s interesting

EffectErase converges with OmnimatteZero: Fast Training-free Omnimatte with Pre-trained Video Diffusion Models on the same underlying claim — that a video DiT’s attention maps already localize the “object + its visual effects” region, and that this region is the right unit of work for removal — but takes the opposite engineering route. OmnimatteZero is training-free, reads attention from a frozen LTX-Video, and isolates the foreground by latent subtraction. EffectErase trains a dedicated DiT and supervises the attention map directly via the L_EC loss, paying training cost up-front in exchange for a sharper, learned attention prior plus the symmetric insertion capability. The two together pin down a useful design axis for Layered Image/Video Decomposition: how much of the work goes into the prior (frozen, read-out) versus into the head (trained, supervised). EffectErase also complements Generative Image Layer Decomposition with Visual Effects (LayerDecomp) (LayerDecomp) on the data side — both papers solve the missing-supervision problem with a synthetic+real hybrid, but EffectErase commits to video and to paired before/after triplets at 60k scale, where LayerDecomp’s photographic counterfactuals are small. Joint training of removal + insertion is the most distinctive piece; it’s a structurally cleaner answer to “where does the supervision come from?” than building a one-way removal dataset.