It Took Me 30 Years to Solve this VFX Problem (CorridorKey)

TL;DR

Corridor Crew’s Niko Pueringer walks through CorridorKey, an open-source neural greenscreen keyer that treats chroma keying as a per-pixel color unmixing problem (recovering straight, un-premultiplied foreground RGB + a linear alpha) rather than a binary segmentation problem. The training data is entirely synthetic: a Houdini + Blender procedural pipeline renders thousands of subjects against green screens in 3D, which gives perfect ground-truth alpha (including the hard cases — hair, motion blur, refractive edges) without any human rotoscoping. The model is paired with an alpha-hint generator (GVM or VideoMaMa) and produces 16/32-bit linear EXRs that drop directly into Nuke / Fusion / Resolve. For a Luma audience the interest isn’t the keyer itself but the synthetic-pair pipeline: a worked example of a generation/edit task where 3D ground truth lets you train a dense regressor without crowdsourced labels.

Key claims

Traditional chroma keyers and “AI roto” segmenters collapse semi-transparent pixels (hair, motion blur, refractive edges) to a binary in/out decision, destroying the unmixed-foreground color needed for a realistic composite [video description; GitHub README §“What it does”].
CorridorKey re-frames chroma keying as an unmixing problem: for every pixel — including ones that are mostly background — the network predicts both the true straight foreground RGB and a clean linear alpha, as if the green screen was never there [GitHub README §“What it does”].
Training data is generated entirely synthetically with a Houdini + Blender procedural pipeline that renders subjects against green screens in 3D, exposing the rendered alpha channel as ground truth — this is what avoids the “manually rotoscope thousands of clips” bottleneck that has historically blocked supervised matting at scale [video, ~12 min mark; pipeline summary in VFXer breakdown].
The synthetic pipeline can deliberately stress-test edge cases (motion blur at different speeds, out-of-focus backgrounds, translucent materials, varied hair, varied green-screen quality) — a curriculum advantage that real-data collection cannot easily replicate [CreativeAI News breakdown].
The model is paired with a separate “alpha hint” generator (GVM = Generative Video Matting from Zhejiang AIM, or VideoMaMa from KAIST CVLAB) that supplies a rough mask; the refiner then unmixes — i.e. the system is a two-stage hint → refine pipeline, not a one-shot matter [GitHub README §“Credits”; EZ-CorridorKey UI flow].
Output is 16/32-bit linear-float EXR (foreground + alpha + premultiplied processed pass) — chosen specifically to slot into Nuke / Fusion / Resolve without conversion or banding [GitHub README §Outputs; CreativeAI News].
The deployed model is small enough to run on 6–8 GB VRAM after development on RTX Pro 6000 (96 GB) hardware [Gigazine writeup].

Method

The system has two coupled components. Component A is an alpha-hint model — third-party generative matting nets (GVM, VideoMaMa) that produce a coarse alpha given the input frame. Component B is the CorridorKey refiner: a transformer-style model (referred to as “GreenFormer” in third-party writeups) that takes the raw frame plus the alpha hint and emits, per pixel, the un-premultiplied foreground RGB and a linear alpha. The refiner is what the synthetic-data pipeline trains.

The training pipeline is the part Paul flagged at the ~12 min mark of the video. The Corridor team (Jordan Allen on the Houdini side) procedurally constructs 3D scenes with subjects in front of green screens, varying material, lighting, camera depth-of-field, motion-blur shutter, hair geometry, and screen contamination. Because the scenes are rendered in 3D, the renderer exposes the perfect per-pixel un-premultiplied foreground colour and the perfect alpha — the (input frame, ground-truth FG, ground-truth alpha) triplets are exact by construction. No crowdsourced labelling, no proxy matting algorithm in the loop. This is the same “use 3D as the oracle for an otherwise-unlabellable supervised problem” pattern that several wiki entries already document for different perception tasks.

Results

The video itself is the primary artifact; reproducible benchmarks are not part of the release. Third-party VFX outlets (Corridor’s own demo, Compositing Academy, VFXer, CreativeAI News) report it cleanly recovers hair, motion blur, and translucent edges where Keylight, Primatte, and “AI Roto” tools collapse to a binary mask. No FID/PSNR-style numbers reported on the project page.
Deployment envelope: 96 GB VRAM RTX Pro 6000 used for development; inference runs on 6–8 GB cards (current-gen gaming GPU) and on Apple M1 [Gigazine].
Adoption signal: third-party forks (EZ-CorridorKey desktop GUI, CorridorKey-Cloud farm) have shipped within a few weeks of release [GitHub forks].

Why it’s interesting

The whole point of Paul’s note is the data pipeline, not the model. CorridorKey is a fielded, working instance of: take a task where the labels are normally collected by paying humans to mask thousands of frames; instead, build a 3D renderer that exposes the labels by construction; train a dense regressor on (rendered input, rendered label) pairs; rely on the renderer’s coverage of edge cases for robustness, not on real-world distribution match. For Luma’s roadmap toward an edit model, this is a directly transferable recipe — any local-edit task whose ground truth is recoverable from a 3D scene (relighting, despill, alpha for layered compositing, segmentation under transparency, deshadowing, defocus removal) can in principle be supervised this way without human annotation.

There are two other Luma-relevant cross-links. First, CorridorKey explicitly uses VideoMaMa: Mask-Guided Video Matting via Generative Prior as one of its two alpha-hint generators — VideoMaMa’s recipe is itself “fine-tune a video diffusion model on synthetic composites, then use it as a pseudo-labeller to bootstrap a feed-forward student,” which is the same synthetic-supervision philosophy applied a level higher. Second, this contrasts cleanly with OmnimatteZero: Fast Training-free Omnimatte with Pre-trained Video Diffusion Models, which gets layer decomposition entirely training-free by exploiting a pretrained video diffusion model’s self-attention — a third route to the same “alpha matte without human labels” outcome, with no rendered data at all.

The synthetic-3D-as-oracle pattern also belongs on Synthetic Training Data alongside the existing entries, but as a new flavour: SpaceTimePilot uses Blender to synthesize a control axis (animation time) that nature can’t provide; NeoVerse simulates degradations on real video; CorridorKey synthesizes the entire (input, label) pair from scratch using 3D rendering as the supervision oracle. Three different reasons to render, three different things the renderer provides.