FIT: A Large-Scale Dataset for Fit-Aware Virtual Try-On

TL;DR

FIT (Fit-Inclusive Try-on) is a synthetic, photorealistic virtual-try-on dataset of 1.06M training triplets and 5K test samples that include precise body and garment measurements — the first VTO dataset that systematically covers “ill-fit” cases (XS body in 3XL shirt, etc.). The pipeline programmatically generates 3D garments with GarmentCode, drapes them via physics simulation on 168 body shapes × 528 poses, then runs a re-texturing model to convert the simulated renders into photorealistic images while strictly preserving geometry. The team also trains Fit-VTO, a FLUX.1-dev MMDiT + LoRA baseline that takes measurements as conditioning embeddings (replacing text embeddings) and reports new SOTA on fit-aware metrics, beating IDM-VTON. Targeted at SIGGRAPH 2026.

Key claims

The dataset is 1,064,824 training + 5,000 test triplets (I_try-on, I_p, I_g, m_p, m_g) covering 168 distinct body shapes (82 men’s + 86 women’s, sizes XS–3XL), 528 body poses, and 158,483 unique top/garment designs ranging from loose to tight [§Dataset, Table 1].
FIT is positioned as the first VTO dataset that simultaneously offers photorealism, ill-fit examples, precise size info, and ground-truth paired-person images at >100K scale — the comparison table shows prior datasets miss at least one of these (e.g., SewFactory has size info at 1M scale but lacks photorealism and ill-fit; ViTON-HD has photorealism but no size info or ill-fit; SIZER and SV-VTO are small) [Table: Dataset comparison].
Data is generated by three coupled stages: (1) GarmentCode + physics-simulation drape, including a “box-mesh realignment” trick that prevents the simulator from failing when garments are cross-draped onto bodies of very different sizes; (2) a re-texturing model that converts synthetic renders into photorealistic images while preserving geometry via composite normal maps (stitched simulated normals + estimated head/feet normals); (3) a paired-person variant of the re-texturing model conditioned on an identity map I_id (garment-masked try-on image), enabling generation of same-person-different-garment pairs for supervised training [§Method, Fig. (a)–(c)].
Fit-VTO uses Flux.1-dev MMDiT + LoRA finetuning; the text-embedding slot is replaced with a learned measurement embedding m_embed computed from concatenated body and garment measurements, layflat-garment and noisy target latents are sequence-concatenated, and person latents are channel-concatenated with the noisy target [§Architecture].
The team claims new state-of-the-art for fit-aware virtual try-on, beating IDM-VTON on fit metrics, demonstrated on both synthetic FIT test images (with measurement conditioning) and real-world VITON-HD images (without measurements) [§Results].

Method

The pipeline solves the central data-collection obstacle — that real garment fit at scale cannot be photographed across the full body-size × garment-size × pose × design grid — by going fully synthetic and then re-rendering to photorealism. GarmentCode produces sewing patterns parameterized by body size; the patterns are draped onto a different target body via physics simulation (with box-mesh realignment to avoid solver failures), and ground-truth measurements are extracted from the resulting mesh. The simulated render is then converted to a photorealistic image by a re-texturing model f_texture, conditioned on (a) a VLM-generated text prompt p describing the garment and person, and (b) a composite normal map I_n that stitches the geometry-accurate normals from simulation with realistic head/feet normals estimated separately. A second model f_paired generates a paired-person image I_p from (I_id, I_n', p'), where I_id is the garment-masked try-on image — this gives the supervised “same person, different garments” pairs that prior datasets lack. Fit-VTO is trained on this corpus with LoRA on Flux.1-dev MMDiT, with measurements injected through a custom embedding that replaces the text-token slot, layflat-garment latents sequence-concatenated to the noise tokens, and person latents channel-concatenated.

Results

The page reports SOTA for fit-aware virtual try-on against IDM-VTON, demonstrated on synthetic FIT-test (measurement-conditioned) and on real-world VITON-HD (no measurements). The headline qualitative claim — illustrated by interactive figures on the project page — is that the same garment design rendered onto the same person at S, M, L, XL conditioning produces visibly different drape and silhouette, whereas prior VTO models hallucinate a well-fit result regardless of input measurements. Quantitative head-to-head numbers are not posted on the project page (paper to come at SIGGRAPH 2026); the claim graph is “Fit-VTO excels on synthetic FIT images with measurement conditioning, as well as real-world VITON-HD images (without measurements).”

Why it’s interesting

The data-pipeline pattern here is the same one tracked on Synthetic Training Data: a sim+VLM+re-render stack with a coupled paired-data generator, applied to a domain (fit-aware try-on) where the real-data ill-fit distribution is essentially unobtainable. The Blender-physics-then-re-texture-to-photorealism recipe is structurally similar to SpaceTimePilot’s CamTime use of Blender to synthesize a control axis that real cameras cannot record (SpaceTimePilot: Generative Rendering of Dynamic Scenes Across Space and Time §3.2.2) — both papers exploit synthesis specifically to supply a control dimension the real-data manifold lacks (here, body/garment size mismatch; there, animation time × camera trajectory). On the modeling side, Fit-VTO is another concrete instance of measurement / structured-condition embeddings replacing the text-prompt slot in an MMDiT — close in spirit to the LoRA-as-task-adaptation recipes filed on Parameter-Efficient Finetuning. For Luma’s layering interest specifically, the relevance is indirect: FIT’s paired generator f_paired (same identity, different garment, geometry-locked via normal maps) is a clean recipe for swapping a single visual stratum in an image while holding the rest fixed — closer to a garment-specialist analog of the masked-region paired training that powers Generative Image Layer Decomposition with Visual Effects (LayerDecomp) than to general layered decomposition.