Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material

TL;DR

Hunyuan3D 2.1 is Tencent’s image-to-3D asset system, framed as the first fully open-source pipeline that produces production-ready PBR-textured meshes. It splits cleanly into Hunyuan3D-DiT (flow-matching shape generation in a 3DShape2VecSet-style ShapeVAE latent) and Hunyuan3D-Paint (a multi-view diffusion model that jointly emits albedo, metallic, and roughness maps). The release ships model weights, training and inference code, and the data pipeline — explicitly positioned as the “Stable Diffusion / LLaMA / HunyuanVideo equivalent” missing from the 3D ecosystem.

Key claims

Hunyuan3D-DiT beats Trellis, TripoSG, Step1X-3D, Craftsman 1.5, Michelangelo, and Direct3D-S2 on ULIP-T/I and Uni3D-T/I shape-similarity metrics (Uni3D-I 0.3213 vs the next-best Step1X-3D 0.3195) [Table 1].
Hunyuan3D-Paint beats SyncMVD-IPA, TexGen, and Hunyuan3D-2.0 on CLIP-FID, CMMD, CLIP-I, and LPIPS for texture synthesis (CLIP-FID 24.78 vs Hunyuan3D-2.0 26.44) [Table 2].
ShapeVAE uses dual point-cloud sampling (uniform surface + importance-sampled high-curvature regions, ~50/50 split) with Farthest Point Sampling to produce a variational latent of up to 3072 tokens, supervised by MSE-on-SDF + KL [§3.1.1].
Hunyuan3D-DiT is a 21-layer transformer that conditions on DINOv2-Giant features (background-removed, resized, centered, white-padded), with cross-attention for image conditioning and an MoE layer for representation capacity, trained with flow matching on an affine OT schedule [§3.1.2].
Hunyuan3D-Paint is built on Stable Diffusion 2.1 (Zero-SNR init), uses a parallel dual-branch UNet for albedo and metallic-roughness, and propagates albedo reference-attention outputs into the MR branch to enforce spatial alignment between the two material channels [§3.2, Fig. 4].
Cross-view consistency in the multi-view PBR diffusion is enforced by 3D-Aware RoPE — downsampled 3D coordinate volumes injected additively into hidden states at each UNet hierarchy level — rather than the standard image-plane RoPE [§3.2].
An illumination-invariant training strategy forces the model to output light- and shadow-free albedo by enforcing consistency loss between two renderings of the same object under different lighting [§3.2].
Training data: 100K+ textured/untextured meshes (ShapeNet, ModelNet40, Thingi10K, Objaverse, Objaverse-XL) for shape; 70K+ human-curated high-quality assets from Objaverse-XL for texture, rendered at 4 elevation angles × 24 azimuths × 512² [§2].
Hunyuan3D-Paint training runs ~180 GPU-days with AdamW @ 1e-5, 2000 warmup steps [§3.2].

Method

Hunyuan3D 2.1 is a two-stage image→textured-mesh pipeline. Stage 1 — Shape: a 3DShape2VecSet-style variational encoder maps a surface+importance-sampled point cloud (with normals) to a continuous latent of up to 3072 tokens; a 21-layer DiT conditioned on DINOv2-Giant features denoises that latent under a flow-matching objective with affine OT scheduling; a perceiver-decoder converts the latent back to an SDF, then marching cubes produces a watertight mesh. The autoencoder loss is MSE-on-SDF + KL; multi-resolution training varies latent length up to the 3072 cap.

Stage 2 — Texture: the generated (or user-supplied) mesh is fed to Hunyuan3D-Paint, a multi-view diffusion model initialized from Stable Diffusion 2.1 Zero-SNR. Geometry-rendered normal maps and CCM (canonical coordinate maps) are concatenated with the latent noise; a reference image is injected via ReferenceNet. A parallel dual-branch UNet generates albedo and metallic-roughness maps, with spatial-aligned multi-attention modules (self-attn + multi-view attn + reference attn) on both branches; albedo reference-attention outputs are propagated to the MR branch to align the two channels in space. 3D-Aware RoPE injects downsampled multi-resolution 3D-coordinate encodings into the multi-view attention to suppress seams. An illumination-invariant consistency loss across two lighting conditions of the same asset enforces light-free albedo. Outputs are albedo + roughness + metallic maps at multiple viewpoints, baked into a textured mesh adhering to the Disney Principled BRDF.

Results

Shape generation (Table 1). Hunyuan3D-DiT vs Michelangelo / Craftsman 1.5 / TripoSG / Step1X-3D / Trellis / Direct3D-S2 on ULIP and Uni3D similarity to input image and caption: ULIP-T 0.0774, ULIP-I 0.1395, Uni3D-T 0.2556, Uni3D-I 0.3213 — first place on all four metrics, ULIP-I gap of +0.01 over Craftsman 1.5 [Table 1].
Texture synthesis (Table 2). Hunyuan3D-Paint vs SyncMVD-IPA / TexGen / Hunyuan3D-2.0: CLIP-FID 24.78 (vs 26.44 / 28.24 / 28.39), CMMD 2.191 (vs 2.318 / 2.448 / 2.397), CLIP-I 0.9207 (vs 0.8893 / 0.8818 / 0.8823), LPIPS 0.1211 (vs 0.1261 / 0.1331 / 0.1423) [Table 2].
End-to-end visual comparison against Step1X-3D, 3DTopia-XL, and two commercial models on image-to-3D with PBR — qualitative claim of “highest fidelity PBR maps” plus reduced sensitivity to upstream geometry quality [Fig. 7].

Why it’s interesting

This is the 3D-asset complement to the rest of the Tencent Hunyuan open-foundation push the wiki has been tracking — HunyuanImage 3.0 Technical Report for image, HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds for full 3D worlds (panoramas → trajectories → 3DGS), and Hunyuan3D 2.1 as the focused single-object PBR-mesh artifact that ships with the strongest open-source positioning in its niche. The architectural choice that’s worth flagging is the same one HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds makes for WorldStereo 2.0: 3D-aware (not image-plane) positional encoding to enforce cross-view consistency in a multi-view diffusion stack — pretty consistent design pattern across the Hunyuan 3D lineage. Compared to TC-AE: Unlocking Token Capacity for Deep Compression Autoencoders and the broader autoencoder-for-diffusion theme, ShapeVAE’s use of FPS + importance sampling + variable token length (up to 3072) is a domain-specific take on the same “compact-yet-expressive latent” question for a non-image modality.