PixelGen: Improving Pixel Diffusion with Perceptual Supervision

TL;DR

PixelGen is a pixel-space diffusion framework that beats strong latent-diffusion baselines on ImageNet-256 (FID 5.11 without CFG, 80 epochs) by replacing the latent-space MSE objective with two complementary perceptual losses: LPIPS for local textures and a Perceptual-DINO (P-DINO) loss for global semantics, applied on top of a JiT (pixel-DiT) backbone with x-prediction and flow-matching reparametrization. The thesis: the reason pixel diffusion historically lagged latent diffusion is not the pixel manifold itself but the loss function — MSE in pixel space wastes capacity on perceptually irrelevant signal, while perceptual losses point the model at the manifold humans care about. No VAE, no latent stage, no auxiliary modules.

Key claims

Pixel diffusion’s prior gap vs. latent diffusion is attributable to optimizing a high-dimensional pixel manifold dominated by perceptually irrelevant signals, not to pixel space being intrinsically harder [Abstract, §1].
LPIPS and P-DINO provide complementary supervision: LPIPS sharpens local textures, P-DINO improves global semantics; ablations show the JiT baseline at FID 23.67 drops to 10.00 with LPIPS and 7.46 with LPIPS + P-DINO (ImageNet-256, no CFG) [Fig. 5, §4 ablation].
PixelGen achieves FID 5.11 on ImageNet-256 without classifier-free guidance in 80 training epochs, surpassing strong latent diffusion baselines [Abstract].
On large-scale text-to-image generation, PixelGen reaches a GenEval score of 0.79, indicating the recipe scales beyond class-conditional ImageNet [Abstract].
The diffusion model uses x-prediction (predicting the clean image) and converts the prediction to velocity to retain a flow-matching training objective [§3 Method].
P-DINO is implemented as patch-level feature alignment against a frozen DINOv2 encoder — this is the same DINO target REPA uses, but applied as an output-space perceptual loss on the decoded image rather than as an intermediate-layer alignment objective [§3 Method].

Method

PixelGen builds directly on JiT (Joint image Transformer — a DiT operating on raw pixel patches with no VAE) and changes only the loss. The diffusion model is trained with x-prediction: at noise level t, the network predicts the clean image $\hat{x}_0$ rather than the noise $\epsilon$ or velocity $v$ . The predicted clean image is then converted into velocity to plug into a flow-matching objective, which preserves flow matching’s training-stability and sampling properties while exposing a clean-image output that perceptual losses can be computed on.

Two perceptual losses are added on top of the flow-matching loss. (1) An LPIPS loss between the predicted clean image and the ground-truth image, which emphasizes local perceptual fidelity (textures, edges, fine detail). (2) A Perceptual-DINO (P-DINO) loss that passes both the predicted and ground-truth images through a frozen DINOv2 encoder and aligns their patch-level features, which emphasizes global semantic structure. The team frames this as projecting the optimization target from the full pixel manifold onto a “perceptual manifold” where most of the gradient signal corresponds to features humans actually perceive. The poster also lists REPA-style alignment as a third ingredient (per the team’s TL;DR), suggesting the final recipe stacks intermediate-layer REPA on top of the two output-space perceptual losses.

Results

On ImageNet-256 class-conditional generation without CFG, the JiT baseline scores FID 23.67; adding LPIPS drops this to 10.00; adding P-DINO on top reaches 7.46; the full PixelGen recipe (the additional ingredients not detailed on the project page) reaches FID 5.11 — all in 80 training epochs [Fig. 5, Abstract]. The team’s headline framing is that this beats latent-diffusion baselines at parity training compute / iterations, though the specific LDM baselines and their epoch budgets are not on the project page.

For text-to-image, PixelGen reports GenEval 0.79 [Abstract], which is competitive with mid-tier T2I systems and shows the recipe is not specific to ImageNet’s class-conditional regime.

Why it’s interesting

PixelGen is the cleanest version yet of the “pixel diffusion is back” thesis emerging from JiT, L2P, and now this work — and it’s the first to argue that the loss function, not the representation, was the actual bottleneck. If correct, this dissolves the original motivation for latent diffusion (compress away perceptually irrelevant signal before training a generative model on it) by moving the compression into the loss (project the gradient onto perceptual features) rather than the encoder (project the pixels onto VAE latents). For Luma, which trains video diffusion models where VAE artifacts directly hurt output quality, this is a serious candidate recipe — perceptual losses on pixel x-prediction sidestep the VAE failure modes (temporal flicker from per-frame VAE codes, color shifts, fine-detail loss) entirely. Also worth noting: P-DINO and REPA both pull DINOv2 features into the training loop but in different places (output space vs intermediate layers), and PixelGen reportedly uses both — that connects to RAEv2’s finding that RAE and REPA are complementary rather than competing, suggesting “DINO features as supervision” is becoming the dominant inductive bias for diffusion training regardless of where in the network you apply it.