x-jepa — Explorations into JEPA approaches (lucidrains)

ListenOne Repo to Bolt the JEPA Zoo Together9:15

Show notes

This week we look at `lucidrains/x-jepa`, Phil Wang's early exploration repo that bundles a cluster of JEPA-adjacent ideas — LeJEPA, LeWorldModel, Next-Latent Prediction Transformers, VISReg, Kimi's Attention Residuals, and jumpy-world-model planning — into one PyTorch codebase, framed by LeCun's 2022 autonomous-intelligence position paper. There are no benchmarks, ablations, or checkpoints yet; the interesting move is treating those separate papers as a single composable design space. We talk through why a minimal third-party reference matters for the team, and what a stable end-to-end pixel-JEPA trainer could mean for video-model work. **Sources** - x-jepa — Explorations into JEPA approaches (lucidrains / Phil Wang) - A Path Towards Autonomous Machine Intelligence (LeCun, 2022) - LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels (Maes et al.) - Next-Latent Prediction Transformers Learn Compact World Models (Teoh et al.) - Learning-to-Optimize via Deep Unfolded Flows (Saravanos et al.) - Compositional Planning with Jumpy World Models (Farebrother et al.) - LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics (Balestriero & LeCun) - VISReg: Variance-Invariance-Sketching Regularization for JEPA training (Wu et al.) - Attention Residuals (Kimi Team)

Subscribe to the channel: https://podcast.lumalabs.link/27a25f51ea81d41a8253f46c6133a35e/feed.xml

TL;DR

lucidrains/x-jepa is a Phil Wang exploration repo that bundles the recent JEPA-adjacent ideas — LeCun & Balestriero’s LeJEPA (isotropic-Gaussian regularizer for stable SSL), Maes et al.’s LeWorldModel (two-loss end-to-end JEPA world model from pixels), Teoh et al.’s Next-Latent Prediction Transformers, Kimi’s Attention Residuals, VISReg (variance-invariance-sketching regularizer for JEPA), and Compositional Planning with Jumpy World Models — into one PyTorch codebase, with LeCun’s 2022 “Path Towards Autonomous Machine Intelligence” position paper as the framing. Very early (“Explorations into…”) — no README beyond the citation list at filing time.

Key claims

The repository is scoped as an exploration of a “more wholistic architecture (JEPA) in general,” with LeCun 2022 as its position anchor [README].
The current bibliography lists seven papers the code aims to combine: LeWorldModel (arXiv 2603.19312), Next-Latent Prediction Transformers (arXiv 2511.05963), Learning-to-Optimize via Deep Unfolded Flows (ICML 2026), Compositional Planning with Jumpy World Models (arXiv 2602.19634), LeJEPA (arXiv 2511.08544), VISReg (arXiv 2606.02572), and Kimi’s Attention Residuals (arXiv 2603.15031) [README, Citations].

Method

The repository is a lucidrains-style single-file PyTorch exploration; no README description of the architecture is published beyond the citation list at filing time. Reading the citation set as a design brief: a JEPA-family encoder+predictor trained end-to-end from pixels (LeWorldModel) with an isotropic-Gaussian latent regularizer (LeJEPA, LeWorldModel’s SIGReg, or VISReg’s sketching variant), a next-latent transformer predictor (Next-Latent Prediction Transformers), attention-residual blocks (Kimi Attention Residuals), and jumpy planning primitives (Compositional Planning with Jumpy World Models). No benchmarks, ablations, or trained checkpoints are shipped at filing.

Results

None released. The repo is code-only, and no experimental numbers or trained models are published at filing time.

Why it’s interesting

lucidrains’s reimpls consistently function as portable minimal codebases for a research idea before its authors ship theirs — the wiki’s existing filings on LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels and V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning have no independent third-party PyTorch reference, so this repo is potentially the shortest path for the team to sanity-check the JEPA-end-to-end-from-pixels recipe on our own data. Two structural observations: (1) the citation set slots directly into the World Foundation Models cluster’s latent-predictive pole (LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels, AdaJEPA: An Adaptive Latent World Model, Back to the Features: DINO as a Foundation for Video World Models) — this is the first filed code artifact that treats those papers as one composable design space rather than separate contributions; (2) the inclusion of Kimi’s Attention Residuals (Attention Residuals (AttnRes / Block AttnRes) — Technical Report) alongside JEPA-family SSL objectives is unusual — most JEPA reimpls stay within the ViT-with-standard-attention envelope. Worth watching to see if the combined recipe yields a stable end-to-end pixel-JEPA trainer that the team could reuse.