Haiyu Wu — critique of reachability-augmented CEM planning in JEPA world models (with VISReg counter-example)

TL;DR

Haiyu Wu (@HaiyuWu1) shares and critiques an unnamed JEPA world-model paper that argues (a) next-frame supervision during training is misaligned with the test-time planning objective, and (b) Euclidean-distance-only CEM action optimization does not guarantee the terminal state is reachable — and adds a reachability score plus K-step rollouts during training to fix both. Wu’s read: he likes the reachability idea but flags two concerns — K-step rollouts scale training time with the planning horizon, and constructing negative reachability pairs across samples-within-a-batch is unsafe when samples aren’t guaranteed different. His counter-datapoint: training LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels with VISReg (arxiv 2606.02572) instead of SIGReg achieves similar or better results than the reachability method on TwoRoom and OGBench Cube — suggesting the underlying problem is a latent-space regularization issue, not a training-objective misalignment.

Key claims

The tweet’s cited paper argues next-frame supervision during training is misaligned with the test-time CEM-planning objective — training predicts one step, planning optimizes over an H-step goal-matching loss.
The cited paper argues pure Euclidean distance in latent space is insufficient for CEM action optimization: the CEM winner can be a latent close to the goal but unreachable from the current state under the learned dynamics.
The cited paper’s fix has two components: (i) K-step teacher-forced rollouts during training with alignment of each predicted latent to the corresponding actual observation; (ii) a learned reachability predictor trained on three negative-pair types (within-sample non-reachable, cross-sample, prediction-vs-GT non-reachable) — used as an auxiliary score in CEM.
The cited paper reports higher success rates than LeWM on 4 of 5 datasets.
Wu’s Concern 1: K-step rollouts during training make training-time cost scale with planning horizon; longer horizons need larger K, breaking training efficiency — Wu does not think this is an ideal solution to the train-test-alignment gap.
Wu’s Concern 2: cross-sample negative pairing for the reachability predictor implicitly assumes all samples within a batch are distinct, which is not guaranteed and is a red flag.
Wu’s counter-datapoint: training LeWM with VISReg (arxiv 2606.02572) instead of the original SIGReg regularizer matches or exceeds the reachability paper’s TwoRoom and OGBench Cube results — evidence that the core issue is latent-space regularization, not training–planning objective mismatch.

Method

Twitter post — no code, no benchmarks of Wu’s own, no images beyond a screenshot of the referenced paper’s Fig. 2 (a diagram showing the CEM-with-reachability-score planning loop). Content is one long-form comment plus a reference to VISReg (arxiv 2606.02572) as a competing regularizer on LeWM. The underlying paper being critiqued is not linked in the tweet and could not be identified from the tweet text alone; the discussion is centered on the ideas rather than on rehosting the artifact.

Results

None reported quantitatively. Wu’s counter-claim (“VISReg on LeWM achieves similar or better performance on TwoRoom and Cube than [the reachability paper]”) is asserted in the tweet without numbers or a linked writeup.

Why it’s interesting

Sits inside the JEPA-latent-world-model design debate the wiki has been tracking through LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels (LeWM’s SIGReg two-loss recipe), V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning (V-JEPA 2.1 dense-token extension), and AdaJEPA: An Adaptive Latent World Model (AdaJEPA’s test-time adaptation of the same JEPA predictor inside the MPC loop). The reachability-score-in-CEM angle is a new axis compared to those: rather than changing the representation or the training loss or the test-time adaptation, the cited paper changes the planning objective itself. Wu’s rebuttal is telling — he argues the improvement disappears once the regularizer is switched, i.e. the “planning objective mismatch” framing may be masking a latent-space-shape problem that a better isotropy regularizer solves for free. Complements x-jepa — Explorations into JEPA approaches (lucidrains) which bundles LeWM, VISReg, and adjacent JEPA variants as one composable design space — Wu’s tweet is a working example of that composability in practice.