Ψ-0.6: Adaptive Streams of Thought in a Unified Vision-Language World Model

TL;DR

Ψ-0.6 (Stanford NeuroAILab) is an 8B transformer that unifies perception, VLM, world modeling, reasoning, and policy under one shared token vocabulary spanning images, text, camera pose, depth, optical flow, and point tracks — any output of one task can be fed back as input to the next, yielding streams of thought that interleave words with geometric and dynamic intermediates. The pitch: text-only chain-of-thought is the wrong substrate for physical reasoning; one model that both generates and interprets geometry/motion lets reasoning move through whichever representation best fits the problem. Reinforcement learning on task feedback jointly optimizes the selection, generation, and use of these intermediate representations, with gains that transfer to unseen VQA benchmarks. Also supports counterfactual simulation — “would moving the box obstruct the pathway?” — by rendering the imagined scene and interpreting it.

Key claims

Perception, language, geometry, and simulation share one 8B transformer and one tokenized interface; every task becomes a token-ordering question answered by the same weights [§Abstract, Thread (1)].
Any output can be the next input: a camera pose is an estimate when inferred from two images, a command when used to render a view, and a thought when used to answer a question — text, depth, flow, and camera pose are treated as complementary projections of the same world [Thread (2)].
Beats leading open-weight generalist VLMs (incl. Qwen3-VL) and spatial specialists on multi-view spatial reasoning (MindCube, zero-shot) and beats other world models at predicting object motion (FlowBench) [§Abstract, Thread].
Counterfactual simulation combines world-model and VLM capabilities in one model: given “move the box 0.8 m right,” Ψ-0.6 predicts the motion, renders the outcome, and answers whether a cup is hidden; trained on simulated rooms, it transfers to real robot tabletops with different scene scales and object sets [Thread (6)].
RL on task feedback jointly optimizes which representation to reason through and how to generate/use it; VQA accuracy gains transfer to unseen benchmarks, suggesting the policy is learning a general modality-selection skill rather than per-benchmark heuristics [§Abstract, Thread (7)].
The recipe is a direct port of a familiar LLM lineage past language: (1) one model with diverse capabilities, (2) a prompt interface that composes them, (3) RL on outcomes shaping intermediate reasoning steps [Thread (3)].

Method

Single 8B transformer with a shared token interface covering images, text, camera pose, depth, optical flow, and point tracks. The model is trained to both interpret and generate each modality, so every task is a reordering of the same tokens. Composition of capabilities is driven by a prompt interface: perception, VLM-style QA, world-model rollouts, and policy outputs all share weights and are distinguished only by what the prompt orders. On top of this, RL on task feedback shapes which intermediate representations the model emits inside its reasoning stream and how it consumes them — the key move being end-to-end optimization of the entire stream, not just the final answer. For counterfactual tasks the model first produces an action-conditioned simulated future (predicted motion + rendered outcome), then re-reads those tokens to answer a VLM-style question — the simulation and interpretation legs are the same model.

Results

Zero-shot MindCube (multi-view spatial reasoning): beats Qwen3-VL and spatial specialists; further gains when reasoning through explicit geometric intermediates (depth / camera pose) [Thread (2)].
FlowBench (object-motion prediction): outperforms other world models at predicting how objects move [Thread (2)].
Counterfactual simulation: trained on simulated rooms, transfers to real robot tabletops at different scene scales and with different objects [Thread (6)].
RL VQA transfer: joint optimization of selection + generation + use of intermediate representations improves accuracy on seen tasks and transfers to unseen benchmarks [§Abstract].
No numeric scoreboard versus closed-flagship omni models (Gemini / Veo / Cosmos 3) is reported in the abstract or launch thread.

Why it’s interesting

Sharpens the substrate axis on Thinking with Modalities: Ψ-0.6 is substrate (a) autoregressive-tagged-latent in the sense that the modality switches happen inside one AR trajectory, but it generalizes the set of latents far past prior filed instances — Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning (grounding/caption on video), Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens (expert-aligned latent tokens), Latent Visual Reasoning (self-supervised ROI patches), and LightNav-0: Scaling Real2Sim2Real for Zero-Shot Generalist Navigation (point CoT) all commit to a single non-text modality; Ψ-0.6 adds depth, flow, camera pose, and rendered imagined scenes as peers inside one trajectory, with an RL policy choosing between them per step. This is a direct empirical test of the “which substrate wins” open question that page has been tracking.
Fits the frozen-nothing / unified-everything extreme of Unified Multimodal Models: contrast with the taxonomy’s three paradigms (E2E AR, AR+MAR, AR+Diffusion), Ψ-0.6 puts the WFM, VLM, perceiver, reasoner, and policy under one 8B AR trunk with a shared vocabulary — adjacent in framing to Rho-1: Collapsing the Multimodal Stack Rho-1’s “collapsing the multimodal stack” bet but with explicit geometric modalities (depth, camera pose) as first-class tokens rather than action streams.
Adds a new row to the WFM-deployment ledger on World Foundation Models: WFM-as-counterfactual-rollout-plus-interpreter within one model — the simulate-then-answer loop is end-to-end differentiable because it never leaves the shared weights, unlike the DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models generator-proposes / verifier-checks split that uses two separate systems.