Scaling Video Pretraining with Imagination Models (Photon-1)

TL;DR

Induction Labs (David & Jonathan Li, a two-person SF team) introduce imagination models — a foundation-model architecture that pretrains on internet-scale video via next-latent-token prediction, without any action labels. Their first model, Photon-1, is a 106B-A5B MoE transformer trained from scratch for one epoch on 575M frames (~18 years of screen recording, 552B tokens, ~30k H200-hours). After a small action-conditioned finetune (<35k trajectories) and online RL on virtual-machine rollouts, Photon-1 reportedly outperforms Gemini 3.1 Flash-Lite on internal computer-use benchmarks with 30× less pretraining compute and 3× lower serving cost, and transfers to unseen dynamics (checkers, billiard physics) with less data than a similarly-sized LLM baseline. The load-bearing engineering is a differential FSQ vision encoder that compresses each frame to 960 discrete tokens (2.2KB, ~100× smaller than typical VLM representations), making autoregressive next-latent prediction tractable at video scale.

Key claims

Imagination models predict future frames autoregressively in latent space using next-latent-token prediction; no pixels are generated during pretraining, and no actions are seen or inferred [§Training imagination models].
The vision encoder uses finite scalar quantization (FSQ) to compress each frame into 960 tokens, each an 8-dim vector with 5 values per dim (vocab 5⁸), totalling 2.2KB — reported ~100× smaller than existing OCR and VLM representations while preserving text, layout, and state-change detail [§Training imagination models].
Frames are encoded differentially in pairs so the latents describe frame-to-frame differences, which is what enables the reported compression rate [§Training imagination models].
Pretraining corpus: 2B publicly-available videos filtered to ~2M screen recordings; a keyframe detector removes redundant frames; final dataset is 575M frames / 552B tokens / ~18 years @ 1 fps [§Training imagination models].
Training compute: 106B-A5B MoE transformer, 32K context, one epoch, ~30,000 H200-hours ≈ 4.4×10²² FLOPs, 40% end-to-end MFU on custom fused kernels [§Training imagination models].
Acting recipe: after pretraining, a small action-conditioned finetune (<35k trajectories) teaches Photon-1 to imagine the next state first, then emit the action to reach it; policy is then improved by online RL on programmatically-verified VM rollouts across five desktop environments (LXQt, Xfce, MATE, GNOME, Plasma) [§Acting from imagination].
After RL, Photon-1 exceeds Gemini 3.1 Flash-Lite on the labs’ internal computer-use benchmarks with ≥30× less pretraining compute and 3× lower per-1M-token weighted inference cost ( $0.11 vs$ 0.36) [§Acting from imagination, table].
Post-RL latents become harder to visualize into screenshots — interpreted as the model repurposing latent tokens for planning/reasoning rather than reconstructable state [§Acting from imagination].
Beyond desktop, Photon-1 finetuned on 20k tournament checkers games outperforms both a same-encoder baseline and a similarly-sized pretrained LLM on world simulation and move quality; finetuned on 10k synthetic billiard games it also beats both baselines on physics simulation [§Generalizing beyond the desktop].
Photon-1 picks up human tool-use priors from the pretraining video — after RL it learns to prompt an internal ChatGPT clone, verify the output, and steer the LLM until the task is done [§Imitating human behavior].
Positional argument vs prior work: IDM-based recipes (extract inferred actions from video, then learn a policy on them) are bottlenecked by the IDM’s coverage and by the requirement that actions be writeable; imagination models sidestep both by never touching an action vocabulary at pretraining time [§Related work].

Method

An imagination model is an autoregressive transformer trained with next-latent-token prediction over sequences of quantized frame tokens. The vision encoder is a differential FSQ VAE: pairs of frames are encoded together so the latent describes their difference, then each frame is represented as 960 discrete tokens with a 5⁸ codebook (2.2KB per frame). Photon-1 is a 106B-total / 5B-active MoE transformer with 32K context, trained from scratch for one epoch on ~552B tokens (575M frames sampled at 1 fps from ~2M filtered computer-use videos, sourced from a 2B-video internal index).

To act, Photon-1 is action-conditioned finetuned on fewer than 35k trajectories, teaching it to produce a latent prediction of the desired next state and then the action to reach it. Online RL is applied on a fleet of Linux VMs with five desktop environments, each with its own Google account and a rate-limit-free internal ChatGPT clone; outcomes are programmatically verified to produce reward. Separately, an off-the-shelf single-stream diffusion transformer is finetuned as a decoder from Photon-1’s latents back to screenshot pixels — used only for visualization, not part of pretraining or inference.

Results

Computer-use benchmarks (internal): Photon-1 post-RL surpasses Gemini 3.1 Flash-Lite; ≥30× less pretraining compute; weighted per-1M-token inference cost $0.11 vs$ 0.36 (~3×) [§Acting from imagination, table].
Checkers (unseen domain): after finetuning on 20k tournament games, Photon-1 beats both a same-encoder baseline and a similarly-sized pretrained LLM on world simulation and move quality [§Generalizing beyond the desktop].
Billiard physics (unseen domain): after 10k synthetic games at 5 fps, Photon-1 beats the same two baselines on simulation accuracy [§Generalizing beyond the desktop].
Compression: 2.2KB per frame, reported ~100× smaller than OCR/VLM representations at comparable text-and-layout fidelity [§Training imagination models].
Training efficiency: 40% end-to-end MFU on a 106B-A5B MoE with custom fused vision-encoder and MoE kernels; 30k H200-hours for the full run [§Training imagination models].

Why it’s interesting

Photon-1 is the label-free extreme of the computer-use pretraining design space. VideoAgentTrek: Computer Use Pretraining from Unlabeled Videos recovers action-and-thought trajectories from YouTube via a grounded VLM before doing masked-text SFT; Computer Use Large: 48k screen recordings (~12,300 hours) of professional software captures 12k hours of screen recordings with manual action labels; The First Fully General Computer Action Model (FDM-1) FDM-1 builds on internal captured demonstrations. Photon-1 skips all of these and does pure next-latent prediction on raw video, learning a policy implicitly and only later grafting on action tokens — a direct instantiation of Sitzmann’s “generative-rollout video pretraining should be the substrate” thesis (The flavor of the bitter lesson for computer vision) in a domain (computer use) where the video is cheap and the environment is closed enough to permit programmatic reward. It sits alongside Video Generation Models are General-Purpose Vision Learners (video-generation pretraining beats V-JEPA / VideoMAE on downstream perception) as a second data point that generative-rollout beats latent-predictive on transfer, extended here into the action regime. Also relevant to Autoregressive Video Generation: the FSQ+differential-encoder recipe reads as a heavy compression cousin of DeltaToken: 100× Token-Efficient Video Representation for World Models‘s ~100× token reduction, applied to next-latent-token policy pretraining rather than reconstruction.

The desktop-to-checkers/billiards transfer, if it holds up under independent evaluation, would be the sharpest evidence to date that latent next-frame prediction on a narrow visual domain still yields a broad dynamics prior — a stronger version of Sitzmann’s argument than the community has previously demonstrated.