Skip to content
Bud's Wiki

Robostral Navigate: single-camera AI navigation

TL;DR

Section titled “TL;DR”

Robostral Navigate is Mistral’s first embodied-navigation model — an 8B VLM-derived policy that takes a single RGB camera stream plus a plain-language instruction (e.g. “leave the lobby, walk through the corridor, enter the supply room, and stop to face the second shelf”) and drives a robot to the target. It reports 76.6% success on R2R-CE validation-unseen, 9.7 points over the best prior single-camera approach and 4.5 points over the best depth-or-multi-camera system, while using no LiDAR / depth / multi-view sensing. The model is built in-house (not initialized from an existing open-source VLM), trained entirely in simulation on ~400K trajectories over 6K scenes with a prefix-cached tree-attention training scheme that shrinks the token budget 22×, and then improved with CISPO online RL for a further +3.2% success.

Key claims

Section titled “Key claims”
  • Robostral Navigate is an 8B model that takes RGB images + a plain-language instruction and moves a robot through an environment [§Summary / §Highlights].
  • It achieves 79.4% success on R2R-CE validation-seen and 76.6% on validation-unseen, using only a single RGB camera and no LiDAR / depth sensors [§Summary / §Highlights].
  • On the same R2R-CE validation-unseen split it beats the best prior single-camera approach by 9.7 points and the best depth/multi-camera system by 4.5 points [§Summary].
  • The model uses pointing-based navigation: given task + observation history it predicts image coordinates of the next waypoint in the current camera view plus a desired arrival orientation — making the policy robust to camera-intrinsic and world-scale changes; when the target is outside the current FOV it falls back to robot-local metric displacements (“Move 2 m forward, 1.5 m left, turn 25° left”) [§Navigation via pointing].
  • The model is built in-house and does not rely on existing open-source VLMs; it is initialized from Mistral’s own grounding-specialized VLM (pointing, counting, object localization) [§Built from the ground up].
  • Training data is generated entirely in simulation via an in-house pipeline: ~400,000 trajectories across ~6,000 scenes [§Built from the ground up].
  • Training uses prefix-caching with a tree-based attention mask that compresses an entire episode into a single sequence with no cross-timestep leakage, reducing training tokens 22× while preserving all learning signal — collapses month-scale runs into days [§Efficient supervised training].
  • After supervised training, an online RL stage with CISPO adds +3.2% success rate on top, learning recovery + exploratory behaviors that mitigate the behavior-cloning distribution-shift problem, with no observed plateau [§Online reinforcement learning].
  • The policy runs on wheeled, legged, and flying robots and generalizes across robot sizes and camera intrinsics [§Highlights].

Method

Section titled “Method”

Robostral Navigate is an 8B language-first policy: the input is (task instruction, observation history of RGB frames) and the output is either a pointed waypoint (image coordinates + arrival orientation) in the current view, or — when the target is off-screen — a fallback metric displacement in the robot’s local frame. The backbone is Mistral’s own grounding VLM (with pointing/counting/localization capabilities), post-trained for navigation. Data is 400K simulated trajectories over 6K scenes, generated by an in-house pipeline that supports rapid iteration on the data itself.

Two training-side ideas do most of the work. First, prefix-cached tree-attention SFT: each simulated episode becomes a single sequence, and a tree-shaped attention mask lets every timestep condition on shared observation prefixes without leaking information across timesteps. This uses ~22× fewer training tokens than the naive one-sample-per-timestep expansion of the same episode. Second, an online-RL polish stage with CISPO on top of the SFT checkpoint, which the post credits with +3.2% absolute success and with the emergence of recovery / exploration behaviors that pure BC does not produce.

Results

Section titled “Results”
  • R2R-CE validation-seen: 79.4% success [§Highlights].
  • R2R-CE validation-unseen: 76.6% success — +9.7 over best prior single-camera baseline, +4.5 over best depth/multi-camera baseline [§Summary].
  • CISPO online-RL stage contributes +3.2% absolute success on top of SFT [§Online reinforcement learning].
  • Prefix-cached tree-attention training uses 22× fewer training tokens vs one-sample-per-timestep, “transforming month-scale runs into days” [§Efficient supervised training].
  • Deployment: reported to run on wheeled, legged, and flying robots and generalize across robot sizes and camera intrinsics [§Highlights].

Why it’s interesting

Section titled “Why it’s interesting”

This is the strongest filed data point yet for the pointing-as-action-space thesis at the navigation level, and the first substantive VLN release from Mistral. It complements Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models and MolmoPoint: Better pointing architecture for vision-language models which argued that a strong VLM-level pointing/grounding prior substitutes for extensive action pretraining — Robostral Navigate is a clean instantiation of the same idea for navigation, and its “grounding VLM → nav policy” recipe mirrors Embodied-R1.5’s “unified VLM with pointing → manipulation VLA” pipeline. Robostral Navigate also fills the concrete gap that Qwen-Robot Suite launch — Qwen-RobotManip, Qwen-RobotNav, Qwen-RobotWorld + RobotClaw framework (Alibaba Qwen) flagged: Qwen-RobotNav announced the same unified single-camera language-first navigation framing at the suite level but shipped no benchmark numbers or technical report — Mistral’s post is now the first filed VLN system to put quantitative R2R-CE numbers behind that framing. On the training side, the 22× prefix-cached tree-attention scheme is a sim-episode-native cousin of the standard prompt-caching used in LLM inference, and the CISPO-on-top-of-SFT loop is another instance of the Reasoning RL pattern (SFT → online RL closes the BC distribution-shift gap) filed today alongside the flow-matching RL debates in VLA Models (RECAP / π*0.6 etc.).

See also

Section titled “See also”