Skip to content
Bud's Wiki

RynnBrain 1.1 — Open Embodied Foundation Models (Alibaba DAMO Academy)

TL;DR

Section titled “TL;DR”

RynnBrain 1.1 is Alibaba DAMO Academy’s second-generation open embodied foundation model, released as an Apache-2.0 bundle of three scales (2B, 9B, and 122B-A10B sparse-MoE) built on Qwen3-VL / Qwen3.5 backbones. The 1.1 release adds two capabilities beyond the 1.0 arxiv paper (arXiv:2602.14979): explicit native 3D grounding (3D bounding boxes from a single RGB image + camera intrinsics) and contact point prediction (instruction-conditioned contact point + in-plane orientation), extending the family from image-plane localization to metric-3D and action-relevant interaction grounding. A downstream RynnBrain-VLA demonstrates real-robot cross-platform transfer to Unitree G1, Astribot, and Tianji-Wuji across humanoid, bimanual, and dexterous-hand tasks. Ships with 2B/9B/122B checkpoints on Hugging Face + ModelScope, a 1.1 technical report PDF, cookbooks for 8 capability categories including the two new 3D/contact tasks, and pretraining/evaluation code via the sibling RynnScale repo.

Key claims

Section titled “Key claims”
  • Three-scale unified release under one training recipe: 2B, 9B, and 122B-A10B (first 122B-scale embodied brain model on this page), built on Qwen3.5-2B / Qwen3.5-9B / Qwen3.5-122B-A10B backbones with Qwen3-VL vision integration [README §Introduction, §Model Zoo].
  • Native 3D grounding is a new post-training task in 1.1: predicts 3D bounding boxes (position, dimensions, orientation) from a single RGB image with camera intrinsics, extending the model past 2D image-plane localization [README §What’s New in 1.1, cookbook 8].
  • Contact point prediction is the other new 1.1 task: instruction-conditioned prediction of a contact point + in-plane orientation on the image, aimed at action-relevant interaction grounding rather than pure perception [README §What’s New in 1.1, cookbook 7].
  • Unified decoder-only vision-language architecture across all three scales, supporting both dense (2B, 9B) and sparse-MoE (122B-A10B) variants; encodes omni-vision inputs with language instructions and emits text + pointing sequences + 3D perception + contact signals from one model [README §Model Architecture].
  • Downstream RynnBrain-VLA demonstrates real-robot performance across three heterogeneous platforms: Unitree G1 (humanoid), Astribot (bimanual), and Tianji-Wuji (dexterous-hand), and reports success-rate improvements over baseline VLAs on all three [README §Real-Robot VLA Evaluation, VLA_results.png].
  • Full open release package: HuggingFace + ModelScope weights for all three scales, 1.1 technical report PDF hosted on the project page, 8 cookbooks (spatial understanding, object understanding, object grounding, area location, affordance location, trajectory location, contact point prediction, 3D grounding), inference recipes for HuggingFace transformers and SGLang, Apache-2.0 licensed [README §Model Zoo, §Quick Start, §Cookbooks, §License].
  • Positioned as a systematic upgrade of RynnBrain 1.0 (arXiv:2602.14979): the 1.0 branch is preserved with its reasoning/navigation/planning fine-tuning recipes and the RynnBrain-Bench dataset (4-dimensional embodied benchmark on HF: Alibaba-DAMO-Academy/RynnBrain-Bench) [README §News, §From RynnBrain 1.0].

Method

Section titled “Method”

RynnBrain 1.1 adopts a single decoder-only vision-language architecture shared across the three scales, following the Qwen3-VL family: a vision encoder (from Qwen3-VL) tokenizes multi-image / video / camera-intrinsics input, tokens are interleaved with a language instruction, and a Qwen3.5 dense-or-MoE backbone (2B, 9B, or 122B-A10B) produces the output stream. The output space is deliberately heterogeneous: plain text for reasoning; special-tagged bounding boxes for 2D grounding (<object>(x1,y1),(x2,y2)</object> with coordinates ∈ [0,1000]); pointing sequences for area/affordance/trajectory tasks; 3D boxes for native-3D grounding conditioned on camera intrinsics; and contact-point-plus-orientation tokens for interaction grounding. All eight capabilities are folded into one autoregressive next-token loss over a shared vocabulary, so the router-of-tasks is the instruction template rather than a separate head — consistent with the “unified VLM as embodied brain” recipe direction the wiki has been tracking. The 122B-A10B variant activates ~10B parameters per token via sparse mixture-of-experts routing, keeping inference cost comparable to dense-10B models. Downstream RynnBrain-VLA attaches an action decoder to the same backbone and is trained separately for real-robot deployment on Unitree G1, Astribot, and Tianji-Wuji embodiments; the README credits π 0.5 and RTC as method influences for that layer.

Results

Section titled “Results”
  • Scale-parity comparisons vs same-tier baselines: RynnBrain 1.1-2B, 1.1-9B, and 1.1-122B are each compared against other 2B / 9B / 122B-scale embodied VLMs on general embodied understanding — bar-chart figures in the README (RynnBrain1.1-2B.png, -9B.png, -122B.png) report the 1.1 family outperforming same-scale baselines on the aggregate score, though specific per-benchmark numbers are in the 1.1 tech report PDF, not in the README [README §Performance, §General Embodied Understanding].
  • Real-robot VLA success rates vs baselines: RynnBrain-VLA outperforms baseline VLAs on all three deployed platforms (Unitree G1, Astribot, Tianji-Wuji) [README §Real-Robot VLA Evaluation, VLA_results.png].
  • 3D grounding accuracy: bar-chart figure shows RynnBrain 1.1 leading baselines on 3D grounding metrics [README §3D Grounding, 3D_chart.png].
  • Contact point prediction: visualized qualitatively; no numeric benchmark accompanies the visualization in the README [README §Contact Point Prediction].
  • Model sizes: RynnBrain 1.1-2B (dense, based on Qwen3.5-2B), 1.1-9B (dense, based on Qwen3.5-9B), 1.1-122B-A10B (122B total, ~10B activated, based on Qwen3.5-122B-A10B) [README §Model Zoo].

Why it’s interesting

Section titled “Why it’s interesting”

RynnBrain 1.1 is the sharpest direct sibling on the wiki to RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination (Tencent, 6.2B unified embodied cognition, filed 3 days earlier) — same “unified embodied VLM emitting text + visual + grounding outputs” recipe, released within the same week from a competing Chinese lab, but Alibaba’s version pushes to 122B-A10B sparse MoE and adds metric-3D + contact-point outputs where RxBrain focuses on interleaved language reasoning + imagined-frame generation. The 3D-grounding-from-single-RGB-plus-intrinsics move directly extends the unified-VLM-with-pointing recipe the vla-models concept catalogues under Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models (Embodied-R1.5) and Robostral Navigate: single-camera AI navigation (Robostral Navigate), and the contact-point-prediction task complements the sensorimotor / tactile counter-position (Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level, Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction, ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control) by attacking the same “action-relevant interaction grounding” problem from the perception-output side rather than the tactile-input side. Alibaba’s broader Rynn* stack now spans world model (RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation — RynnWorld-4D), teleop world model (RynnWorld-Teleop, referenced in README), embodied VLM (RynnBrain), and VLA (RynnVLA-001, RynnVLA-002, referenced in README) — a vertically-integrated open-weights answer to Tencent’s Hy-* stack (Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stack, HY-World 2.0) and Xiaomi’s XR-1/U0 stack (Xiaomi-Robotics-1 (XR-1) — Scaling VLA Foundation Models with 100K Hours of Embodiment-Free UMI Pre-training, Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model).

See also

Section titled “See also”