OpenDM — DM0.5: An Open-World Foundation Model for General-Purpose Embodied Intelligence

TL;DR

OpenDM is Dexmal’s release repo for DM0.5, a next-generation open-weight Vision-Language-Action (VLA) foundation model for open-world robot control — a Gemma3-4B VLM backbone paired with a 680M flow-matching Action Expert, trained on the “native embodied modeling” recipe DM0 introduced. Headline design bets: (1) long historical visual context (up to ~60 seconds per external coverage), used both for long-horizon coherence and for in-episode learning from a start-of-episode human demonstration; (2) separate learning-rate groups for VLM backbone vs Action Expert plus an internal automatic VL data generation pipeline, to fight catastrophic forgetting of open-world vision-language capability during action co-training; (3) policy robustness demos where the top-view camera is physically perturbed during an episode and the policy re-adapts. Reported benchmarks against π0 / π0.5 / GR00T-N1.7: LIBERO 99.0 SR, RoboTwin2.0 Clean 93.6 / Rand 93.3, VLA-Arena L0/L1/L2 89.0/53.6/44.1, RoboDojo-Sim SR 19.34%, and RoboChallenge Table30V2 real-world SR 43.0% — with π0.5 as the closest reported baseline on every table (the Slack sharer notes the community framing “pi0.5 of China”).

Key claims

Base architecture is a Gemma3-4B VLM + 680M Action Expert trained with Flow Matching for continuous robot actions, and is designed for natural-language manipulation, zero-shot generalization, efficient downstream fine-tuning, long-horizon historical context, robust policy behavior, and cross-embodiment transfer [Hugging Face DM05-robotwin2 card via web search; consistent with README §Introduction].
The RoboTwin 2.0 fine-tuned checkpoint controls the ALOHA bimanual embodiment using three RGB camera views (“Head”, “Left wrist”, “Right wrist”) and generates 14-dim joint-position action chunks [README §Inference example; DM05-robotwin2 card].
Long historical visual context is claimed to enable in-episode learning from a single human demonstration: the human first demonstrates a placement rule at the start of the episode, and the robot must observe that demonstration and preserve the rule during its own later execution phase within the same context window [Slack quote from blog].
Policy robustness under mid-episode disturbances: demos show the top-view camera being perturbed and the policy adapting during the same episode without a reset [Slack sharer note].
Embodied chain-of-thought is used as an intermediate representation between perception and action; the sharer flags a recurring theme that “action completion is more important than directly using [the CoT]” as an inference-time signal [Slack notes; consistent with the DM0 embodied-CoT lineage].
Optimization uses separate learning-rate groups for the VLM backbone and the Action Expert; the VLM backbone is trained with a smaller learning rate to reduce catastrophic forgetting and preserve general vision-language capabilities [Slack blog quote].
The VL side is fed by an internal automatic data generation pipeline over large-scale image, video, and visual-instruction data, further strengthening scene semantics, physical dynamics, and task progress [Slack blog quote].
Benchmark table (as reported): LIBERO SR 99.0 (π0 94.4 / π0.5 96.9 / GR00T-N1.7 97.0); RoboTwin2.0 Clean 93.6 / Rand 93.3 (π0 65.9/58.4; π0.5 82.7/76.8); VLA-Arena L0/L1/L2 89.0/53.6/44.1 (π0.5 64.3/35.6/24.5); RoboDojo-Sim Score 24.90 / SR 19.34% (π0 3.48/1.53%, π0.5 11.41/6.91%, GR00T-N1.7 2.85/1.31%); RoboChallenge Table30V2 real-world Score 54.42 / SR 43.0% (π0.5 31.48/14.3%) [README §Benchmark Results].
The MEM-Robodojo-Sim variant targets ARX X5 bimanual manipulation through the XPolicyLab evaluation integration; this is where the RoboDojo-Sim scores come from [README §News, 2026-08-26; PR #101].
Cross-embodiment support ships as per-benchmark fine-tuned checkpoints rather than a single generalist deployment weight — DM05 base, DM05-libero, DM05-robotwin2, DM05-SO101-Pick-Cube (LoRA SFT), DM05-Vla-Arena, DM05-MEM-Robodojo-Sim, DM05-Table30v2 collection — with robot_type (e.g. RobotType.ALOHA) and state_desc selecting the normalization profile per embodiment [README §Models, §Training / dataset registration].
Recommended training footprint is 8 GPUs (RTX 4090 / A100 / H100 / H20); 1 GPU is sufficient for deployment inference [README §Requirements].
The fast inference backend is not a best-effort acceleration toggle — it builds or loads a TensorRT vision engine, dispatches Triton prefix/suffix kernels, and forces the LLM attention backend to flex_attention; these three are hard prerequisites for --inference-config.backend fast [README §Fast Backend Environment Layer].
License is Apache-2.0 on the code; the reported model checkpoints on Hugging Face carry the Gemma license (as expected from the Gemma3 backbone).

Method

DM0.5 keeps DM0’s native embodied modeling skeleton — a Gemma3-4B VLM backbone with a 680M Action Expert head trained by Flow Matching for continuous action generation — and layers on four systematic changes aimed at open-ended instructions, long horizons, mid-episode disturbances, and cross-embodiment control.

Long visual history as a first-class input. DM0.5 conditions on a long window of historical visual context (up to ~60 seconds per external coverage of the release), which is what enables the “learning from a human demonstration” experiment: the human demonstrates a placement rule at the start; the robot must both observe that demonstration and preserve the rule across its own subsequent execution phase, all within the same context window. This makes historical context the substrate for both long-horizon coherence and a form of in-episode ICL that requires no weight update — sibling in effect to MEM: Multi-Scale Embodied Memory for Vision Language Action Models MEM’s dual-timescale memory and to the KV-cache side of Context Length / Quality Trade-off in Video Generation, but obtained by scaling the KV-cache directly rather than adding external memory modules or TTT weight updates in the RoboTTT — 8k-timestep robot policy via test-time training (Jim Fan / NVIDIA GEAR) RoboTTT / WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time WAM-TTT line.

Embodied CoT as an intermediate representation. The Slack note flags embodied CoT explicitly, and combined with the sharer’s observation that action completion matters more than directly using [CoT] at inference time, DM0.5 appears to use embodied CoT primarily as a training-time shaping signal for the shared representation rather than as a runtime autoregressive planner in the loop — closer to R&B-EnCoRe: Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoning R&B-EnCoRe’s variational treatment of CoT as an action-predictive latent than to a full ReAct-style thinking loop.

Separate learning-rate groups for VLM backbone vs Action Expert. The blog quote is explicit: “the VLM backbone and the Action Expert use separate learning-rate groups. The VLM backbone is trained with a smaller learning rate to reduce catastrophic forgetting and preserve general vision-language capabilities.” This is the same failure mode Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models‘s VLM2VLA writeup calls the “grounding wall” and that Anchor-Align: Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment Anchor-Align attacks with a distillation-against-frozen-VLM loss. DM0.5’s answer is the simpler LR-group knob rather than an auxiliary loss.

Internal automatic VL data generation pipeline. Rather than only lowering the VLM’s LR, DM0.5 also feeds the VL side during action co-training: “we incorporate large-scale image, video, and visual instruction data, and further strengthen scene semantics, physical dynamics, and task progress through an internal automatic data generation pipeline.” The three explicit targets (scene semantics, physical dynamics, task progress) are the axes that would degrade if the VLM branch were left to catastrophically forget while the Action Expert trains. Combining the LR-group knob with a synthesized-VL-data pipeline gives two independent knobs for the same problem.

Systems / release surface. OpenDM ships a full open stack (README §Models, §Training, §Inference): base DM05 weights plus benchmark-fine-tuned checkpoints (LIBERO, RoboTwin2.0, VLA-Arena, SO101 pick-cube LoRA, MEM-Robodojo-Sim ARX X5 bimanual, Table30V2 collection); a dataset-registration API tied to embodiment-specific state descriptions (RobotType.ALOHA, RobotStateDesc.JOINT/GRIPPER, per-image prompt labels like “Head” / “Left wrist” / “Right wrist”); automatic per-experiment normalization statistics computed from the current experiment data + action mode + chunk size and cached under ./norm_stats/; and a fast inference backend that requires TensorRT + Triton + PyTorch flex_attention (torch>=2.5) with a first-launch ONNX export + engine build cost. The default --inference-config.image-prompts "Head" "Left wrist" "Right wrist" example ships a 14-dim output action dim, matching the ALOHA-family bimanual convention.

Results

Reported benchmark comparisons vs π0 / π0.5 / GR00T-N1.7 (higher is better), from the README §Benchmark Results table:

LIBERO (SR): DM0.5 99.0 vs π0 94.4, π0.5 96.9, GR00T-N1.7 97.0.
RoboTwin2.0 (SR): Clean DM0.5 93.6 vs π0 65.9 / π0.5 82.7; Rand DM0.5 93.3 vs π0 58.4 / π0.5 76.8. Widest reported margin on the Rand split (+16.5 pp over π0.5), the split closest to the policy-robustness-under-perturbation claim.
VLA-Arena (SR): L0 89.0 / L1 53.6 / L2 44.1 vs π0 82.3/32.2/11.4 and π0.5 64.3/35.6/24.5. Margin over π0.5 widens with instruction difficulty (+24.7 / +18.0 / +19.6 pp across L0/L1/L2) — the direction the long-context + embodied-CoT bet predicts.
RoboDojo-Sim: DM0.5 Score 24.90 / SR 19.34% vs π0 3.48/1.53%, π0.5 11.41/6.91%, GR00T-N1.7 2.85/1.31%. Reported through the MEM-Robodojo-Sim ARX X5 checkpoint via XPolicyLab PR #101.
RoboChallenge Table30V2 (real-world): DM0.5 Score 54.42 / SR 43.0% vs π0.5 31.48 / 14.3%. The only real-world benchmark on the reported table; π0.5 is the only baseline listed. Sharpest single reported margin (+28.7 pp SR vs π0.5).

Numbers are as reported by the authors and have not been independently verified. External coverage highlights the same 99.0 / 93.5 / 54.42 headline triple for LIBERO / RoboTwin2.0 / Table30V2.

Beyond the tables, the qualitative claims from the blog and Ishaan’s Slack note that are load-bearing for the paper’s positioning are: (i) mid-episode adaptation to a physical top-view camera perturbation without a reset; (ii) in-episode learning from a start-of-episode human demonstration; (iii) preserved general vision-language capability under separate-LR co-training and synthetic-VL feeding; (iv) cross-embodiment reuse (ALOHA bimanual, ARX X5 bimanual, SO101 single-arm, RoboChallenge Table30V2 real-world platform). None of these carry per-episode statistics in the materials fetched.

Why it’s interesting

DM0.5 sits at a distinctive spot on the recipe-lever board this wiki tracks for VLAs. On the architecture side, it looks like π*0.6: a VLA That Learns From Experience (RECAP) π*0.6 — a large VLM backbone with a smaller flow-matching Action Expert — but on the training-recipe side it stakes out a different answer to the same problem: the reported gains come from long-context conditioning + separate-LR co-training + an internal synthetic VL data pipeline, not from RECAP-style RL post-training. That makes DM0.5 the cleanest filed instance of the “keep VL capability strong while learning to act” bet — the same problem Anchor-Align: Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment Anchor-Align attacks with a distillation loss, Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models‘s VLM2VLA writeup frames as catastrophic forgetting during VLM→VLA transition, and Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models Embodied-R1.5 attacks by starting from a stronger embodied VLM to begin with.

On the long-context axis, DM0.5’s ~60 s KV-cache substrate is a different answer to the same problem MEM: Multi-Scale Embodied Memory for Vision Language Action Models MEM attacks with dual-timescale memory and that RoboTTT — 8k-timestep robot policy via test-time training (Jim Fan / NVIDIA GEAR) RoboTTT / WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time WAM-TTT attack with test-time weight updates: DM0.5 stays inside the standard context window and pays the attention cost, MEM adds a compressed long-horizon memory branch, and TTT-family recipes push memory into weights. The three levers have never been benchmarked head-to-head at matched horizons — DM0.5 supplies the “just make the window bigger” datapoint.

Finally, the reported benchmark headline positions DM0.5 as a direct π0.5 challenger from a Chinese lab — the “pi0.5 of China” framing Ishaan flagged. The RoboChallenge Table30V2 real-world 43.0% vs 14.3% margin over π0.5 is the sharpest single reported datapoint, and the RoboTwin2.0 Rand robustness split (+16.5 pp) plus the VLA-Arena L2 hardest-tier lift (+19.6 pp) are the two most useful for stress-testing the “long context + separate-LR + synthetic VL” claim once independent evaluations land. Real-world Table30V2 with π0.5 as the only listed baseline is also the axis most likely to be renegotiated by third-party runs.