OSCAR: Omni-Embodiment Action-Conditioned World Model for Robotics

TL;DR

OSCAR is an action-conditioned video world model for robotics that finetunes NVIDIA’s Cosmos-Predict2.5-2B on a curated multi-embodiment corpus (robot arms and egocentric human data) using 2D kinematic skeleton renderings as the unified conditioning representation. Training runs on a single GH200 GPU. The headline use case is policy evaluation in the generated world rather than data generation: rollouts of RoboArena policies inside OSCAR show significant correlation with the corresponding real-world evaluation outcomes, positioning the model as an inexpensive virtual stand-in for the real-robot benchmark.

Key claims

A standardized data pipeline curates, filters, and deduplicates broad robotics and egocentric human datasets into a clean joint-training corpus spanning diverse tasks, scenarios, actions, and robot embodiments [Abstract].
2D kinematic skeleton rendering is adopted as a unified conditioning representation that generalizes across different robot arms and even human hands, sidestepping per-embodiment action spaces [Abstract].
The model is built by finetuning Cosmos-Predict2.5-2B on a single GH200 GPU and reports significant improvements over baselines on action following, appearance quality, and motion consistency despite baselines being either much larger or requiring more GPUs [Abstract].
OSCAR is deployed to evaluate RoboArena robot policies, and the in-OSCAR virtual evaluation correlates significantly with the real-world evaluation outcomes [Abstract].

Method

OSCAR finetunes Cosmos-Predict2.5-2B — NVIDIA’s 2B-parameter video world model — on a curated multi-source corpus that combines robotics datasets across embodiments with egocentric human-hand data. The conditioning interface is a 2D kinematic-skeleton rendering of the controllable agent (robot arm or human hand) overlaid onto the current observation; this lets a single backbone consume actions from disparate embodiments without per-robot action heads, and it lets human videos contribute as additional embodiment data rather than being held out. Training is performed on a single GH200 GPU, which the paper emphasizes as a contrast with the multi-GPU / larger-model baselines it compares against. For policy evaluation, RoboArena policies are rolled out in OSCAR’s generated worlds and their virtual outcomes are compared against the corresponding real-world RoboArena results.

Results

The abstract reports two quantitative claims without specific numbers. First, OSCAR improves on existing baselines on three axes — action following, appearance quality, and motion consistency — while using either a smaller model or fewer GPUs than the baselines it compares against. Second, the in-OSCAR virtual policy evaluation of RoboArena policies shows significant correlation with the real-world RoboArena evaluation. The body of the paper presumably contains the breakdown by policy and task; numbers are not surfaced in the abstract.

Why it’s interesting

OSCAR is a third-party instance of the WFM-as-policy-evaluator pattern that Evaluating Gemini Robotics Policies in a Veo World Simulator established for Veo at Google DeepMind — but built on the open Cosmos stack instead of a closed flagship, and explicitly cross-embodiment via the skeleton-rendering trick. The Veo work uses pose-overlay conditioning + multi-view tiling on top of a closed Veo backbone with undisclosed compute; OSCAR uses skeleton-overlay conditioning on top of open Cosmos-Predict2.5-2B with a single-GH200 finetune. The two together start to define the design space of “specialize a flagship video model into a manipulation simulator for policy ranking”.

The policy-side coupling is tighter than Veo’s: OSCAR validates against RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies, the actual decentralized real-robot ranking system that the wiki already tracks as the trust contract for generalist policies. That pairing is the natural next move — RoboArena says “the only trustworthy ranking is real-world distributed evaluation”, and OSCAR claims to reproduce that ranking inside a generative simulator. If the correlation is strong enough, this is a concrete answer to the Sitzmann action-conditional-WFM chicken-and-egg (World Foundation Models open questions): policy evaluation, not just data generation, is the high-leverage downstream use.

The skeleton-rendering conditioning is also worth flagging against Cosmos 3: Omnimodal World Models for Physical AI (Cosmos 3’s native numerical-action channel) — OSCAR sticks with a visual action representation (skeletons rendered to pixels) on the 2.5-generation backbone, which is what makes the cross-embodiment + human-hand transfer work without a unified numerical action space.