Do robot policies transfer between physics engines?

TL;DR

Manda Robotics trains the same policy with the same assets across four physics engines (MuJoCo, PhysX / Isaac Sim, Newton / MuJoCo-Warp, Genesis) and measures what breaks when training in one and evaluating in another. Across peg insertion, ANYmal-D quadruped locomotion in Isaac Lab, Isaac Lab’s grasp-and-lift task with Franka, and finetuning π0.5 inside Isaac Lab’s RoboLab, the choice of engine materially changes success rates, learned gait, and even which seeds survive — identically authored foot friction is resolved to 0.8/0.6 in PhysX but 1.0 in Newton, and a single PhysX solver swap (TGS 100/1 → PGS) moves a “fragile” policy from 20% to 84% success without changing weights. Even validated cross-engine transfer recipes still lose 20–29 points, and generalist π0.5 shows failed-grasp divergence (41 vs 13 matched episodes) between Newton and PhysX. Second post in a three-part Manda series on simulation engines for robotics; part one covered scripted controllers, part three (sim-to-real transfer) is still pending.

Key claims

A policy that solves 49/50 MuJoCo insertions can collapse to 11/50 in PhysX GPU on the worst training seed, with mean success dropping from ~99.6% (MuJoCo / Newton / Genesis) to 74.4% in PhysX — all PhysX failures are overload stops, not depth misses [§1 Insertion results table].
Within a single engine, one solver setting moves the worst policy from 5/25 to 21/25 success (PhysX PGS vs TGS 100/1) at Holm-corrected p=0.001 on held-out starts, and TGS 4/0 (Isaac Lab’s locomotion default) gives 19/25 at p=0.003 — more solver iterations do not make transfer safer [§2 Solver setting table].
Identically authored foot friction is resolved to 0.8 static / 0.6 dynamic in PhysX (foot × ground product) but to 1.0 in Newton (max of the two coefficients), from the same URDF — a silent engine-level divergence that only shows up by reading values back after scene load [§3 Isaac Lab defaults table].
Training engine imprints a persistent gait signature: every PhysX-trained ANYmal-D seed uses more joint torque than every Newton-trained seed, in both evaluation engines, after matching friction, and under Newton’s hard-contact Kamino solver (exact permutation test p=0.008, smallest possible with 5 seeds/group) [§3 Locomotion].
Isaac Lab’s own validated cross-backend recipe still loses 20–29 success points in grasp-and-lift: Newton→PhysX drops from 88% to 68%, PhysX→Newton from 82% to 53%, with intra-engine scores varying 55–88% just from evaluation-condition changes [§4 Grasp-and-Lift].
Generalist π0.5 succeeds in 80–100% of episodes in both PhysX and Newton inside RoboLab, yet 41 matched episodes see more failed grasps in Newton against 13 in PhysX — and scenes saved at rest in PhysX are not at rest in Newton (17 vs 4 successes in 40 episodes) [§5 RoboLab].
MuJoCo’s contact time constant is the one knob whose variation reproduces PhysX’s failure mode on seed 4 (99.6% → fails via overload stops) — stiffening MuJoCo to 2 ms gives seed 4 scores of 14/50, matching PhysX GPU’s 11/50; step size, friction, impedance curve, friction cone, impratio, noslip all leave seed 4 at nominal [§2 Contact-stiffness results].

Method

The study holds assets, PPO hyperparameters, actor/critic architecture (64×64 tanh), control rates (50 Hz policy, 1 kHz impedance / torque), and reward functions constant across engines; only the physics backend changes. Four tasks are run: (1) a Panda rod insertion with 0.10/0.25 mm radial clearances, trained 5 seeds in MuJoCo at 262K transitions each, evaluated on 50 fixed new starts in all four engines; (2) ANYmal-D locomotion in Isaac Lab 3.0 early access, 5 seeds × 2 engines × 1,000 iterations with Isaac Lab’s stock task/reward, evaluated at 5 commanded velocities with 320 robots per cell; (3) Isaac Lab’s grasp-and-lift Franka task (8 objects, zero-grav curriculum, randomized friction/mass/joint gains, no body-velocity observations); (4) π0.5 finetuned in RoboLab with scenes saved at rest in PhysX and transferred to Newton. Every requested solver / step / friction / contact-stiffness value is read back from the compiled MuJoCo model or live PhysX stage before the first episode, with the run halting if the value does not match — a discipline the paper argues makes the comparison meaningful. Statistical tests include exact McNemar with Holm correction for the solver-settings study, Fisher’s exact for pass/fall counts, and exact-permutation tests for across-seed gait differences.

Results

Headline numbers are in Key claims. Three further points worth filing: (a) training directly in PhysX with the same recipe does not fix the problem — a MuJoCo-trained baseline scores 8/8 on PhysX GPU validation starts, while a PhysX-trained baseline scores 1/8, so the gap is in the learning setup under the PhysX contact model, not in a sim2sim transfer artifact; (b) a force-penalty reward rescues the worst MuJoCo-trained seed in PhysX (11/50 → 37/50) but most other seeds regress slightly — a counterexample to the hope that one reward shaping term universally improves transfer; (c) at coarser physics steps (10 ms, 20 ms), both engines produce policies that never learn to walk — feet sink or jitter through the ground — a settings regime where the learning setup breaks before any transfer measurement is meaningful. All simulated contact loads are measurements, not ground truth: a passing depth criterion does not certify a penetration-free fit, and agreement between simulators says nothing about agreement with the real robot (which is the subject of the promised part three).

Why it’s interesting

For RL Environment Platforms, this is the sharpest filed negative result on the “swap simulators freely” assumption many entries on that page implicitly rely on — the vendor-aligned MuJoCo stacks (Microduck RL — RL training environments for a ~800 g open-source bipedal robot, Dex-SuperMimic — Reproducing SuperMimic on Dexterous Hand Manipulation) that keep sim and real on one physics core now have a concrete number for how badly the alternative (train in one engine, eval in another) can leak. It also quantifies what HydroShear: Hydroelastic Shear Simulation for Tactile Sim-to-Real Reinforcement Learning HydroShear and Genesis World — Simulation platform for general-purpose robotics & embodied AI learning Genesis implicitly bet on by shipping their own physics: identically authored assets are not identical scenes, and the engine-level resolution differences (friction product vs max, contact stiffness clamp, friction cone) show up as policy-level success rate swings. For Simulator-in-the-Loop Refinement, the solver-swap result (20% → 84% from TGS→PGS alone) is a cautionary datapoint: a sim-in-the-loop refinement loop that holds the engine fixed but is unaware of its solver settings could silently flatter or sabotage every candidate it evaluates.