Microduck RL — RL training environments for a ~800 g open-source bipedal robot

ListenThe 800-Gram Biped and the Actuator That Ate the Sim2Real Gap9:51

Show notes

A look at Pollen Robotics' **Microduck RL** — a fully open sim2real reinforcement-learning stack for a ~800 g, 14-servo bipedal robot. We dig into why actuator fidelity dominates at this scale, how a shared 61-dimensional observation contract lets the robot hot-swap between walking, fall recovery, and roller-skating policies, and what the "backlash twin" trick teaches about honest domain randomization. We close with what this complete recipe means for anyone shipping learned policies to real hardware. **Sources** - Microduck RL — RL training environments for a ~800 g open-source bipedal robot (Pollen Robotics)

Subscribe to the channel: https://podcast.lumalabs.link/27a25f51ea81d41a8253f46c6133a35e/feed.xml

TL;DR

Pollen Robotics open-sources the full sim2real RL stack for Microduck, a ~800 g, ~25 cm tall bipedal robot driven by 14 Dynamixel XL330 servos. The repo trains PPO policies at 50 Hz on top of mjlab (MuJoCo Warp), covering 13 task families — walking, fall recovery, standup, sit/stand, ball-kick, forward-roll, and roller-skating variants — plus per-task Backlash twins that simulate ±1° gear play. All policies share a 61-dim observation contract so the onboard runtime can hot-swap policies mid-execution. The stack encodes the BAM actuator model (voltage control law + back-EMF + Coulomb/Stribeck/load-dependent friction) with per-env domain randomization on battery voltage, sag, command delay, and friction magnitude, plus MJCF models auto-exported from Onshape via onshape-to-robot. Notable for a Luma reader: it is a rare complete recipe — sim, actuator physics, DR, reward design (documented in AGENTS.md), ONNX export, and the deployed runtime link — for a sub-1 kg biped, at a scale where actuator fidelity is most of the sim2real gap.

Key claims

The full pipeline — MJCF export from Onshape, MuJoCo Warp training via mjlab + PPO, ONNX export with baked-in observation normalizer, and CPU MuJoCo inference for validation — runs from four uv run commands and is designed to yield a usable gait at 4096 parallel envs in ~1–2 h on a single CUDA GPU [§Quickstart].
13 task families are registered, all sharing a 61-dim actor observation layout (48 proprioception + 3 twist + 4 head pose + 6 body pose) so the deployed runtime hot-swaps policies (walk / recover / trick) behind a single interface; unused command slots are zero-padded rather than dropped [§Tasks, §Conventions].
Every main task has a Backlash twin that inserts an unactuated passive_<joint>_backlash hinge in series with each servo, and both the firmware PD emulation (BacklashEncoderBamActuator) and joint_pos / joint_vel observations read through the play (qpos[servo] + qpos[backlash]); observation / action dims are unchanged so ONNX + runtime need no code changes [§Backlash variants].
The actuator uses the BAM M6 model for the Dynamixel XL330 — voltage control law, back-EMF, Coulomb + Stribeck + load-dependent friction — with per-env DR on battery voltage, voltage sag under load, command delay, and friction magnitude, on the argument that at ~800 g body mass driven by tiny servos, actuator fidelity is most of the sim2real gap [§Actuator model].
Four MJCF variants are shipped for different physical regimes: robot_walk.xml (stripped trunk/head contacts, cheap falls), robot_allcollisions.xml (VelStand / StandUp / SitStand / Roulade — body physically contacts the ground), robot_allcollisions_rollers.xml (passive-wheel roller tasks), and generated robot_*_backlash.xml twins via add_backlash.py [§Robot models].
A “roller” sub-family — Velocity-Flat-Rollers, Swizzle, RollerCrouch, RollerSlope, RollerStandUp, Spin — trains the same PPO stack to skate on passive-wheel end-effectors, i.e. the same actuator + observation contract generalized to a non-standard bipedal-locomotion regime [§Tasks].

Method

The repository is built as an application of mjlab — a MuJoCo Warp + rsl_rl training framework — plus a custom actuator (FrictionDRBamActuator), a set of task cfg modules (microduck_*_env_cfg.py), and MJCF assets exported from Onshape. Training is uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096; the exporter (scripts/export.py) bakes the observation normalizer into the ONNX graph, and scripts/infer_policy.py runs the ONNX policy inside CPU MuJoCo for CPU-only debugging or sim2real gap analysis with --debug --save-csv --record. The observation contract (61-dim actor obs, matching command layout across tasks) is what makes runtime policy hot-swapping between walk / recover / trick policies possible — the same interface is consumed by the runtime in the sibling pollen-robotics/microduck repo. Backlash is modeled at the physics level rather than as observation noise: a passive_<joint>_backlash hinge sits between each servo and the output side of the encoder, so the PD firmware emulation reads through the play, matching the real hardware where the encoder is on the output side of the gear train.

Results

The repo is a code + assets release with a demonstrator video montage (real robot walking, standup, forward-roll, roller-skating) rather than a paper-style eval table; no quantitative benchmark numbers are reported in the README. The claimed operational envelope — usable gait at 4096 envs in ~1–2 h on a single CUDA GPU, ONNX export end-to-end, real-robot deployment via the sibling runtime — is documented as a workflow rather than as a benchmarked headline number.

Why it’s interesting

The Microduck stack is a rare complete sim2real recipe for a sub-1 kg humanoid where every piece — MJCF, actuator physics, DR, reward-design playbook (documented in AGENTS.md), ONNX export, and deployed runtime — is open and reproducible. It complements mjswan: real-time interactive MuJoCo + ONNX policy simulation in the browser, which is the browser-hosted inference side of a MuJoCo + ONNX policy pipeline — Microduck is what happens upstream of that: the training pipeline that produces the ONNX. Compared to AI Sapiens Open Source — ROBOTIS Docs Directory — ROBOTIS’s larger open-source K1 humanoid with URDF + MuJoCo assets + cyclo_lab sim training entry point — Microduck operates at a scale (~800 g, 14 servos) where actuator fidelity dominates over kinematic accuracy, and the BAM voltage-control-law actuator model is a much stronger sim2real prior than the ideal-PD model most humanoid stacks use. The unified 61-dim observation contract across 13 policy classes is also unusually disciplined for an open release, and directly enables the runtime policy hot-swapping story that Humanity's Last Machine: A Deep Dive on Humanoid Hardware identifies as an emerging humanoid-software-stack pattern.