ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning

ListenOne Prior to Grasp Them All: RL Dexterity with ADEPT10:30

Show notes

This episode digs into ADEPT, a large-scale RL framework from NVIDIA and the University of Michigan for sim-to-real dexterity on high-DoF multi-fingered hands. We unpack the core idea — pretrain one generic object-reposing prior, then adapt it with a careful three-step recipe that avoids erasing what it learned — plus the joint-space Geometric Fabric and the teacher-to-student distillation that lands it on real hardware. We also draw the obvious parallels to pretrain-then-adapt thinking in image and video generative modeling. **Sources** - ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning

Subscribe to the channel: https://podcast.lumalabs.link/27a25f51ea81d41a8253f46c6133a35e/feed.xml

TL;DR

ADEPT (NVIDIA + U. Michigan; Lee, Yin et al.) is a large-scale RL framework for sim-to-real transferable dexterity on high-DoF multi-fingered hands that separates a reusable dexterous prior from task-specific post-training. A single generic object-reposing policy is pretrained across 16 primitive shapes with automatic domain randomization, then adapted to contact-rich downstream tasks via a stable three-step recipe (behavior-cloning distillation of the prior, critic warm-up with the policy frozen, and conservative PPO with reduced clip range and learning rate) that avoids erasing the pretrained behavior. Post-trained teachers are then distilled into perceptive students that consume raw RGB (and vision-based tactile) and zero-shot sim-to-real on two very different embodiments — 23-DoF Kuka-Allegro and 29-DoF Flexiv-Sharpa. A joint-space Geometric Fabric mediates between the RL policy and the robot, enforcing joint limits and collision safety in the same controller for sim and real.

Key claims

Pre-training on a generic object-reposing task over 16 randomized primitives amortizes the cost of learning reusable dexterous behavior — each new task then post-trains in ~3B environment steps vs ~9B needed from scratch, which “rarely succeeds” on multi-fingered robots [project page, §1].
Naïve RL fine-tuning of the pretrained prior rapidly degrades zero-shot reposing capability during transfer to a contact-rich downstream task [Abstract].
A three-component post-training recipe stabilizes the transfer: (i) behavior-cloning distillation from the pretrained prior πpre into πpost, (ii) critic warm-up where Vpost trains while πpost is frozen, then (iii) conservative on-policy PPO with reduced clip ε and reduced learning rate [project page, “The ADEPT recipe” figure].
A joint-space Geometric Fabric sits between the RL policy and the robot in both simulation and hardware, enforcing joint limits and collision safety while exposing the arm-hand system’s full kinematic dexterity to the policy [Abstract, project page].
Post-trained teachers are distilled into stereo-RGB perceptive students via a two-stage curriculum combining a BC loss and an auxiliary loss, yielding a single continuous policy from raw perception to 23-D joint-space actions [project page pipeline figure].
The recipe transfers across two embodiments with different sensor suites: 23-DoF Kuka-Allegro with two RGB cameras, and 29-DoF Flexiv-Sharpa with two RGB cameras + five vision-based tactile sensors [Abstract].
The pre-training curriculum uses only 16 primitive shapes at randomized scales (50 mm sphere through 250 mm rod) — no scripting, no demonstrations, no additional task-specific rewards [project page “Pre-training diet”].
The post-trained specialists reach human-level speed on long-horizon contact-rich insertion tasks from challenging initial states, executed zero-shot on real hardware from raw visuo-tactile observations [Abstract].

Method

ADEPT is organized as four stages sharing the same joint-space Geometric Fabric controller. Stage 1 (pre-training). A single dexterous prior πpre is trained end-to-end with PPO under Automatic Domain Randomization on a generic object-reposing task: each episode picks one of 16 primitive shapes at a randomized scale, and the reward rpre drives the hand to move the object to a target pose. Because the task is generic, the same policy learns a broad kinematic repertoire (reach, grasp, lift, reorient, transport) that carries into any downstream contact-rich task starting from a reposing sub-goal.

Stage 2 (post-training). Adapting πpre to a specialist πpost happens in three sub-steps designed to never erase the prior. First, behavior-cloning distillation regularizes πpost toward πpre. Second, a critic-warm-up phase trains Vpost while πpost is frozen so PPO’s advantage estimate is well-calibrated before any policy update. Third, conservative PPO fine-tunes πpost with a reduced clip range and reduced learning rate — the specific choices the paper credits with letting the specialist add contact-rich behavior on top of the prior rather than overwriting it.

Stage 3 (distillation to perception). The post-trained state-based specialist becomes the teacher for a stereo-RGB (optionally vision-based-tactile) student trained with a BC loss plus an auxiliary loss. The student consumes raw images and outputs 23-D joint-space actions.

Stage 4 (sim-to-real). The student runs zero-shot on real hardware, with the same joint-space Geometric Fabric mediating between policy commands and physical actuation, enforcing joint limits and collision avoidance while exposing the full kinematic DoF.

Results

Numeric per-task success rates and speed comparisons are not tabulated in the arxiv abstract or the project page text available at filing. The reported claims are:

Post-trained perceptive students zero-shot sim-to-real transfer on both 23-DoF Kuka-Allegro (two RGB) and 29-DoF Flexiv-Sharpa (two RGB + five vision-based tactile sensors) [Abstract].
The perceptive students solve long-horizon tasks from challenging initial states with dexterity at human-level speed [Abstract].
Per-task post-training cost is ~3B environment steps vs the “around 9B” a from-scratch task-specific policy requires, and from-scratch attempts “rarely succeed” [project page “The idea”].

Why it’s interesting

ADEPT stakes out a pretraining-plus-conservative-RL-adaptation recipe row for the VLA Models concept board that sits distinctively next to π*0.6 / RECAP’s advantage-conditioned SFT (π*0.6: a VLA That Learns From Experience (RECAP)) — both are RL-with-a-value-function stories for contact-rich manipulation, but ADEPT trains one generic prior over primitives and then adapts, while RECAP fine-tunes a flow-matching VLA in place with a binarized-value text token. It also complements Simulation Pre-training for Dexterity (SPD) (SPD) on the Synthetic Training Data axis: SPD collects sim VR-teleop as the pre-training substrate and fine-tunes with real teleop, whereas ADEPT keeps everything in sim, generates behavior with RL over primitives, and closes the loop via teacher→student distillation. On Tactile sensing for manipulation, the Flexiv-Sharpa deployment is the first filed instance on the wiki where five vision-based tactile sensors are consumed by a policy that was trained entirely in simulation and zero-shots to hardware — a concrete existence proof for the sim-first side of the tactile-integration debate that Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks and Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map have been arguing for on the tooling side.