Evo-RL: Open Real-World Offline RL on SO-101 and AgileX PiPER

TL;DR

Evo-RL is an Apache-2.0 open-source toolkit from SJTU and Evo-Tech that ports the RECAP iterated-offline-RL recipe (π*0.6: a VLA That Learns From Experience (RECAP)) to two commodity robot platforms — Hugging Face’s SO-101 (single and bimanual) and AgileX PiPER / PiPER-X — built on top of LeRobot 0.4.4. The full closed loop is shipped as four CLI commands (lerobot-human-inloop-record, lerobot-value-train, lerobot-value-infer, lerobot-train) wrapping the same architecture diagrammed in π*0.6: collect demos + human-in-loop interventions, train a multi-task distributional value head (default --value.type=pistar06), binarize advantages into an indicator field, then SFT-fine-tune the policy on text-injected positive/negative tags. A companion RW-RL dataset is released on Hugging Face; the actual VLA backbone and trained model weights are listed as “coming soon.”

Key claims

Evo-RL replicates the π*0.6 RECAP loop end-to-end on two non-Physical-Intelligence robot platforms — SO-101 and AgileX PiPER — and ships it as a single LeRobot fork with CLI entry points covering the entire pipeline (Quick Start, §4-§7 in README).
The default value function type is --value.type=pistar06, explicitly linking the toolkit to Physical Intelligence’s π*0.6 design (README §4); the README also documents a pluggable extension path for alternate value heads via src/lerobot/values/<your_value>/{configuration,modeling,processor}_<your_value>.py (README §4 “To plug in a different value function”).
Advantage-conditioned policy training is preserved as the policy-extraction trick from RECAP: the value-inference step writes back complementary_info.advantage_<TAG> and complementary_info.acp_indicator_<TAG> columns, and policy training conditions on the binarized indicator with a configurable text-tag dropout (--acp.indicator_dropout_prob=0.3 default) (README §5, §6).
Advantage binarization uses an n-step return horizon (--acp.n_step=50) and a per-task positive-ratio cutoff (--acp.positive_ratio=0.3, top 30%), rather than the per-task threshold ε_ℓ described in the π*0.6 paper — a concrete reproducibility decision (README §5).
The toolkit imposes a hard requirement that the trained policy support text/task input, since advantage-conditioning tags are injected into the task text rather than as a separate input head (README §6 “Policy requirement”).
Human-in-the-loop intervention is first-class: the data-collection CLI exposes hotkey-driven intervention toggling (i), per-episode success/failure marking (s/f), and writes back complementary_info.is_intervention, complementary_info.state, complementary_info.collector_policy_id, and an episode-level episode_success label (README §3, §7).
The dataset schema and iterative loop are explicitly designed so each new policy round (π_k → π_{k+1}) just appends to the data pool with --resume=true, then re-runs value-training + advantage-inference + policy-training (README §7 “Iterative training loop”).
Bimanual operation is supported on both platforms with mirrored leader-arm hardware on SO-101 and CAN-bus dual-arm config on PiPER, with the same lerobot-human-inloop-record interface (README §2, §3).
The accompanying RW-RL dataset on Hugging Face (MINT-SJTU/RW-RL-Dataset) bundles teleoperation demos, intervention data, policy rollout traces, success/failure labels, and value-modeling complementary signals — explicitly framed as offline-RL companion data (README “Model & Dataset”).

Method

Evo-RL is structured as a LeRobot fork (lerobot==0.4.4) wrapping the existing recording / training CLIs with three new ones: lerobot-human-inloop-record (interventionable data collection), lerobot-value-train (multi-task distributional value head), and lerobot-value-infer (write-back of value / advantage / indicator columns into the dataset). The policy is then trained with the standard lerobot-train plus new --acp.* flags that inject the binarized indicator into the task-text prefix at training time. Multi-GPU is via accelerate launch --multi_gpu. The iterative loop diagram in README §7 mirrors the RECAP pseudocode from π*0.6 nearly one-to-one: deploy → collect new data → merge → value-train → advantage-infer → policy-train → repeat.

Hardware support is the differentiator. SO-101 uses USB-serial leader/follower arms (recommended /dev/serial/by-id/) and OpenCV or RealSense cameras (recommended /dev/v4l/by-id/ or /dev/v4l/by-path/, with fourcc: "MJPG" for OpenCV and warmup_s for RealSense). AgileX PiPER / PiPER-X use CAN-bus interfaces (lerobot-setup-can) and require firmware ≥ 1.8.5 with all arms in follower mode (0xFC). The bimanual config layout (bi_so101_follower_{left,right}.json calibration files, separate left_arm_config.port / right_arm_config.port flags) is shipped as a working template.

Results

The repository ships no quantitative benchmarks at filing time — the model card and trained weights are both listed as “coming soon,” and the paper badge reads “Coming Soon.” Visual qualitative results are GIFs in website/assets/gifs/: value-overlay success and failure episodes (frames 0405 and 0697), policy-rollout examples, and human-in-the-loop sessions. The news log records two reproducibility milestones: 2026-02-26 (first SO-101 RL baseline) and 2026-03-07 (AgileX PiPER/PiPER-X support added).

Why it’s interesting

This is the first filed open-source community implementation of the π*0.6: a VLA That Learns From Experience (RECAP) RECAP recipe on commodity hardware that the broader robotics community already owns — SO-101 (Hugging Face’s $250 educational arm) and AgileX PiPER (a mid-tier industrial bimanual platform). π*0.6’s paper is a strong recipe but locked to PI’s proprietary VLA backbone and proprietary in-house tasks; Evo-RL is the open-recipe + open-hardware counterpart, which is what would have to exist for a reproducibility-and-extension community to form around advantage-conditioned VLA training. Worth contrasting with Spirit-v1.5: A Robotic Foundation Model by Spirit AI (Spirit-v1.5’s “clean teleop is the lever” position on Qwen3-VL+DiT) and RAIDEN: A Toolkit for Policy Learning with YAM Bimanual Robot Arms (RAIDEN’s data-collection-only stack on YAM bimanual arms) — those are the data-side opens; Evo-RL is the RL-recipe open.

The choice to make --value.type=pistar06 the default value function — and to document the plug-in path for swapping it out — is a deliberate scaffold for the next round of RECAP-variant research: anyone with a different value-function design (n-step vs. Monte Carlo, distributional vs. scalar, language-conditioned vs. not) can drop it in and re-run the loop on the same SO-101 hardware. The decision to threshold advantages by a per-task positive ratio (top 30%) instead of π*0.6’s per-task ε_ℓ is also a concrete, ablatable design point — the paper itself notes ε_ℓ was tuned per task, and Evo-RL’s ratio-based cutoff sidesteps that tuning at the cost of assuming a fixed positive fraction across tasks.

The CFGRL connection is preserved but implicit: the indicator-dropout flag (--acp.indicator_dropout_prob=0.3) is doing the same job as classifier-free guidance’s unconditional-dropout — training the policy to operate both with and without the advantage tag, which is the Classifier-Free Guidance variants primitive applied to policy extraction.