Wentao Zhu — GPT-6 Astra autonomously trains an Isaac Lab RL policy for pen spinning on the Sharpa hand over 1.5 days

TL;DR

Wentao Zhu (EIT-HAI, first author of Thea: Towards the Harness of Embodied Agents) reports the fourth entry in his GPT-6 Astra test series: hand Astra a one-line prompt asking for pen-spinning on a Sharpa dexterous hand — with “Use Isaac Lab for RL training, use the Sharpa hand, and create the pen mesh yourself” — and let it run autonomously for ~1.5 days. Astra returns a trained RL policy and a visualization video. Under the hood the run reportedly includes web search, paper download, mesh authoring, Isaac Lab environment setup, and the actual RL training loop, all inside a single codex-app session with the researcher as spectator. Credit to student Chengyang Li. This is the first filed instance where a frontier LLM as a tool-using agent has autonomously run an end-to-end physical-robotics ML research project on the timescale of a small paper, not just a control-loop demo.

Key claims

Prompt was a single natural-language instruction: implement pen spinning with a dexterous hand, use Isaac Lab for RL, use the Sharpa hand, author the pen mesh, and produce a trained policy + visualization video [Post 1/thread].
Astra ran autonomously for ~1.5 days end-to-end, including the RL training step, with no reported mid-session intervention from the researcher [Post 1/thread].
The scope explicitly included web search and paper/asset download as sub-tasks Astra was free to invoke as needed [Post 1/thread].
Deliverables at the end of the run were a trained RL policy and a visualization video showing the policy executing pen spinning [Post 1/thread + attached video].
The researcher frames this as “GPT-6 Astra test 4/n” — a series of qualitative capability probes rather than a controlled study [Post 1/thread].

Method

Single-post tweet (with a companion video slot) documenting a one-shot demo. The prompt is quoted verbatim in the tweet body. Wentao Zhu is spectator; his student Chengyang Li is credited but the tweet does not specify Li’s role (harness author? evaluator? scaffolder? unclear). No paper, no code, no released harness, no logs. Hardware side is the Sharpa 22-DoF five-finger tactile hand (same platform as the Sharpa releases the Origami Dataset for the IROS 2026 Robotic Origami Challenge Origami Dataset release), simulated in NVIDIA Isaac Lab; the pen is a mesh Astra was instructed to author itself. RL algorithm, reward function, curriculum, sim-to-real gap, tokens consumed, GPU time consumed, and cost are all unreported. The single downstream artifact is a visualization video; whether the policy transfers to real hardware, generalizes across pen geometries, or matches a hand-tuned baseline is not addressed.

Results

Qualitative single-case demo. One trained RL policy for pen spinning on a simulated Sharpa hand, delivered inside a ~1.5-day autonomous run initiated by a single-sentence prompt. No baseline (hand-tuned researcher policy, prior Astra attempts, alternative LLM controllers), no ablation, no success metric on the pen-spinning task itself beyond “here is the video,” no report of how many failed attempts the run went through, no reproduction. No indication whether the run was cherry-picked from among multiple Astra sessions.

Why it’s interesting

This is the point on the AI-for-AI Research map where the autonomous-research loop closes through simulated physical robotics end-to-end, and the missing seat between two clusters the wiki already tracks. Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster and AlphaGo Moment for Model Architecture Discovery closed autonomous loops on ML-training scripts; ENPIRE: Agentic Robot Policy Self-Improvement in the Real World and GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks closed autonomous loops on real-robot policy improvement given a working policy scaffold. This thread’s Astra run collapses both into a single 1.5-day session: define the task, choose the simulator, author the assets, set up the RL environment, train the policy, produce the deliverable. Two moves are worth flagging for the internal recipe-lever conversation: (i) it is the physical-simulated counterpart to Wenli Xiao — GPT-6 Astra does physical in-context learning from a single human video via a motion-planning tool harness‘s no-training LLM-as-controller pattern — same model, same series-author framing, but the LLM is running the RL loop rather than replacing it, extending the Tool-Use Agents “code + web + sim as tools” substrate to Isaac Lab as an interactive tool surface; (ii) the choice of pen spinning — a canonical open-loop dexterity benchmark that classical RL groups spend paper-length effort on — plus the Sharpa releases the Origami Dataset for the IROS 2026 Robotic Origami Challenge Sharpa hand + Isaac Lab combination lands directly on the same hardware/simulator stack the wiki tracks elsewhere, so the run is in principle reproducible by any lab with the same substrate. The productive-tension counterpoint is the same one every Astra-as-agent thread carries: no baselines, no metrics, no comparison to a researcher-directed run at matched compute, and the failure-mode literature (contact-rich manipulation as the LLM ceiling — Comparing Fable 5.1 to Fable 5 on robotic manipulation, Gemini Robotics-ER 2 vs Claude Opus 5 clapboard eval — ER2 threw robot arms off table, hallucinated success (Robocurve / Jay Chooi)) applies even more strongly to training a contact-rich policy than to executing one. Wentao Zhu’s own Thea: Towards the Harness of Embodied Agents Thea position paper names “evaluation as exit codes” as the missing primitive for embodied agents; this thread demos exactly the pattern Thea argues for — no exit codes exposed to the reader.