Astranomy: An Empirical Study of GPT-6 Astra as a Native Robot Policy

ListenThirty-Six Dollars a Grasp: GPT-6 Astra as a Robot Policy12:43

Show notes

A deep look at **Astranomy**, an NVIDIA SRL empirical study of GPT-6 Astra used directly as a language-conditioned manipulation policy on the full RoboLab-120 benchmark. We walk through the three policy forms — Direct, Agentic, and Orchestrated — the ablations that show persistent history is the real lever, and the brutal capability–latency–cost trade-off: 74.8% success at roughly thirty-six dollars and eighteen-plus seconds per call. We close on what the LLM-as-policy pattern means for how we think about reasoning wrappers in visual generation. **Sources** - Astranomy: An Empirical Study of GPT-6 Astra as a Native Robot Policy (Jenai Xuning Yang, Ashna Khetan, Shun Zhang)

Subscribe to the channel: https://podcast.lumalabs.link/27a25f51ea81d41a8253f46c6133a35e/feed.xml

TL;DR

Astranomy (NVIDIA SRL; Yang, Khetan, Zhang) is a controlled empirical study of GPT-6 Astra as a native language-conditioned manipulation policy on the full RoboLab-120 benchmark, with the action representation, geometry context, and observation history specified entirely at inference time. Three policy forms are measured — Direct (plain API call per action chunk), Agentic (persistent Codex session with history/tools), and Orchestrated (Astra reviewing Cosmos3-Nano-Policy proposals and intervening) — with ablations across action space, dynamic horizon, prompt geometry, observation/text history, task-specification channel (language vs in-context video demo), and model tier (Astra / Sol / Luna / Opus 5.5). Agentic Policy tops out at 74.8% on RoboLab-120. Inference averages 18.56 s/call for Direct EEF and inference accounts for 94.9–97.8% of estimated episode duration across configurations; successful Agentic episodes cost ~$36.40 and 30.9M tokens each. The paper is the first filed controlled, large-scale, same-benchmark characterization of the LLM-as-robot-policy pattern — pinning the capability–latency–cost trade-off on the same substrate (RoboLab — Task-Based Evaluation Benchmark for Robot Manipulation Policies on Isaac Lab (NVLabs)) that other VLA/WAM recipes are measured against.

Key claims

Direct Policy with absolute end-effector targets outperforms joint positions and B-spline joint trajectories (72.7% vs 48.7% vs 37.3% on the 30-task ablation suite); under Agentic Policy, the EEF↔joint-position gap narrows to 2 pp (86.0% vs 84.0%), i.e. action representation matters less once persistent context is in the loop [§3.1.1].
Dynamic action horizon: increasing the chunk range from 3–15 to 16–32 cuts inference calls on successful episodes by 62% at similar wall time but drops success by ~6 pp — shorter chunks give the policy room to correct mid-execution [§3.1.1].
Geometry prompts (gripper + camera descriptions) are load-bearing for Direct Policy and much less so for Agentic Policy — removing them reduces Agentic success by 2.7 pp while using 14.7% more control steps, suggesting the agent reconstructs or substitutes geometric information from history + incremental correction [§3.1.2].
Observation horizon: current-observation-only Direct Policy is the worst configuration; adding any short history helps, but longer/denser history does not add further gains [§3.1.3].
Context history for Agentic Policy: retaining full text history + current images outperforms visual-history-only; full text + visual history reaches 100% on a 23-task subset. Text history is more valuable than visual history in isolation [§3.1.3].
Task specification: default language instructions beat vague instructions by 8.7 pp (the model does not reason to overcome specificity); in-context learning from a sampled successful rollout performs on-par with language but produces 8.8% longer episodes [§3.1.4].
Model tier under Direct Policy: Astra 72.7%, Sol 33.3%, Luna 2.0%; increasing reasoning effort from medium to xhigh changes success little but adds latency and tokens. Astra is the only tested general model that works as a direct policy here [§3.1.5].
Agentic tool access (code execution, image inspection, scratch workspace) is used aggressively — 19.7 tool-calls per episode on average, including stereo triangulation and IK computation — but does not clearly improve success over the no-tools Agentic variant [§3.2.1].
Explicit written planning is not required: removing the planning instruction nearly eliminates standalone plan commentary but leaves progress-tracking and next-subgoal reasoning intact inside action explanations [§3.2.2].
Motion-tracking error feedback has little effect — explicit references appear in 51% of episodes but only in 1.14% of next-action explanations; Astra identifies and responds to tracking failures from retained history + fresh robot-state alone [§3.2.3].
Orchestrated Policy interventions on Cosmos3-Nano-Policy proposals account for 14.5% of simulation steps; 64.4% of intervention blocks address grasping problems and 20.2% address object-selection/planning mistakes — the specific stages where the base WAM is weakest and the general reasoner adds the most [§3.3].
Headline RoboLab-120 result: Agentic Policy 74.8% best among the three forms; Direct and Orchestrated numbers reported in the full-benchmark section [§3.4, Table 2].
Runtime: Direct Policy EEF averages 18.56 s per inference call; Agentic 13.37 s; Orchestrated 11.50 s (Astra) + 2.74 s (Cosmos). Inference accounts for 94.9–97.8% of estimated episode duration across all three Astra forms. Reference points: FLUX 3 Action 397.74 ms, Cosmos3-Nano-Policy 713.76 ms, π0.5 46.96 ms per chunk on H200 BF16 [§4.1, Table 3].
Token cost per successful episode (Oct 1 2026 pricing): Direct $2.26 (Astra), Orchestrated$ 17.72, Agentic $36.40 — output tokens including reported reasoning dominate every tier. Sol runs the same Direct recipe for$ 0.77 at 33.3% success, Luna for $0.03 at 2.0% success [§4.2, Table 4].
Dominant failure modes concentrate at contact establishment and placement completion — gripper repeatedly adjusts around an object without securing it, or objects are displaced without reaching their intended destination; Agentic improves overall reliability but these failures persist, as do incorrect receptacle selection, obstructive grasps, dropped-during-transport, and ineffective recovery loops [§6].

Method

Astranomy evaluates GPT-6 Astra on RoboLab-120 (RoboLab — Task-Based Evaluation Benchmark for Robot Manipulation Policies on Isaac Lab (NVLabs)) under three fixed policy forms. Direct Policy makes stateless API calls returning an action chunk (EEF pose, joint target, or clamped cubic B-spline over 6 × 8-D coefficient vectors) with dynamic chunk length within a prompted range (3–15 or 16–32), executed open-loop before the next observation. Agentic Policy runs inside a persistent Codex session that retains full conversation history and exposes shell/code execution, image inspection, and a scratch workspace; each decision emits one EEF target executed for 1–5 control steps. Orchestrated Policy uses Astra inside the same Codex harness but with native tools disabled and bounded memory — Astra reviews a 15-step action trajectory proposed by Cosmos3-Nano-Policy (a WAM finetuned on DROID) and either accepts it or intervenes with a shorter edited trajectory or a direct EEF target. Simulation runs on NVIDIA L40s GPUs at 15 Hz with the simulator paused during inference, 5 fixed-seed episodes per task, 8,192 completion tokens per call, no retry limit, three camera views (left shoulder, right shoulder, wrist) at 384×216. Ablations are conducted on a 30-task capability-stratified subset covering 10 manipulation groups (object selection/container placement, multi-object sorting, varied-shape grasping, narrow-opening placement, reorientation, stacking, shelf manipulation, spatial relations, counting/alternatives/sequencing, long-horizon cleanup with exclusions), then the strongest configuration per policy is run on all 120 tasks. Headline metrics: Success Rate and partial Score (100 for success). Model comparison includes GPT-6 Astra / Sol / Luna and Claude Opus 5.5 under Direct Policy at matched settings, plus medium vs xhigh reasoning effort on Astra.

Results

Best-of-three on RoboLab-120 is Agentic Policy at 74.8%; Direct and Orchestrated numbers are tabulated per policy in §3.4 (Table 2 renders client-side on the project page and did not resolve in the fetched body). The 30-task ablation suite is where the method-level evidence lives: Direct EEF 72.7% vs joint-pos 48.7% vs B-spline 37.3%; Agentic EEF 86.0% vs joint-pos 84.0%; model-tier Direct Astra 72.7% / Sol 33.3% / Luna 2.0%; Agentic full-history 100% on a 23-task subset. Orchestrated interventions address grasping 64.4% and object-selection/planning 20.2% of intervention blocks. Runtime: Direct EEF 18.56 s/call, Agentic 13.37 s, Orchestrated 11.50 + 2.74 s, inference 94.9–97.8% of estimated episode duration across Astra configurations vs 4.6–25.6% for the three reference WAM/VLA baselines (π0.5, Cosmos3-Nano-Policy, FLUX 3 Action). Token cost: Direct Astra $2.26 / episode, Orchestrated$ 17.72, Agentic $36.40 — a two-order-of-magnitude cost premium over the specialized-model baselines whose per-chunk latency is already published in FLUX 3 Action: A 7B World Action Model for Robot Control. A yogurt-task example completes in ~3:57 estimated wall time of which ~5.2 s is action execution.

Why it’s interesting

Astranomy pins the LLM-as-robot-policy thread on this wiki — Wenli Xiao — GPT-6 Astra does physical in-context learning from a single human video via a motion-planning tool harness (Astra + IK first-pass ICL), Jay Chooi (Robocurve) — Astra vs Fable 5.1 on real-robot manipulation via IK / Comparing Fable 5.1 to Fable 5 on robotic manipulation (Robocurve Astra vs Fable), thijs — Astra paints the Golden Gate Bridge with a real robot arm (Astra paints Golden Gate Bridge), Wentao Zhu — GPT-6 Astra autonomously trains an Isaac Lab RL policy for pen spinning on the Sharpa hand over 1.5 days (day-scale Astra driving Isaac Lab), HomeBody: A Humanoid That Explores, Remembers, and Acts on Its Own (HomeBody G1 whole-body skill library), Agent as Policy for Robotic Manipulation (AGP paper-scale version), OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies (OpenRUA zero-abstraction ROS 2), RobotUse: Allocating Computation, Context, and Decisions (RobotUse three-role factorization) — to the same measured substrate (RoboLab-120) that other recipe-lever-board entries report against. The capability number (74.8% Agentic) is directly comparable to RobotUse’s 45% on the same benchmark and the ARC + π0.5 real-robot number of 91.7% (ARC: A Reasoning Recipe for Robot Foundation Models) — making this the first filed paper that lets a reader read the LLM-as-policy, VLA-with-reasoning-wrapper, and WAM-with-companion-reasoner recipes against each other at a common axis. The two most load-bearing structural findings for the recipe-lever board tracked on VLA Models: (i) persistent conversation history dominates agentic tool-use and explicit planning as the lever that lifts Direct → Agentic from 72.7 → 86.0% on the ablation suite (relevant to the memory + ACM thread on Tool-Use Agents); (ii) the two-order-of-magnitude cost premium — $36.40/success for Agentic vs ~sub-second specialized VLA inference — is the sharpest quantified datapoint yet behind Isola’s “robot-use agents” positional thesis (Robot-Use Agents) that per-robot performance stays below dedicated policies but intelligence diffuses through the whole ecosystem. The failure-mode taxonomy (grasp establishment and placement completion) reproduces the Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level sensorimotor counter-position prediction and complements the Robocurve Comparing Fable 5.1 to Fable 5 on robotic manipulation contact-stall observation on the same hardware family — now with large-sample empirical weight behind a specific failure mode rather than qualitative anecdote.