Skip to content
Bud's Wiki

Jay Chooi (Robocurve) — Astra vs Fable 5.1 on real-robot manipulation via IK

TL;DR

Section titled “TL;DR”

Jay Chooi (Robocurve co-founder) posts a rapid-response Inspect Robots evaluation pitting OpenAI’s newly released GPT-6 Astra against Anthropic’s Claude Fable 5.1 on real bimanual YAM-arm manipulation. Both models write end-effector poses that an IK solver turns into joint commands. On the harder precision-oriented subset, Astra ties Fable 5.1 at 2/20 success but uses 3.9× fewer output tokens and is 1.6× cheaper per rollout; Kyle’s Slack note references a 95% claimed success rate on an easier task set from earlier in the thread. Chooi projects (from published LLM token-output-speed trends: 2.1× per month at Fable-5 tier) that frontier LLMs could drive robot arms in real time by end of 2026, or 2029 at the latest.

Key claims

Section titled “Key claims”
  • On the harder precision-oriented Robocurve task set, Astra matches Fable 5.1 at 2/20 success rate but with 3.9× fewer output tokens and 1.6× lower cost per rollout [tweet body, image 2].
  • Both models control YAM arms via specifying end-effector pose passed through an automatic IK solver — LLMs can also drive joint angles directly (as in Robocurve’s earlier work) [tweet body].
  • Frontier-tier LLM output token speed doubles roughly every month at the Fable-5 level (2.1×/month); if trends hold, LLMs could drive robots in real time by end of 2026, or 2029 at the latest [tweet body, image 8].
  • Robocurve released all 120 rollout traces (transcripts, camera recordings, trajectories) publicly and runs the evaluation on the fully open-source Inspect Robots harness [tweet body].

Method

Section titled “Method”

Robocurve ran the same Inspect Robots evaluation harness described in Inspect Robots — An Open-Source Evaluation Framework for Physical AI (Robocurve) against a paired set of YAM arms. Both Astra (GPT-6) and Fable 5.1 (Claude) were given the manipulation task and asked to emit end-effector poses; an automatic inverse-kinematics solver translated those into joint commands executed on the physical arms. Each model ran 20 trials on the harder precision-oriented subset. Metrics captured per rollout: success/failure, total output tokens burned, dollar cost, and full transcript + camera recordings + trajectory logs. The scaling chart references public Artificial Analysis speed data binned by intelligence tier, showing the Fable-5 tier doubling output tokens/sec roughly every month over 2024-2026.

Results

Section titled “Results”
  • Astra vs Fable 5.1 on harder precision tasks: both 2/20, but Astra uses 3.9× fewer output tokens and costs 1.6× less per rollout [tweet body, image 2].
  • Fable-5-tier LLM output speed frontier: 2.1× per month (compared to 3.4×/year at GPT-5 tier, 7.8×/year at o1 tier, 2.3×/year at GPT-3.5 Turbo tier) [image 8].
  • All 120 traces released publicly, including transcripts, camera recordings, and trajectories [tweet body].

Why it’s interesting

Section titled “Why it’s interesting”

Directly extends the same-week Comparing Fable 5.1 to Fable 5 on robotic manipulation (Fable 5.1 vs Fable 5) with a same-Robocurve, same-harness, same-YAM-arms comparison against OpenAI’s freshly released Astra — Kyle is threading this against that Anthropic-authored comparison to note the OpenAI side is winning on cost per success and (per an earlier tweet in the thread) hitting 95% on easier tasks. Continues the multi-month Robocurve series of running each new frontier LLM as a robot arm driver — Claude Opus 5 driving robot arms zero-shot via Inspect Robots (Jay Chooi / Robocurve) (Opus 5 zero-shot), Gemini Robotics-ER 2 vs Claude Opus 5 clapboard eval — ER2 threw robot arms off table, hallucinated success (Robocurve / Jay Chooi) (Gemini Robotics-ER 2 vs Opus 5), Gemini 3.7 Flash saturates Robocurve physical tool-use benchmark at 92% vs 32% for 3.6 Flash (three weeks apart) (Gemini 3.7 Flash saturating at 92%) — so the pattern of general LLMs as competitive robot controllers is now three quarters deep with independent evidence. The token-speed scaling claim is a direct forecast argument for the VLA Models discussion: if the compute-per-decision curve keeps doubling monthly at frontier tier, closed-loop LLM control at 20-30 Hz becomes plausible before dedicated action-chunked VLAs finish maturing — which is consistent with Sentdex — general multimodal LLMs may sidestep VLA/WAM research for high-level robotics intelligence‘s take that general multimodal LLMs may sidestep the VLA/WAM research stack for high-level robotics intelligence.

See also

Section titled “See also”