Embodied Analysis — GPT-6 Astra vs Claude Opus 5 on test-tube rack placement, scored on execution directness

TL;DR

Embodied Analysis posts a side-by-side comparison of GPT-6 Astra and Claude Opus 5 as LLM controllers on a real bimanual test-tube-rack placement task: both models complete the task (3/3 tubes upright in the rack), but Astra reaches that state in ~12 seconds while Opus 5 takes ~19 seconds after returning mid-run to correct a leaning tube. The reviewer awards Astra a WIN label for the more direct placement sequence despite both runs scoring 3/3 on visible task completion, and immediately annotates the uncertainty: these are single rollouts with a 3-second result-card overhang on Astra’s clip, so the comparison shows an execution-style difference rather than a reliability measurement. Another individual data point on the growing LLM-as-controller row of the wiki’s VLA board, with the useful new wrinkle that the reviewer explicitly calls out the overclaim that two single runs would represent model-level reliability.

Key claims

Astra has all 3 tubes upright in the rack by ~12 seconds; Opus 5 reaches the same state by ~19 seconds after returning to correct a leaning tube [tweet body].
Both runs are scored 3/3 for visible task completion; the WIN/LOSS labels on the clips distinguish the execution style within these two clips, not a completed vs unfinished task [tweet body, reply 1].
Astra’s shorter recording holds its final frame with a “Clip ended” overlay and the closing result card adds ~3 seconds, so the raw clip-length delta overstates the execution-time delta; and because these are two individual runs, they do not establish how consistently either model completes the task without corrections [tweet body, reply 2].

Method

The reviewer runs the same bimanual test-tube placement task twice — once with GPT-6 Astra as the controller, once with Claude Opus 5 — and posts the two rollout videos side by side with WIN/LOSS annotations and a written commentary explaining the scoring. The comparison axis is placement directness: whether the controller sequences the three-tube rack placement in one pass or has to backtrack. Scoring is qualitative with a 3/3 completion check for both runs. No aggregate statistics (per-model success rate, time-to-completion distributions, repeat trials, or stress tests with adversarial tube initial poses) are reported — this is explicitly framed by the author as a two-rollout qualitative exhibit, with the follow-up replies walking back any implicit reliability claim.

Results

Astra: 3 tubes upright by ~12 seconds, direct placement sequence [tweet body].
Opus 5: 3 tubes upright by ~19 seconds, with one mid-run correction of a leaning tube [tweet body].
Review verdict: Astra receives the WIN label on this pair of rollouts for the more direct placement sequence; both runs are scored 3/3 on visible task completion [reply 1].

Why it’s interesting

This fits cleanly onto the LLM-as-controller row of the VLA Models recipe-lever board alongside the growing cluster of recent Astra-vs-Claude physical-controller demos — Jay Chooi (Robocurve) — Astra vs Fable 5.1 on real-robot manipulation via IK (Astra vs Fable 5.1 on real YAM arms via IK), Jiafei Duan — Astra vs Opus 5.5 in MuJoCo Ring Fence sumo, Astra wins in 31 seconds (Astra vs Opus 5.5 MuJoCo G1 humanoid sumo), thijs — Astra paints the Golden Gate Bridge with a real robot arm (Astra painting with a real arm), and Wenli Xiao — GPT-6 Astra does physical in-context learning from a single human video via a motion-planning tool harness (Astra physical in-context learning from one video). What this post adds to the pile is less about the raw result (Astra wins again on a single rollout) and more about the reviewer’s self-correction: the two follow-up replies explicitly warn that a ~3-second result-card overhang inflates the raw time delta and that two individual runs don’t speak to reliability — a rare example of an informal eval publicly annotating its own confidence bounds. For VLM-as-Evaluator that’s a useful pointer at the “informal human-plus-rubric” end of the evaluator spectrum, and it contrasts with the controlled large-sample Astra study Astranomy: An Empirical Study of GPT-6 Astra as a Native Robot Policy and the 12-benchmark Has GPT-6-Astra Achieved Embodied Intelligence? probe — the structured studies find Astra uneven, the viral single-clip demos keep showing Astra winning, and this post is on the demo end but with better epistemic hygiene than most.