Scale Robot Policy Evaluation with Ray (Distributed Sim-Eval on Anyscale)

TL;DR

Anyscale’s engineering blog lays out a Ray + Anyscale reference architecture for large-scale robot-policy evaluation in simulation: run the VLA policy (NVIDIA GR00T-N1.7-3B in the worked example) as an autoscaling Ray Serve deployment on its own GPU, run each Isaac Lab simulator as an isolated Ray task on a separate GPU, and connect them over HTTP with action-chunking to amortize the network hop. The stated design principle is disaggregation of two GPU-bound workloads that would otherwise contend on the same device — policy inference and Isaac Sim’s physics/rendering — so each fleet scales independently. The example drives a Unitree G1 humanoid through a pick-and-place scene and fans out 100 concurrent rollouts against a single shared policy replica; scaling to more rollouts or more policy replicas is a one-line change.

Key claims

Simulator-based policy evaluation is a closed-loop, distributed problem, not an offline batch pass: each simulator alternates physics steps with policy queries, and hundreds of rollouts must share a small fleet of policy replicas rather than each rollout loading its own 3B model [§Introduction].
Co-locating Isaac Sim’s GPU physics/rendering and a 3B VLA on the same GPU or same Python process causes VRAM and compute contention that stalls both workloads; disaggregation onto different processes and different GPUs is the recommended pattern [§Introduction, §Part 2].
The policy is wrapped in a Ray Serve deployment (@serve.deployment + @serve.ingress(FastAPI())), loaded once per replica on a GPU, and exposed over POST /predict / GET /stats; scaling the fleet is a single num_replicas=N change with automatic client-side load balancing [§Part 1].
The health_check_timeout_s default of 30 s is too short for a 3B model load and must be raised (300 s in the example) or the replica is marked unhealthy before it finishes loading; the HF gated backbone also needs HF_TOKEN passed via runtime_env [§Part 1].
Action chunking is what makes the disaggregated HTTP design economical: GR00T returns a 40-step chunk per ~1.6 s forward pass, and executing action_horizon = 8 physics steps between policy calls amortizes one inference across eight steps — the same GPU can then serve eight times as many simulators as a naive query-every-step loop [§Part 1, §Part 2].
The schema-translation glue between GR00T’s REAL_G1 modality dict (2-frame ego_view stack, 9-D end-effector poses = 3 translation + first two rows of rotation matrix, per-limb state) and Isaac Lab’s flat (1, 28) joint-action tensor is the “genuinely fiddly part” of wiring any VLA into a simulator [§Part 2, _flatten_action code block].
Each simulator runs as a Ray remote task launching Isaac Sim in its own subprocess (Isaac Sim needs a clean interpreter and event loop), receives a policy URL, and writes its own GIF + metrics to disk; ray.get([run_sim_rollout.remote(POLICY_URL, seed=s) for s in range(100)]) is the entire fan-out [§Part 3].
Simulators and policy replicas scale independently: if rollouts are starved on inference, add Serve replicas; if the policy fleet is underutilized, fan out more simulator tasks and Anyscale scales the underlying GPU cluster. Neither change touches the other side’s code [§Part 3, §The payoff].
The runtime_env mechanism propagates the gated-model HF_TOKEN to both the Serve replica and every simulator task uniformly, avoiding manual per-worker credential setup [§Part 3].
Reported per-inference numbers on the example workload: 1619 ms average latency, 1 total call, 26.79 s model load time, 3.144 B parameters, embodiment tag REAL_G1, device cuda:0 [§The payoff, /stats JSON].
Framing claim: robotics policy evaluation is “evolving into an infrastructure problem” and the natural design pattern is the same one that scaled LLM inference — disaggregated GPU inference fleets, autoscaling services, heterogeneous scheduling — now applied to closed-loop rollouts [§What’s Next].

Method

The reference architecture has two stages that run on the same Anyscale Ray cluster but on separate GPU workers:

Policy as a Ray Serve deployment. GR00TPolicyServer is a plain Python class decorated with @serve.deployment(ray_actor_options={"num_gpus": 1}, max_ongoing_requests=16) and @serve.ingress(FastAPI()). Its constructor loads nvidia/GR00T-N1.7-3B onto cuda:0 via Gr00tPolicy. The POST /predict handler pickle-loads the observation, runs one forward pass under torch.no_grad(), and returns a pickled {"action": …} payload; a GET /stats handler reports parameter count, device, cumulative calls, average latency, and load time. serve.run(deployment, name="gr00t-policy") stands the HTTP server up on a GPU worker.
Simulators as Ray remote tasks. @ray.remote(num_gpus=1, runtime_env={"env_vars": {"HF_TOKEN": …}}) decorates run_sim_rollout(policy_url, seed=0), which uses subprocess.run to launch a sim_worker.py process — Isaac Sim needs a clean interpreter and event loop, hence the subprocess rather than an in-process call. Inside the worker, the control loop calls query_policy(policy_url, obs) only when the current action chunk is exhausted, executes action_horizon = 8 physics steps per chunk, and re-queries when chunk_idx >= action_horizon.

The schema-translation glue lives on both sides of the HTTP boundary. On the way in, Isaac Lab’s observation dict is repackaged into GR00T’s nested REAL_G1 modality dict — a two-frame ego_view image stack, 9-D end-effector poses (3 translation values plus the first two rows of the rotation matrix) for each wrist, per-limb arm/hand joint vectors, a 3-D waist vector, and a natural-language instruction. On the way out, GR00T’s nine-key, 40-step action chunk is flattened by _flatten_action(chunk, step_idx) into Isaac Lab’s (1, 28) joint command: 7 + 7 arm joints, 7 + 7 hand joints.

Scaling out is expressed as a list comprehension: ray.get([run_sim_rollout.remote(POLICY_URL, seed=s) for s in range(100)]). Increasing rollouts extends the list; increasing serving capacity raises num_replicas on the Serve deployment; Anyscale scales the underlying GPU cluster to match either.

Results

The post is an architecture walkthrough with a single worked example rather than a benchmarking study, so headline numbers are limited to what the /stats endpoint reports on the demonstration run:

Model: nvidia/GR00T-N1.7-3B, 3.144 B parameters, embodiment tag REAL_G1, cuda:0 [§The payoff].
Load time: 26.79 s (which is why health_check_timeout_s=300 is necessary — the Serve default of 30 s would fail the replica before it finishes loading) [§Part 1, §The payoff].
Inference latency: ~1.6 s per forward pass returning a 40-step action chunk across nine action keys (left_wrist_eef_9d, right_wrist_eef_9d, left_hand, right_hand, left_arm, right_arm, waist, base_height_command, navigate_command) [§Part 1].
Action chunking amortization: at action_horizon = 8, one 1.6 s inference covers eight physics steps, so a single policy replica can feed ~8× as many concurrent simulators as a naive query-every-step design would allow [§Part 1, §Part 2].
Fan-out: 100 concurrent run_sim_rollout tasks against a single Serve replica in the example, with the fan-out extensible by extending the list comprehension and the serving side by raising num_replicas [§Part 3].

The post does not report end-to-end rollout throughput, cluster-level GPU utilization, or a head-to-head against alternative designs (e.g. co-located policy + sim, or a mono-GPU implementation).

Why it’s interesting

This is the first filed artifact that puts a named serving architecture on the robot-policy-evaluation problem the RL Environment Platforms cluster has been circling — where SETA / Toolathlon-GYM / OpenReward standardize the task side, and RoboArena, RoboCasa365, ABC-Sim, and Genesis World standardize the evaluation substrate, Anyscale’s post standardizes the rollout runtime. The disaggregation pattern (policy fleet on one GPU pool, simulator fleet on another, HTTP boundary between them, action-chunk amortization across the hop) is the natural continuation of the “verifier-cheap, action-expensive” design target the cluster’s other entries already commit to — here the action cost is GPU physics + rendering rather than Docker sandbox setup, but the scheduling consequences are the same.

It also cross-cuts with VLA Models: the worked example uses GR00T-N1.7-3B as a served policy, sharpening a practical constraint the recent scale-of-VLA recipes (π*0.6: a VLA That Learns From Experience (RECAP), Spirit-v1.5: A Robotic Foundation Model by Spirit AI, Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models) understate — a 3B model has 26.79 s cold-start and 1.6 s per forward pass, so evaluation throughput is bounded not just by rollout count but by how effectively N simulators share M replicas. Complements SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation (zero-shot real-to-sim scene construction) and RoboCasa365 — Large-Scale Simulation of Everyday Tasks for Generalist Robots (365-task manipulation benchmark) by providing the runtime substrate those task surfaces would be evaluated on at scale.

Finally, the streaming/async-RL primitives the wiki tracks under Distributed training parallelism — Fireworks’ streaming pipeline schedule, AReaL’s StalenessManager, KnapFormer’s per-step compute bags — are all variations of the training-side decoupling of producer and consumer that Anyscale’s post applies on the evaluation-side (many-simulator producer, few-replica consumer, HTTP-mediated). The runtime_env credential propagation detail is a small-but-load-bearing operational note that the training-infra papers don’t have to solve because they don’t span gated-HF-weights + sandboxed sim workers the way policy-eval does.