OSGym: Scalable OS Infra for Computer Use Agents

TL;DR

OSGym is an open-source distributed data engine for training general-purpose computer-use agents on full OS replicas (not specialized sandboxes), scaling to 1024+ parallel Docker-based Ubuntu instances at $0.23 / replica / day. Three architectural moves drive the cost down: (1) fully decentralized per-replica state managers, (2) RAM-bound rather than CPU-bound orchestration (packing K=64 replicas per large-RAM server collapses cost ~10×), and (3) a single-entry asynchronous Python data-server abstraction over the fleet. On a 1024-replica deployment it collects 1420 multi-turn trajectories per minute, and the authors use it to SFT+PPO Qwen-2.5-VL 7B to 34.5% Pass@1 / 47.3% Pass@5 on an OSWorld subset — competitive with Agent-S2, UI-TARS-1.5-7B, and Claude Computer-Use.

Key claims

A fully decentralized state-manager design (one per replica) eliminates the centralized-bottleneck and single-point-of-failure modes that limit semi-decentralized and centralized alternatives [§3.1].
The dominant scaling bottleneck for OS-replica fleets is CPU when K=1 (one replica per server) but flips to RAM when K is large because peak CPU usage of co-located replicas does not fully overlap in time [§3.2].
Exploiting the K-flip: 128 replicas on an 88-core E5-2699 / 768 GB DDR4 server cost $30/day (≈$ 0.23/replica) vs $300/day for K=1, a ~10× cost reduction at fixed N [§3.2, Table 1].
Step throughput scales near-linearly with parallelization size; per-replica step latency grows only marginally as replicas grow exponentially, validating the decentralized + RAM-packing design [§4.1, Fig. 6 left/middle].
Self-recovery is decentralized: each per-replica manager isolates faults locally and autonomously restarts the OS replica; OSGym restores from a fully-crashed initial state to a healthy fleet within a short recovery window [§4.1, Fig. 6 right].
Data generation is fast and cheap: 1024 replicas collect ≈1420 multi-turn trajectories/minute; a 244-task / ~2863-trajectory office+browser+VSCode+GIMP corpus is generated in 121 seconds at $43 total cloud spend [§4.2, Tables 2-3].
The shipped pipeline is SFT-then-RL: Qwen-2.5-VL 7B is SFT’d on the synthesized corpus with (task, screenshot_1, thought_1, action_1, …, screenshot_C, thought_C, action_C) sequences and softmax-CE loss over thoughts+actions, then PPO’d via a semi-online async loop that decouples rollouts from updates [§4.2].
On a manually-verified 89-task OSWorld subset (25-step cap), the OSGym-trained model reaches Pass@1 = 34.54 / Pass@5 = 47.34, beating UI-TARS-1.5-7B (32.68 / 44.63), Claude Computer-Use (24.17 / 30.02), Aguvis-72B (13.24 / 18.96), CogAgent-9B (11.09 / 13.78), and GPT-4o (8.83 / 9.26), and edging Agent-S2 (33.93 / 38.02) [§4.2, Table 4].
The unified per-task execution flow is Configure → Reset → Operate → Evaluate, with the evaluation function user-defined for each task — making OSGym training/recipe-agnostic [§3.3, §3.5].
The centralized data-server class exposes batched reset / async step over the fleet with internal queuing, dynamic load balancing, and built-in fault recovery, abstracting per-replica orchestration from the trainer [§3.4].

Method

OSGym is structured as N independent OS replicas (Docker containers running OSWorld-style Ubuntu desktops, one OS per container) packed onto M physical servers at group size K = N/M. Each replica is owned by its own decentralized state manager process that handles its lifecycle (configure / reset / operate / evaluate), monitors health, and self-recovers on failure. Above the per-replica managers sits a single centralized data-server Python class that gives the trainer one entry point: batched reset, asynchronous step, internal queuing, load balancing, and fault tolerance — without exposing the fleet to the trainer.

The training pipeline shipped with the paper has three stages. (1) Data generation: 244 hand-written OSWorld-style tasks across LibreOffice (Writer/Calc/Impress), Chrome, ThunderBird, VLC, VS Code, GIMP, system configuration, and multi-app workflows are executed simultaneously by 1024 OS replicas running Agent-S2 and UI-TARS as demonstration agents, yielding ~2863 trajectories (10–25 steps each) in 121 seconds for $43. (2) SFT: Qwen-2.5-VL 7B is fine-tuned on interleaved (screenshot, thought, action) sequences with Adam at LR for ~12h on 8×H100. (3) RL: a semi-online async PPO loop decouples a continuous data-rollout worker (osgym_dataloader.async_step) from a model-update worker that samples from the replay buffer and runs standard PPO objectives; trained 5000 steps at batch 64.

Results

Throughput scales near-linearly to 1024 replicas; per-replica latency grows only marginally as concurrent replicas grow exponentially [§4.1].
Cost per replica per day: $2.10 at 36-replica K (8275CL / 192 GB) →$ 0.78 at K=128 (8259CL / 768 GB) → $0.23 at K=128 (E5-2699 / 768 GB) [Table 1].
Synthetic dataset generation: 1024-replica parallelization compresses 115,654 s of sequential generation to 121 s; net cost $43 [Table 2 + 3].
OSWorld-89 Pass@1 / Pass@5 (25-step cap): OSGym Demo Model 34.54 / 47.34, Agent-S2 33.93 / 38.02, UI-TARS-1.5-7B 32.68 / 44.63, Claude Computer-Use 24.17 / 30.02, Aguvis-72B 13.24 / 18.96, CogAgent-9B 11.09 / 13.78, GPT-4o 8.83 / 9.26 [Table 4].
Self-recovery from a fully-crashed initial state to a healthy fleet completes within a short window (figure-only, no specific seconds reported) [Fig. 6 right].

Why it’s interesting

OSGym is a strong fourth datapoint for RL Environment Platforms — the first one that ships full-OS replicas rather than terminal or MCP-bounded sandboxes. Against SETA: Scaling Environments for Terminal Agents (env repo) (≈400 Terminal-Bench tasks), Toolathlon-GYM: Large-Scale Long-Horizon Environments for Tool-Use Agents (503 MCP/PostgreSQL workflow tasks), and OpenReward — 330+ RL Environments through one API (General Reasoning announcement) (330+ envs × 4.5M tasks behind an API), OSGym pushes the action space from terminal/structured-tool to arbitrary GUI — closing the gap to the screen-recording corpora behind The First Fully General Computer Action Model (FDM-1) (FDM-1’s 80k forking Ubuntu VMs) and Computer Use Large: 48k screen recordings (~12,300 hours) of professional software (12k hours of professional-software screen recordings). The K=64 RAM-packing insight is the clean throughput-economics counterpart to FDM-1’s “1 H100 controls 42 VMs” — both papers independently land on “OS sandbox fleets are RAM-bound and that’s a cheaper bottleneck than people assumed.” And the Pass@1 number on OSWorld-89 (34.54) makes this the first openly-reproducible recipe that approaches Agent-S2’s number with a 7B Qwen-VL base, which is directly relevant to Tool-Use Agents discussions of base-model capability vs. recipe contribution.