Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem

TL;DR

Alibaba’s ROLL team releases ROME (Qwen3-MoE-based agent LLM) trained inside a full-stack Agentic Learning Ecosystem (ALE) built specifically for long-horizon terminal/SWE agentic RL. ALE bundles three co-designed systems: ROLL (agentic RL post-training framework with fine-grained rollout + asynchronous training + train-rollout multiplexing), ROCK (sandboxed environment execution engine speaking the GEM API), and iFlow CLI (the agent framework that owns context engineering and is the same context manager used in both training and deployment). The headline algorithmic contribution is IPA (Interaction-Perceptive Agentic Policy Optimization) — credit assignment over semantic interaction chunks rather than individual tokens, intended to stabilize long-horizon RL. ROME reports 57.4% on SWE-bench Verified and 24.7% on Terminal-Bench v2.0, plus the release of Terminal Bench Pro as a contamination-controlled successor benchmark. The “bitter lesson” framing of the companion blog: in agentic settings the bottleneck is no longer data scale but co-design of training infra, executable environments, and evaluation protocols.

Key claims

Vanilla RLVR operates more like an in-context bandit (single response → immediate reward → update); agentic RL is a multi-step MDP with sparse, delayed rewards over extended trajectories — different infra is required [companion blog §Introduction].
The Agentic Learning Ecosystem (ALE) is three co-designed components: ROLL (training framework), ROCK (sandbox/env execution), iFlow CLI (agent framework / context manager) [§2.1].
ROLL decomposes agentic RL into specialized worker roles (LLM inference, environment interaction, reward computation, parameter updates) under a Cluster abstraction with a single-controller programming model [§2.2].
Rollout dominates RL cost — prior work reports ~70% of end-to-end overhead, and in agentic settings rollouts can last hundreds of seconds with environment interaction alone consuming >15% of total training time [§2.2 Agentic Training Pipeline].
ROLL supports fine-grained rollout: decomposing each rollout into three sample-level pipelined phases (LLM generation / environment interaction / reward computation) so the lifecycle of each sample can be scheduled independently [§2.2 Fine-grained Rollout].
Asynchronous training decouples rollout (producer) from training (consumer) across devices via a sample buffer + an “asynchronous ratio” cap on per-sample staleness; samples beyond the staleness bound are discarded to preserve accuracy [§2.2 Asynchronous Training].
Train–rollout multiplexing: dynamic GPU partition between rollout and training, exploiting the bursty-vs-long-tail demand mismatch — all GPUs are first assigned to rollout, then a shrink op reallocates a subset to training during straggler tails, then expand returns them for the next peak [§2.2 Train–Rollout Multiplexing, Fig. 3].
ROCK is framework-agnostic (works with veRL, OpenRLHF, Tinker via the GEM API), provides streamlined SDK control (make/reset/step/close), seamless multi-agent scaling, fault isolation per sandbox, and elastic scheduling to tens of thousands of concurrent environments [§2.3].
Agent Native Mode: a ModelProxyService inside ROCK intercepts LLM requests already orchestrated by iFlow CLI, so context management is owned by the agent framework — not duplicated inside the RL trainer. Eliminates the train/deployment context-handling mismatch and supports multiple scaffolds (iFlow CLI, SWE-Agent, OpenHands) behind one interface [§2.3 Agent Native Mode].
iFlow CLI uses a single-agent orchestrator-worker loop with sub-agents implemented as specialized tools with bounded context rather than via agent handoffs — explicitly motivated by Anthropic’s effective-agentic-systems guidance and Sutton’s Bitter Lesson [§2.4].
iFlow CLI implements five context-engineering techniques: persistent memory (todo file as external memory), context isolation (sub-agent dedicated contexts), context retrieval (agent search + semantic vectors + KB integrations like DeepWiki), context compression (lossy + lossless), and context enhancement (user-highlighted critical signals) [§2.4.0.3].
IPA (Interaction-Perceptive Agentic Policy Optimization): novel policy-optimization algorithm assigning credit over semantic interaction chunks rather than individual tokens, claimed to improve long-horizon stability [Abstract, §1].
ROME is built on Qwen3-MoE, trained on >1M trajectories through continual pre-training → SFT → RL, with all data passing rigorous safety/security/validity verification inside ROCK [Abstract, §1].
ROME scores 57.4% on SWE-bench Verified and 24.7% on Terminal-Bench v2.0, outperforming similarly-sized models and rivaling >100B-parameter models [Abstract, §1].
Terminal Bench Pro is introduced as a contamination-controlled successor to Terminal-Bench v2.0, with improved scale and domain coverage, on which ROME remains competitive among similar-scale open models [Abstract, §1].
ROME has been deployed in production through iFlow CLI — explicitly framed as a real-world validation of the ALE stack [§1].

Method

ALE is a closed end-to-end loop. ROLL is the RL post-training framework: it exposes a Cluster abstraction with a single-controller programming model (similar to veRL / OpenRLHF), but decomposes agentic post-training into independently-scaling worker roles (LLM inference, environment interaction, reward, parameter update). Its key optimizations are (i) fine-grained rollout — sample-level pipelining of LLM generation, environment interaction, and reward; (ii) asynchronous training — disaggregated producer/consumer with a sample buffer and a configurable per-sample staleness cap; (iii) train-rollout multiplexing — dynamically repartitioning GPUs between rollout (long-tailed, bursty post weight-sync) and training (short, bursty) via shrink/expand operations to reduce the pipeline bubble.

ROCK is a client-server sandbox execution engine. The server tier has an Admin control plane (orchestration, admission, scheduling), Worker nodes (sandbox runtime per machine), and Rocklet (a lightweight proxy that mediates SDK-to-sandbox calls and governs network egress). EnvHub is the centralized environment-image registry. ROCK exposes a Sandbox API (lifecycle: provisioning / monitoring / persistence) and the GEM API (make/reset/step/close) — the GEM compatibility is what makes ROCK framework-agnostic across veRL, OpenRLHF, Tinker, and ROLL itself. The Agent Native Mode is the architectural punchline: rather than reimplementing iFlow CLI’s context-handling logic inside ROLL, a ModelProxyService inside the sandbox intercepts the LLM call (already-orchestrated context inclusive) and forwards it to either a ROLL inference worker (training) or an external API (deployment). This decouples training-framework maintenance from agent-framework evolution.

iFlow CLI is the orchestrator. Following the Bitter Lesson, it deliberately uses a single-agent control loop with sub-agents implemented as bounded-context tools. Its context-engineering primitives are persistent memory (todo file), context isolation (sub-agent contexts), context retrieval (agent search + semantic vectors + DeepWiki-style KBs), context compression (lossy + lossless), and context enhancement (user-highlighted signals). Open-configuration extensibility (System Prompt, Workflow/Spec, Tool Set) lets the same agent loop be specialized to e.g. WeChat Mini-program or iOS development workflows.

The model — ROME — is built on Qwen3-MoE and trained in three stages (continual pre-training → SFT → RL) on a curated multi-source, multilingual, tool-grounded trajectory corpus of over one million trajectories generated and validated inside ROCK sandboxes. The RL stage uses the proposed IPA algorithm, which performs credit assignment over semantic interaction chunks (turns or tool-call boundaries) rather than per-token, intended to address the token-level credit instability that becomes acute on long horizons with sparse, delayed rewards.

Results

SWE-bench Verified: 57.4%; Terminal-Bench v2.0: 24.7% — claimed to outperform similarly-sized models and rival >100B-parameter models [Abstract, §1].
Terminal Bench Pro (introduced in this report) — ROME remains competitive among similar-scale open-source models [Abstract, §1]. The benchmark itself is the contamination-controlled successor proposed for the community.
ROME deployed in production through iFlow CLI as the real-world validation [§1].
Training-side claims live in companion ROLL / ROLL-Flash work (cited as rollflash) — the report references prior empirical results that asynchronous training balances accuracy and throughput; quantitative ablations of fine-grained rollout, asynchronous ratio, and train-rollout multiplexing are deferred to that work.

(Note: this paper page is filed from the abstract + §1–§2.4 of the technical report; downstream sections on data composition, training recipe details, and IPA ablations were not fully captured in the fetched material and remain to be added on /bud refresh once a richer fetch is available.)

Why it’s interesting

ALE is the third filed instance of a tightly co-designed training/environment/agent-framework stack for agentic RL, alongside the patterns documented on Agentic Software Engineering (Qwen3-Coder-Next’s Kubernetes/MegaFlow + Mini-SWE-agent stack) and Tool-Use Agents (MiroThinker’s streaming-rollout GRPO + recency-retention). It sharpens two points the existing wiki entries flag as open: (a) the agent-native context-management decoupling (ROCK’s ModelProxyService) directly answers the “training–deployment context mismatch” risk that MiroThinker and Step 3.5 Flash both gesture at, and offers a clean architectural pattern; (b) IPA’s chunk-level credit assignment is a new datum on the question of how the GRPO-family scales to long-horizon agentic trajectories — a question raised under both reasoning-rl (Critical Steps / PARL) and tool-use-agents (interaction depth scaling). The release of Terminal Bench Pro as a contamination-controlled benchmark is also significant for agentic-swe, which currently treats Terminal-Bench v2.0 as a co-headline alongside SWE-Bench Pro.