Tool-Use Agents

TL;DR

Tool-use agents are LLMs that interleave reasoning with external tool invocations (code interpreters, web search, MCP servers, custom APIs) via a ReAct-style loop. Across filed papers, the post-training recipe has converged on GRPO over (sometimes synthesized) trajectories with correctness + format rewards, evaluated on BFCL / WebWalkerQA / BrowseComp / AIME-with-tools. The design surface is splitting in two directions: (a) parameter-space approaches that fine-tune mid-size (30–72B) models on large self-synthesized trajectory corpora (InfTool, MiroThinker), and (b) context-space approaches that keep a frontier model frozen and store learned tool-use experience as a natural-language prior (Training-Free GRPO). The two camps trade GPU spend for API spend and differ sharply on which dimension scales — interaction budget per task vs. base-model capability.

Key claims

ReAct (thought / action / observation triples) with a small fixed toolset — usually a code interpreter, web search, and a sandboxed shell — is the universal substrate; differences across papers are in the reward, the training data, and the optimization target, not the agent loop (Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing §Method, MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §3, Training-Free Group Relative Policy Optimization §3.1). VisGym ports the same multi-turn (instruction, obs_t, act_t, fb_t) history loop to vision, with the toolset becoming task-specific perception/manipulation primitives (swap, rotate, move, mark, gripper) declared via natural-language function instructions (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §2). Kimi K2.5 extends the loop to a multi-agent substrate: a single orchestrator agent dynamically instantiates specialized sub-agents (e.g. AI Researcher, Physics Researcher, Fact Checker) and decomposes the task into parallelizable sub-tasks, each executed concurrently — up to 100 sub-agents across up to 1,500 coordinated tool calls (Kimi K2.5: Visual Agentic Intelligence §Agent Swarm).
GRPO + correctness + format rewards is the de facto post-training objective: InfTool’s gated rewards over verified synthetic trajectories lift a 32B base 19.8% → 70.9% on BFCL (Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing Abstract); MiroThinker uses online GRPO with a correctness-plus-format reward as the third stage of SFT→DPO→GRPO on 72B and reaches 47.1% on BrowseComp (MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §5.3); Training-Free GRPO uses the same group-relative comparison primitive but materializes the “advantage” as a natural-language entry written to an experience library on a frozen DeepSeek-V3.1-Terminus, hitting 82.7 / 73.3 on AIME24/25 and 67.8 on WebWalkerQA (Training-Free Group Relative Policy Optimization §3, Tables 3–4). Kimi K2.5’s PARL is the parallel-agent extension of the same family: an orchestrator RL recipe with shaped rewards that defeat serial-collapse (orchestrator defaulting to single-agent execution) by incentivizing sub-agent instantiation, and defeat spurious-parallelism (spawning sub-agents without meaningful decomposition) by rewarding completed sub-tasks (Kimi K2.5: Visual Agentic Intelligence §PARL).
Self-synthesized trajectories are sufficient supervision when verifiable: InfTool closes a data→model→data loop with three role-playing agents (User Simulator + Tool-Calling Assistant + MCP Server) and no human labels (Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing Abstract); MiroThinker synthesizes ReAct and MiroFlow multi-agent traces from multi-hop QA built off Wikipedia/CommonCrawl subgraphs (MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §4.2); Training-Free GRPO needs only ~100 ground-truth-tagged samples (DAPO-100 / AFM-100) and works in a no-ground-truth variant too (Training-Free Group Relative Policy Optimization §3, Table 2). VisGym extends this to visual tool-use by shipping a per-task heuristic solver whose stochastic rollouts become the SFT corpus (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §2, §5.1) — no LLM-in-the-loop synthesizer needed.
“Interaction depth” — number of tool calls per task — is proposed as an independent scaling axis: MiroThinker’s 256K context + recency-based retention enables up to ~600 tool calls per task vs. <100 for prior open agents, with accuracy curves that don’t saturate (MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling §3.3, §1). Conflict: Training-Free GRPO observes the opposite at training time — the average tool-call count decreases across optimization steps even as accuracy increases, suggesting the learned experiences encode shortcuts that reduce required interaction depth (Training-Free Group Relative Policy Optimization §3.1, Fig. 4). The two findings can coexist if interaction depth is upper-bounded by task hardness and the recipe in question moves the agent up or down a “calls-per-task / accuracy” curve, but no filed paper isolates this. VisGym adds a third datapoint at a much smaller scale (20-30 step budgets) where the bottleneck is visual grounding before interaction depth is even reached — most models’ successful trajectories peak at 3-5 steps and drop off sharply (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §3.2). New dimension: Kimi K2.5 introduces parallel interaction-depth scaling — up to 100 concurrent sub-agents × ≤1,500 coordinated tool calls, evaluated against a latency-oriented Critical Steps metric (longest path through the orchestrator’s execution graph) rather than total step count, with a reported 3–4.5× Critical-Steps savings and up to 80% wall-clock runtime reduction over single-agent execution on wide-search scenarios (Kimi K2.5: Visual Agentic Intelligence §Agent Swarm, §PARL).
Cost asymmetry between the two camps is two orders of magnitude: parameter-space RL on a 32B model (ReTool) costs ~ $10K in GPU spend per training run, while Training-Free GRPO costs ~$ 18 in API spend for a comparable or better result on AIME — at the price of higher per-query inference cost ( $0.005 self-hosted vs$ 0.02 API) (Training-Free Group Relative Policy Optimization §4.2).
Cross-domain transfer is a key differentiator: parameter-space fine-tuning sacrifices the source domain — math-tuned ReTool collapses from AIME 67.0 → WebWalker 18.3, and web-tuned MiroThinker drops from WebWalker 53.6 → AIME 43.5 / 36.8 (Training-Free Group Relative Policy Optimization Table 6) — while a frozen base + swap-able experience library preserves both (Training-Free Group Relative Policy Optimization Table 6).
Base-model capability is a hard prerequisite for context-space methods: Training-Free GRPO regresses below ReAct baseline on QwQ-32B (25.5 vs 27.5 pass@1 on WebWalkerQA) and shows only marginal gains on Qwen2.5-72B-Instruct (+1.4 / +1.8 on AIME), in stark contrast to its +2.7 / +5.4 on DeepSeek-V3.1-Terminus (Training-Free Group Relative Policy Optimization Tables 3, 5).
For visual tool-use, the dominant failure modes are different from text agents: (a) restricted-action-space / action-looping (single repeated primitive instead of converging magnitude), (b) state mismanagement (ignoring feedback, revisiting cells, repeating invalid moves), (c) early termination before reaching the goal, (d) outright failure to use the visual observation (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §3.2). Critically, removing the textual feedback channel and forcing models to infer action validity from visual transitions alone causes consistent drops across Maze 2D/3D, Sliding Block, Matchstick Equation — showing the multi-turn loop in visual tool-use still relies on text feedback even when the observation is an image (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §4.3).
Parallel-agent orchestration introduces new failure modes orthogonal to sequential tool-use: (a) serial collapse, where the orchestrator defaults to single-agent execution despite parallel capacity; (b) spurious parallelism, where the orchestrator inflates parallel metrics by spawning many sub-agents without meaningful task decomposition (Kimi K2.5: Visual Agentic Intelligence §PARL). PARL addresses both with separate shaped-reward terms (sub-agent instantiation incentive vs sub-task completion reward) and a staged-reward schedule that shifts weight from parallelism early to task success late. The reported empirical signature is a smooth reward curve with monotonically increasing parallelism through training (Kimi K2.5: Visual Agentic Intelligence §PARL).
Agent memory is a design axis distinct from RAG, context engineering, and parametric memory — the recent surveys factor it as Forms × Functions × Dynamics (Memory in the Age of AI Agents) or substrate × cognitive mechanism × subject (Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey). Under either taxonomy, Training-Free Group Relative Policy Optimization‘s experience library is external-text + experiential + evolution-heavy; MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling‘s 256K context is latent + working + retrieval-heavy; Recursive Language Models‘s Python REPL is external-structural + working + formation-heavy.

Recent contributions

[2026-07-16] Kimi K3 — Open Frontier Intelligence (2.8T MoE with KDA + AttnRes): Kimi K3 targets long-horizon terminal-tool orchestration with the KimiCode harness and preserved-thinking-history mode; Moonshot explicitly warns that mid-session model swaps destabilize generation, arguing the harness/model contract is now part of the tool-use spec.
[2026-07-16] VIA: Visual Interface Agent for Robot Control: VIA (Stanford; Hu, Sundaresan, Gao, Sadigh) extends the visual tool-use loop of VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents to real manipulator control: a browser-based 3D interface exposes the robot as a computer-use surface, and an off-the-shelf coding agent (Claude Code / Codex / Fable 5) operates it via screenshots + intuitive interface commands with no fine-tuning and no privileged state. Fable 5 reaches 96.7% on three LIBERO-Goal tasks and 100% on a long-horizon rainbow assembly. Sharpens the concept: capabilities usually attributed to specialized robot-training pipelines are inherited from generalist software-agent training given the right interface — a “software-agent scaffold as substrate for embodied control” datapoint that complements Kimi K2.5’s parallel-agent orchestration and Training-Free GRPO’s context-space experience library.
[2026-07-14] How Claude Performs on Robotics Tasks: Anthropic’s Claude-Plays-Robotics report treats VLA supervision as a first-class tool-use loop with accept/edit/replace as the action space over MolmoAct-proposed 7-DoF end-effector commands, and separately treats Go2 joystick-policy control as high-level tool invocation over a pretrained gait. Introduces a follow-rate metric (fraction of proposed VLA actions passed through unchanged) that quantifies orchestrator deference on trusted vs failing tools — Opus 4.5/4.6 follow MolmoAct more than GPT-5.4/Gemini 3.1 on standard LIBERO but defer less on novel tasks the VLA cannot solve, a “good taste” signature the concept previously had no filed metric for. Also finds simple orientation tools (compass yaw in degrees, cursor-annotated gripper point with object identity + distance) uplift every configuration far more than richer perception aids — a physical-world analogue to the “learned experiences reduce required interaction depth” finding of Training-Free GRPO.
[2026-07-14] GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks: GaP (Berkeley/NVIDIA/CMU; Goldberg, Fan, Zhu) pushes the “structured trajectory generator” idea into the output space: an orchestrator agent partitions a robot task and dispatches skill agents to synthesize localized subgraphs from a 51-skill MORSL library, wired into one statically-type-checked directed graph. The graph is not training data — it is the policy. Two ablations show both moves are load-bearing: collapsing to a single LLM emitting raw Python drops success to zero (interface/syntax errors), and collapsing multi-agent decomposition to one LLM also drops to zero (structural verification fails). Sibling in spirit to SkillSynth (Toward Scalable Terminal Task Synthesis via Skill Graphs) and InfTool (Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing) — same “specialized-agents + verification gate + graph structure” pattern, now with a robot simulator as the verifier.
[2026-07-14] RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks: RoboClaw is a robotics-native instantiation of the MCP tool-invocation loop: a VLM meta-controller with structured memory (role identity, task-level memory, working memory) does chain-of-thought reasoning over visual observations and calls a fixed toolkit — Start/Terminate/Change Policy, Env Summary, Fetch Robot Stats, Call Human — to orchestrate low-level VLA policies during both autonomous data collection (via Entangled Action Pairs self-resetting loops) and long-horizon task execution on an Agibot G01 dual-arm mobile platform. Extends the pattern from purely-digital tool use (web search, code interp, other LLMs) to physical-world tool use where the “tools” are calibrated VLA skills and environment queries, with runtime status monitoring and human-escalation as first-class MCP capabilities.
[2026-07-09] Introducing Muse Spark 1.1: Muse Spark 1.1 claims zero-shot generalization to new native tools, MCP servers, and custom skills, and is trained to act as both main agent (planning + delegating) and subagent (scoped execution with escalation) inside multi-agent systems.
[2026-07-09] Benchmarking Coding Agents on Databricks' Multi-Million Line Codebase: Databricks’ internal coding-agent benchmark isolates the harness as a cost axis independent of model choice: running the same model + same thinking effort through Claude Code / Codex vs Pi shifts per-task cost by >2× at held-constant quality, driven by Pi sending ~3× less context per turn. Generalizes the Training-Free Group Relative Policy Optimization “learned experiences reduce required interaction depth” finding to harness-level context engineering — the tool-use loop’s per-turn context budget dominates cost at fixed capability, and is a knob orthogonal to the model.
[2026-07-07] Muse Video preview — MSL video generation model announcement (Alexandr Wang): Muse Image treats web search and prompt-planning (via Muse Spark) as pre-generation tools called before pixels are produced — an agentic wrapper around image generation, targeted at “first-try correctness.”
[2026-07-05] Agent² RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?: Adds the “agent engineers an interactive RL pipeline against a tool-use environment” surface: DeepSearchQA is a ReAct-style search-and-judge task the agent must both consume (as a training target) and construct trajectory collection for. Best score across all seven agent stacks is 15.0 (+2.75 over baseline within 3pp evaluation noise), even with 145 submissions and every alignment method tried — a structural-intractability datapoint for search-agent RL training that complements the concept’s existing ResearchRubrics / MiroThinker measurements of search-agent inference-time performance.
[2026-07-01] DimOS — The Agentive Operating System for Physical Space (v0.0.13): DimOS v0.0.13 ports the ReAct-over-MCP pattern to embodied agents: every @skill-decorated method on a robot Module is auto-exposed as an MCP HTTP tool that external coding agents (Claude Code, OpenClaw, Cursor) invoke via dimos mcp call <skill> --arg ..., with an AGENTS.md onboarding contract that lets the coding agent bootstrap itself into building/running/debugging DimOS applications end-to-end; concrete deployment-side instance of the VoLo “physical orchestrator over interruptible tools” thesis for tool-use-agent design.
[2026-06-30] ASPIRE — self-evolving sensorimotor skill library that compounds across robots and the sim-to-real gap (Jim Fan / NVIDIA GEAR): Jim Fan announces NVIDIA GEAR’s ASPIRE — coding agents observe sensory traces from sim+real robots, run evolutionary search over control programs, and distill the best know-how into a self-evolving skill library; reframes “trained model” as a code repo of sensorimotor skills shipped as HTML rather than weights. Generalizes the Code-as-Policy + persistent-skill-library pattern from Playful Agentic Robot Learning and ENPIRE: Agentic Robot Policy Self-Improvement in the Real World with an evolutionary-search inner loop and an explicit sim-to-real / cross-embodiment know-how-transfer framing (claimed ~10× fewer “transfer learning tokens” single-arm → bimanual).
[2026-06-30] Agents-A1: A 35B MoE Agentic Foundation Model: Agents-A1 (InternScience, 35B MoE) targets tool calling alongside long-horizon search, engineering, and research in a single base; serving recipe uses the qwen3_coder tool-call parser, suggesting Qwen3-Coder XML-tag tool emission.
[2026-06-29] Flexion Reflect v1.0: The Path Towards Long-Horizon Autonomous Humanoid Work: Flexion Reflect v1.0’s mission controller is a fine-tuned VLM that issues structured tool calls (semantic-map query, navigation, manipulation skills, elevator interaction) over an egocentric camera feed; finds off-the-shelf VLMs act too eagerly (emit next-tool-call before visually verifying preconditions), and RL fine-tuning is required to fix this — extending the tool-use-agent loop to long-horizon embodied execution.
[2026-06-27] Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond: Survey explicitly folds web/GUI/tool-use agents into a unified “world modeling” frame as the digital-world regime of an L1/L2/L3 capability hierarchy — frames tool-use environments (browsers, terminals, GUIs) as constraint regimes governed by program semantics, on the same footing as physical / social / scientific worlds. Provides vocabulary for distinguishing one-step tool-prediction (L1) from multi-step environment rollouts that respect program semantics (L2) from agents that revise their environment models from deployment failures (L3).
[2026-06-25] ZeroDex: Zero-Shot Long-Horizon Dexterous Manipulation via Multi-View 3D-Grounded VLM Reasoning: ZeroDex pushes the embodied tool-use-agent pattern further than VoLo by removing the VLA from the loop: the VLM directly indexes into an object-centric library of atomic actions (each storing a 6D tool trajectory), aligns the retrieved trajectory to the lifted 3D keypoints, and runs closed-loop status verification with replan over the resulting primitive sequence — zero training, no fine-tuning.
[2026-06-25] OpenThoughts-Agent: Data Recipes for Agentic Models: OpenThoughts-Agent ships a fully open SFT-data recipe targeting cross-benchmark agentic generalization over 7 agentic benchmarks rather than a single target — 100+ ablations isolate task source and diversity as load-bearing, Qwen3-32B fine-tune reaches 44.8% (+3.9 pp over Nemotron-Terminal-32B). Pairs with Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing as the SFT-curation counterpart to closed-loop tool-trajectory synthesis.
[2026-06-24] Qwen-AgentWorld: Language World Models for General Agents: Qwen-AgentWorld introduces a new design axis to the page: instead of training agents to act in environments, train an LLM (35B-A3B or 397B-A17B MoE) to be the environment — a language world model simulating browser/terminal/code-interpreter responses via long CoT, used as a decoupled simulator for downstream agentic RL with reported gains exceeding real-environment training. World-model training also acts as a warm-up that improves downstream tool-use performance across 7 agentic benchmarks. Token-space analog to LingBot-VA’s world-model-as-policy-internal-substrate pattern.
[2026-06-23] Seed2.1: A Next-Generation Agent for Real-World Productivity: Seed2.1 release reports “with tool” parenthetical numbers alongside standalone scores on MathVision (92.6→94.5), MMMU-Pro (81.6→82.7), ZEROBench (18.0→22.0), CharXiv-RQ (85.4→86.4) — tool-augmented inference treated as a first-class evaluation mode rather than a separate benchmark.
[2026-06-20] Playful Agentic Robot Learning: RATs extends the Code-as-Policy + persistent skill library pattern from CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation with a self-directed play stage that grows the skill library before downstream tasks arrive — same Berkeley/Goldberg authors, same LIBERO-PRO benchmark, but the skill library is now learned via task-proposer-driven exploration rather than extracted from successful downstream rollouts. The library transfers to other inference-time Code-as-Policy agents (+8.9 pp RoboSuite, +8.8 pp real-world) without finetuning, evidence that the toolset itself — not the agent — is the carrier of capability.
[2026-06-19] Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering: Treats a frozen VLA as a callable tool whose only input is language, and learns the calling policy (LFP) plus a conformalized abstention head — a tool-use loop where the tool is a robot policy and the safety property is “don’t make task success worse than the base instruction”, a cleaner formalization of the orchestrator-over-VLA pattern introduced by VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation.
[2026-06-16] Qwen-Robot Suite launch — Qwen-RobotManip, Qwen-RobotNav, Qwen-RobotWorld + RobotClaw framework (Alibaba Qwen): Qwen-RobotClaw framework lets general Qwen VLM agents invoke the three Qwen-Robot Suite models (Manip, Nav, World) as tools with explicit context/memory management for long-horizon embodied tasks.
[2026-06-15] VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation: VoLo introduces Physical Orchestration — a VLM orchestrator that runs a closed agent loop over interruptible robot tools (VLA/WAM, vision models, action primitives), distinguishing the embodied regime from text-agent ReAct loops by the fact that the world does not pause for deliberation, so tool calls must be steerable mid-rollout rather than blocking.
[2026-06-13] InterleaveThinker: Reinforcing Agentic Interleaved Generation: InterleaveThinker treats a frozen image generator as the action space for a planner+critic agentic loop with >25 generator calls per trajectory; step-wise reward design for long-horizon RL with expensive per-step environment calls.
[2026-06-03] LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks: LEAP applies the same iterative tool-use loop to formal theorem proving, with Lean as the tool — a frozen general-purpose foundation model decomposes proofs into sub-goals, drafts tactics, and refines against compiler feedback, reaching 12/12 on Putnam 2025 and 70% on Lean-IMO-Bench (vs 48% for a specialized gold-medal-caliber IMO formal system). Adds Lean compiler as the cleanest possible verifier-tool to the page’s tool taxonomy: total correctness signal, no observation ambiguity, and no trained verifier required (contrast DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning).
[2026-05-25] Memory in the Age of AI Agents: Second comprehensive agent-memory survey filed (after Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey) — proposes a Forms × Functions × Dynamics taxonomy that adds Dynamics (formation / evolution / retrieval) as a first-class axis, explicitly distinguishing agent memory from RAG (a Forms-axis primitive), context engineering (provisioning, not memory), and LLM/parametric memory (the substrate, not the function). The companion paper list at https://github.com/Shichun-Liu/Agent-Memory-Paper-List is the canonical reading list for the agent-memory subfield as of late 2025.
[2026-05-22] Kimi K2.5: Visual Agentic Intelligence: Kimi K2.5 introduces Agent Swarm mode and the Parallel-Agent Reinforcement Learning (PARL) recipe — an open-source production-scale datapoint for learned parallel-agent orchestration in tool-use. The orchestrator decomposes tasks into parallelizable sub-tasks and dynamically instantiates specialized sub-agents (≤100 sub-agents × ≤1,500 tool calls), trained with shaped rewards defeating two newly named failure modes (serial collapse, spurious parallelism) and a Critical-Steps metric replacing total step count. Reported 3–4.5× Critical-Steps savings, up to 80% wall-clock reduction, and +3.5 absolute BrowseComp points over the single-agent context-managed inference baseline (74.9 → 78.4). Adds the parallel-execution axis to a design space that previously contained sequential interaction-depth (MiroThinker), sub-LM REPL orchestration (RLM), and context-space optimization (Training-Free GRPO).
[2026-05-22] VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents: First filed visual-tool-use entry. Same multi-turn (instruction, obs, action, feedback) history loop as the text-tool-use papers, with the toolset declared per-task as natural-language-described function calls (swap, rotate, move, gripper, etc.). Frontier models (Gemini 3 Pro, GPT-5, Claude 4, Qwen3-VL-235B) top out at 46.6% (Easy) / 26.0% (Hard) average across 17 long-horizon environments. Adds dimensions the text-agent papers don’t ablate: reversed-U over history length (~4 turns optimal, unbounded worse); ASCII-vs-image swap shows visual grounding (not reasoning) is GPT-5’s primary bottleneck; removing textual feedback degrades all models — visual-only feedback is insufficient. SFT recipe diverges from the GRPO consensus on this page: solver-generated demonstrations with information-revealing structure (exploratory moves that probe action-perception correspondence) substantially outperform goal-directed demonstrations under partial observability or unknown dynamics.
[2026-05-22] Training-Free Group Relative Policy Optimization: Adds the context-space pole to the design space — GRPO’s group-relative advantage primitive applied to a frozen LLM, with the “policy update” being an Add/Delete/Modify/Keep edit on a natural-language experience library. Matches/beats 32B GRPO-fine-tuned baselines (ReTool, AFM) at ~500× lower cost on AIME and WebWalkerQA. Preserves cross-domain generalization. Capability-gated: fails on QwQ-32B.
[2026-05-22] MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling: Proposes “interaction depth” as a third scaling axis alongside model size and context length; 256K context + recency-retention enables ~600 tool calls per task. Three-stage SFT → DPO + auxiliary-SFT-loss → online GRPO with streaming rollouts on synthesized multi-hop trajectories. 72B reaches 47.1 / 55.6 / 37.7 / 81.9 on BrowseComp / BrowseComp-ZH / HLE / GAIA-Text-Only.
[2026-05-22] Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing: InfTool closes a data→model→data loop — three role-playing agents synthesize tool-use trajectories from raw MCP API specs with no human labels; GRPO + gated rewards trains a 32B base to 70.9% BFCL (from 19.8%), claimed competitive with Claude-Opus.

Open questions

Does the “interaction depth scales monotonically” claim (MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling) hold up once tool-call shortcuts (as in Training-Free Group Relative Policy Optimization‘s learned experiences) are accounted for? It is possible both papers describe the same accuracy/calls curve from opposite ends — MiroThinker by extending the budget, Training-Free GRPO by lowering the calls-per-task at fixed accuracy. VisGym’s reverse-U over history length (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §4.1) is a third orientation on the same family of curves. K2.5’s PARL is a fourth — it claims monotone Critical-Steps improvement under parallel execution, but does not measure how parallel and sequential interaction-depth interact (does PARL training collapse to fewer sequential calls per sub-agent, or are gains purely from concurrency?).
Where is the capability cliff for context-space methods? Training-Free Group Relative Policy Optimization shows DeepSeek-V3.1-Terminus benefits sharply, Qwen2.5-72B-Instruct marginally, QwQ-32B regresses — but the threshold is not characterized along model size, instruction-tuning quality, or native tool-use proficiency.
Self-generated trajectory data risks overfitting to its own synthesizer (Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing BFCL scores are within-distribution); no filed paper measures held-out transfer to non-BFCL tool distributions. VisGym (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §5.4) supplies an adjacent finding from the visual side — heuristic-solver demonstrations alone underperform if they’re not structured to reveal hidden state, suggesting the “synthesize trajectories, then SFT” pattern is brittle to demonstration structure on partial-observability tasks.
Does the “frozen base + swap-able context” recipe (Training-Free Group Relative Policy Optimization §4.1) compose? E.g. can math + web experience libraries be concatenated for tasks that span both, or do they interfere? No filed paper tests compositional or zero-experience regimes.
For the parameter-space camp, is the dominant gain coming from RL or from the trajectory-curation pipelines (MiroThinker’s filtering of noisy-correct + trivially-incorrect samples; InfTool’s verification step)? No filed paper ablates RL vs. SFT on the same curated trajectory pool. VisGym is SFT-only and shows large gains over no-fine-tune (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §5.1), which is a partial data point in favor of “curation + SFT is most of the win”.
For visual tool-use specifically: is the text-feedback-dependence finding (VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents §4.3) a transient artifact of current VLMs, or a structural property of the multi-turn-with-image-observation loop? If structural, environments that do not provide textual feedback (most real-world robotic settings) will need a different agent architecture entirely.
For parallel-agent orchestration: do the serial-collapse and spurious-parallelism failure modes (Kimi K2.5: Visual Agentic Intelligence §PARL) generalize beyond K2.5’s specific reward formulation, or are they artifacts of how the K2.5 reward weights the parallelism incentive? PARL is the only published recipe in this family — there is no comparison point yet.
For parallel-agent orchestration: is the Critical-Steps metric actually well-correlated with downstream wall-clock latency, or does it reward graph structures that don’t realize their parallelism gains at inference time (e.g. sub-agents that wait on shared resources)? K2.5 claims 4.5× Critical-Steps reduction translates to 4.5× wall-clock reduction in wide-search scenarios but doesn’t validate this in tasks with tighter cross-sub-agent coupling.
Which agent-memory taxonomy survives? Memory in the Age of AI Agents (Forms × Functions × Dynamics) and Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey (substrate × cognitive mechanism × subject) overlap heavily but disagree on which axes are first-class. Open whether the “Dynamics” axis (formation / evolution / retrieval) carves new territory or recapitulates the older “memory operations” sub-axis under a different name.

Papers

2026-11

Kimi K2 Thinking — 1T MoE thinking model with native INT4 QAT and 200–300-step tool use — Kimi K2 Thinking — 1T MoE thinking model with native INT4 QAT and 200–300-step tool use (published 2026-11-06)

2026-05

Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters — Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters (published 2026-05-23)
Qwen3.7-Max: The Agent Frontier (Qwen) — Qwen3.7-Max: The Agent Frontier (Qwen) (published 2026-05-20)
SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation — SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation (published 2026-05-08)

2026-04

Toward Scalable Terminal Task Synthesis via Skill Graphs — Toward Scalable Terminal Task Synthesis via Skill Graphs (published 2026-04-28)
Kimi K2.6: Advancing Open-Source Coding — Kimi K2.6: Advancing Open-Source Coding (published 2026-04-20)
DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models — DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models (published 2026-04-15)
Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence? — Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence? (published 2026-04-03)
Trinity-Large-Thinking: Scaling an Open Source Frontier Agent — Trinity-Large-Thinking: Scaling an Open Source Frontier Agent (published 2026-04-01)

2026-03

Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Design — Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Design (published 2026-03-30)
Chroma Context-1 — 20B-parameter open-source agentic search model (Apache 2.0) — Chroma Context-1 — 20B-parameter open-source agentic search model (Apache 2.0) (published 2026-03-26)
OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis — OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis (published 2026-03-24)
OpenReward — 330+ RL Environments through one API (General Reasoning announcement) — OpenReward — 330+ RL Environments through one API (General Reasoning announcement) (published 2026-03-24)
CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation — CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation (published 2026-03-23)
MiniMax M2.7: Early Echoes of Self-Evolution — MiniMax M2.7: Early Echoes of Self-Evolution (published 2026-03-18)
Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster — Scaling Karpathy’s Autoresearch: What Happens When the Agent Gets a GPU Cluster (published 2026-03-18)
MiroThinker 1.7 release announcement (MiroMind open research agents) — MiroThinker 1.7 release announcement (MiroMind open research agents) (published 2026-03-11)
ArxivQA: Training Retrieval Agents for arXiv Search — ArxivQA: Training Retrieval Agents for arXiv Search (published 2026-03-10)
Introducing PostTrainBench (Thoughtful Lab) — Introducing PostTrainBench (Thoughtful Lab) (published 2026-03-10)
PostTrainBench: Can LLM Agents Automate LLM Post-Training? — PostTrainBench: Can LLM Agents Automate LLM Post-Training? (published 2026-03-09)
SWE-Vision: A Minimal Agent for Advancing Visual Intelligence — SWE-Vision: A Minimal Agent for Advancing Visual Intelligence (published 2026-03-09)
PSDesigner: Automated Graphic Design with a Human-Like Creative Workflow — PSDesigner: Automated Graphic Design with a Human-Like Creative Workflow (published 2026-03-01)

2026-02

DeerFlow 2.0: Deep Exploration and Efficient Research Flow (Super Agent Harness) — DeerFlow 2.0: Deep Exploration and Efficient Research Flow (Super Agent Harness) (published 2026-02-28)
CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation — CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation (published 2026-02-27)
Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization — Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization (published 2026-02-26)
DeepPresenter: Environment-Grounded Reflection for Agentic Presentation Generation — DeepPresenter: Environment-Grounded Reflection for Agentic Presentation Generation (published 2026-02-26)
REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents — REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents (published 2026-02-14)
Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity — Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity (published 2026-02-14)
Forge: Scalable Agent RL Framework and Algorithm — Forge: Scalable Agent RL Framework and Algorithm (published 2026-02-13)
MiniMax-M2.5 — MiniMax-M2.5 (published 2026-02-12)
Step 3.5 Flash — Step 3.5 Flash (published 2026-02-12)
FARS: Fully Automated Research System — FARS: Fully Automated Research System (published 2026-02-11)
Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning — Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning (published 2026-02-10)
SimpleTool: Parallel Decoding for Real-Time LLM Function Calling — SimpleTool: Parallel Decoding for Real-Time LLM Function Calling (published 2026-02-04)
Qwen3-Coder-Next Technical Report — Qwen3-Coder-Next Technical Report (published 2026-02-03)

2026-01

Introducing Agentic Vision in Gemini 3 Flash — Introducing Agentic Vision in Gemini 3 Flash (published 2026-01-27)
Yunjue Agent Tech Report: A Fully Reproducible, Zero-Start In-Situ Self-Evolving Agent System for Open-Ended Tasks — Yunjue Agent Tech Report: A Fully Reproducible, Zero-Start In-Situ Self-Evolving Agent System for Open-Ended Tasks (published 2026-01-26)
Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents — Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents (published 2026-01-26)
Kimi K2.5: Visual Agentic Intelligence — Kimi K2.5: Visual Agentic Intelligence (published 2026-01-26)
LongCat-Flash-Thinking-2601 Technical Report — LongCat-Flash-Thinking-2601 Technical Report (published 2026-01-23)
VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents — VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents (published 2026-01-23)
Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey — Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey (published 2026-01-14)
SETA: Scaling Environments for Terminal Agents (env repo) — SETA: Scaling Environments for Terminal Agents (env repo) (published 2026-01-09)
Toolathlon-GYM: Large-Scale Long-Horizon Environments for Tool-Use Agents — Toolathlon-GYM: Large-Scale Long-Horizon Environments for Tool-Use Agents (published 2026-01-01)

2025-12

Recursive Language Models — Recursive Language Models (published 2025-12-31)
Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem — Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem (published 2025-12-31)
Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing — Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing (published 2025-12-29)
Step-DeepResearch Technical Report — Step-DeepResearch Technical Report (published 2025-12-23)
AdaTooler-V: Adaptive Tool-Use for Images and Videos — AdaTooler-V: Adaptive Tool-Use for Images and Videos (published 2025-12-18)
Memory in the Age of AI Agents — Memory in the Age of AI Agents (published 2025-12-15)
Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch — Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch (published 2025-12-02)
PaperBanana: Automating Academic Illustration for AI Scientists — PaperBanana: Automating Academic Illustration for AI Scientists (published 2025-12-01)
Cut the Bill, Keep the Turns: Affordable Multi-Turn Search RL — Cut the Bill, Keep the Turns: Affordable Multi-Turn Search RL (published 2025-12-01)

2025-11

Describe Anything Anywhere At Any Moment — Describe Anything Anywhere At Any Moment (published 2025-11-29)
ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration — ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration (published 2025-11-26)
Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performance — Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performance (published 2025-11-17)
MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling — MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling (published 2025-11-14)
OSGym: Scalable OS Infra for Computer Use Agents — OSGym: Scalable OS Infra for Computer Use Agents (published 2025-11-11)
Kosmos: An AI Scientist for Autonomous Discovery — Kosmos: An AI Scientist for Autonomous Discovery (published 2025-11-04)

2025-10

Context-Bench: Benchmarking LLMs on Agentic Context Engineering — Context-Bench: Benchmarking LLMs on Agentic Context Engineering (published 2025-10-30)
VideoAgentTrek: Computer Use Pretraining from Unlabeled Videos — VideoAgentTrek: Computer Use Pretraining from Unlabeled Videos (published 2025-10-22)
PokeeResearch-7B: Deep Research Agent via RLAIF and Robust Reasoning Scaffold — PokeeResearch-7B: Deep Research Agent via RLAIF and Robust Reasoning Scaffold (published 2025-10-21)
Training-Free Group Relative Policy Optimization — Training-Free Group Relative Policy Optimization (published 2025-10-09)
Agentic AI's OODA Loop Problem — Agentic AI’s OODA Loop Problem (published 2025-10-01)

2025-09

Gemini Robotics 1.5 brings AI agents into the physical world — Gemini Robotics 1.5 brings AI agents into the physical world (published 2025-09-25)
Tongyi DeepResearch: A New Era of Open-Source AI Researchers — Tongyi DeepResearch: A New Era of Open-Source AI Researchers (published 2025-09-16)

2025-08

DeepSeek-V3.1 — hybrid thinking + non-thinking model, smarter tool calling, faster thinking — DeepSeek-V3.1 — hybrid thinking + non-thinking model, smarter tool calling, faster thinking (published 2025-08-21)

2025-07

Kimi K2: Open Agentic Intelligence — Kimi K2: Open Agentic Intelligence (published 2025-07-28)
Noam Brown — 'your fancy AI scaffolds will be washed away by scale' (Latent Space) — Noam Brown — ‘your fancy AI scaffolds will be washed away by scale’ (Latent Space) (published 2025-07-13)
Kimi K2: Open Agentic Intelligence — Kimi K2: Open Agentic Intelligence (published 2025-07-11)
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities — Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities (published 2025-07-07)

Unknown date

Edit Banana: image-to-editable-DrawIO conversion via SAM3 + VLM + OCR — Edit Banana: image-to-editable-DrawIO conversion via SAM3 + VLM + OCR (published unknown)
CLI-Anything: Making ALL Software Agent-Native — CLI-Anything: Making ALL Software Agent-Native (published unknown)
ECHO — Environment Prediction Loss for GRPO CLI Agents (microsoft/echo-rl) — ECHO — Environment Prediction Loss for GRPO CLI Agents (microsoft/echo-rl) (published unknown)
MiniMax-M2: a Mini model built for Max coding & agentic workflows — MiniMax-M2: a Mini model built for Max coding & agentic workflows (published unknown)