Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing

TL;DR

InfTool is a self-evolving, fully-autonomous pipeline for synthesizing tool-use training trajectories without human annotation. Three role-playing agents (User Simulator, Tool-Calling Assistant, MCP Server) generate verified trajectories from raw API specs; the data trains a 32B base model via GRPO with gated rewards, and the improved model feeds back into the next round of synthesis. Reported result: a 32B base goes from 19.8% → 70.9% on the Berkeley Function-Calling Leaderboard (BFCL), claimed to rival Claude-Opus on that benchmark, using only synthetic data.

Key claims

A multi-agent role-playing setup (User Simulator + Tool-Calling Assistant + MCP Server) can produce diverse, verified tool-use trajectories from raw API specifications alone, with no human-annotated seeds [Abstract].
GRPO with gated rewards over the synthetic trajectories closes a data→model→data loop where the improved model targets prior capability gaps in the next synthesis round [Abstract].
The closed-loop pipeline lifts a 32B base model from 19.8% to 70.9% accuracy on BFCL (+258% relative), reported to surpass models 10× larger and rival Claude-Opus [Abstract].

Method

InfTool orchestrates three collaborative agents: a User Simulator that issues realistic queries, a Tool-Calling Assistant (the model being trained) that decides tool invocations, and an MCP Server that executes calls and returns results. Trajectories range from single-turn calls to multi-step workflows. Synthesized trajectories are filtered/verified, then used as training data under Group Relative Policy Optimization (GRPO) with gated rewards. After each training round, the improved model is plugged back in as the Tool-Calling Assistant to generate the next batch — explicitly targeting prior failure modes — yielding a closed loop that iterates without human intervention.

Results

The headline number is on the Berkeley Function-Calling Leaderboard: starting from a 32B base at 19.8%, InfTool reaches 70.9% (+258% relative, +51.1 absolute) using only synthetic data. The paper claims this surpasses models ~10× larger and is competitive with Claude-Opus on BFCL. Detailed per-category breakdowns (single-turn, multi-turn, parallel calls, etc.) are in the full paper.

Why it’s interesting

The bottleneck for tool-use agents has been the cost and coverage of supervised trajectories; if a closed multi-agent loop genuinely lifts a mid-size model past frontier-scale models on BFCL using no human labels, that materially changes the cost curve for building tool-using agents. The “data→model→data” framing is also a clean instance of self-play translated from games to API-call distributions, and the gated-reward GRPO setup is one of the more concrete recipes for stabilizing RL on synthetic agent traces. Worth watching how the BFCL numbers hold up against held-out, non-BFCL tool distributions — self-generated data can overfit to its own simulator’s quirks.