PokeeResearch-7B: Deep Research Agent via RLAIF and Robust Reasoning Scaffold

TL;DR

PokeeResearch-7B is a 7B Qwen2.5-7B-Instruct-based deep-research agent from Pokee AI, post-trained with REINFORCE Leave-One-Out (RLOO) against a reward combining an LLM-judge semantic-equivalence signal (RLAIF) and a format-adherence term. It pairs this with a “robust reasoning scaffold” that wraps tool calls, answers, and verification in structured tags and runs multiple independent research threads at inference time (the RTS variant). The release reports state-of-the-art mean@4 accuracy among 7B open agents across 10 QA/research benchmarks including BrowseComp, GAIA, and Humanity’s Last Exam, with weights, training code, and a technical report shipped together (Apache 2.0).

Key claims

Post-training uses RLOO (an unbiased REINFORCE variant) rather than PPO/GRPO, with no entropy or KL regularization, on 8 research threads per prompt for 140 iterations at batch size 64 and learning rate 3e-6 [§Training Hyperparameters].
The reward is a combination of an external-LLM-judge semantic-equivalence signal (RLAIF) and a format-adherence term enforcing the <tool_call> / <answer> / <verification> schema [§Reward Design].
Training data is the MiroRL-GenQA multi-turn QA dataset (from MiroMind AI), with explicit exclusion of any benchmark test data [§Training Data].
Headline mean@4 accuracy of the RTS variant (PR+): 17.6 HLE, 41.3 GAIA, 8.4 BrowseComp, 75.0 BAMB, 75.0 2WIKI, 91.8 TQ, 75.0 NQ, 60.0 POPQA, 41.4 MUSIQUE, 71.6 HOTPOTQA — beating R1searcher, SearchR1, ZeroSearch, ASearcher, and DeepResearcher at 7B scale across all 10 benchmarks [Results table].
“Research Threads Synthesis” (RTS) — running multiple independent reasoning threads at inference and synthesizing — gives a consistent +1–4 point absolute lift over the base PokeeResearch variant on every benchmark [Results table, §Glossary].
Training cost: ~5 days on 8× A100 80GB; 32,768-token context limit during training; bf16 mixed precision [§Speeds, Sizes, Times].
Companion package: Apache-2.0 weights on HuggingFace, training code at github.com/Pokee-AI/PokeeResearchOSS, and a technical report at arXiv:2510.15862 [§Model Sources, §Citation].

Method

The agent operates as a tool-augmented LLM in a ReAct-style loop with explicit structured outputs: <tool_call> for external retrieval, <answer> for candidate responses, and <verification> for self-correction passes against the retrieved evidence. Post-training is on top of Qwen2.5-7B-Instruct with the MiroRL-GenQA corpus of multi-step QA pairs. The optimizer is RLOO — a REINFORCE variant that uses leave-one-out baselines from a group of 8 rollouts per prompt — without KL or entropy regularization. The reward is RLAIF semantic correctness + format adherence, with the RLAIF judge being an external LLM checking whether the agent’s answer is semantically equivalent to the gold answer.

At inference time the RTS (“Research Threads Synthesis”) variant runs multiple independent research threads in parallel and synthesizes their conclusions — a thin parallel-scaffolding layer on top of the same trained policy.

Results

On 10 open-domain QA/research benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, Musique, Bamboogle, GAIA, BrowseComp, Humanity’s Last Exam), PokeeResearch-7B and its RTS variant outperform all five filed 7B baselines (R1searcher, SearchR1, ZeroSearch, ASearcher, DeepResearcher) on every benchmark. Largest gaps over the strongest prior baseline (ASearcher): GAIA 41.3 vs 22.1 (+19.2), BrowseComp 8.4 vs 3.2 (+5.2), HOTPOTQA 71.6 vs 71.0 (+0.6, narrowest). HLE moves from 13.8 (ASearcher) to 17.6 (PR+) [Results table].

For context — though the page does not compare directly — the larger 72B MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling reports 47.1 BrowseComp and 81.9 GAIA-Text-Only at much higher cost, so PokeeResearch-7B is the strongest 7B point on these benchmarks but is still well below the frontier 70B+ research agents.

Why it’s interesting

The most direct relative is MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling (MiroThinker) — both are open deep-research agents using ReAct + multi-stage post-training on synthesized multi-hop QA, and PokeeResearch is explicitly trained on the MiroRL-GenQA dataset from the same MiroMind ecosystem. The contrast is in the RL recipe: MiroThinker uses SFT → DPO → online GRPO with streaming rollouts and 256K context to push interaction depth, while PokeeResearch uses a single RLOO stage with 32K context and shifts the inference-time scaling to RTS parallel-thread synthesis. This makes it a useful smaller-scale ablation of the compute-allocation question: training-time interaction depth versus inference-time thread parallelism.

The RLAIF + format reward composition complements the gated/shaped reward designs catalogued in Reasoning RL — PokeeResearch is closer to Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing (InfTool, GRPO + gated rewards on a 32B base) in pipeline structure than to the GRPO-with-correctness recipes, but distinguishes itself by using RLOO rather than GRPO and by dropping KL/entropy regularization entirely (matching the DanceGRPO/HuDA trend noted on the concept page).