PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning (PaCoRe-Train-8k release)

TL;DR

PaCoRe (Parallel Coordinated Reasoning) is StepFun + Tsinghua’s framework that scales test-time compute via coordinated parallel breadth rather than sequential depth: each round launches massive parallel exploration trajectories, then a learned message-passing step compacts them into concise messages that condition the next round. Trained with large-scale outcome-based RL on the released PaCoRe-Train-8k corpus (math + code), an 8B model reaches 94.5% on HMMT 2025 — beating GPT-5’s 93.2% — by scaling effective TTC to roughly 2M tokens per problem. The artifact at this URL is the training dataset (3k Stage 1 + 5k Stage 2), with the model checkpoint and inference pipeline released alongside.

Key claims

An 8B PaCoRe model reaches 94.5% on HMMT 2025, surpassing GPT-5’s reported 93.2%, while consuming ~1.8M tokens of total TTC per problem at the “high” inference config $\vec{K}=[32,4]$ [Table 1].
Without message-passing compaction, performance flatlines at the model’s context limit; the compaction step is what unlocks scaling beyond a single round’s window [Key Findings].
Coordinated parallel reasoning (“breadth”) delivers higher returns per unit compute than extending a single chain (“depth”) on the reported math + code benchmarks [Key Findings].
The Pareto-optimal inference configs combine parallel-trajectory width $K$ with multi-round coordination $\hat R$ ; reported configs are $\vec K=[4]$ (low), $[16]$ (medium), $[32,4]$ (high) [Table 1 caption].
The RLVR-8B baseline (their pre-PaCoRe starting checkpoint, strong-reasoning post-trained on Qwen3-8B-Base) reaches 75.4% on HMMT 2025 at 48k tokens; PaCoRe-8B at “low” reaches 88.2% at 243k tokens — i.e. +12.8 absolute points from the parallel-coordination recipe at ~5× the TTC [Table 1].
The training data is structured as (conversation, responses, ground_truth) triples where the responses field is a list of cached generated trajectories used as input messages $M$ during PaCoRe training — i.e. the training procedure conditions on synthesized parallel rollouts rather than generating them online [Getting Started → Data].
Training is staged: Stage 1 (3k examples) and Stage 2 (5k examples), spanning opensource_math, public_mathcontest, synthetic_math, and code [Releases].
Training is reportedly stable with monotone increase in both training reward and response length across the run, evaluated single-round with $\vec K=[16]$ on HMMT 2025 and LiveCodeBench (2408–2505) [Figure 2].
Gains are uneven across benchmark types: HMMT 2025 +12.8/+17.5/+19.1 (low/med/high vs RLVR-8B), but HLE-text only +3.7/+5.3/+6.9 and MultiChallenge +8.5/+12.4/+13.7, with PaCoRe-8B still trailing all listed frontier closed/open models on HLE and MultiChallenge [Table 1].

Method

PaCoRe’s inference loop has three stages per round: (i) Think in Parallel — launch $K$ exploration trajectories from the current state; (ii) Compact — message-passing reduces the $K$ trajectories to concise messages; (iii) Coordinate — feed the compacted messages plus the original question forward to the next round. Repeating this $\hat R$ times gives multi-million-token effective TTC while respecting a fixed per-call context limit, with the final compacted message serving as the answer.

Training is outcome-based RL (RLVR-style — built on top of RLVR-8B-0926, their strong-reasoning post-trained checkpoint on Qwen3-8B-Base) on the released 8k corpus. Each training instance is a conversation plus a cached list of parallel response trajectories plus a verifiable ground-truth answer; the policy learns the Reasoning Synthesis skill of reconciling diverse parallel insights into the next message. The model is served via standard vllm serve; the bespoke inference pipeline (the multi-round dispatch + compaction loop) is announced as “coming soon” at filing time.

Results

HMMT 2025: GPT-5 93.2 (16k tokens) · Kimi-K2-Thinking 86.5 (33k) · RLVR-8B 75.4 (48k) · PaCoRe-8B 88.2/92.9/94.5 at low/medium/high (243k / 869k / 1796k tokens) [Table 1].
LiveCodeBench: GPT-5 83.5 (13k) · PaCoRe-8B 75.8/76.7/78.2 (188k / 659k / 1391k); 8B PaCoRe does not surpass GPT-5 on code [Table 1].
HLE-text: GPT-5 26.0 · PaCoRe-8B 13.0/14.6/16.2; PaCoRe-8B remains well below all listed frontier models on HLE [Table 1].
MultiChallenge: GPT-5 71.1 · PaCoRe-8B 41.8/45.7/47.0; same pattern as HLE [Table 1].
Headline framing: an 8B model surpassing GPT-5 specifically on HMMT 2025 by scaling parallel TTC ~110× (1796k vs 16k tokens) — the gain does not transfer at the same magnitude to less mathematical benchmarks.

Why it’s interesting

PaCoRe is the first filed concrete instance of a trained multi-round parallel-coordinated reasoner where the message-passing/synthesis step is the learned object — distinct from Kimi K2.5: Visual Agentic Intelligence §PARL which trains a parallel-agent orchestrator over tool-use sub-agents (different sub-task structure, different reward shaping), and distinct from Recursive Language Models / Recursive Language Models: the paradigm of 2026 where the orchestrator is a fixed Python program around a frozen LM. It also complements Inference-Time Hyper-Scaling with KV Cache Compression, which scales the same single-chain-plus-parallel-chains axis at the KV-cache layer rather than the trajectory-coordination layer — the two look stackable.

The HMMT-vs-HLE gap in Table 1 is worth flagging on its own: heavy parallel TTC scaling at 8B beats GPT-5 on a contest math benchmark but doesn’t close most of the gap on HLE-text or MultiChallenge, suggesting parallel breadth pays off where independent attempts at a checkable answer compose well, and pays off much less where the bottleneck is broad knowledge or long-form dialogue coherence.