HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning

TL;DR

HopChain is a scalable framework for synthesizing multi-hop vision-language reasoning data for RLVR (reinforcement learning with verifiable rewards) training of VLMs. The premise is that current RLVR data for VLMs lacks logically dependent chains that stay grounded in visual evidence across intermediate steps, so failure modes (perception, reasoning, knowledge, hallucination errors) compound silently rather than being exposed and shaped by the reward. HopChain queries are instance-grounded chains where earlier hops establish the objects / sets / conditions needed by later hops, and the final answer is a specific, unambiguous number — suitable for verifiable rewards.

Key claims

Long chain-of-thought on VLMs exposes diverse failure modes — perception, reasoning, knowledge, hallucination — that compound across intermediate steps [Abstract].
Most existing RLVR data for VLMs does not involve complex reasoning chains that rely on visual evidence throughout, leaving these weaknesses largely unexposed [Abstract].
HopChain synthesizes multi-hop queries as logically dependent chains where earlier hops establish instances / sets / conditions needed by later hops [Abstract].
Final answers in HopChain queries remain specific, unambiguous numbers, making the data compatible with RLVR’s verifiable-reward gate [Abstract].

Method

HopChain is a data-synthesis framework, not a model. Each synthesized query forms a chain of instance-grounded hops: the first hop identifies one or more visual instances or sets in an image; each subsequent hop adds a constraint or operation that depends on the output of the prior hop; the final hop reduces the chain to a numerical answer that can be exact-checked. This design serves two ends — (a) the chain forces the model to keep its CoT tied to visual evidence at every step, exposing perception / hallucination errors a single-step verifier would miss; (b) the numerical answer admits the standard RLVR reward gate. The framework is positioned as scalable, i.e. the chain length, instance count, and constraint mix are knobs the synthesis pipeline turns.

(Note: full methodological detail beyond the abstract was not accessible at filing time due to an arXiv rate-limit; see the paper for hop-template design, source-image selection, and the RLVR training recipe.)

Results

The arXiv abstract reports the framing and motivation but specific RLVR benchmark numbers were not accessible at filing time. The paper is 28 pages with 8 figures and 2 tables [arXiv listing].

Why it’s interesting

Two threads on this wiki converge here. First, Synthetic Training Data has documented that the verification gate is the load-bearing piece in synthetic-data recipes (Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing, Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning); HopChain’s contribution is to design the data so the gate is trivial — the final answer is a number — but the chain enforces visual grounding throughout, which is the part that usually goes unverified. Second, Reasoning RL has tracked GRPO + shaped rewards spreading from LLMs to VLMs (vlm-gym: RL gym for vision language models (JAX), Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning); HopChain attacks a complementary problem — what data should RLVR consume so that the resulting policy generalizes? Contrasts with Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling which optimizes the generator of synthetic reasoning data; HopChain instead optimizes the structure of each example.