cs.CVMar 17, 2026

HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning

Authors: Shenzhi Wang, Shixuan Liu, Jing Zhou, Chang Gao, Xiong-Hui Chen, Binghai Wang, An Yang, Shiji Song, 

Organizations: Qwen Team, Alibaba Inc. · LeapLab, Tsinghua University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Vision-language models (VLMs) show strong multimodal capabilities but still struggle with fine-grained vision-language reasoning. We find that long chain-of-thought (CoT) reasoning exposes diverse failure modes, including perception, reasoning, knowledge, and hallucination errors, which can compound across intermediate steps. However, most existing vision-language data used for reinforcement learning with verifiable rewards (RLVR) does not involve complex reasoning chains that rely on visual evidence throughout, leaving these weaknesses largely unexposed. We therefore propose HopChain, a scalable framework for synthesizing multi-hop vision-language reasoning data for RLVR training of VLMs. Each synthesized multi-hop query forms a logically dependent chain of instance-grounded hops, where earlier hops establish the instances, sets, or conditions needed for later hops, while the final answer remains a specific, unambiguous number suitable for verifiable rewards. We train Qwen3.5-35B-A3B and Qwen3.5-397B-A17B under two RLVR settings: the original data alone, and the original data plus HopChain's multi-hop data, and compare them across 24 benchmarks spanning STEM and Puzzle, General VQA, Text Recognition and Document Understanding, and Video Understanding. Although this multi-hop data is not synthesized for any specific benchmark, it improves 20 of 24 benchmarks on both models, indicating broad and generalizable gains. Consistently, replacing full chained queries with half-multi-hop or single-hop variants reduces the average score across five representative benchmarks from 70.4 to 66.7 and 64.3, respectively. Notably, multi-hop gains peak in long-CoT vision-language reasoning, exceeding 50 points in the ultra-long-CoT regime. These experiments establish HopChain as an effective, scalable framework for synthesizing multi-hop data that improves generalizable vision-language reasoning.

Figures & tables

Figure 1: Overview of HopChain and the motivation for multi-hop vision-language reasoning data. (a) HopChain synthesizes multi-hop data through four stages: category identification, instance segmentation, multi-hop query generation, and ground-truth annotation with difficulty calibration. (b) Typical vision-language training data often does not require complex reasoning chains that rely on visual evidence throughout; on hard problems, minor mistakes introduced at intermediate long-CoT steps can compound into final errors. (c) In contrast, the multi-hop data synthesized by HopChain forms logically dependent hop chains in which later hops depend on instances, sets, or conditions established by earlier hops, while nearly every hop requires fresh visual re-grounding. This training signal encourages continual visual evidence seeking during long-CoT reasoning, improving robustness and reducing compounding errors.
Figure 2: Error-type distributions before and after adding multi-hop data in RLVR. Subfigure (a) shows the error-type distribution of RLVR w/o Multi-Hop on the benchmarks in Tables 1 and 2 . Subfigure (b) shows the distribution of baseline error types mitigated after adding multi-hop data (i.e., cases corrected by RLVR w/ Multi-Hop ). In (a), errors are diverse, with perception and reasoning errors as the dominant categories. In (b), the distribution remains similar to (a), indicating that multi-hop data mitigates failures in a generalizable way rather than improving only a narrow error type. Figure 3 provides representative perception reasoning failures, together with responses corrected after adding multi-hop data.
Figure 3: Qualitative examples of unreliable visual perception in long CoT reasoning. We show failure cases from the benchmarks in Tables 1 and 2 using “RVLR w/o Multi-Hop” (baseline), alongside correct answers from “RVLR w/ Multi-Hop” (ours). For brevity, unimportant parts of the long CoT are omitted with [… ][\dots] . The baseline error is highlighted in red, and the failure reason is given in the error analysis.
Figure 4: Examples of synthesized multi-hop data. In each query, purple-highlighted text denotes the instance chain, meaning that the instance required at the current hop can only be identified from instances established in earlier hops. In the corresponding image, we mark the instance regions involved in each hop with colored rectangles, and the rectangle colors are aligned with the text colors of the corresponding hop-wise ground-truth answers.
Benchmark Before RLVR RLVR w/o Multi-Hop RLVR w/ Multi-Hop
STEM and Puzzle
MathVision61.9773.7176.05
MMMU Pro66.0769.2570.64
MMMU75.8978.8978.33
Mathvista(mini)82.8085.5085.00
BabyVision14.9521.9122.68
Table 1: Results of Qwen3.5-35B-A3B. Before RLVR denotes the model after SFT but before RLVR. RLVR w/o Multi-Hop denotes RLVR training on the original RLVR data only. RLVR w/ Multi-Hop denotes RLVR training on the original RLVR data plus the multi-hop data synthesized by HopChain.
Benchmark Before RLVR RLVR w/o Multi-Hop RLVR w/ Multi-Hop
STEM and Puzzle
MathVision77.3881.6883.71
MMMU Pro73.0375.0676.47
MMMU79.7881.6782.89
Mathvista(mini)87.5088.3089.00
BabyVision17.0128.6132.22
Table 2: Results of Qwen3.5-397B-A17B, the larger model counterpart of Table 1 . Before RLVR denotes the model after SFT but before RLVR. RLVR w/o Multi-Hop denotes RLVR training on the original RLVR data only. RLVR w/ Multi-Hop denotes RLVR training on the original RLVR data plus the multi-hop data synthesized by HopChain.
Figure 5: Benchmark-level comparison on Qwen3.5-35B-A3B under three training-query settings. RLVR w/ Single Hop simplifies each multi-hop training query to only its final hop, RLVR w/ Half-Multi-Hop removes the first half of the hops and keeps only the latter half, and RLVR w/ Multi-Hop uses the full multi-hop training queries. We evaluate the resulting models on five representative benchmarks, namely MathVision, MMMU Pro, RealWorldQA, ERQA, and VideoMMMU, and plot the benchmark score for each setting. Across all five benchmarks, the full multi-hop setting performs best, showing that preserving the complete multi-hop structure during training is more effective than shortening the query chain.
Figure 6: Performance as a function of response length, aggregated from the benchmark evaluation results of Qwen3.5-397B-A17B in Table 2 . The blue dashed line and the orange solid line show the accuracies of RLVR w/o Multi-Hop and RLVR w/ Multi-Hop , respectively, across bins of increasing response token count, while the bars show the corresponding accuracy improvement of RLVR w/ Multi-Hop over RLVR w/o Multi-Hop on the right axis. The advantage of multi-hop training remains evident on long responses and is especially pronounced in the ultra-long-CoT region.
Figure 7: Query-level success-rate distributions for Qwen3.5-35B-A3B and Qwen3.5-397B-A17B. For each multi-hop query, we independently sample eight responses and bucket the query by how many of the eight are correct, from 0/8 to 8/8 . For both models, more than half of the queries fall into the Partially Correct regime, and the queries are distributed across multiple success buckets. This suggests that the synthesized multi-hop data spans a broad difficulty range and is suitable for RLVR training across models of different sizes and capability levels.
Figure 8: Error distribution of the original RLVR w/o Multi-Hop errors among the cases where RLVR w/ Multi-Hop improves over RLVR w/o Multi-Hop on the benchmarks in Tables 1 and 2 . Center pie chart: the major error categories of the corrected cases, indicating which types of baseline errors are most often fixed after adding multi-hop data. Left pie chart: the subtype breakdown of corrected perception errors. Right pie chart: the subtype breakdown of corrected reasoning errors. With Figure 2 , this figure shows that the multi-hop data synthesized by HopChain can improve models in broad, generalizable domains.
More in this topic

Reasoning for VLMs

See all 39 papers

Explore similar work

CardsList
Sep 22, 2026

Video-HopChain: Multi-Hop Questions and Confidence-Gated Exploration for Video Reasoning Models

HopChain has shown on still images that multi-hop data synthesis improves vision-language reasoning, because long chain-of-thought reasoning exposes errors that compound across steps, while most data used for reinforcement learning with verifiable rewards (RLVR) rarely demands a chain of visual evidence, so these weaknesses are likely to stay unexposed. We observe the same problem in video, where this framework has not yet been explored. We therefore build Video-HopChain, a dataset of 22,550 multi-hop video questions over 13,378 videos, together with a held-out benchmark of 1,000 questions. Each question chains three to six yes/no questions about moments in one video, and each yields one of two integers depending on its answer. The final answer is the sum of these integers, so an exact match on that sum gives the verifiable reward that RLVR needs. We first train Qwen3-VL-8B with GRPO on a standard video dataset, and a second stage on Video-HopChain then raises the mean over eight video understanding and reasoning benchmarks from 55.4 to 57.9 and improves every one of them. Training on such a dataset, however, exposes a known limitation of GRPO: its learning signal comes from the reward variance within a group, so hard questions whose rollouts are all incorrect and easy questions whose rollouts are all correct both leave the group with no gradient. To recover these groups at the same compute budget, we introduce Confidence-Gated Exploration (CGE). With 8 rollouts per question, CGE samples the first 4 as usual. If these 4 are either all correct or all incorrect, it samples the last 4 with the policy's most confident token masked inside the reasoning span, and removes the masked positions from the loss while all 8 rollouts enter the advantage. With CGE, the mean rises further to 59.3. We release the dataset, the checkpoint, and the data generation and training code.

Trung Nguyen Quang, Yuhao Dong, Shuo Sun +4

S-Lab, Nanyang Technological University (NTU) · Johns Hopkins University · NTU

Group Relative Policy OptimizationVideo ReasoningReinforcement Learning with Verifiable RewardsRL for VLMs
AskPDF
Apr 8, 2026

Decompose, Look, and Reason: Reinforced Latent Reasoning for VLMs

Vision-Language Models often struggle with complex visual reasoning due to the visual information loss in textual CoT. Existing methods either add the cost of tool calls or rely on localized patch-based embeddings that are insufficient to extract semantics in multi-step reasoning. We propose "Decompose, Look, and Reason" (DLR), a reinforced latent reasoning framework that dynamically decomposes queries into textual premises, extracts premise-conditioned continuous visual latents, and deduces answers through grounded rationales. We introduce a three-stage training pipeline and propose a novel Spherical Gaussian Latent Policy, to enable effective exploration in the latent space. Extensive experiments on vision-centric benchmarks show that DLR consistently outperforms strong baselines, including text-only, interleaved multimodal CoT, and latent reasoning methods, while providing superior stepwise interpretability.

Mengdan Zhu, Senhao Cheng, Liang Zhao

Emory University · University of Michigan, Ann Arbor

Vision-Language GroundingVLM InterpretabilityLatent Visual ReasoningRL for VLMsVLM ReasoningVisually Grounded Reasoning+1
AskPDF
Oct 11, 2026

Genesis: Visual Augmentation Primitives for Multimodal Chain-of-Thought Reasoning

Recent multimodal large language models achieve promising results on visual reasoning tasks, approaching human-level performance on existing STEM benchmarks. However, these capabilities degrade under complex real-world scenes. We identify that chain-of-thought reasoning remains largely confined to the linguistic space, rarely anchoring each step in visual evidence. The main cause is the absence of such data during training. In this work, we introduce a training data augmentation pipeline, Visual Augmentation Primitives (VAP), which leverages state-of-the-art vision generative models to synthesize diverse photorealistic training samples and corresponding reasoning text. VAP introduces some basic operators and applies them to a seed image and gets a variant. It also constructs coherent multimodal reasoning chains throughout the generation process. Such training data force models to perceive visual content and perform logical reasoning according to them at every step. We further introduce multimodal chain-of-thought rubric-based reinforcement learning to enhance this. It rewards not only text reasoning but also visual reasoning by scoring intermediate reasoning steps for logical validity, completeness, and consistency. Moreover, we construct the UNREAL benchmark to systematically evaluate visual reasoning on hyperrealistic scenes across controllable difficulty levels. Experiments show that a 4B model built on this framework surpasses 8B baselines, with performance improving consistently as the model scales to 32B, validating the scaling law for visual reasoning.

Fan Yang, Lifeng Chen, Yipu Wang +14

Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Beijing Jiaotong University +4

VLM ReasoningMultimodal CoT ReasoningVisually Grounded ReasoningSynthetic Data AugmentationRL for VLMs
AskPDF