Decoding the Pulse of Reasoning VLMs in Multi-Image Understanding Tasks

Chenjun Li

cs.CVMar 4, 2026

Decoding the Pulse of Reasoning VLMs in Multi-Image Understanding Tasks

Authors: Chenjun Li

Organizations: School of Electrical and Computer Engineering, Cornell University

Abstract

Multi-image reasoning remains a significant challenge for vision-language models (VLMs). We investigate a previously overlooked phenomenon: during chain-of-thought (CoT) generation, the text-to-image (T2I) attention of reasoning VLMs exhibits diffuse "pulses": sporadic and unfocused attention patterns that fail to concentrate on task-relevant images. We further reveal a systematic positional bias in attention allocation across images. Motivated by these observations, we propose PulseFocus, a training-free, inference-time method that structures CoT reasoning into interleaved plan/focus blocks with soft attention gating. By forcing the model to explicitly plan which image to examine and then gating decode-time attention to the referenced image, PulseFocus sharpens attention focus and yields consistent improvements on multi-image benchmarks like BLINK benchmark (+3.7%) and MuirBench (+1.07%).

VLM Reasoning

Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.

See all 846 papers

Distilling Visual Reasoning into Text Space
Sep 28, 2026Wenhan Yang, Nilay Naharas, Ali Payani +1CoT Distillation VLM Distillation
University of California, Los Angeles · Cisco Research
TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding
Jun 7, 2026Lianyu Hu, Xiaoyu Ma, Zeqin Liao +1Multimodal Large Language Models Multimodal Reasoning
College of Computing and Data Science of Nanyang Technological University, Singapore.
Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text
Jun 8, 2026Yutong Bian, Dongjie Cheng, Heming Xia +2Visual Reasoning Multimodal Reasoning
The Hong Kong Polytechnic University

Sep 28, 2026J/K move · Enter open · S save

Distilling Visual Reasoning into Text Space

Wenhan Yang, Nilay Naharas, Ali Payani, Baharan Mirzasoleiman

University of California, Los Angeles · Cisco Research

CoT Distillation VLM Distillation Large Vision-Language Models VLM Reasoning

Open paper AskPDF arXiv

Large Vision-Language Models (LVLMs) have shown strong promise for multimodal reasoning, yet often struggle with tasks requiring concepts beyond what is directly observable in the input image. Existing methods generate intermediate images or latent visual tokens to guide reasoning, but these representations can introduce errors and increasingly interfere with textual reasoning as reasoning progresses. We propose Visual-to-Text Chain-of-Thought Distillation (V2T), a framework that enables LVLMs to internalize visual reasoning without generating intermediate visual representations at inference time. V2T first trains a teacher LVLM using interleaved visual and textual chains of thought, and then uses knowledge distillation to train a student LVLM using the teacher's logits and cross-entropy supervision from ground-truth textual reasoning. When reasoning images can be mapped to the original image, V2T can additionally distill the teacher's attention to corresponding regions, while ground-truth bounding boxes can further guide a subsequent reinforcement learning stage. Experiments across multiple multimodal reasoning benchmarks show that V2T consistently outperforms the teacher and existing baselines, improving average accuracy by 14.3% on a held-out set and 2.7% on the broader visual evaluation suite. Moreover, lightweight SFT and substantially reduced RL make V2T up to 42x faster to train than state-of-the-art baselines.

Abstract

VLM Reasoning

Explore similar work

Distilling Visual Reasoning into Text Space

TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding

Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text