Let's Think with Images Efficiently! An Interleaved-Modal Chain-of-Thought Reasoning Framework with Dynamic and Precise Visual Thoughts

Xu Liu; Yongheng Zhang; Qiguang Chen; Yao Li; Sheng Wang; Libo Qin

cs.CVMar 23, 2026

Let's Think with Images Efficiently! An Interleaved-Modal Chain-of-Thought Reasoning Framework with Dynamic and Precise Visual Thoughts

Authors: Xu Liu, Yongheng Zhang, Qiguang Chen, Yao Li, Sheng Wang, Libo Qin

Organizations: Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen · School of Computer Science and Engineering, Central South University · Text Computing and Cognitive Intelligence Ministry of Education Engineering Research Center, Guizhou University · Shanghai Aviation Electric Co., Ltd, Aviation Industry Corporation of China, Shanghai

Abstract

Recently, Interleaved-modal Chain-of-Thought (ICoT) reasoning has achieved remarkable success by leveraging both multimodal inputs and outputs, attracting increasing attention. While achieving promising performance, current ICoT methods still suffer from two major limitations: (1) Static Visual Thought Positioning, which statically inserts visual information at fixed steps, resulting in inefficient and inflexible reasoning; and (2) Broken Visual Thought Representation, which involves discontinuous and semantically incoherent visual tokens. To address these limitations, we introduce Interleaved-modal Chain-of-Thought reasoning with Dynamic and Precise Visual Thoughts (DaP-ICoT), which incorporates two key components: (1) Dynamic Visual Thought Integration adaptively introduces visual inputs based on reasoning needs, reducing redundancy and improving efficiency. (2) Precise Visual Thought Guidance ensures visual semantically coherent and contextually aligned representations. Experiments across multiple benchmarks and models demonstrate that DaP-ICoT achieves state-of-the-art performance. In addition, DaP-ICoT significantly reduces the number of inserted images, leading to a 72.6% decrease in token consumption, enabling more efficient ICoT reasoning.

Figures & tables

Figure 1: (a) Current ICoT: While supporting multimodal inputs and outputs, it suffers from Static Visual Thought Integration , which requires the insertion of visual information after each step, and Broken Visual Thought Representation , in which the inserted visual tokens are lack coherence, resulting in inefficient reasoning. (b) Our DaP-ICoT : It provides Dynamic Visual Thought Integration and Precise Visual Thought Guidance , enabling efficient reasoning.

Figure 2: An overview of Interleaved-modal Chain-of-Thought reasoning with D ynamic a nd P recise Visual Thoughts ( DaP-ICoT ), including Dynamic Visual Thought Integration ( $\lx@sectionsign\ref{DVTI}$ ), and Precise Visual Thought Guidance ( $\lx@sectionsign\ref{PVTG}$ ).

Figure 3: Ablation Study on Qwen2-VL-7B : “w/o PVTG” indicates removal of Precise Visual Thought Guidance for Visual Cues, and “w/o DVTI” indicates removal of Dynamic Visual Thought Integration for Adaptive Reasoning

Figure 4: A comparison of the total token consumption between DaP-ICoT and baseline methods on the M 3 CoT benchmark using the Qwen2-VL-7B . DaP-ICoT achieves a 72.6% reduction in token consumption compared to ICoT.

Figure 5: A comparison of image insertion frequency and the number of inserted image tokens between DaP-ICoT and ICoT on the Qwen2-VL-7B model.

Figure 6: The proportion of samples where the confidence level increases after image insertion, as well as the average confidence value improvement, for DaP-ICoT and ICoT. The Blue Color denotes the baseline method ICoT, while the Green Color represents our DaP-ICoT .

Figure 7: Performance comparison across the Confidence Threshold

\tau

range of

(0,1]

: Optimal model performance achieved at

\tau=0.2

, with 57.2% Accuracy in the 0-shot setting and 58.7% in the 1-shot setting, effectively balancing visual integration and textual reasoning.

Figure 8: The case study. Figure (a) illustrates the reasoning process of the current ICoT, where after each reasoning step, it is necessary to insert image tokens that are most similar to the previous text. However, these tokens are broken image tokens, which ultimately leads to the incorrect answer (D) . Figure (b) illustrates our DaP-ICoT . In contrast, DaP-ICoT only inserts images when the text confidence is low. Furthermore, the inserted images are precise and complete , having been segmented using the SAM2 model. After efficient interleaved visual-textual reasoning, the final correct answer (A) is obtained.

Multimodal CoT Reasoning

Multimodal chain-of-thought reasoning studies how AI models combine visual, audio, and textual evidence through intermediate reasoning to solve complex tasks.

See all 143 papers

Thinking with Images without Coordinates
Oct 9, 2026Meng Cao, Haoze Zhao, Yi Wang +2VLM Reasoning Visually Grounded Reasoning
Mohamed bin Zayed University of Artificial Intelligence · Shanghai AI Lab · Sun Yat-sen University
Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text
Jun 8, 2026Yutong Bian, Dongjie Cheng, Heming Xia +2Visual Reasoning Multimodal Reasoning
The Hong Kong Polytechnic University
TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding
Jun 7, 2026Lianyu Hu, Xiaoyu Ma, Zeqin Liao +1Multimodal Large Language Models Multimodal Reasoning
College of Computing and Data Science of Nanyang Technological University, Singapore.

Oct 9, 2026J/K move · Enter open · S save

Thinking with Images without Coordinates

Meng Cao, Haoze Zhao, Yi Wang, Ian Reid+1

Mohamed bin Zayed University of Artificial Intelligence · Shanghai AI Lab · Sun Yat-sen University

VLM Reasoning Visually Grounded Reasoning Latent Visual Reasoning Visual Grounding Efficient VLM Inference

Open paper AskPDF arXiv

Recent advances in multi-modal large language models have shifted toward the "thinking with images" paradigm to improve reasoning faithfulness, yet existing methods rely heavily on explicit coordinate- based grounding. We identify two fundamental limitations in this approach, i.e., a semantic gap where sparse regressions miss high-dimensional latent structures, and a resolution gap due to the granularity mismatch between patch-level tokens and pixel-level coordinates. To address these, we propose TiCO (Thinking with images without COordinates). It reformulates visual reasoning as a semantic evidence discovery process within the latent space, and evidence is injected as token-level regions without pixel coordinates. Specifically, TiCO utilizes a dedicated probe token to estimate similarity distributions and performs importance sampling over spatially connected components to generate diverse, semantically dominant evidence rollouts. Extensive experiments across multiple high-resolution and general visual reasoning benchmarks demonstrate that our TiCO consistently outperforms state-of-the-art coordinate-based methods with significantly improved inference efficiency.

Abstract

Multimodal CoT Reasoning

Explore similar work

Thinking with Images without Coordinates

Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text

TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding