Organizations: Nanyang Technological University · Chalmers University of Technology and University of Gothenburg · University of Science and Technology of China
Recent multimodal large language models (MLLMs) increasingly rely on visual chain-of-thought to perform region-grounded reasoning over images. However, existing approaches ground regions via either textified coordinates-causing modality mismatch and semantic fragmentation or fixed-granularity patches that both limit precise region selection and often require non-trivial architectural changes. In this paper, we propose Numerical Visual Chain-of-Thought (NV-CoT), a framework that enables MLLMs to reason over images using continuous numerical coordinates. NV-CoT expands the MLLM action space from discrete vocabulary tokens to a continuous Euclidean space, allowing models to directly generate bounding-box coordinates as actions with only minimal architectural modification. The framework supports both supervised fine-tuning and reinforcement learning. In particular, we replace categorical token policies with a Gaussian (or Laplace) policy over coordinates and introduce stochasticity via reparameterized sampling, making NV-CoT fully compatible with GRPO-style policy optimization. Extensive experiments on three benchmarks against eight representative visual reasoning baselines demonstrate that NV-CoT significantly improves localization precision and final answer accuracy, while also accelerating training convergence, validating the effectiveness of continuous-action visual reasoning in MLLMs. The code is available in https://github.com/kesenzhao/NV-CoT.
Figures & tables
Figure 1: Our NV-CoT outperforms text-based visual CoT models (Vis-CoT ( Shao et al., 2024a ) and DeepEyes ( Zheng et al., 2025 ) ) in localization precision , answer accuracy , and convergence speed across both SFT and RL. SFT-based models are evaluated on the Vis-CoT-363K dataset ( Shao et al., 2024a ) , where ground-truth bounding boxes are available, while RL-based models are evaluated on the DeepEyes-47K dataset ( Zheng et al., 2025 ) . We only replace the text-space discrete coordinate objective with our Euclidean-space continuous one, while keeping all other training configurations unchanged for a fair comparison.
Figure 2: Comparison of different paradigms for thinking with images. (a) Text-based approaches represent localized regions as discrete coordinate tokens, leading to modality mismatch and fragmented semantics. (b) Patch-based approaches reason directly over fine-grained visual tokens but are constrained by the fixed spatial granularity of the vision backbone. (c) Our NV-CoT predicts region coordinates in continuous space, enabling flexible and precise localization.
Model
V∗ Bench
HR Bench 4K
HR Bench 8K
Attr
Spatial
Overall
FSP
FCP
Overall
FSP
FCP
Overall
LLaVA-OneVision-7B
75.7
75.0
75.4
71.7
53.3
62.5
69.1
52.1
60.6
Qwen2.5-VL-7B
73.9
67.1
71.2
84.7
52.9
68.8
78.7
51.8
65.3
Qwen2.5-VL-32B
87.8
88.1
87.9
89.4
58.3
73.9
83.6
56.9
70.3
PaDT-7B
75.7
73.6
74.9
86.0
54.8
70.4
79.0
53.8
66.4
LVR-7B
84.4
76.3
81.2
88.3
56.8
72.5
82.5
54.5
68.5
Table 1: Overall comparison of different models on three benchmarks. The best results are in bold. Our NV-CoT consistently improves performance under both SFT and RL settings. We use NV-CoT (SFT) with an ℓ1 loss and NV-CoT (RL) with a Laplace policy. DeepEyes-7B † and NV-CoT † denote results under multi-step tool use (multiple zoom-in calls). For a fair comparison, all other visual CoT methods are evaluated with a single zoom-in call, since some baselines do not support multiple tool invocations.
Model
V∗ Bench
HR Bench 4K
HR Bench 8K
Attr
Spatial
Overall
FSP
FCP
Overall
FSP
FCP
Overall
SFT
ℓ22 loss
83.5
76.3
80.6
81.5
53.5
67.5
76.0
54.3
65.1
ℓ1 loss
84.3
78.9
82.2
82.0
55.3
68.6
77.3
54.5
65.9
RL
Gaussian policy
88.7
85.5
87.4
90.3
58.3
74.3
85.0
58.0
71.5
+ Independent σ
87.8
86.8
87.4
90.0
58.3
74.1
85.5
57.8
71.6
Laplace policy
90.4
86.8
89.0
91.0
58.5
74.8
85.8
58.3
72.0
Table 2: Ablation on coordinate policy (Gaussian ( ℓ22 ) vs. Laplace ( ℓ1 )) and uncertainty parameterization (shared σ/α vs. per-coordinate σ/α ). Non-shared σ/α : per-coordinate scale (standard deviation) predicted by four heads. Laplace policy consistently performs better than Gaussian policy, while using a shared versus per-coordinate σ/α yields comparable results.
Vis-CoT
NV-CoT ( ℓ22 )
NV-CoT ( ℓ1 )
Bbox IoU
47.3
57.4
59.5
Table 3: Bounding-box IoU on the Vis-CoT-363K dataset. NV-CoT substantially improves localization precision over Vis-CoT, with the ℓ1 loss outperforming the ℓ22 loss.
Figure 3: Behavior of α . Successful trajectories exhibit smaller α than failed ones, reflecting higher confidence.
Figure 4: Effect of λ . Performance peaks at λ=0.3 and NV-CoT consistently outperforms the baseline across all values.
Figure 5: Visualization of bounding boxes. NV-CoT produces more accurate bounding boxes (shown in red ) compared to the backbone model (shown in blue ), demonstrating improved localization capability.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 6: More visualization results of bounding boxes. NV-CoT produces more accurate bounding boxes (shown in red ) compared to the backbone model (shown in blue ), demonstrating improved localization capability.
Chain-of-thought (CoT) reasoning has proven effective for enhancing problem-solving in large language models. However, when applied to multimodal LLMs (MLLMs), existing CoT approaches suffer from a fundamental limitation: they perform reasoning entirely in text without accessing visual features during the reasoning process. After initial visual encoding, image information becomes inaccessible, forcing models to reason based solely on whatever was captured in the initial description, which forms a `vision-blind reasoning' paradigm that limits fine-grained visual extraction, error verification, and adaptive attention. We propose Text-Visual Interleaved Chain-of-Thought (TVI-CoT), a framework that enables explicit interleaving of textual reasoning and visual feature access through learnable control tokens <THINK>, <LOOK> and <ANSWER>. These tokens allow dynamic switching between reasoning and visual grounding, attending to relevant image regions conditioned on the evolving reasoning state. Experiments on eight benchmarks demonstrate state-of-the-art results among MLLM-based CoT methods and notable performance boost compared to the baseline: +6.1% on MMMU, +3.8% on MathVerse, +3.4% on MathVista, and +3.4% on ScienceQA. Code is available at https://github.com/hulianyuyy/TVI-CoT.
Lianyu Hu, Xiaoyu Ma, Zeqin Liao +1
College of Computing and Data Science of Nanyang Technological University, Singapore.
The effectiveness of Chain-of-Thought (CoT) prompting in Multimodal Large Language Models (MLLMs) remains uncertain: across several visual reasoning benchmarks, CoT prompting often degrades performance compared to direct prompting. In this paper, we provide a systematic analysis of CoT behavior in three modern MLLM families across model scales on datasets requiring step-wise visual evidence. Our analysis identifies two recurring failure modes: premature answer commitment and limited direct visual-token access during rationale generation. We further find that standard CoT-style Supervised Fine-Tuning (CoT-SFT) can mitigate these issues only partially, while often increasing reliance on textual priors and reducing counterfactual visual dependence. Motivated by these findings, we propose Attentive-CoT (Att-CoT), an attention-guided fine-tuning objective that encourages CoT trajectories to delay answer commitment while maintaining sustained visual-token access. Att-CoT can be plugged into any CoT-SFT training run without architectural changes. Experiments on three visual reasoning benchmarks across six MLLMs show that Att-CoT enhances CoT performance over standard fine-tuning.
Multimodal large language models (MLLMs) excel at visual reasoning but rely on text-based chain-of-thought (CoT), lacking interpretable visual intermediates. Existing methods use opaque tokens or external tools, missing key properties. We propose Gen-VCoT, a framework using expert vision models to generate RGB images as reasoning intermediates. It has three stages: visual grounding (SAM segmentation), geometric reasoning (Marigold depth maps), and semantic reasoning (Qwen2-VL integration). An adaptive router selects reasoning depth. Evaluations show Gen-VCoT improves spatial (25% better) and depth (50% better) questions, but may hurt simple factual queries. Text CoT outperforms visual intermediates on CLEVR (91.2% vs 62.5%), showing task-dependent optimal representations. Gen-VCoT establishes a new paradigm for interpretable multimodal reasoning.
Zhiqiang Zhou, Junliang Dai, Xu ling
Hunan Chemical Industry Vocational and Technical College, Zhuzhou, Hunan 412000, China