Visual chain-of-thought (visual-CoT), also known as "thinking with images", equips LVLMs with the ability to locate and use key visual evidence for reasoning. However, acquiring such visual-CoT capabilities requires costly post-training and tightly couples them to a specific backbone, making transfer to other LVLMs difficult. Existing cross-model transfer methods typically treat visual-CoT as a monolithic capability, using a unified representation that ignores the functional distinction between grounding and reasoning and introduces stage-irrelevant interventions. To address this problem, we propose Viscot-steer, a training-free activation steering framework for transferring visual-CoT capabilities across heterogeneous LVLMs. Specifically, Viscot-steer first disentangles visual-CoT into grounding and reasoning vectors and extracts them from the source-model layers associated with the corresponding capabilities. It then aligns the vectors with the activation space of the target model and steers the grounding and reasoning layers during bounding-box prediction and reasoning-trace generation, respectively. Across benchmarks covering high-resolution visual understanding, general visual perception and reasoning, and mathematical visual reasoning, Viscot-steer transfers capabilities from different visual-CoT models to architecturally diverse LVLMs and outperforms the strongest capability-transfer baseline by an average of 1.86 percentage points on the primary evaluation metrics.
Figures & tables
Figure 1: Disentangling and transferring visual-CoT with our Viscot-steer . (a) Existing methods extract a unified visual-CoT vector, whereas our Viscot-steer extracts separate grounding and reasoning vectors using capability-specific contrastive prompts. (b) Existing methods inject a unified vector at a fixed layer, whereas our Viscot-steer steers each vector at its corresponding layer and generation phase. (c) Activation distributions and strengths of grounding, reasoning, and undisentangled visual-CoT at the early (12th) and later (22nd) layers during the grounding and reasoning stages. (d) Our Viscot-steer outperforms the existing baselines by a substantial margin.
Figure 2: Overview of our Viscot-steer framework. (a) Disentangled vector extraction. Disentangled reasoning and grounding vectors are extracted from capability-specific contrastive prompts. (b) Phase-aware activation steering. The reasoning and grounding vectors are routed only to their corresponding generation phases and injected at capability-specific target layers.
Model
Method
V ∗
HR-Bench 4K
HR-Bench 8K
Overall
Attr.
Spatial
Overall
FSP
FCP
Overall
FSP
FCP
DeepEyes-7B
Base
88.48
90.44
85.53
74.00
91.25
56.75
71.13
84.50
57.75
Qwen2.5-VL-7B
Vanilla
75.92
71.30
82.90
68.13
80.50
55.75
62.88
76.50
49.25
L2V-CoT
84.29
86.96
80.26
71.50
83.25
59.75
67.50
81.25
53.75
Command-V
86.91
86.09
88.16
68.75
81.00
56.50
64.63
76.50
52.75
Ours
93.72
91.30
97.37
73.13
88.00
58.25
66.88
80.25
53.50
Table 1: Main results on high-resolution visual understanding benchmarks. We transfer visual-CoT capabilities from DeepEyes-7B to Qwen2.5-VL-7B, Qwen3-VL-4B and InternVL2.5-8B. Results for additional target LVLMs (Vanilla) are reported in Table 10 . We compare against the base target LVLMs, L2V-CoT, Command-V, and Transport and Merge on V* and HR-Bench. Best and second-best results are in bold and underlined , respectively.
Model
Method
Overall
Perception
Reasoning
OCR
RS
DT
MO
AD
OCR
DT
MO
AD
DeepEyes-7B
Base
52.27
90.00
50.67
89.00
44.51
34.00
73.00
70.00
46.00
35.00
Qwen2.5-VL-7B
Vanilla
38.56
82.00
33.33
87.00
22.26
20.86
66.00
70.00
26.67
19.50
L2V-CoT
41.17
87.20
34.67
85.00
24.76
21.43
62.00
62.00
34.00
26.50
Command-V
42.05
86.40
40.67
82.00
26.65
22.29
71.00
68.00
30.67
25.00
Ours
43.30
84.80
44.67
86.00
26.65
27.71
67.00
64.00
31.33
26.50
Table 2: Main results on MME-RealWorld-Lite for general visual perception and reasoning.
Model
Method
MathVista
MathVerse
Overall
General
Math
Overall
TD
TL
VI
VD
VO
DeepEyes-7B
Base
66.5
69.35
64.07
43.35
50.76
44.67
42.01
39.72
39.59
Qwen2.5-VL-7B
Vanilla
64.9
69.78
60.74
38.93
45.69
38.32
36.29
35.66
38.71
L2V-CoT
67.2
70.00
64.81
40.25
47.46
41.12
38.07
38.07
36.55
Command-V
64.1
68.48
60.37
39.06
46.19
39.09
36.55
35.53
37.94
Ours
65.6
71.30
60.74
42.89
49.37
45.18
39.85
40.74
39.34
Table 3: Main results on mathematical visual reasoning benchmarks.
Method
V*
MME-RealWorld-Lite
MathVista
Overall
Attr.
Spatial
Overall
Percep.
Reason.
Overall
General
Math
Vanilla
86.39
89.57
81.58
49.66
52.78
44.80
66.0
62.39
69.07
+vg
86.91
89.56
82.90
49.08
52.70
43.47
64.4
61.74
66.67
+vr
92.15
95.65
86.84
50.03
52.78
45.73
66.3
63.70
68.52
+v
88.48
90.44
85.53
50.44
53.98
44.93
66.8
65.65
67.78
+vg,+vr
92.67
93.91
90.79
50.70
53.29
46.67
66.9
65.43
68.15
Table 4: Ablation study of disentangled capability vectors on Qwen3-VL-4B. vg and vr denote the grounding and reasoning capability vectors, respectively, while v denotes the unified visual-CoT capability vector.
Figure 7
Figure 5: Visualization of the responses generated by the base LVLM and Viscot-steer .
Figure 9
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Model
V*
HR-Bench
MME-RW-Lite
MathVista
MathVerse
ℓtg
ℓtr
α
ℓtg
ℓtr
α
ℓtg
ℓtr
α
ℓtg
ℓtr
α
ℓtg
ℓtr
α
Qwen2.5-VL-7B
12
22
1.0
12
22
1.0
12
22
1.0
12
25
1.0
12
25
1.0
Qwen3-VL-4B
9
20
0.5
6
25
0.5
9
20
0.5
15
23
0.5
15
23
0.5
InternVL2.5-8B
12
23
0.5
12
23
0.5
15
27
0.5
16
22
0.5
16
22
0.5
Appendix
Table 7: Hyperparameter settings of Viscot-steer . ℓtg and ℓtr denote the target grounding and reasoning steering layers, respectively. α denotes the steering strength, where α=αg=αr . The source extraction layers are fixed to ℓsg=12 and ℓsr=22 for all settings. MME-RW-Lite denotes MME-RealWorld-Lite.
Target model
V*
HR-Bench 4K
HR-Bench 8K
MME-RW-Lite
MathVista
MathVerse
ℓs
ℓt
α
ℓs
ℓt
α
ℓs
ℓt
α
ℓs
ℓt
α
ℓs
ℓt
α
ℓs
ℓt
α
Qwen2.5-VL-7B
8
8
1.0
8
8
1.0
9
9
1.0
9
9
1.0
9
9
1.0
9
9
1.0
Qwen3-VL-4B
8
14
0.5
8
13
0.5
9
7
0.5
9
8
0.5
9
13
0.5
9
10
0.5
InternVL2.5-8B
8
19
0.5
8
15
0.5
9
20
0.5
9
18
0.5
9
14
0.5
9
18
0.5
Appendix
Table 8: Hyperparameter settings of L2V-CoT ( Zhan et al., 2026 ) . ℓs and ℓt denote the source extraction and target injection layers, respectively, and α denotes the injection strength.
Target model
V*
HR-Bench
MME-RW-Lite
MathVista
MathVerse
ℓ
α
ℓ
α
ℓ
α
ℓ
α
ℓ
α
Qwen2.5-VL-7B
12
2.0
12
2.0
12
2.0
12
2.0
12
2.0
Qwen3-VL-4B
3
2.0
5
2.0
5
2.0
9
2.0
9
2.0
InternVL2.5-8B
10
2.0
10
2.0
10
2.0
10
2.0
10
2.0
Appendix
Table 9: Hyperparameter settings of Command-V ( Wang et al., 2026a ) . ℓ and α denote the injection layer and injection strength, respectively.
Model
Method
V ∗
HR-Bench 4K
HR-Bench 8K
Overall
Attr.
Spatial
Overall
FSP
FCP
Overall
FSP
FCP
DeepEyes-7B
Base
88.48
90.44
85.53
74.00
91.25
56.75
71.13
84.50
57.75
Qwen2.5-VL-3B
Vanilla
54.45
53.91
55.26
48.13
50.25
46.00
41.75
45.75
37.75
L2V-CoT
63.35
65.21
60.53
44.50
49.50
39.50
45.63
50.00
41.25
Command-V
57.59
56.52
59.21
49.13
52.00
46.25
43.75
48.25
39.25
Transport and Merge
53.93
53.91
53.95
47.25
50.75
43.75
39.38
43.75
35.00
Appendix
Table 10: Additional results on high-resolution visual understanding benchmarks. We transfer visual-CoT capabilities from DeepEyes-7B to Qwen2.5-VL-3B and Qwen3-VL-32B. We compare against the base target LVLMs (Vanilla), L2V-CoT, Command-V, and Transport and Merge on V* and HR-Bench.
Model
Method
Overall
Perception
Reasoning
OCR
RS
DT
MO
AD
OCR
DT
MO
AD
DeepEyes-7B
Base
52.27
90.00
50.67
89.00
44.51
34.00
73.00
70.00
46.00
35.00
Qwen2.5-VL-3B
Vanilla
33.72
68.80
31.33
44.00
25.71
29.14
58.00
33.00
28.67
16.50
L2V-CoT
35.64
69.60
36.00
49.00
25.39
32.29
59.00
38.00
30.00
17.75
Command-V
33.98
69.20
34.00
44.00
25.71
29.14
58.00
33.00
28.67
16.50
Transport and Merge
31.74
65.60
27.33
35.00
26.33
27.43
55.00
34.00
25.33
15.50
Appendix
Table 11: Additional results on MME-RealWorld-Lite.
Model
Method
MathVista
MathVerse
Overall
General
Math
Overall
TD
TL
VI
VD
VO
DeepEyes-7B
Base
66.5
69.35
64.07
43.35
50.76
44.67
42.01
39.72
39.59
Qwen2.5-VL-3B
Vanilla
51.9
60.87
44.26
32.11
37.44
33.12
30.84
29.95
29.19
L2V-CoT
54.3
62.39
47.41
31.93
38.45
31.85
31.09
30.96
27.28
Command-V
52.5
61.30
45.00
32.03
36.42
33.38
30.84
30.58
28.93
Transport and Merge
52.2
59.35
46.11
32.64
38.45
33.38
31.09
31.47
28.81
Appendix
Table 12: Additional results on mathematical visual reasoning benchmarks.
Method
V*
MME-RealWorld-Lite
MathVista
Overall
Attr.
Spatial
Overall
Percep.
Reason.
Overall
General
Math
Vanilla
69.63
66.96
73.68
44.09
47.48
38.80
58.0
61.74
54.81
+vg
70.68
67.83
75.00
44.14
46.96
39.73
58.3
60.87
56.11
+vr
70.68
67.83
75.00
44.92
48.33
39.60
59.9
64.78
55.74
+v
70.16
67.83
73.68
45.23
48.16
40.67
60.0
63.48
57.04
+vg,+vr
72.25
69.57
76.32
46.17
49.27
41.33
61.5
65.22
58.32
Appendix
Table 13: Additional ablation study of disentangled capability vectors on InternVL2.5-8B. vg and vr denote the grounding and reasoning capability vectors, respectively, while v denotes the unified visual-CoT capability vector.
Table 17
Method
Acc.
ms/token
Latency ratio
Vanilla
86.39
19.96
1.00×
Ours
92.67
21.77
1.09×
Appendix
Table 17: Inference latency on V* using Qwen3-VL-4B.
Large Vision-Language Models (LVLMs) have shown strong promise for multimodal reasoning, yet often struggle with tasks requiring concepts beyond what is directly observable in the input image. Existing methods generate intermediate images or latent visual tokens to guide reasoning, but these representations can introduce errors and increasingly interfere with textual reasoning as reasoning progresses. We propose Visual-to-Text Chain-of-Thought Distillation (V2T), a framework that enables LVLMs to internalize visual reasoning without generating intermediate visual representations at inference time. V2T first trains a teacher LVLM using interleaved visual and textual chains of thought, and then uses knowledge distillation to train a student LVLM using the teacher's logits and cross-entropy supervision from ground-truth textual reasoning. When reasoning images can be mapped to the original image, V2T can additionally distill the teacher's attention to corresponding regions, while ground-truth bounding boxes can further guide a subsequent reinforcement learning stage. Experiments across multiple multimodal reasoning benchmarks show that V2T consistently outperforms the teacher and existing baselines, improving average accuracy by 14.3% on a held-out set and 2.7% on the broader visual evaluation suite. Moreover, lightweight SFT and substantially reduced RL make V2T up to 42x faster to train than state-of-the-art baselines.
Wenhan Yang, Nilay Naharas, Ali Payani +1
University of California, Los Angeles · Cisco Research
The effectiveness of Chain-of-Thought (CoT) prompting in Multimodal Large Language Models (MLLMs) remains uncertain: across several visual reasoning benchmarks, CoT prompting often degrades performance compared to direct prompting. In this paper, we provide a systematic analysis of CoT behavior in three modern MLLM families across model scales on datasets requiring step-wise visual evidence. Our analysis identifies two recurring failure modes: premature answer commitment and limited direct visual-token access during rationale generation. We further find that standard CoT-style Supervised Fine-Tuning (CoT-SFT) can mitigate these issues only partially, while often increasing reliance on textual priors and reducing counterfactual visual dependence. Motivated by these findings, we propose Attentive-CoT (Att-CoT), an attention-guided fine-tuning objective that encourages CoT trajectories to delay answer commitment while maintaining sustained visual-token access. Att-CoT can be plugged into any CoT-SFT training run without architectural changes. Experiments on three visual reasoning benchmarks across six MLLMs show that Att-CoT enhances CoT performance over standard fine-tuning.
Chain-of-thought (CoT) reasoning has proven effective for enhancing problem-solving in large language models. However, when applied to multimodal LLMs (MLLMs), existing CoT approaches suffer from a fundamental limitation: they perform reasoning entirely in text without accessing visual features during the reasoning process. After initial visual encoding, image information becomes inaccessible, forcing models to reason based solely on whatever was captured in the initial description, which forms a `vision-blind reasoning' paradigm that limits fine-grained visual extraction, error verification, and adaptive attention. We propose Text-Visual Interleaved Chain-of-Thought (TVI-CoT), a framework that enables explicit interleaving of textual reasoning and visual feature access through learnable control tokens <THINK>, <LOOK> and <ANSWER>. These tokens allow dynamic switching between reasoning and visual grounding, attending to relevant image regions conditioned on the evolving reasoning state. Experiments on eight benchmarks demonstrate state-of-the-art results among MLLM-based CoT methods and notable performance boost compared to the baseline: +6.1% on MMMU, +3.8% on MathVerse, +3.4% on MathVista, and +3.4% on ScienceQA. Code is available at https://github.com/hulianyuyy/TVI-CoT.
Lianyu Hu, Xiaoyu Ma, Zeqin Liao +1
College of Computing and Data Science of Nanyang Technological University, Singapore.