Multimodal Large Language Models (MLLMs) inherit strong textual reasoning capabilities from their language backbones and text pretraining, but often fail to apply the same level of competency to visual inputs. We show that this degradation cannot be attributed to only perception failures, by evaluating models on semantically-equivalent text and rendered-text image inputs: models can accurately read text in the images and recover near-text performance when provided their own transcriptions, but just cannot reason well on the images directly. We characterize this discrepancy as a modality reasoning-access gap, where models fail to route perceived visual content into existing reasoning machinery accessible with text. To bridge this, we propose TIGER (Text-to-Image Gap-targeted training for Enhanced Reasoning), which first mines synthetic modality counterfactuals where a model succeeds on a text problem but fails when provided the corresponding rendered image, and post-trains the model on the image input via Group Relative Policy Optimization (GRPO), along with SFT and DPO variants. Importantly, although trained exclusively on synthetic rendered-text images with no multimodal annotations, TIGER results in model performance improvements across diverse real-world visual benchmarks including geometric diagrams (MathVerse), multi-disciplinary sciences (EMMA, MMMU), complex charts (ChartQAPro), and embodied spatial reasoning (RoboSpatial-Home). TIGER also complements existing state-of-the-art RLVR visual reasoning methods, yielding further performance gains. Analysis involving reasoning-subspace activation and activation patching provide support indicating that TIGER helps to align visual-to-reasoning pathways toward text reasoning states. Our implementation is available at https://github.com/minhngh/Tiger.
Figures & tables
Figure 1: Overview of the proposed pipeline. In Stage 1, we construct training samples by querying a MLLM with text and image versions of the same problem, then verifying and filtering the resulting outputs to retain pairs with correct text-conditioned answers and incorrect image-conditioned answers. In Stage 2, the resulting data is used for gap-targeted training using GRPO to improve image-conditioned reasoning and comparing with alternatives like SFT and DPO.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Accuracy gap (%) between text and image modalities on MATH-500-I across different sampling temperatures. Shaded regions denote standard deviation. Despite some variability, all models consistently exhibit a non-trivial modality gap across the full temperature range.
Model
Dataset
Generation Length
InternVL3.5-1B
MATH-500
706.9 ± 546.3
InternVL3.5-1B
MATH-500-I
689.8 ± 537.7
InternVL3.5-1B w/ TIGER
MATH-500-I
655.0 ± 354.7
Qwen2.5-VL-3B
MATH-500
547.9 ± 416.9
Qwen2.5-VL-3B
MATH-500-I
518.7 ± 377.5
Qwen2.5-VL-3B w/ TIGER
MATH-500-I
577.2 ± 333.9
Appendix
Table 15: Generation lengths (mean ± standard deviation) of base and GRPO-finetuned models on MATH-500 and MATH-500-I, showing comparable verbosity across datasets and model-dependent effects of GRPO.
Figure 3: Token length distribution comparison between Image and Text inputs for InternVL3.5-1B. The analysis includes a histogram of probability density, box plots detailing quartile spread and outliers, a cumulative distribution function (CDF), and summary statistics.
Figure 4: Token length distribution comparison between Image and Text inputs for Qwen2.5-VL-3B. The analysis includes a histogram of probability density, box plots detailing quartile spread and outliers, a cumulative distribution function (CDF), and summary statistics.
Figure 5: Token length distribution comparison between Image and Text inputs for Qwen2.5-VL-7B. The analysis includes a histogram of probability density, box plots detailing quartile spread and outliers, a cumulative distribution function (CDF), and summary statistics.
Figure 6: Token length distribution comparison between Image and Text inputs for Qwen3-VL-8B. The analysis includes a histogram of probability density, box plots detailing quartile spread and outliers, a cumulative distribution function (CDF), and summary statistics.
Model
Training data size
InternVL3.5-1B
1586
Qwen2.5-VL-3B
4043
Qwen2.5-VL-7B
3701
Qwen3-VL-8B
3401
Appendix
Table 16: Training dataset sizes for each model after applying the data construction procedure described in Subsection 4.1 .
Figure 7: Image of case study 1.
Figure 8: Image of case study 2.
Figure 9: Image of case study 3.
Model
Input
MATH-500-I
GPQA Diamond-I
LLaVA-OneVision-1.5-4B
Text Input
34.6±0.2
33.0±0.8
Image Input
29.8±0.5
23.9±0.5
Appendix
Table 18: Accuracy on MATH-500-I and GPQA Diamond-I under text and image inputs.
Model
MMStar
ChartQAPro
MMMU VAL
RoboSpatial-Home
Avg.
LLaVA-OneVision- 1.5-4B
61.9
23.6
50.1
35.4
42.8
w/ TIGER
62.7 (+0.8)
24.9 (+1.3)
50.6 (+0.5)
36.9 (+1.5)
43.8 (+1.0)
Appendix
Table 19: Additional generalization results of LLaVA-OneVision-1.5-4B on diverse multimodal evaluation benchmarks
Model
MMStar
ChartQAPro
MMMU VAL
RoboSpatial-Home
Avg.
R1-OneVision
61.3
42.0
51.4
38.0
48.2
w/ TIGER
62.5 (+1.2)
44.1 (+2.1)
52.8 (+1.4)
40.3 (+2.3)
49.9 (+1.7)
Vision-SR1
65.1
28.9
56.1
34.9
46.3
w/ TIGER
65.6 (+0.5)
29.1 (+0.2)
56.6 (+0.5)
38.9 (+4.0)
47.6 (+1.3)
Perception-R1
66.7
39.7
57.4
33.1
49.2
w/ TIGER
66.8 (+0.1)
40.3 (+0.6)
58.3 (+0.9)
38.6 (+5.5)
51.0 (+1.8)
Appendix
Table 21: Additional generalization results of RLVR-based methods on diverse multimodal evaluation benchmarks.
Method
Number of Epochs
MathVista
MathVerse
MathVision
EMMA
Avg.
SFT
3
69.1
45.7
26.8
22.9
41.1
SFT
5
69.2
45.1
26.7
23.5
41.1
SFT
10
68.7
44.4
26.2
23.4
40.7
SFT
15
67.7
45.2
27.1
23.3
40.8
DPO
1
70.4
44.6
26.7
23.0
41.2
DPO
5
69.5
45.6
26.8
22.1
41.0
Appendix
Table 23: Performance of SFT and DPO under matched training epochs with GRPO on Qwen2.5-VL-7B. Increasing the number of training epochs does not yield performance gains for either method.
Method
Number of Epochs
MathVista
MathVerse
MathVision
EMMA
Avg.
SFT
3
63.0
37.1
24.6
20.8
36.4
SFT
5
61.4
36.2
24.0
21.7
35.8
SFT
10
61.2
32.7
21.0
22.7
34.4
SFT
15
60.5
33.8
21.6
22.1
34.5
DPO
1
63.1
34.8
22.4
21.3
35.4
DPO
5
61.6
34.0
24.8
19.5
35.0
Appendix
Table 24: Performance of SFT and DPO under matched training epochs with GRPO on Qwen2.5-VL-3B. Longer training schedules provide no improvement over the default settings.
Figure 10: Layer-wise changes in attention between the finetuned and base Qwen3-VL-8B models.
Figure 11: Relative projection magnitude of image-conditioned representations onto the text-derived reasoning subspace of Qwen2.5-VL-3B on MATH-500-I
Figure 12: Relative projection magnitude of image-conditioned representations onto the text-derived reasoning subspace of Qwen3-VL-8B on MathVision
Figure 13: Relative projection magnitude of image-conditioned representations onto the text-derived reasoning subspace of Qwen3-VL-8B on MathVerse
Figure 14: Relative projection magnitude of image-conditioned representations onto the text-derived reasoning subspace of Qwen2.5-VL-7B on MathVision
Figure 15: Relative projection magnitude of image-conditioned representations onto the text-derived reasoning subspace of Qwen2.5-VL-7B on MathVerse
rpb
Qwen3-VL-8B
0.43
w/ SFT
0.43
w/ DPO
0.42
w/ GRPO text
0.37
w/ GRPO
0.43
Appendix
Table 25: Point-biserial correlation rpb between the relative projection norm ∥PRhiu∥/∥hiu∥ and the answer correctness of Qwen3-VL-8B on MATH-500-I.
Figure 16: Layer-wise changes in attention between the finetuned and base Qwen2.5-VL-7B models.
MATH-500-I
Qwen2.5-VL-7B
59.2
w/ SFT
61.8 (+2.6)
w/ DPO
59.2 (+0.0)
w/ GRPO
63.4 (+4.2)
Appendix
Table 26: Activation patching results across post-training methods for Qwen2.5-VL-7B.
Large Vision-Language Models (LVLMs) have shown strong promise for multimodal reasoning, yet often struggle with tasks requiring concepts beyond what is directly observable in the input image. Existing methods generate intermediate images or latent visual tokens to guide reasoning, but these representations can introduce errors and increasingly interfere with textual reasoning as reasoning progresses. We propose Visual-to-Text Chain-of-Thought Distillation (V2T), a framework that enables LVLMs to internalize visual reasoning without generating intermediate visual representations at inference time. V2T first trains a teacher LVLM using interleaved visual and textual chains of thought, and then uses knowledge distillation to train a student LVLM using the teacher's logits and cross-entropy supervision from ground-truth textual reasoning. When reasoning images can be mapped to the original image, V2T can additionally distill the teacher's attention to corresponding regions, while ground-truth bounding boxes can further guide a subsequent reinforcement learning stage. Experiments across multiple multimodal reasoning benchmarks show that V2T consistently outperforms the teacher and existing baselines, improving average accuracy by 14.3% on a held-out set and 2.7% on the broader visual evaluation suite. Moreover, lightweight SFT and substantially reduced RL make V2T up to 42x faster to train than state-of-the-art baselines.
Wenhan Yang, Nilay Naharas, Ali Payani +1
University of California, Los Angeles · Cisco Research
Multimodal large language models (MLLMs) can process text presented as images, yet they often perform worse than when the same content is provided as textual tokens. We systematically diagnose this "modality gap" by evaluating seven MLLMs across seven benchmarks in five input modes, spanning both synthetically rendered text and realistic document images from arXiv PDFs to Wikipedia pages. We find that the gap is highly sensitive to rendering choices such as font and resolution, and that natural document images often exhibit much smaller gaps, suggesting the performance difference partly reflects evaluation artifacts rather than fundamental limitations. Through a grounded-theory error analysis of over 4,000 examples, we identify the primary cause: image input alone suppresses reasoning effort, with models producing 5--19x shorter outputs that skip step-by-step computation or reasoning. The reluctance to reason, not a failure of perception or knowledge retrieval, drives the performance gap, particularly on tasks requiring multi-step reasoning. We show that a simple, lightweight on-policy self-distillation method by fine-tuning models on their own text-mode reasoning traces paired with image inputs closes this gap, raising image-mode accuracy to match or exceed text-mode performance with over 50% improvement, and the gains transfer to unseen benchmarks without catastrophic forgetting. Overall, our results and analyses provide a systematic understanding of the modality gap and suggest a practical path toward improving visual text understanding in multimodal language models.
Kaiser Sun, Xiaochuang Yuan, Hongjun Liu +4
Johns Hopkins University · Amazon · New York University +1
Chain-of-Thought (CoT) improves the performance of Large Language Models (LLMs) and has been extended to Multimodal Large Language Models (MLLMs). More recent work further moves from text-based multimodal reasoning toward interleaved-modal reasoning, where intermediate steps can incorporate both textual rationales and visual evidence. In this work, we propose a bolder and more ambitious idea: could images alone serve as the reasoning medium for both language and multimodal tasks? To explore this, we propose optical reasoning, which treats images as a standalone reasoning medium. We instantiate this concept with two variants: typographic-based optical reasoning, which optimizes visual layouts for compact rationale rendering, and graphical-based optical reasoning, which composes text and graphical elements into structured visual rationales. Across mathematical, scientific, and interleaved-modal reasoning benchmarks, optical reasoning can match or even exceed traditional text reasoning while reducing reasoning tokens by an average of 28.57% on language tasks and 16% on multimodal tasks, achieving 1.96 times the token efficiency of text reasoning. These results show that images can effectively and efficiently encode rationales while providing a unified visual canvas for reasoning.