Recent multimodal large language models (MLLMs) advance visual reasoning by strengthening both perception and reasoning, implicitly assuming a process that transitions seamlessly from perception to reasoning. However, we observe a counterintuitive phenomenon that challenges this assumption: holding the model, question, and decoding fixed, we replace images with their caption or code representations (symbolic views), which seems to be redundant given the clear image structures, but the performance surprisingly improves by 10.2% to 23.6% across model scales and datasets. We term this performance gap as the Symbolic Visual Gap and then take a closer look at it. Through experiments, we find that although the visual evidence can already appear in the reasoning trace for the image-input model, the symbolic-view-input model shows much higher attention to the correct evidence than the image-input model. This suggests that despite good capabilities from current works in perception and reasoning themselves, another bottleneck exists between perception and reasoning in selecting perceived visual information as appropriate evidence for subsequent reasoning. To handle this bottleneck, since the symbolic view steers attention toward correct evidence and is readily obtained at scale, it provides supervision for evidence selection without manually labeled evidence. Building on this, we introduce MM-OPD, a multimodal on-policy self-distillation framework for symbolic-to-visual correction that transfers guidance from symbolic-conditioned behavior to the image-conditioned policy through residual token-level targets, steering the model toward correct visual evidence. Experiments across benchmarks and model scales show that MM-OPD improves a broad range of multimodal abilities, with gains in visual perception, chart and document understanding, mathematical reasoning, and general VQA.
Figures & tables
Figure 1: (Left) A case of replacing the question image with its symbolic view, under the same model, question, and decoding procedure. (Right) The Symbolic Visual Gap : symbolic views outperform visual inputs on every model and dataset pair from 4B to 9B.
Figure 2: A case study. Task-relevant visual information is correctly perceived and verbalized, yet distracting evidence receives more attention and is selected as the premise for subsequent reasoning.
Figure 3: Answer-to-evidence attention under image and symbolic conditioning. (a,b) Layer-wise results at the observed full-attention layers. (c) Overall mean attention to correct and distracting evidence, together with their ratio. (d) The signed score F=log2(C/X) ; positive and negative values indicate stronger association with correct and distracting evidence, respectively.
Figure 4: Overview of MM-OPD , a multimodal OPSD framework for symbolic-to-visual correction. Symbolic views provide privileged supervision, while residual targets discount token preferences induced by these views alone and guide image-conditioned reasoning toward correct visual evidence.
Category
Benchmark
Qwen3.5-4B
Qwen3.5-9B
Baseline
SFT
GRPO
GSPO
DAPO
OPSD
MM-OPD
Baseline
SFT
GRPO
GSPO
DAPO
OPSD
MM-OPD
General VQA
MMStar
72.9
70.1
73.8
72.1
68.7
72.8
74.6
74.9
70.5
73.4
74.5
73.0
75.4
75.9
RealWorldQA
75.7
75.4
77.8
76.5
74.2
76.1
77.4
78.0
78.3
76.1
76.2
76.1
77.9
77.5
MMBench v1.1
85.7
83.5
86.5
86.3
84.8
85.2
86.6
87.2
84.4
86.1
85.7
86.0
87.2
87.7
SimpleVQA
50.7
49.9
51.2
47.5
52.2
48.4
51.7
51.6
49.2
52.4
53.3
52.9
53.2
53.2
Reasoning Tasks
MathVista mini
81.4
75.9
81.8
80.5
75.7
81.8
82.2
82.9
77.9
83.2
82.9
83.4
83.0
84.1
Table 1: Main results across diverse multimodal benchmarks. We compare different post-training methods on the Qwen3.5-4B and Qwen3.5-9B backbones. Benchmarks are grouped into general VQA, reasoning, perception, and document understanding. Results are obtained with VLMEvalKit using each benchmark’s evaluation metric. The best result in is shown in bold and the second best is underlined .
Divergence
V* Bench
HR-Bench 4K
HR-Bench 8K
MMStar
MathVista mini
LogicVista
Average
Forward KL
84.3
84.8
79.5
74.4
82.0
67.6
78.8
Reverse KL
86.4
85.3
79.6
74.5
82.7
70.0
79.8
JSD
88.5
85.0
80.5
74.6
82.2
69.4
80.0
Table 2: Effect of the distillation objective on Qwen3.5-4B. Forward and reverse KL denote DKL(qt⋆∥St) and DKL(St∥qt⋆) , respectively. The residual target construction is unchanged.
Teacher Update
V* Bench
HR-Bench 4K
HR-Bench 8K
MMStar
MathVista mini
LogicVista
Average
EMA
81.1
85.8
79.8
75.0
81.6
67.1
78.4
Frozen
88.5
85.0
80.5
74.6
82.2
69.4
80.0
Table 3: Effect of the teacher update rule on Qwen3.5-4B. Current uses the latest student weights, EMA tracks their exponential moving average, and Frozen retains the initial weights.
Response Length
V* Bench
HR-Bench 4K
HR-Bench 8K
MMStar
MathVista mini
LogicVista
Average
1024
84.3
83.5
78.6
75.0
81.4
66.2
78.2
2048
84.8
84.1
79.1
75.4
81.7
66.4
78.6
4096
86.4
84.2
79.9
73.9
82.1
69.6
79.4
8192
88.5
85.0
80.5
74.6
82.2
69.4
80.0
16384
86.9
85.2
80.5
74.3
82.7
68.1
79.6
Table 4: Effect of the maximum on-policy response length. We vary the response length on Qwen3.5-4B while fixing the other settings as default.
Rollout Number
V* Bench
HR-Bench 4K
HR-Bench 8K
MMStar
MathVista mini
LogicVista
Average
1
84.3
83.5
81.0
74.1
82.9
68.9
79.1
2
88.5
85.0
80.5
74.6
82.2
69.4
80.0
4
86.4
85.5
79.3
74.4
82.2
69.1
79.5
8
86.9
85.0
79.1
74.5
82.0
68.7
79.4
Table 5: Effect of the number of on-policy rollouts. We vary the number of rollouts per prompt on Qwen3.5-4B while keeping all other settings at their default values.
Figure 5: Effectiveness of Symbolic-to-Visual Correction. (Left) Layer-wise answer-to-evidence preference F=log2(C/X) under image conditioning, where positive values favor correct evidence. (Right) The Symbolic Visual Gap before and after training across three task families.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Evidence selection in a radial bar chart. Relevant values are available, but distracting evidence is used as the premise for the derivation.
Figure 7: Evidence selection in a Venn diagram. The shared factor is verbalized but omitted from the final comparison.
Figure 8: Evidence selection in a line chart. The required values are listed, but the computation uses incorrectly selected extrema.
On-Policy Self-Distillation (OPSD) has become a standard post-training approach for improving visual reasoning in multimodal large language models (MLLMs). Existing methods draw privileged information from diverse input sources to guide self-distillation. Yet these designs overlook Modality Imbalance, a challenge inherent to MLLM reasoning. When textual information dominates generation, the model cannot fully integrate its multimodal input. Consequently, carefully designed privileged information remains underused, limiting the effectiveness of OPSD. To examine this limitation, we construct a Positive Teacher with the Zoom-In Image and a Negative Teacher with the Mask Image, which exhibit different degrees of Modality Imbalance. Changes in their reasoning correctness and token logits reveal that Modality Balance can itself serve as privileged information. Motivated by this finding, we introduce OPD-V, a visual OPSD paradigm that instantiates such information through the Positive Teacher and Negative Teacher. Positive Modality-Balance Logits Margins define a Modality-Balance Trust Region that selects the on-policy tokens used for self-distillation. Experiments across 6 benchmarks, 4 MLLM backbones, and 5 post-training methods show that OPD-V consistently improves reasoning performance while reducing training cost.
Aniri, Jinhe Bi, Peng Liao +5
2Ludwig Maximilian University of Munich · 3Munich Center for Machine Learning · 1National University of Singapore +1
On-policy self-distillation (OPSD) trains a model on its own rollouts and uses a frozen copy to provide dense token-level targets conditioned on a reference target. This works well for LLM reasoning, but a direct extension to multimodal large language models (MLLMs) can create a shortcut: the privileged target may guide tokens mainly based on the text reference target rather than the image. We propose ViGOS, a visually grounded OPSD framework for MLLM post-training. The student first writes a visual description and then reasons toward the final answer. For valid rollouts, an image-only perception teacher supervises the description, while a privileged reasoning teacher supervises the reasoning and final answer on the same student prefix. A reference teacher is used only for invalid rollouts to recover the output format. Across general vision-language, expert reasoning, visual math, spatial grounding, and visual-language-prior benchmarks, ViGOS keeps the main benefits of OPSD and improves image-grounded behavior in shortcut-prone settings.
Sihan Wang, Xiyao Liu, Lianqing Liu +1
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences
Multimodal Large Language Models (MLLMs) still struggle with fine-grained visual understanding, where answers often depend on small but decisive evidence in the full image. We observe a regional-to-global perception gap: the same MLLM answers fine-grained questions more accurately when conditioned on evidence-centered crops than on the corresponding full images, suggesting that many failures stem from difficulty to focus on relevant evidence rather than insufficient local recognition ability. Motivated by this observation, we propose Vision-OPD (Vision On-Policy Distillation), a regional-to-global self-distillation framework that transfers the model's own privileged regional perception to its full-image policy. Vision-OPD instantiates two conditional policies from the same MLLM: a crop-conditioned teacher and a full-image-conditioned student. The student generates on-policy rollouts, and Vision-OPD minimizes token-level divergence between the teacher and student next-token distributions along these rollouts. This enables the model to internalize the benefit of visual zooming without external teacher models, ground-truth labels, reward verifiers, or inference-time tool use. Experiments on multiple fine-grained visual understanding benchmarks show that Vision-OPD models achieve competitive or superior performance against much larger open-source, closed-source, and "Thinking-with-Images" agentic models. The code is available at https://github.com/VisionOPD/Vision-OPD
Qianhao Yuan, Jie Lou, Xing Yu +4
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Xiaohongshu Inc.