Perception or Reasoning? Diagnosing and Tackling Reinforcement Learning Bottlenecks in VLMs

Zeyuan Yang; Zhuohao Chen; Jiaben Chen; Yao Zhang; Maohao Shen; Chuang Gan

cs.CVOct 15, 2026

Perception or Reasoning? Diagnosing and Tackling Reinforcement Learning Bottlenecks in VLMs

Authors: Zeyuan Yang, Zhuohao Chen, Jiaben Chen, Yao Zhang, Maohao Shen, Chuang Gan

Organizations: UMass Amherst · UIUC · MIT

Abstract

Reinforcement learning has brought strong gains to language model reasoning, but its improvements on visual tasks remain less consistent. One difficulty is that visual task failures can arise from different sources: the same benchmark may be limited by perception for one model and by reasoning for another. We introduce Perception and Reasoning Attribution (PARA), which measures the remaining perceptual and reasoning headroom of each model--benchmark pair by comparing how many current errors are recovered with more accurate visual information versus a stronger reasoner. Across 4 vision-language models and 17 multimodal benchmarks, PARA broadly agrees with the expected characteristics of existing benchmarks while revealing substantial model-specific variation. More importantly, it shows that RL gains become smaller as perceptual headroom increases, but larger when reasoning explains more of the recoverable errors, suggesting that current RL exploits reasoning headroom more effectively than perceptual headroom. Motivated by this gap, we analyze how existing RL objectives affect visual perception and find that reward- and token-level signals influence visual attention only indirectly. We therefore propose GRAD, a group-relative attention distillation objective that uses the best rollout in each group to directly guide the visual attention of weaker rollouts. GRAD consistently improves performance across two model scales and seven benchmarks, and further complements existing RL methods.

Figures & tables

Figure 1: The same task can expose different failure modes across models. One model may fail to perceive the relevant evidence, while another perceives it correctly but fails to reason from it.

Figure 2: (a) Illustration of PARA . Starting from a model’s current failures, PARA measures the fractions recoverable by selectively strengthening perception or reasoning. (b) Representative Headroom Measurements. Results exhibit broad benchmark-level trends but substantial model-specific variation, showing that effective bottlenecks cannot be inferred by benchmark labels alone.

Figure 3: (a) PARA validation. Self-description accuracy closely tracks direct image accuracy. (b)(c)(d) Perceptual supervision comparison. Only direct attention supervision exhibits a coefficient-dependent response to perceptual headroom

\mathrm{PARA}_{\mathrm{PH}}

Figure 4: Asymmetric RL gains. Across models and algorithms , RL gains (a) decrease with perceptual headroom

\mathrm{PARA}_{\mathrm{PH}}

and (b) increase with the reasoning bottleneck index

\mathrm{PARA}_{\mathrm{BI}_{R}}

	Method	MME	MMMU	MMMU-Pro	MathVis	PercepBench	ScienceQA	VisNum.	Avg.
Qwen3-VL-2B	Base	2044.0	46.6	30.8	13.5	22.7	86.1	35.1	44.0
	GRPO	2217.6	49.1	34.3	18.1	21.3	88.2	35.3	46.5
	Vision-SR1	2223.2	48.3	34.8	20.7	23.7	87.5	41.1	47.9
	VPPO	2212.0	50.8	33.8	19.1	22.8	88.2	37.6	47.3
	OPAD	2256.8	49.6	32.9	16.1	20.3	87.1	36.3	46.1
	Ours	2290.4	52.1	35.1	21.1	24.4	90.3	40.8	49.4

Table 1: Results across seven visual benchmarks. MME is reported on its native scale, while Avg. rescales it with

1/28

. The best and second-best results are shown in bold and underline , respectively.

Method	MME	MMMU	MMMU-Pro	MathVis	PercepBench	ScienceQA	VisNum.	Avg.
Vision-SR1 w/ Ours	+44.8	+2.8	+0.2	+4.3	+0.8	+2.4	+2.6	+2.1
VPPO w/ Ours	+19.6	+0.5	+0.5	+1.3	-0.9	+1.6	+1.3	+0.7
DAPO w/ Ours	+123.2	-0.2	+0.6	+2.3	+1.7	+2.3	+1.8	+1.9

Table 2: Complementarity with existing methods. Each entry reports the performance integrating GRAD to corresponding baseline on Qwen3-VL-2B. The largest gain at each benchmark is bolded .

Figure 5: (a) Changes in perception and reasoning headroom relative to Qwen3-VL-4B. GRAD reduces perceptual headroom consistently over GRPO while making more residual errors recoverable through reasoning intervention. (b) Visualization of visual attention map. GRAD focuses on the correct visual evidence while GRPO fails.

Method	MME	MMMU	MMMU-Pro	MathVis	PercepBench	ScienceQA	VisNum.	Avg.
Ours	2290.4	52.1	35.1	21.1	24.4	90.3	40.8	49.4
- w/ Attention Mass.	2265.2	50.7	34.6	21.2	21.7	88.8	40.2	48.3
- w/o Percep. Only	2228.8	52.7	34.2	19.1	21.2	90.1	38.2	47.9
- w/o Best Sibling	2284.8	50.7	32.4	13.5	19.1	87.3	33.6	45.5

Table 3: Ablation study results. Both components work jointly to achieve better performance.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Qwen3-VL-2B				Qwen3-VL-4B				Llama-4-Scout-17B				Gemma-3-27B
	$\mathrm{PH}$	$\mathrm{RH}$	$\mathrm{BI}_{P}$	$\mathrm{BI}_{R}$	$\mathrm{PH}$	$\mathrm{RH}$	$\mathrm{BI}_{P}$	$\mathrm{BI}_{R}$	$\mathrm{PH}$	$\mathrm{RH}$	$\mathrm{BI}_{P}$	$\mathrm{BI}_{R}$	$\mathrm{PH}$	$\mathrm{RH}$	$\mathrm{BI}_{P}$	$\mathrm{BI}_{R}$
BLINK	0.54	0.35	0.77	0.50	0.54	0.24	0.86	0.38	0.62	0.26	0.96	0.40	0.61	0.39	0.85	0.54
VisNumBench	0.54	0.42	0.76	0.58	0.54	0.42	0.79	0.61	0.53	0.46	0.74	0.64	0.60	0.44	0.81	0.59
RealWorldQA	0.54	0.31	0.97	0.57	0.46	0.46	0.73	0.73	0.50	0.33	0.86	0.57	0.67	0.42	0.98	0.61
PerceptionBench	0.43	0.20	0.89	0.41	0.42	0.13	0.87	0.27	0.38	0.19	0.86	0.42	0.35	0.16	0.80	0.37
MMStar	0.62	0.49	0.83	0.65	0.50	0.32	0.83	0.53	0.47	0.31	0.81	0.53	0.58	0.36	0.88	0.56

Appendix

Table 4: Full PARA results across models and benchmarks. PH and RH denote perception and reasoning headroom, and

\mathrm{BI}_{P}

and

\mathrm{BI}_{R}

denote their directional bottleneck indices. The complete results further support the benchmark-level trends alongside distinct patterns across models.

Template	Prompt
Question answering	You FIRST think about the reasoning process as an internal monologue and then provide the final answer. The reasoning process MUST BE enclosed within <think> </think> tags. The final answer MUST BE put in \boxed{} . For multiple-choice questions, put the letter of the correct option in \boxed{} , e.g., \boxed{A} .
Question-blind description	Describe this image in exhaustive detail for someone who cannot see it. Report every object, text, number, axis label, chart value, spatial relationship, color, and fine visual detail you can discern. Do NOT answer any question or guess what is being asked—only describe what is literally visible.
Description-conditioned answer	Visual description: [visual description] Question: [question] [Question answering instruction]

Appendix

Table 5: Prompt templates used for PARA .

Visual Input	Question-blind Description
PerceptionBench	The image is a $3\!\times\!4$ grid composed of 12 distinct black shapes on a plain white background. Each cell contains a single solid black shape. The top row contains a circle, square, upward-pointing triangle, and diamond. The middle row contains a circle, diamond, upward-pointing triangle, and square. The bottom row repeats the middle row: a circle, diamond, upward-pointing triangle, and square. Each shape is centered within its cell, and the shapes do not overlap.
MMMU	The image displays a table with two rows and six columns. The top row is labeled $x$ , followed by the values $-1$ , $-0.6$ , $-0.2$ , $0.2$ , and $0.6$ . The bottom row is labeled $f^{\prime}(x)$ , followed by the corresponding values $1$ , $2$ , $-0.5$ , $-1.5$ , and $1.2$ . The values are centered in their cells, with black text and lines on a white background.
MathVista	The image displays a single solid-colored rectangle centered on a plain white background. The rectangle is filled with a uniform muted purple color and is wider than it is tall. The top and bottom sides are each labeled “3 cm,” while the left and right sides are each labeled “2 cm.” The labels are positioned outside the corresponding sides, and the rectangle has sharp right-angled corners.
MME	The image displays a broom resting on a green grassy surface. It has a long, slender, light-colored wooden handle with a bright yellow rounded cap. The handle connects to a wide, dark-blue rectangular head with a raised central joint. Densely packed, light-colored straw bristles fan outward below the head and appear slightly worn at their tips. The grass contains scattered brown patches, and a partially visible dark object with a small red or orange area appears near the upper-right edge.

Appendix

Table 6: Representative examples of question-blind descriptions. Each description is generated by Qwen3-VL-4B without seeing the question.

Visual Input	Claude Opus 4.8 Description
PerceptionBench	The image is a black-and-white line drawing of two dice positioned side by side. The left die shows a front face with two diagonal dots and a top face with one dot. The right die shows a front face with six dots, a top face with three dots, and a right face with two dots. The dice touch along their adjacent edges on a plain white background.
MMMU	The image shows a table titled “Cash Flows ( $)” in white text on a blue banner. Its columns are Project and$ C_{0} $through$ C_{5} $. Project A contains$ -1{,}000 $,$ +1{,}000 $, and four zeros. Project B contains$ -2{,}000 $,$ +1{,}000 $,$ +1{,}000 $,$ +4{,}000 $,$ +1{,}000 $, and$ +1{,}000 $. Project C contains$ -3{,}000 $,$ +1{,}000 $,$ +1{,}000 $, 0,$ +1{,}000 $, and$ +1{,}000$ .
MathVista	The image is a horizontal bar chart titled “Title.” Three medium-blue bars correspond to “bull,” “wish,” and “heap.” The x-axis is labeled “Values” and runs from 0 to 10 in increments of 2. The bars reach approximately 4, 9, and 1, respectively. The white plotting area contains faint grid lines and a rectangular border.
MME	The image shows off-white ruled paper with two gray dashed horizontal guide lines. Centered between them is the handwritten expression “4 + 5 =” in dark ink. The handwriting is informal, and the space to the right of the equals sign is blank. No other text, numbers, or objects are visible.

Appendix

Table 7: Representative question-blind descriptions from Claude Opus 4.8.

Visual Input	GPT-5.6-Luna Description
PerceptionBench	The image is a rectangular table with a dark-gray header and three data rows. The first and third data rows have light-gray backgrounds, while the middle row is white. Its columns are “Test,” “Patient’s Value,” and “Reference Range.” The rows report pH as 7.20 and 7.35–7.45; Bicarbonate as 20 mEq/L and 22–29 mEq/L; and Creatinine as 1.80 mg/dL and 0.62–1.10 mg/dL.
MMMU	The image is a black-and-white chemical reaction diagram. “Glyceraldehyde 3-phosphate + H 2 O” appears to the left of a horizontal reaction arrow, while “3-phosphoglycerate” appears below the product side. Above the arrow, NAD + and NADH + H + are connected by a curved arrow that bends downward and rises toward NADH + H + . All text and lines are dark against a plain white background.
MathVista	The image shows a large upright triangle centered on a white background. Its interior is uniformly light blue, with a brighter blue outline. The triangle is approximately equilateral, with a sharp top vertex and a horizontal base. Each of its three sides is labeled “3 cm” in dark-gray text, and no other objects or chart elements are visible.
MME	The image is a wide photograph of a lake under an overcast sky. Layered gray and lavender clouds fill the upper half. A tree-covered shoreline and low rounded ridge extend across the center, with autumn foliage in green, brown, rust, orange, and muted red. Blue-gray water with fine horizontal ripples fills the foreground; no people, buildings, or boats are visible.

Appendix

Table 8: Representative question-blind descriptions from GPT-5.6-Luna.

Visual Input

What is the size of real GDP in the short-run equilibrium? (A) 500 (B) 450 (C) 400 (D) 600

Base Model Generation

The short-run equilibrium occurs where the aggregate demand (AD) curve intersects the short-run aggregate supply (SRAS) curve. The intersection appears at

(500,600)

, so the size of real GDP is 500 billion. Therefore, the answer is

\boxed{\mathrm{A}}

GPT-5.6-Luna Description

The image is a square economic graph on a white background. The vertical axis is labeled “Price Level,” and the horizontal axis is labeled “Real GDP.” Both axes have tick marks and numeric labels at intervals of 100. A blue downward-sloping line labeled “AD” begins at approximately

(0,1000)

, passes through approximately

(400,600)

, and crosses the horizontal axis at approximately

(1000,0)

. A red upward-sloping line labeled “SRAS” begins at approximately

(0,200)

and passes through approximately

(400,600)

. A vertical black line labeled “LRAS” is located at real GDP 500.

Recovered Answer

Appendix

Table 9: Example of perception recovery. Qwen3-VL-4B answers the same MMMU question before and after receiving the question-blind description generated by GPT-5.6-Luna, recovering the perception failure.

Benchmark	Qwen3-VL-2B	Qwen3-VL-4B	Llama-4-Scout-17B	Gemma-3-27B
BLINK †	0.39	0.30	0.30	0.39
VisNumBench †	0.43	0.44	0.46	0.42
RealWorldQA	0.37	0.50	0.40	0.38
PerceptionBench	0.31	0.24	0.33	0.32
MMStar †	0.44	0.39	0.40	0.39
ChartQA	0.52	0.50	0.36	0.43

Appendix

Table 10: Reasoning Share across models. RS reports

R/(P+R)

for the four models and

17

benchmarks in the main PARA evaluation. † represents multiple-choice benchmarks.

$\lambda$	MME	MMMU	MMMU-Pro	MathVis	PercepBench	ScienceQA	VisNum.	Avg.
$0.2$	2248.4	52.7	34.2	19.1	20.6	90.1	38.2	47.9
$1.0$	2290.4	52.1	35.1	21.1	24.4	90.3	40.8	49.4
$5.0$	2228.8	49.4	34.5	19.7	20.9	88.8	36.1	47.0

Appendix

Table 11: Sensitivity to the attention-loss coefficient. We vary

\lambda

on Qwen3-VL-2B.

	MME	MMMU	MMMU-Pro	MathVis	PercepBench	ScienceQA	VisNum.	Avg.
Final layer	2290.4	52.1	35.1	21.1	24.4	90.3	40.8	49.4
Top 4	2259.6	48.0	34.3	18.4	19.5	88.5	36.3	46.5
Top 8	2242.8	50.9	33.9	20.4	20.1	89.7	36.1	47.3
Top 16	2245.6	47.1	29.9	14.8	18.8	85.9	33.2	44.3

Appendix

Table 12: Effect of supervised attention layers. We extend the final-layer objective to the top 4, 8, 16, or all layers of Qwen3-VL-2B.

Attention-Guided Distillation

See all 11 papers

Improving Vision-language Models with Perception-centric Process Reward Models
Apr 27, 2026Yingqian Min, Kun Zhou, Yifan Li +6Process Reward Models Object Hallucination in VLMs
Gaoling School of Artificial Intelligence, Renmin University of China. · Bytedance. · University of California, San Diego. +1
Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning
May 13, 2026Haozhe Wang, Qixin Xu, Changpeng Wang +4Multimodal Reasoning VLM Reasoning
1Hong Kong University of Science and Technology (HKUST) · 2Tsinghua University · 3Zhejiang University +2
MIRL: Mutual Information-Guided Reinforcement Learning for Vision-Language Models
May 2, 2026Yin Zhang, Jiaxuan Zhao, Zonghan Wu +5VLM Reasoning Vision-Language Models
School of Mathematics, Tianjin University, Tianjin, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Shanghai Advanced Institute of Finance (SAIFS), East China Normal University, Shanghai, China +4

Apr 27, 2026J/K move · Enter open · S save

Improving Vision-language Models with Perception-centric Process Reward Models

Yingqian Min, Kun Zhou, Yifan Li, Yuhuan Wu+5

Gaoling School of Artificial Intelligence, Renmin University of China. · Bytedance. · University of California, San Diego.+1

Process Reward Models Object Hallucination in VLMs VLM Reasoning Visually Grounded Reasoning RL for VLMs

Open paper AskPDF arXiv

Recent advancements in reinforcement learning with verifiable rewards (RLVR) have significantly improved the complex reasoning ability of vision-language models (VLMs). However, its outcome-level supervision is too coarse to diagnose and correct errors within the reasoning chain. To this end, we propose Perceval, a process reward model (PRM) that enables token-level error grounding, which can extract image-related claims from the response and compare them one by one with the visual evidence in the image, ultimately returning claims that contain perceptual errors. Perceval is trained with perception-intensive supervised training data. We then integrate Perceval into the RL training process to train the policy models. Specifically, compared to traditional GRPO, which applies sequence-level advantages, we apply token-level advantages by targeting penalties on hallucinated spans identified by Perceval, thus enabling fine-grained supervision signals. In addition to augmenting the training process, Perceval can also assist VLMs during the inference stage. Using Perceval, we can truncate the erroneous portions of the model's response, and then either have the model regenerate the response directly or induce the model to reflect on its previous output. This process can be repeated multiple times to achieve test-time scaling. Experiments show significant improvements on benchmarks from various domains across multiple reasoning VLMs trained with RL, highlighting the promise of perception-centric supervision as a general-purpose strategy. For test-time scaling, it also demonstrates consistent performance gains over other strategies, such as major voting. Our code and data will be publicly released at https://github.com/RUCAIBox/Perceval.

Abstract

Attention-Guided Distillation

Explore similar work

Improving Vision-language Models with Perception-centric Process Reward Models

Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning

MIRL: Mutual Information-Guided Reinforcement Learning for Vision-Language Models