Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning

Ziqi Miao; Haonan Jia; Lijun Li; Chen Qian; Yuan Xiong; Wenting Yan; Jing Shao

cs.AIMar 30, 2026

Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning

Authors: Ziqi Miao, Haonan Jia, Lijun Li, Chen Qian, Yuan Xiong, Wenting Yan, Jing Shao

Abstract

Reinforcement learning with verifiable rewards (RLVR) has substantially enhanced the reasoning capabilities of multimodal large language models (MLLMs). However, existing RLVR approaches typically rely on outcome-driven optimization that updates both perception and reasoning using a shared reward based solely on the final answer. This shared reward blurs credit assignment, frequently improving reasoning patterns while failing to reliably enhance the accuracy of upstream visual evidence extraction. To address this perception bottleneck, we introduce PRCO (Perception-Reasoning Coevolution), a dual-role RLVR framework with a shared policy. PRCO consists of two cooperative roles: an Observer that generates an evidence caption tailored to the question and a Solver that predicts the final answer based on this caption. Crucially, PRCO employs role-specific reward signals: the Solver is optimized using verifiable outcome rewards on the final answer, while the Observer receives a utility reward derived from the Solver's downstream success. Extensive experiments across eight challenging multimodal reasoning benchmarks demonstrate that PRCO yields consistent improvements across model scales by over 7 points on average accuracy compared to the base model, outperforming prior open-source RL-tuned baselines.

Figures & tables

Figure 1: Diagnostic analysis of GRPO on WeMath Qiao et al. (2025) . Left: GRPO reduces reasoning errors much more than perception errors. Right: a representative failure case caused by incorrect perception.

Figure 2: Overview of PRCO. A shared policy alternates between an Observer for question-conditioned evidence captioning and a Solver for evidence-conditioned reasoning. The two roles are jointly optimized with role-specific learning signals and group-relative advantages, enabling perception–reasoning coevolution under a shared policy.

Based on Qwen2.5-VL-3B
Model	Math-Related					General Task			Overall
	MathVerse	MathVision	MathVista	WeMath	DynaMath	LogicVista	MMMU-Pro	MMStar	Avg.
Base Model	34.13	22.50	65.00	23.52	12.37	38.70	26.76	56.06	34.88
GRPO	36.29	24.70	67.40	30.57	17.96	38.92	29.88	58.00	37.97
DAPO	40.98	27.40	70.20	35.14	20.35	43.62	31.73	60.40	41.23
PAPO-G-3B	37.56	23.61	67.60	31.05	17.96	40.26	29.01	58.86	38.24

Table 1: Main results on eight multimodal reasoning benchmarks with Qwen2.5-VL-3B and Qwen2.5-VL-7B backbones. We report benchmark scores on math-related benchmarks, general-task benchmarks, and their overall average. The best and second-best results within each backbone are highlighted in bold and underlined , respectively.

Model	Math	$\Delta$	General	$\Delta$	Avg	$\Delta$
Qwen2.5-VL-3B	31.50	-	40.51	-	34.88	-
+ PRCO w/o warmup	39.45	+7.95	45.88	+5.37	41.86	+6.98
+ PRCO w/o Observer	39.48	+7.98	45.02	+4.51	41.56	+6.68
+ PRCO w/o Solver	33.46	+1.96	41.55	+1.04	36.49	+1.61
+ PRCO	40.49	+8.99	45.94	+5.43	42.53	+7.65
Qwen2.5-VL-7B	38.89	-	48.39	-	42.45	-

Table 2: Ablation study of PRCO on Qwen2.5-VL-3B and Qwen2.5-VL-7B. We report average scores on math-related benchmarks, general-task benchmarks, and all benchmarks;

\Delta

denotes improvement over the corresponding base model.

Figure 3: Training reward curves of PRCO and its role-ablation variants with Qwen2.5-VL-3B and Qwen2.5-VL-7B as backbones.

Figure 4: Pass@

k

comparison on WeMath and MMStar for PRCO-7B, DAPO-7B, and VPPO-7B under different inference-time sampling budgets.

Figure 5: Error category analysis on WeMath and MathVista. Compared with Qwen2.5-VL-7B, PRCO-7B reduces both perception and reasoning errors. For presentation clarity, Knowledge and Extraction errors are grouped into the Other category.

Figure 6: Effect of solver rollout group size

G_{S}

on Qwen2.5-VL-7B. We vary

G_{S}

among {4, 8, 12} and compare PRCO with DAPO on Math, General, and Avg. The dashed line denotes the base model performance.

Figure 7: Qualitative analysis of PRCO-7B on two representative cases. For each case, we show the Observer output together with an attention overlay obtained by averaging attention to image tokens across all layers.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Evaluation Split	Num. of Samples	Metric
MathVerse ( Zhang et al., 2024 )	MathVerse_MINI_Vision_Only	788	Overall
MathVista ( Lu et al., 2023 )	MathVista_MINI	1000	acc
MathVision ( Wang et al., 2024 )	MathVision	3040	acc
WeMath ( Qiao et al., 2025 )	WeMath	1740	Score (Strict)
DynaMath ( Zou et al., 2024 )	DynaMath	5010	Overall (Worst Case)
LogicVista ( Xiao et al., 2024 )	LogicVista	447	Overall

Appendix

Table 3: Details of the benchmarks we evaluate. The evaluation splits and reported metrics follow the settings in VLMEvalKit ( Duan et al., 2024 ) . We report single-sample greedy scores under each benchmark’s official VLMEvalKit metric, which we denote as accuracy for simplicity.

Method	lr	Max Len.	Steps	Warmup	Opt.	Rollout BS	Freeze VT	Temp.	top- $p$	top- $k$
GRPO	1e-6	2048	200	–	AdamW	384	False	1	1.0	-1
DAPO	1e-6	2048	200	–	AdamW	384	False	1	1.0	-1
PRCO	1e-6	1024 / 2048	200	40	AdamW	384	False	1	1.0	-1

Appendix

Table 4: Training hyperparameters used in our experiments. For GRPO and DAPO, Max Len. denotes the maximum rollout length of the single policy. For PRCO, it denotes Observer / Solver maximum rollout lengths.

Setting	MathVerse	MathVision	MathVista	WeMath	DynaMath	LogicVista	MMMU-Pro	MMStar	Avg.
Qwen2.5-VL-7B (Base)	43.02	25.46	70.20	35.43	20.35	45.41	35.49	64.26	42.45
PRCO-7B (Ours)	49.49	30.86	77.10	50.29	29.74	49.66	42.08	67.80	49.63
PRCO w/ $I^{S}=\emptyset$	47.34	28.82	75.20	43.81	28.94	48.77	42.89	66.13	47.74 $\downarrow$ 1.89
PRCO w/ Fixed Utility Estimator	49.24	29.67	76.70	45.52	29.74	48.55	40.17	66.20	48.22 $\downarrow$ 1.41
PRCO w/o Leakage Checker	50.00	29.31	76.00	44.19	29.54	51.45	41.33	67.60	48.68 $\downarrow$ 0.95

Appendix

Table 5: Additional ablations of PRCO-7B. We report benchmark scores on eight benchmarks. Red downward arrows in the Avg. column indicate the drop relative to PRCO-7B (ours).

Figure 8: Ablation of the observer rollout group size

G_{O}

in PRCO on Qwen2.5-VL-7B. Bars show different

G_{O}

settings (

2,4,8

) on Math, General, and Avg, and the dashed line indicates the base model performance.

Figure 9: Training caption reward standard deviation curves of PRCO and its W/O warmup variant with Qwen2.5-VL-3B and Qwen2.5-VL-7B as backbones.

Setting	MathVerse (Vision Only)	MathVision	MathVista	WeMath	DynaMath	LogicVista	MMMU-Pro-V	MMStar	Avg.
Qwen3-VL-8B-Instruct (Base)	54.57	36.64	74.90	51.33	38.12	53.02	41.27	69.40	52.41
GRPO	64.59	46.94	78.30	59.62	41.12	61.75	52.72	73.00	59.75
DAPO	65.61	50.13	78.30	66.86	43.31	62.19	53.87	74.07	61.79
PRCO (Ours)	69.67	51.78	79.00	68.57	46.11	60.40	54.57	74.27	63.05

Appendix

Table 6: Comparison of PRCO with GRPO and DAPO on Qwen3-VL-8B-Instruct. We report benchmark scores on eight benchmarks. The best and second-best results within each backbone are highlighted in bold and underlined .

Panel of Figure 10 — Figure 10: Training and inference prompt templates of the PRCO Observer and Solver, GRPO, and DAPO.

Panel of Figure 11 — Figure 11: Prompt templates for leakage checking and error-type classification.

Figure 12: Qualitative example of PRCO-7B on synthetic object filtering. The Observer enumerates the relevant objects together with size, color, material, shape, and coarse spatial cues, converting the scene into a question-conditioned object inventory. PRCO performs discrete filtering and counting from explicit visual evidence.

Figure 13: Qualitative example of PRCO-7B on bar-chart threshold counting. The Observer transcribes item-wise values across stores into explicit textual evidence, and the Solver then checks the threshold condition and aggregates over items. PRCO supports accurate chart value extraction and evidence-conditioned counting through a compact intermediate representation.

Figure 14: Qualitative example of PRCO-7B on table-based option selection. The Observer extracts only the question-relevant entries in the Rec column, rather than transcribing the full table, after which the Solver identifies the maximum and maps it to the correct answer option. PRCO focuses on question-relevant evidence while avoiding unnecessary visual details.

Figure 15: Qualitative example of PRCO-7B on diagram-grounded geometry reasoning. The Observer localizes points, coordinates, and segment relations from the diagram, and the Solver then uses this structured evidence to derive $\angle BOA$ , $OC$ , and $AB$ .

Explore similar work

PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping
Jun 7, 2026Qiming Li, Tianlun Li, Xiaolong Cheng +5Token-Level Credit Assignment VLM Reasoning
Amap CV Lab, Alibaba Group · Peking University
Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
Mar 26, 2026Jinda Lu, Junkang Wu, Jinghan Li +6RL for VLMs VLM Reasoning
University of Science and Technology of China · Peking University · Independent Researcher
Structured Role-Aware Policy Optimization for Multimodal Reasoning
May 8, 2026Bingqing Jiang, Difan ZouToken-Level Credit Assignment Multimodal Reasoning
School of Computing & Data Science, The University of Hong Kong · School of Computing & Data Science and Institute of Data Science, The University of Hong Kong

Jun 7, 2026J/K move · Enter open · S save

PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping

Qiming Li, Tianlun Li, Xiaolong Cheng, Hangyu Li+4

Amap CV Lab, Alibaba Group · Peking University

Token-Level Credit Assignment VLM Reasoning Visual Grounding RL for VLMs

Open paper AskPDF arXiv

Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective paradigm for improving the reasoning capability of Large Vision-Language Models (LVLMs). However, existing RLVR methods primarily rely on trajectory-level outcome rewards, which assign identical learning signals across all generated tokens. This coarse-grained credit assignment is fundamentally mismatched to multimodal reasoning, where only a sparse subset of tokens is causally grounded in visual evidence. Consequently, these pivotal perceptual tokens receive weak supervision and are often overwhelmed by language priors or reasoning-template tokens. To address this limitation, we propose Perception-Reinforced Policy Optimization (PRPO), a token-level reinforcement learning framework that explicitly identifies and reinforces pivotal perceptual tokens within long-horizon multimodal reasoning trajectories. PRPO introduces Robust Visual Dependency (RVD), a principled metric that identifies tokens whose predictions are both visually grounded and perturbation-stable, filtering out brittle or noisy visual tokens. Based on RVD, we further propose Perceptual Advantage Reshaping (PAR), a token-level credit assignment technique that amplifies perceptually informative tokens while preserving stable gradients for non-perceptual tokens. Extensive experiments on seven multimodal reasoning benchmarks demonstrate that PRPO consistently outperforms strong LVLM baselines across both 3B and 7B model scales, achieving average gains of 23.3% and 21.1%, respectively. PRPO achieves state-of-the-art performance with improved training efficiency and stronger cross-task generalization. Our findings highlight the importance of fine-grained credit assignment for scalable multimodal reinforcement learning.

Abstract

Explore similar work

PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping

Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs

Structured Role-Aware Policy Optimization for Multimodal Reasoning