PerceptRL: Making Visual Perception Rewardable in MLLMs from Answers to Interactions

Tongkun Guan; Zhibo Yang; Mingkun Yang; Haocheng Wang; Wei Shen; Xiaokang Yang

cs.CVOct 27, 2026

PerceptRL: Making Visual Perception Rewardable in MLLMs from Answers to Interactions

Authors: Tongkun Guan, Zhibo Yang, Mingkun Yang, Haocheng Wang, Wei Shen, Xiaokang Yang

Organizations: MoE Key Laboratory of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China · Qwen

Abstract

Fine-grained visual perception remains a bottleneck during post-training of multimodal large language models (MLLMs), as perceptual errors can constrain gains from reasoning-focused optimization. Although image-caption pre-training improves perception, its influence can weaken as subsequent stages change data mixtures and objectives. Continuing caption supervision during post-training is challenging because subjective, information-dense descriptions complicate verifiable reward design. We present PerceptRL, which strengthens perception at two complementary levels. Perception RL uses a multi-agent pipeline to extract verified errors from dense image captions and convert the corresponding visual facts into QA pairs with fixed answer labels and rule-based rewards. Chain QA links correlated errors, while multiple-choice QA combines unrelated errors. A specialized route targets high-error domains with tailored verification, and a domain-agnostic general route covers errors beyond these domains. Perception Agentic RL addresses fine-grained and dense cases unresolved by single-turn Perception RL through multi-turn inspection with visual tools. We observe coordinate drift even in models trained to use tools, where incorrect crops or annotations distort evidence and propagate errors. Coordinate Injection corrects spatial operations in cold-start trajectories, while a grounding-aware process reward supervises intermediate spatial fidelity alongside final-answer correctness. Training proceeds through shared cold-start SFT, single-turn Perception RL, and multi-turn Perception Agentic RL. Across seven benchmarks under a shared tool-use protocol, PerceptRL achieves 71.6% average accuracy, surpassing the base model with the same tools and the strongest evaluated frontier VLM by 15.7 and 5.2 percentage points, respectively.

Figures & tables

Fig. 1 : PerceptRL : making real-world perception rewardable, and training it at two levels. Center: How it becomes rewardable. Open-ended captions contain useful perceptual information but are difficult to score with direct rule-based rewards. We extract and verify perceptual facts from these captions, then formulate QA pairs with fixed answer labels for rule-based training (top). The pipeline organizes tasks from multiple perception families ( e.g., color, spatial position, counting, functional state, orientation), using a perception tree to guide data construction (Section III-B ). The difficulty axis is what divides the two levels. Level 1: Perception RL. A single look suffices here, so we train single-turn visual QA under a rule-based outcome reward. The left panel shows what this must fix: an error entering at the perception level is inherited by an otherwise correct reasoning chain, so the answer is wrong for a purely perceptual reason. Level 2: Perception Agentic RL. This tail, where objects are small, scenes are cluttered, or details require magnification, resists a single look, so the model inspects the image over multiple grounded turns with a code interpreter. The right panel illustrates how a grounding error, where a predicted crop or annotation misses its target, can affect subsequent observations. Its propagation and accumulation across successive tool interactions constitute coordinate drift . Coordinate Injection corrects these grounding errors during data construction, while a grounding-aware process reward supervises intermediate spatial fidelity (Section III-C ).

Fig. 2 : Perception RL: from perceptual errors to rewardable QA. (01) Perception tree construction: A hierarchical capability map and model probing reveal perceptual weaknesses, guiding specialized and general data-construction routes. Appendix expands the displayed tree with additional skills and associated failure modes. (02) Rewardable QA construction: A multi-agent pipeline mines and verifies errors in the base model’s captions, then composes them into questions with unambiguous answers. MCQs combine unrelated errors, while chain QA connects correlated errors. (03) Rule-based reward for RL: The resulting QA pairs provide exact-match rewards for training the model’s visual perception. The error distribution and example errors are illustrative.

Fig. 3 : Perception Agentic RL: correcting and supervising spatial grounding during multi-turn visual inspection. (01) Coordinate drift. A misplaced annotation changes the visual evidence available to later turns. Subsequent spatial operations can inherit this error, leading to repeated or missed counts. (02) Coordinate Injection (CI). During trajectory construction, CI identifies the intended target of each spatial operation, corrects its coordinates using a VLM judge or the LocateAnything detector, and re-executes the code before the next turn. The corrected trajectories provide cold-start SFT data. (03) Grounding-aware process reward. During RL, a fidelity judge evaluates the predicted boxes and points against their intended targets in the original image. The resulting per-turn rewards complement the final-answer reward and supervise intermediate spatial accuracy (Equation 17 ).

Method	RealWorldQA	MMStar	BLINK	TreeBench	ERQA	RefSpatial	CountQA	RealWorldCounting	Avg.
Frontier VLMs
Qwen3.5-Plus [ 53 ]	82.2	82.2	73.4	62.2	62.7	69.7	71.7	35.4	67.4
Gemini-3-Flash [ 54 ]	84.6	81.5	76.3	58.3	65.2	63.4	70.4	34.6	66.8
Doubao-Seed-2.0-mini [ 55 ]	82.2	77.5	73.8	56.3	52.2	63.4	70.2	37.9	64.2
Qwen3.5-Flash [ 53 ]	72.8	74.9	66.3	58.8	61.3	57.9	67.8	22.0	60.2
Kimi-K2.6 [ 56 ]	80.0	79.2	74.4	55.6	48.8	43.3	43.2	12.9	54.7

TABLE I : Main results on eight perception benchmarks, scored by accuracy (

\%

) under each benchmark’s official protocol, comparing PerceptRL against proprietary and open frontier VLMs. Avg. averages over the eight reported benchmarks; best per column in bold .

Model (agentic)	RealWorldQA	MMStar	BLINK	TreeBench	ERQA	CountQA	RealWorldCounting	Avg
Frontier VLMs $+$ code interpreter
Gemini-3-Flash [ 54 ]	84.4	82.8	76.9	62.2	71.8	53.7	33.2	66.4
Doubao-Seed-2.0-mini [ 55 ]	78.3	77.7	73.1	54.6	54.0	65.8	33.7	62.5
Qwen3.5-Flash [ 53 ]	74.6	77.9	44.6	56.3	54.0	44.1	17.7	52.7
Kimi-K2.6 [ 56 ]	79.1	78.5	73.7	54.8	62.3	54.1	25.0	61.1
Claude-Opus-4.8 [ 58 ]	80.7	79.9	76.6	59.0	61.3	53.1	29.2	62.8

TABLE II : Comparison of multi-turn tool-use capabilities across frontier VLMs. Cold-start SFT denotes the shared SFT checkpoint before Level-1 and Level-2 RL; its single-turn scores are taken from Table I . Accuracy is reported in

\%

. Avg is the mean over the seven benchmarks shown. Best per column in bold .

Route	Domain	Single-turn QA	Agentic QA
Specialized	Counting	14,109	9,783
	Color	3,710	377
	Spatial	3,965	1,150
	State	3,331	2,483
General	General Perception	11,330	6,017
Total		36,445	19,810

TABLE III : Perception training QA across domains. Single-turn QA is the full pool used for Perception RL (Level 1). Agentic QA is a strict subset selected from this pool after Level-1 training: the trained model generates eight rollouts per query, and queries with empirical accuracy

<5/8

are retained for Perception Agentic RL (Level 2).

Method	RealWorldQA	MMStar	BLINK	TreeBench	ERQA	RefSpatial	CountQA	RealWorldCounting
SFT cold-start	83.3	77.9	68.6	56.8	53.8	67.7	57.9	29.3
$+$ Specialized route only	83.6	78.9	72.0	63.2	61.0	72.4	70.2	40.0
$+$ General route only	84.8	81.7	73.6	59.5	57.2	69.6	63.5	33.8
$+$ Both routes ( Ours )	85.2	82.6	75.1	64.7	62.8	73.8	71.9	42.1

TABLE IV : Coverage of the two Level-1 construction routes on the Qwen3.6-Flash (35A3) backbone. All rows share the same SFT cold-start and RL recipe under a matched total query budget , differing only in which route supplies the RL data; Both is our full Perception RL. Best per column in bold .

Setting	RealWorldQA	TreeBench	PerceptionBench	RealWorldCounting
Gemini-3.5-Flash (baseline)	85.4	60.5	69.4	33.0
$+$ with Python (uncorrected coordinates)	86.0 (+0.6)	67.2 (+6.7)	70.0 (+0.6)	37.4 (+4.4)
$+$ with Python ( Ours )	87.8 (+2.4)	68.9 (+8.4)	74.4 (+5.0)	45.8 (+12.8)

TABLE V : Effect of inference-time coordinate correction on final-answer accuracy using Gemini-3.5-Flash (no training).

+

with Python acts on uncorrected coordinates; Ours applies Coordinate-Injection correction at inference. Parentheses show gains over the baseline; best per column in bold .

Step	0	20	40	60	80	100	120	140	160	180	200
Outcome-only ( $\bar{G}$ )	0.55	0.57	0.63	0.59	0.66	0.58	0.64	0.60	0.66	0.59	0.63
$+$ Grounding-aware process reward ( Ours , $\bar{G}$ )	0.55	0.61	0.66	0.70	0.73	0.76	0.73	0.80	0.81	0.84	0.81

TABLE VI : Grounding fidelity

\bar{G}

during Level-2 RL with outcome-only or grounding-aware process reward. Higher is better.

Model	V*	HR-Bench		MMStar	TreeBench
Model	V*	4K	8K	MMStar	TreeBench
DeepEyes-7B [ 15 ]	85.6	75.1	72.6	—	37.5
DeepEyesV2-7B [ 16 ]	81.8	77.9	73.8	—	42.5
Pixel-Reasoner-7B [ 75 ]	84.3	74.0	66.9	—	39.0
ReVPT-3B [ 17 ]	—	—	—	53.9	—
ReVPT-7B [ 17 ]	—	—	—	61.1	—

TABLE VII : Comparison with specialized visual tool-use models on perception benchmarks. Accuracy is reported in

\%

Level-2 reward	RealWorldQA	BLINK	TreeBench	RealWorld Counting
Outcome-only ( $R_{\mathrm{acc}}$ )	85.1	73.4	63.5	45.5
$+$ Process reward ( Ours )	87.1	76.4	68.6	53.9

TABLE VIII : Effect of the grounding-aware process reward at Level 2, on the Qwen3.6-Flash (35A3) backbone. Accuracy (

\%

); best per column in bold .

Vision-Language Models

Vision-language models connect visual inputs with text for retrieval, question answering, generation, and grounded multimodal reasoning.

See all 736 papers

Apr 14, 2026

Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs

Multimodal language models (MLLMs) are increasingly paired with vision tools (e.g., depth, flow, correspondence) to enhance visual reasoning. However, despite access to these tool-generated visual cues, MLLMs often fail to benefit from them. Existing approaches typically feed raw tool outputs into the model, but these dense, pixel-level representations are misaligned with the language-native reasoning strengths of LLMs, leading to weak perception and reliance on language priors. We argue that, in problems where vision tools can provide the necessary visual cues, the bottleneck is not more tool calls or larger MLLMs, it is how tool outputs are represented. We introduce Perception Programs (P

^2

), a training-free, model-agnostic method that rewrites tool outputs into compact, structured, language-native summaries that MLLMs can directly parse and reason over. Across six perception-centric tasks in BLINK, P

^2

consistently yields large improvements over base models and raw tool-augmented baselines. With GPT-5 Mini as the base model, P

^2

raises its accuracy from 41.35% to 86.47% on multi-view reasoning, from 52.42% to 81.45% on relative depth, and achieves a 22% average gain across tasks, setting new state-of-the-art results. Even on smaller MLLMs, e.g., InternVL3.5-4B and Qwen3VL-4B, we observe 15-40% absolute gains from P

^2

, surpassing prior agentic, supervised, and RL-based tool-use methods-without any training or model modifications.

Muhammad Kamran Janjua, Hugo Silva, Di Niu +1

Huawei Technologies, Canada · University of Alberta, Canada

Tool Use in VLMs VLM Reasoning Large Vision-Language Models

Ask PDF

Apr 27, 2026

Improving Vision-language Models with Perception-centric Process Reward Models

Recent advancements in reinforcement learning with verifiable rewards (RLVR) have significantly improved the complex reasoning ability of vision-language models (VLMs). However, its outcome-level supervision is too coarse to diagnose and correct errors within the reasoning chain. To this end, we propose Perceval, a process reward model (PRM) that enables token-level error grounding, which can extract image-related claims from the response and compare them one by one with the visual evidence in the image, ultimately returning claims that contain perceptual errors. Perceval is trained with perception-intensive supervised training data. We then integrate Perceval into the RL training process to train the policy models. Specifically, compared to traditional GRPO, which applies sequence-level advantages, we apply token-level advantages by targeting penalties on hallucinated spans identified by Perceval, thus enabling fine-grained supervision signals. In addition to augmenting the training process, Perceval can also assist VLMs during the inference stage. Using Perceval, we can truncate the erroneous portions of the model's response, and then either have the model regenerate the response directly or induce the model to reflect on its previous output. This process can be repeated multiple times to achieve test-time scaling. Experiments show significant improvements on benchmarks from various domains across multiple reasoning VLMs trained with RL, highlighting the promise of perception-centric supervision as a general-purpose strategy. For test-time scaling, it also demonstrates consistent performance gains over other strategies, such as major voting. Our code and data will be publicly released at https://github.com/RUCAIBox/Perceval.

Yingqian Min, Kun Zhou, Yifan Li +6

Gaoling School of Artificial Intelligence, Renmin University of China. · Bytedance. · University of California, San Diego. +1

Process Reward Models Object Hallucination in VLMs VLM Reasoning Visually Grounded Reasoning RL for VLMs

Ask PDF

Jul 27, 2026

PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models

We introduce PerceptionBench, a benchmark specifically designed to evaluate the atomic visual perception capabilities of Multimodal Large Language Models (MLLMs). Existing benchmarks often fail to isolate perception: holistic evaluations conflate perceptual errors with failures in reasoning or domain knowledge, while application-driven benchmarks only cover narrow, fragmented domains shaped by heuristic designs. To address these limitations, PerceptionBench adopts a bottom-up approach: by diagnosing the earliest failure points in the responses of frontier MLLMs across 42 existing benchmarks, we construct an error taxonomy whose perception branch defines ten atomic perceptual capabilities. Guided by this taxonomy, we construct 3,000 verified questions with short, unambiguous answers, each isolating a single capability, with difficulty stemming from perception rather than reasoning or knowledge. Benchmark results across sixteen frontier MLLMs reveal that atomic perception remains largely unsolved---no model reaches 60% accuracy, perception-related hallucination is the weakest capability on average, and similar overall scores conceal sharply divergent capability profiles. PerceptionBench thus provides a capability-level standard for measuring and diagnosing the visual perception boundaries of MLLMs.

Zichao Lin, Yifeng Xie, Bowen Qu +30

Moonshot AI

Object Hallucination in VLMs Multimodal Large Language Models Multimodal Model Evaluation Fine-Grained Visual Perception

Ask PDF

Abstract

Vision-Language Models

Explore similar work

Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs

Improving Vision-language Models with Perception-centric Process Reward Models

PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models