Organizations: Qwen Large Model Application Team, Alibaba · Institute of Automation, Chinese Academy of Sciences · Beijing University of Posts and Telecommunications
Vision-language process reward models (VL-PRMs) are increasingly used to score intermediate reasoning steps and rerank candidates under test-time scaling. However, they often function as black-box judges: a low step score may reflect a genuine reasoning mistake or simply the verifier's misperception of the image. This entanglement between perception and reasoning leads to systematic false positives (rewarding hallucinated visual premises) and false negatives (penalizing correct grounded statements), undermining both reranking and error localization. We introduce Explicit Visual Premise Verification (EVPV), a lightweight verification interface that conditions step scoring on the reliability of the visual premises a step depends on. The policy is prompted to produce a step-wise visual checklist that makes required visual facts explicit, while a constraint extractor independently derives structured visual constraints from the input image. EVPV matches checklist claims against these constraints to compute a scalar visual reliability signal, and calibrates PRM step rewards via reliability gating: rewards for visually dependent steps are attenuated when reliability is low and preserved when reliability is high. This decouples perceptual uncertainty from logical evaluation without per-step tool calls. Experiments on VisualProcessBench and six multimodal reasoning benchmarks show that EVPV improves step-level verification and consistently boosts Best-of-N reranking accuracy over strong baselines. Furthermore, injecting controlled corruption into the extracted constraints produces monotonic performance degradation, providing causal evidence that the gains arise from constraint fidelity and explicit premise verification rather than incidental prompt effects. Code is available at: https://github.com/Qwen-Applications/EVPV-PRM
Figures & tables
Figure 1: EVPV for reliable multimodal PRMs. (A) A motivating failure case (hallucinated visual premise). (B) Step errors are dominated by visual misinterpretation on VisualProcessBench. (C) EVPV-PRM improves step-level verification (Macro-F1). (D) EVPV-PRM improves Best-of- 8 reranking (BoN@8) across InternVL2.5 scales.
Figure 2: Overview of EVPV-PRM. Given an image I and question q , the policy model generates a step-by-step solution and, for each step, declares whether it depends on visual evidence, forming a visual checklist of explicit claims. In parallel, a constraint extractor predicts a structured set of visual facts C (numeric readings, geometric relations, and compositional structure). We compute a visual reliability score r by matching checklist claims against C to obtain support scores and aggregating them into a single confidence signal. A step verifier then produces base step rewards, which are calibrated by reliability gating: rewards for non-visual steps are kept unchanged, while rewards for visually dependent steps are down-weighted when r is low and preserved when r is high. The resulting reliability-gated step rewards are aggregated for Best-of- N reranking and process diagnosis.
Figure 3: Training and inference pipeline of EVPV. EVPV trains a constraint extractor Eϕ and a step verifier Vθ . The policy is not trained and is only prompted to produce step-wise solutions with a visual checklist at inference time.
Model
EVPV
DynaMath
MMMU
MathVerse
MathVision
WeMath
Overall
Proprietary Models
gpt-4o-mini
✗
56.57
54.08
52.53
51.42
56.74
53.57
✓
58.13
53.20
54.09
52.07
54.62
54.29
Δ
+1.56
-0.88
+1.56
+0.65
-2.12
+0.72
doubao-seed-1.6-vision
✗
66.19
59.47
63.12
61.07
62.74
62.77
✓
68.66
61.86
65.57
62.51
64.62
64.91
Table 1: VisualProcessBench Macro-F1 (%). For external judge models, ✗ uses the judge alone; ✓ attaches the EVPV plug-in (constraints-based reliability + gating, with the same constraints provided as evidence). Δ = ✓ − ✗ (points); positive Δ is shown in blue and negative Δ in red.
Model
MathVista
MathVision
MathVerse-VO
WeMath
LogicVista
MMMU
Overall
Proprietary Models
GPT-4o
60.00
31.20
40.60
45.80
52.80
70.70
47.90
Gemini-2.0-Flash
70.40
43.60
47.80
47.40
52.30
69.90
53.40
Claude-3.5-Sonnet
65.30
35.60
46.30
44.00
60.40
66.40
50.50
Open-source Models
InternVL2.5-8B
64.50
17.00
22.80
23.50
36.38
56.20
32.84
Table 2: Downstream Best-of-8 reranking with InternVL2.5 policies. BoN@8 accuracy (%) after reranking with different PRMs; numbers denote Δ8 (BoN@8 − Pass@1) for our PRM.
Variant
DynaMath
MMMU
MathVerse
MathVision
WeMath
Overall
Δ
Full Method
Full (EVPV + gating)
69.57
68.86
67.09
65.27
69.11
67.46
+0.00
Evidence / structure ablations
w/o structured facts (caption-only)
67.75
58.09
63.48
60.68
67.10
63.38
-4.08
w/o constraints (facts = ∅ )
66.66
55.80
62.61
59.13
65.81
62.11
-5.35
w/ shuffled facts (structure corrupted)
62.86
52.57
59.81
58.52
64.77
59.82
-7.64
Table 3: Key ablations on VisualProcessBench (Macro-F1; higher is better). Δ is relative to the full method.
Figure 4: Constraint quality–performance causal curves under controlled noise.
Variant
DynaMath
MMMU
MathVerse
MathVision
WeMath
Overall
Δ
Full Method
Full (EVPV + gating)
69.57
68.86
67.09
65.27
69.11
67.46
+0.00
Evidence / structure ablations
w/o structured facts (caption-only)
67.75
58.09
63.48
60.68
67.10
63.38
-4.08
w/o constraints (facts = ∅ )
66.66
55.80
62.61
59.13
65.81
62.11
-5.35
w/ shuffled facts (structure corrupted)
62.86
52.57
59.81
58.52
64.77
59.82
-7.64
Table 4: Key ablations on VisualProcessBench (Macro-F1; higher is better). Δ is relative to the full method.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: A representative structured constraint set C serialized as JSON.
Category
Key fields
Description
numeric
entity , attribute , value , unit , confidence *
A measurable fact associated with a named visual entity. entity is a label or description of the object (e.g., "segment AB" ); attribute names the quantity (e.g., "length" , "angle" , "count" ); value is a numeric literal; unit is optional (e.g., "cm" , "degrees" ).
relation
type , entities , direction , confidence
A geometric or logical relationship between two or more entities. type encodes one of { parallel , perpendicular , equal , subset , incident , adjacent , greater , less }. entities is an ordered list of participants; direction is optional (e.g., "AB → CD" ).
structure
type , parts , attachment , adjacency , confidence
Compositional or topological description of a multi-part figure. type is one of { composite , graph , table , sequence }. parts lists sub-components; attachment and adjacency are optional relational lists specifying how parts connect.
Appendix
Table 5: Top-level fields for each constraint category. * confidence is a model-estimated reliability weight in [0,1] and is used during matching (Appendix B ).
Figure 6: Policy prompt used to generate step-by-step reasoning traces with explicit visualdependency checklist items.
Figure 7: Teacher prompt used to distill pseudo-gold structured constraints C⋆ for training Eϕ .
Figure 8: Optional inference-time prompt for generating constraints in JSON form (used only when Eϕ is instantiated via prompting).
Figure 9: Optional prompt for external step judges in the EVPV plug-in setting. The judge’s output is treated as Rtbase∈{−1,+1} and then calibrated by EVPV gating.
Figure 10: Prompt used for error-type classification of incorrect steps in VisualProcessBench.
MathVista
MathVision
MathVerse-VO
WeMath
LogicVista
Overall
Strategy
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
Geometric Mean
64.5
76.3
+11.8
17.0
22.1
+5.1
22.8
29.5
+6.7
23.5
37.5
+14.0
36.4
45.3
+8.9
32.8
41.7
+8.9
Correctness Rate
64.5
75.1
+10.6
17.0
21.4
+4.4
22.8
28.9
+6.1
23.5
36.8
+13.3
36.4
44.6
+8.2
32.8
41.0
+8.2
Streak Score
64.5
74.8
+10.3
17.0
21.9
+4.9
22.8
28.6
+5.8
23.5
36.5
+13.0
36.4
44.3
+7.9
32.8
40.7
+7.9
Weighted Correctness
64.5
73.2
+8.7
17.0
20.5
+3.5
22.8
27.4
+4.6
23.5
35.1
+11.6
36.4
43.1
+6.7
32.8
39.5
+6.7
First-Error Position
64.5
75.7
+11.2
17.0
22.0
+5.0
22.8
29.0
+6.2
23.5
37.1
+13.6
36.4
44.9
+8.5
32.8
41.3
+8.5
Appendix
Table 7: Best-of-8 reranking under five aggregation strategies, InternVL2.5-8B policy. Pass@1 is the same across strategies; BoN@8 and Δ8 vary.
MathVista
MathVision
MathVerse-VO
WeMath
LogicVista
Overall
Strategy
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
Geometric Mean
68.2
79.6
+11.4
23.4
28.1
+4.7
24.0
32.5
+8.5
30.9
42.1
+11.2
39.6
51.7
+12.1
37.2
46.8
+9.6
Correctness Rate
68.2
78.4
+10.2
23.4
27.5
+4.1
24.0
31.8
+7.8
30.9
41.3
+10.4
39.6
50.9
+11.3
37.2
45.8
+8.6
Streak Score
68.2
78.0
+9.8
23.4
27.2
+3.8
24.0
31.4
+7.4
30.9
41.0
+10.1
39.6
50.5
+10.9
37.2
45.4
+8.2
Weighted Correctness
68.2
76.5
+8.3
23.4
26.0
+2.6
24.0
30.1
+6.1
30.9
39.6
+8.7
39.6
49.1
+9.5
37.2
44.0
+6.8
First-Error Position
68.2
79.0
+10.8
23.4
27.9
+4.5
24.0
32.2
+8.2
30.9
41.7
+10.8
39.6
51.2
+11.6
37.2
46.3
+9.1
Appendix
Table 8: Best-of-8 reranking under five aggregation strategies, InternVL2.5-26B policy.
MathVista
MathVision
MathVerse-VO
WeMath
LogicVista
Overall
Strategy
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
P@1
B@8
Δ8
Geometric Mean
71.9
83.5
+11.6
32.2
37.6
+5.4
36.9
47.7
+10.8
38.3
50.0
+11.7
47.9
58.7
+10.8
45.4
55.2
+9.8
Correctness Rate
71.9
82.3
+10.4
32.2
36.8
+4.6
36.9
46.8
+9.9
38.3
49.1
+10.8
47.9
57.8
+9.9
45.4
54.3
+8.9
Streak Score
71.9
81.9
+10.0
32.2
36.4
+4.2
36.9
46.5
+9.6
38.3
48.8
+10.5
47.9
57.5
+9.6
45.4
54.0
+8.6
Weighted Correctness
71.9
80.4
+8.5
32.2
35.0
+2.8
36.9
45.1
+8.2
38.3
47.4
+9.1
47.9
56.1
+8.2
45.4
52.5
+7.1
First-Error Position
71.9
83.0
+11.1
32.2
37.3
+5.1
36.9
47.3
+10.4
38.3
49.6
+11.3
47.9
58.3
+10.4
45.4
54.7
+9.3
Appendix
Table 9: Best-of-8 reranking under five aggregation strategies, InternVL2.5-38B policy.
Split
#Units
#Steps
#ShouldVis
#ModelVis
#Omit
OmissionRate
Completeness
Overall
TOTAL
415
1780
1305
1314
77
0.0590
0.9410
By policy file
InternVL2.5-8B
153
689
453
419
48
0.1060
0.8940
InternVL2.5-26B
159
666
507
518
25
0.0493
0.9507
InternVL2.5-38B
103
425
345
377
4
0.0116
0.9884
Appendix
Table 10: Checklist completeness audit. Under-reporting (omissions) and over-reporting (false positives) of visualdependency compared to an independent lenient visual-necessity annotation.
Type
TP
FP
FN
Prec
Rec
F1
LogicVista
numeric
19
6
3
0.7600
0.8636
0.8085
relation
23
11
12
0.6765
0.6571
0.6667
structure
40
4
4
0.9091
0.9091
0.9091
OVERALL
82
21
19
0.7961
0.8119
0.8039
MMMU
Appendix
Table 11: Human-annotated fidelity of extracted constraints. Precision/Recall/F1 of constraints predicted by Eϕ , evaluated on 600 sampled instances (534 valid), reported by constraint type and dataset.
Table 13: Complete ablation results on VisualProcessBench (Macro-F1, %). Δ = variant − Full Method. Best per group is highlighted by the best score in that group (see group-wise shading).
Recent advancements in reinforcement learning with verifiable rewards (RLVR) have significantly improved the complex reasoning ability of vision-language models (VLMs). However, its outcome-level supervision is too coarse to diagnose and correct errors within the reasoning chain. To this end, we propose Perceval, a process reward model (PRM) that enables token-level error grounding, which can extract image-related claims from the response and compare them one by one with the visual evidence in the image, ultimately returning claims that contain perceptual errors. Perceval is trained with perception-intensive supervised training data. We then integrate Perceval into the RL training process to train the policy models. Specifically, compared to traditional GRPO, which applies sequence-level advantages, we apply token-level advantages by targeting penalties on hallucinated spans identified by Perceval, thus enabling fine-grained supervision signals. In addition to augmenting the training process, Perceval can also assist VLMs during the inference stage. Using Perceval, we can truncate the erroneous portions of the model's response, and then either have the model regenerate the response directly or induce the model to reflect on its previous output. This process can be repeated multiple times to achieve test-time scaling. Experiments show significant improvements on benchmarks from various domains across multiple reasoning VLMs trained with RL, highlighting the promise of perception-centric supervision as a general-purpose strategy. For test-time scaling, it also demonstrates consistent performance gains over other strategies, such as major voting. Our code and data will be publicly released at https://github.com/RUCAIBox/Perceval.
Yingqian Min, Kun Zhou, Yifan Li +6
Gaoling School of Artificial Intelligence, Renmin University of China. · Bytedance. · University of California, San Diego. +1
Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.
Shulin Tian, Minglun Li, Yuhao Dong +6
S-Lab, Nanyang Technological University · A*STAR · Independent Researcher +1
Vision-language models (VLMs) perform strongly on visual question answering benchmarks, yet often make decisions that contradict visual evidence they have already identified correctly. We distinguish perceptual failure, where relevant evidence is not recognized, from process failure, where recognized evidence fails to constrain the final decision. We introduce VPAC-Bench, a benchmark spanning nine real-image process families, with each image annotated by its current activity stage and nearby stage transition. We also propose State-Relevance-Target (SRT), a family of structured process-prior interventions that requires models to connect visible evidence to the relevant process state before answering. Across multiple VLMs, process failure is widespread: models that correctly enumerate visual candidates still over-commit to a single answer in more than 95% of ambiguous cases. An explicit process-structured intervention reduces this rate to below 13% without degrading performance on unambiguous cases. However, the transfer of process priors is model-dependent, and generic SRT does not consistently outperform strong chain-of-thought baselines. When the relevant stage transition is known, boundary-aligned SRT substantially outperforms generic process prompting and all tested chain-of-thought baselines across assembly, physical state transition, navigation and traffic, and object-use affordance tasks. These results show that process priors are most useful when aligned with the scene's specific decision boundary, motivating boundary-aware prior selection for process-grounded visual reasoning.