Reinforcement learning with verifiable rewards (RLVR) has become a powerful tool for improving multimodal reasoning. However, standard outcome supervision suffers from credit assignment ambiguity, as a model may receive positive rewards by exploiting language priors or dataset shortcuts rather than grounding its reasoning in the requisite visual evidence. This makes high-quality negative responses critical, since effective negatives should remain plausible under the current policy while exposing the shortcut that should be suppressed. In this paper, we propose EVAN, a multimodal reinforcement learning framework utilizing EVidence-Ablated Negatives to train LMMs against their own language-prior failures. EVAN first identifies highly visually dependent examples using a scaled image information gain metric and preserves data diversity with a mixed-ranking strategy. It then constructs evidence-ablated images by masking the most discriminative image patches, eliciting plausible but visually ungrounded responses from the current policy. During policy optimization, these negative responses are incorporated as bounded auxiliary signals under the clean image, penalizing shortcut learning while maintaining RL stability. Experiments across challenging multimodal reasoning benchmarks demonstrate that EVAN mitigates modality imbalance and improves over strong RLVR baselines. Our code is available at https://github.com/DawnLIU35/EVAN.git.
Figures & tables
Model
Geometry3K
MathVista
MathVerse
We-Math
DynaMath
MathVision
MMK12
MMMU-Pro
LogicVista
Avg.
Qwen2.5-VL-3B-Instruct
PAPO-3B
36.9
63.3
60.5
62.4
57.2
26.2
62.9
28.4
43.2
49.0
Qwen2.5-VL-3B
19.4
46.3
33.5
29.9
40.4
18.2
36.3
19.8
32.1
30.7
+DAPO
38.8
62.4
60.4
60.5
59.3
26.5
72.2
29.9
43.0
50.3
+ EVAN (ours)
39.9
62.7
62.8
64.6
60.6
27.6
71.5
31.0
43.3
51.6
Qwen2.5-VL-7B-Instruct
Table 1: Comparison with state-of-the-art open-source reasoning LMMs on multimodal reasoning benchmarks. All methods are evaluated under the same evaluation configuration. The best performance is marked in bold .
Variant
Geometry3K
MathVista
MathVerse
We-Math
DynaMath
MathVision
MMK12
MMMU-Pro
LogicVista
Avg.
Random
38.5
62.4
62.5
65.1
60.1
27.1
72.2
30.0
42.0
51.1
Global
39.8
60.8
60.9
61.9
59.2
27.5
72.3
28.8
41.7
50.3
Category
38.4
63.5
63.5
64.8
59.6
26.8
72.7
29.3
42.9
51.3
RankMix ( EVAN )
39.9
62.7
62.8
64.6
60.6
27.6
71.5
31.0
43.3
51.6
Table 2: Ablation study of RankMix Selection with Qwen2.5-VL-3B. All variants use the same 20% data budget and differ only in the selection strategy.
Variant
Geometry3K
MathVista
MathVerse
We-Math
DynaMath
MathVision
MMK12
MMMU-Pro
LogicVista
Avg.
Clean Image
39.8
63.5
60.8
61.3
59.8
26.9
72.8
29.2
42.4
50.7
Random Patch
39.9
62.0
62.1
61.4
59.6
27.2
72.9
29.9
42.8
50.8
FGSM
39.6
61.8
60.6
64.1
59.8
26.5
73.2
29.6
42.3
50.8
Adv. mask ( EVAN )
39.9
62.7
62.8
64.6
60.6
27.6
71.5
31.0
43.3
51.6
Table 3: Ablation on evidence-ablated image construction with Qwen2.5-VL-3B. All variants use the same selected examples and differ only in the image used for evidence-ablated rollouts.
Variant
Geometry3K
MathVista
MathVerse
We-Math
DynaMath
MathVision
MMK12
MMMU-Pro
LogicVista
Avg.
EVAN ( ρ=25% )
39.6
59.9
61.5
64.9
61.0
26.8
72.7
31.1
42.8
51.1
EVAN ( ρ=50% )
39.9
62.7
62.8
64.6
60.6
27.6
71.5
31.0
43.3
51.6
EVAN ( ρ=75% )
39.4
61.6
62.2
63.7
60.2
27.9
73.0
30.7
43.5
51.4
EVAN ( ρ=100% )
39.5
63.2
63.1
65.0
59.1
26.5
71.3
29.4
41.0
50.9
Table 4: Effect of the masking ratio ρ on Qwen2.5-VL-3B. All variants use the same training recipe and differ only in the fraction of masked patches. ρ=100% masks the entire image.
Figure 1: (Left) Bounded negative advantage (BNA) ablation on Qwen2.5-VL-3B, with entropy on the left axis and gradient norm on the right axis. (Right) rollout filtering ablation on Qwen2.5-VL-7B, with PPO KL Divergence on the left axis and response length on the right axis.
Figure 2: Training dynamics of the EVidence-Ablated Negatives branch. (Left) Number of prompts with prepared evidence-ablated images in each training step. (Right) Number of retained evidence-ablated negative responses appended to the clean-image update after filtering.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Correct clean responses
1
2
3
4
5
6
7
Clean correct response
2.65
1.73
1.29
1.00
0.77
0.58
0.38
Clean incorrect response
−0.38
−0.58
−0.77
−1.00
−1.29
−1.73
−2.65
Evidence-ablated negative
−0.38
−0.58
−0.77
−1.00
−0.77
−0.58
−0.38
Appendix
Table 5: Advantages under binary rewards for a clean rollout group of G=8 responses. Each column corresponds to a number of correct responses in the clean rollout group, and each row gives the advantage assigned to one type of response.
Model
Geometry3K
MathVista
MathVerse
We-Math
DynaMath
MathVision
MMK12
MMMU-Pro
LogicVista
Avg.
Qwen2.5-VL-3B-Instruct
DAPO
38.8 ± 0.80
62.4 ± 0.52
60.4 ± 0.28
60.5 ± 0.47
59.3 ± 0.29
26.5 ± 0.26
72.2 ± 0.24
29.9 ± 0.46
43.0 ± 0.93
50.3 ± 0.25
EVAN
39.9 ± 0.66
62.7 ± 0.45
62.8 ± 0.50
64.6 ± 0.42
60.6 ± 0.17
27.6 ± 0.58
71.5 ± 0.40
31.0 ± 0.61
43.3 ± 1.14
51.6 ± 0.18
Qwen2.5-VL-7B-Instruct
DAPO
45.5 ± 0.99
66.2 ± 0.58
69.2 ± 0.46
68.4 ± 0.32
65.8 ± 0.12
29.0 ± 0.26
81.9 ± 0.24
37.0 ± 0.44
45.8 ± 1.19
56.5 ± 0.16
EVAN
46.0 ± 1.08
69.0 ± 0.46
69.1 ± 0.58
69.7 ± 0.33
66.5 ± 0.45
31.3 ± 0.35
82.2 ± 0.24
38.8 ± 0.50
48.8 ± 1.07
57.9 ± 0.23
Appendix
Table 6: 95% confidence intervals of DAPO and EVAN on Qwen2.5-VL-3B and Qwen2.5-VL-7B. Each interval is computed from the eight stochastic pass@1 evaluations of the same checkpoint.
Model
ChartQA
GQA
Visual CounterFact (CF+this)
Color
Size
Pooled
DAPO
81.74
54.46
84.61
79.57
81.61
EVAN
82.55
55.83
85.04
84.99
85.01
Appendix
Table 7: Transfer of the Qwen2.5-VL-3B DAPO and EVAN checkpoints beyond structured reasoning.
Method
Total training time (h)
Cumulative rollout time (h)
DAPO
36.05
≈ 22.0
EVAN
36.74
≈ 21.2
Appendix
Table 8: Training cost of the matched 300-step Qwen2.5-VL-7B runs on 4 × NVIDIA H200 GPUs, excluding the one-time preprocessing of EVAN .
Figure 3: Rollout filtering ratios during training. The fraction of candidates that pass the length filter (left) and the clean-image likelihood filter (right) for EVAN -3B and EVAN -7B.
Responses
Reviewer
Audited
Pathological
Degenerate text
Describes mask
Retained
GPT-5.5
250
0
0
0
Retained
Human
50
0
0
0
Removed by likelihood filter
GPT-5.5
100
40
30
10
Appendix
Table 9: Semantic audit of evidence-ablated responses from the EVAN run on Qwen2.5-VL-3B. Retained responses pass all filters and are used for training. Pathological responses contain degenerate text or explicitly describe the mask.
Variant
Geometry3K
MathVista
MathVerse
We-Math
DynaMath
MathVision
MMK12
MMMU-Pro
LogicVista
Avg.
EVAN ( τ=10% )
39.5
63.2
63.2
63.9
59.3
27.4
72.4
30.1
44.6
51.5
EVAN ( τ=20% )
39.9
62.7
62.8
64.6
60.6
27.6
71.5
31.0
43.3
51.6
EVAN ( τ=30% )
38.8
63.7
60.3
60.3
61.1
27.5
72.1
30.3
43.0
50.8
Appendix
Table 10: Effect of the RankMix selection ratio τ on Qwen2.5-VL-3B. All variants use the same training recipe and differ only in the fraction of selected visually dependent examples.
Variant
Scaled-IIG Before
Scaled-IIG After
Acc. Before (%)
Acc. After (%)
Adv. mask
0.595
0.455
88.3
66.3
FGSM
0.595
0.093
88.3
35.3
Appendix
Table 11: Scaled-IIG and model accuracy before and after image perturbation on the RankMix 20% selected data.
Model
AUC
Top 20%
Bottom 20%
Gap
Qwen2.5-VL-3B-Instruct
0.566
43.6
28.5
15.1
Qwen2.5-VL-7B-Instruct
0.583
63.5
41.0
22.6
Qwen3-VL-8B-Instruct
0.592
85.8
71.7
14.1
Qwen3-VL-8B-Thinking
0.600
86.6
70.5
16.2
Mean
0.585
69.9
52.9
17.0
Appendix
Table 12: Scaled-IIG and response correctness on 5,000 ViRL39K questions with five sampled responses per question. AUC measures how well the scaled-IIG of each response predicts its correctness. Top 20% and Bottom 20% report the accuracy (%) of the responses with the highest and lowest scaled-IIG, and Gap is their difference.
Selection
Non-Geo Math
Geometric
Tables/Charts
Science
Spatial
Broader STEM
Social Science
Commonsense
Total
Full dataset
14901
10631
6189
3475
1922
956
676
120
38870
RankMix 10%
1122
382
1349
428
298
164
121
23
3887
RankMix 20%
2471
1131
2235
857
521
304
214
41
7774
RankMix 30%
3860
2092
2974
1241
720
423
295
56
11661
Global Top-20%
1244
306
4125
720
529
463
307
80
7774
Appendix
Table 13: Category distribution of selected training examples.
Figure 4: Case study on a train-schedule question. Left: clean image. Right: evidence-ablated image after adversarial masking.
Figure 5: Case study on a counting question. Top: clean image. Bottom: evidence-ablated image after adversarial masking.
Figure 6: Case study on a table question. Left: clean image. Right: evidence-ablated image after adversarial masking.
Figure 7: Case study on a genetics question. Top: clean image. Bottom: evidence-ablated image after adversarial masking.
Reinforcement learning with verifiable rewards (RLVR) improves vision-language models (VLMs) by optimizing outcome rewards derived from final answers. However, such outcome-only rewards do not tell the model which image regions justify an answer. For questions that require visual grounding, these rewards cannot distinguish responses supported by relevant visual evidence from those produced by language-prior shortcuts or lucky guesses. We introduce EASE (Evidence-Anchored Spatial Attention), which augments multimodal RLVR with visual-evidence process supervision. EASE converts annotated evidence regions into a smoothed visual-token target and uses it to guide response-to-image attention during RL training, but only on high-reward trajectories. The annotations are used solely as privileged training labels, while inference requires only the original image and question. Across Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3-VL-8B, EASE raises average scores over DAPO by 2.5 to 3.1 points on perception, hallucination, visual math, and multimodal reasoning benchmarks. Diagnostics and ablations show that EASE better aligns visual attention with annotated evidence regions.
Ruina Hu, Chen Wang, Lai Wei +5
1Harbin Institute of Technology · 2Zhongguancun Academy · 4Nankai University +4
Reinforcement learning with verifiable rewards (RLVR) has substantially enhanced the reasoning capabilities of multimodal large language models (MLLMs). However, existing RLVR approaches typically rely on outcome-driven optimization that updates both perception and reasoning using a shared reward based solely on the final answer. This shared reward blurs credit assignment, frequently improving reasoning patterns while failing to reliably enhance the accuracy of upstream visual evidence extraction. To address this perception bottleneck, we introduce PRCO (Perception-Reasoning Coevolution), a dual-role RLVR framework with a shared policy. PRCO consists of two cooperative roles: an Observer that generates an evidence caption tailored to the question and a Solver that predicts the final answer based on this caption. Crucially, PRCO employs role-specific reward signals: the Solver is optimized using verifiable outcome rewards on the final answer, while the Observer receives a utility reward derived from the Solver's downstream success. Extensive experiments across eight challenging multimodal reasoning benchmarks demonstrate that PRCO yields consistent improvements across model scales by over 7 points on average accuracy compared to the base model, outperforming prior open-source RL-tuned baselines.
Reinforcement Learning with Verifiable Rewards (RLVR) improves multimodal reasoning by rewarding verifiable final answers. Yet answer-correct trajectories may still rely on incomplete derivations, weak evidence, or statements that contradict their conclusions. This gap between answer correctness and reasoning validity, which we call reasoning-answer inconsistency, motivates trajectory supervision in multimodal RL. We compare two main approaches: reward models (RMs), and Generative Rewards (GRs). RMs are efficient and help early in training, but their gains weaken as the policy distribution shifts; GRs improve performance, but may give unstable rewards and computationally expensive. We therefore propose Groupwise Ranking Reward, which ranks verifier-passed trajectories for the same prompt in one pass and redistributes reward accordingly. Groupwise comparison better separates stronger and weaker correct trajectories with lower judge overhead than GRs. Experiments show that RLVR aggravates reasoning-answer inconsistency, while trajectory supervision alleviates it. Groupwise Ranking Reward performs best overall, improving reliability-conditioned accuracy from 47.4% to 54.7% over RLVR.