Reinforcement learning with verifiable rewards (RLVR) improves vision-language benchmark scores even without visual information during training. With images at test, blind-trained models recover roughly half of the real-image gain at 3B and nearly four fifths at 7B. Prolonged real-image training can erode grounding while benchmark gains persist. Both findings expose the same gap: an image in the prompt is not an image in the learning signal. Our design rule, visual resolvability, asks that visual evidence be necessary for a correct answer and that the task remain learnable. We test it on counterfactual coordinate scenes in which the question stays fixed and the target is never named, so a correct answer requires finding the target in the image. With standard GRPO and correctness-and-format rewards, a 7B model raises its accuracy at finding the target (discovery) from 0.425 to 0.875 on held-out scenes denser than any it trained on, and it improves on question types it never trained on. Two controls locate the source of the gain. Replacing test images with gray canvases drops discovery to zero; training on gray canvases instead, at matched step 30 and in each of four seeds, yields essentially none of the gain even when the model is then tested with real images. The learned skill carries over to grounding tasks built independently of the training corpus. A caption that answers the training question, added to the same images, reward and budget, cuts the gain by nearly two thirds. Changing what reward requires changes what RL learns.
Figures & tables
Figure 1: Same learning rule, different lessons. Left: shortcut-accessible tasks allow reward without visual evidence; gains in the reported 3B comparisons mainly favor cued readout. Right: constructed tasks demand visual evidence and relation-based target identification; discovery, cued readout, and named-target grounding improve. The hierarchy distinguishes supplied locations, supplied identities, and relation-defined targets. GRPO is shared across the study; rewards stay fixed within matched comparisons.
Evaluated with the real image
Test image removed
Trained with
gain over base
share of Real ’s gain
gray canvas
no image
base model (accuracy)
0.175
—
0.090
0.068
Real
+ 0.238
1 (reference)
+ 0.019
+ 0.036
Caption
+ 0.171
0.72
+ 0.024
+ 0.037
trained without visual information
None
+ 0.137
0.57
+ 0.015
+ 0.044
Table 1: Blind gains. Training without visual information recovers more than half of Real ’s benchmark gain when real images are supplied at test; with the test image removed, no condition gains more than 0.05. Base row: accuracy; other cells: gain over that base under the same evaluation; shares: fraction of Real ’s gain (three-seed pooled). All cells use the same answer matcher.
Figure 2: Benchmark gains are not visual acquisition. (a) Gains by operation: readout and grounding for Geometry3K-trained 3B models (three-seed means), discovery for ViRL39K-trained models (two-seed means; 7B: Geometry3K-trained pair). (b) Benchmark accuracy and grounding pair accuracy under prolonged Real training with a trainable encoder. (c) Grounding pairs lost by three Gray -trained runs, and their wrong answers. All evaluations use real images; error bars are 95% intervals and shading marks the grounding equivalence band.
Figure 3: Measure and shape. (a) Audited visual benefit against initial success, constructed corpus and ViRL39K. (b) The K8 scene and its counterfactual twin. (c) Its target-cue hierarchy.
Figure 4: The regime flips. (a) Discovery member accuracy on the development set (inset: confirmatory 20-point result). (b) Step-30 discovery gains under Real and Gray training, evaluated with real images. (c) Step-100 change in pair accuracy. Error bars: 95% intervals over scene programs in (a, b) and over pairs in (c).
Figure 5: Composition changes the gain; model size changes the trajectory. (a) Step-30 discovery gain of each training mixture against the constructed fraction f . (b) Discovery accuracy of 3B and 7B models trained on the constructed corpus. Error bars: 95% intervals over scene programs.
Appendix figures & tables33 assets
Supplementary material from the paper’s appendix.
Appendix
benchmark (format)
n
with image
blind
chance
naive retention
corrected retention
BLINK (MC, pooled)
1901
0.493
0.409
0.377
0.829 [0.781, 0.879]
0.271 [0.089, 0.454]
MathVerse (MC, pooled)
2180
0.465
0.394
0.260
0.848 [0.807, 0.889]
0.655 [0.568, 0.743]
MathVerse (free-form)
1755
0.055
0.019
0.000
0.340 [0.240, 0.458]
0.340 [0.240, 0.458]
MMVP (MC, k=2 )
300
0.660
0.500
0.500
0.758 [0.668, 0.854]
0.000 [ − 0.417, 0.342]
MMMU dev+val (MC, pooled)
988
0.506
0.413
0.263
0.816 [0.764, 0.873]
0.617 [0.512, 0.729]
MMMU dev+val (free-form)
62
0.097
0.048
0.000
0.500 [0.000, 1.000]
0.500 [0.000, 1.000]
Appendix
Table A.1: Visual-necessity audit of the untrained Qwen2.5-VL-3B model on seven benchmarks: accuracy with the image and with the image removed (blind), the chance level (mean of the item-specific 1/k for multiple choice; 0 for free-form answers), naive retention (blind ÷ with-image) and chance-corrected retention (blind−chance)/(with-image−chance) , with 95% item-paired bootstrap intervals (10,000 draws). Mixed-format benchmarks are reported by format; HallusionBench is reported under both plausible chance levels.
model
pool
n
with image
blind
chance
naive retention
corrected retention
Gemma-3
ViRL39K free-form
2789
0.430
0.312
0.000
0.727 [0.690, 0.765]
0.727 [0.690, 0.765]
Gemma-3
ViRL39K multiple choice
1215
0.135
0.086
0.268
0.634 [0.533, 0.744]
undefined
InternVL3-9B
ViRL39K free-form
2789
0.269
0.130
0.000
0.485 [0.439, 0.533]
0.485 [0.439, 0.533]
InternVL3-9B
ViRL39K multiple choice
1215
0.294
0.205
0.268
0.697 [0.619, 0.782]
− 2.439 [ − 17.961, 6.505]
Appendix
Table A.2: The same audit for two other model families on a 4,096-item ViRL39K sample (the image is removed rather than replaced): with-image and blind accuracy, the chance level, and naive and chance-corrected retention with 95% item-paired bootstrap intervals. Corrected retention is defined only when with-image accuracy exceeds the chance level (§ 2.2 ). On the multiple-choice pool Gemma-3 scores below chance with the image, so its ratio is undefined; InternVL3-9B exceeds chance by only 0.026, so its ratio is defined but unstable and is reported unclipped. Without the image, both models score 0.000 pair accuracy on the grounding task and its twin (§ 2.2 ).
level
question
visual cue
gold (twin a)
gold (twin b)
L1 (arrow cue; position given)
Point K8 has the smallest y -coordinate. What is the x -coordinate of point K8?
offset arrow at K8
−1
6
L2 (target named; identity given)
Point K8 has the smallest y -coordinate. What is the x -coordinate of point K8?
none
−1
6
L3 (discover + ground + read; nothing given)
Consider the point with the smallest y -coordinate. What is its x -coordinate?
none
−1
6
probe (identification only)
Which labeled point has the smallest y -coordinate?
none
K8
K8
Appendix
Table B.1: The three levels and the identification probe on one development scene (eight labeled points; the K8 scene of Figure 3 (b)): the question templates, the cue each level supplies, and the gold answers on the two twins, which differ in one fact (K8 moves from (−1,−1) to (6,−1) ). L2 and the probe use the L3 image; L1 adds only the offset arrow.
model
discovery 8-pt
12-pt
20-pt
probe 8-pt
12-pt
20-pt
3B, ViRL39K-trained (step 100); two seeds
untrained 3B
0.330
0.260
0.245
0.705
0.680
0.630
Real , two-seed mean
0.383
0.338
0.330
0.728
0.700
0.610
seeds 1 / 2
0.380 / 0.385
0.330 / 0.345
0.330 / 0.330
0.725 / 0.730
0.710 / 0.690
0.620 / 0.600
Caption , two-seed mean
0.370
0.323
0.340
0.728
0.715
0.635
seeds 1 / 2
0.365 / 0.375
0.325 / 0.320
0.345 / 0.335
0.715 / 0.740
0.710 / 0.720
0.610 / 0.660
Appendix
Table B.2: Discovery on the coordinate development set (member accuracy with real images) for the models trained with ordinary rewards: the ViRL39K-trained 3B conditions (two seeds; step 100) and the Geometry3K-trained 7B pair (step 100), with the untrained models. Discovery (L3) and the identification probe at 8, 12, and 20 points. Two-seed means are computed from the per-seed counts; intervals are not reported for these values.
condition
cued readout: gain [95% CI]
find-and-bind
header-cued table
untrained 3B (pair accuracy)
0.320
0.455
0.867
Real
+ 0.143 [0.102, 0.188]
+ 0.018 [ − 0.007, 0.044]
+ 0.019 [ − 0.002, 0.042]
seeds 1 / 2 / 3
+ 0.157 / + 0.127 / + 0.147
+ 0.012 / + 0.022 / + 0.022
+ 0.030 / + 0.013 / + 0.013
Caption
+ 0.107 [0.066, 0.149]
+ 0.006 [ − 0.018, 0.030]
+ 0.021 [ − 0.001, 0.044]
seeds 1 / 2 / 3
+ 0.130 / + 0.113 / + 0.077
+ 0.003 / 0.000 / + 0.013
+ 0.023 / + 0.027 / + 0.013
None
+ 0.108 [0.070, 0.148]
− 0.013 [ − 0.039, 0.013]
+ 0.023 [0.000, 0.049]
Appendix
Table B.3: Task-layer gains of the Geometry3K-trained 3B conditions at step 100 (pair accuracy; three-seed means with 95% paired-item intervals; per-seed gains beneath). The equivalence test for the grounding task is in Appendix I .
(4) caption-only accuracy
set
density
discovery (L3)
accuracy
limit ( 21 L3)
outcome
(5) artifact screen
criteria met
development
8-pt
0.660
0.580
0.330
not met
met
4 of 5
12-pt
0.575
0.445
0.288
not met
not met: 0.557 in 1 of 3 seeds
3 of 5
20-pt
0.470
0.185
0.235
met
not met: 0.565 in 1 of 3 seeds
4 of 5
confirmatory
8-pt
0.585
0.570
0.293
not met
met
4 of 5
12-pt
0.560
0.410
0.280
not met
met
4 of 5
Appendix
Table B.4: Outcome of the five evaluation criteria of § 5.3 on the untrained 7B model, for every density of the development set (used to track learning) and of the separate confirmatory set: (1) discovery accuracy between 0.20 and 0.80; (2) discovery at least 0.05 below both cued levels; (3) exactly zero accuracy on discovery and its probe with a gray canvas and with no image; (4) caption-only accuracy at most half of with-image discovery accuracy; (5) the artifact screen of Appendix B : out-of-fold max(AUC,1−AUC) of four classifiers at most 0.55, and the upper end of the unfolded interval at most 0.62, at each of three cross-validation seeds. Criteria 1–3 are met in every cell. Only the 20-point confirmatory cell meets all five; it supplies the confirmatory result.
task
pairs
role
untrained 3B
untrained 7B [95% CI]
grounding task (named-point coordinate pairs)
600
find-and-bind (primary)
0.455
0.768 [0.733, 0.802]
header-cued table task
300
cued control, high baseline
0.867
0.993 [0.983, 1.000]
cued-readout task (marked-plot pairs)
300
cued readout, location marked
0.320
0.673 [0.620, 0.727]
twin of the grounding task
600
regenerated twin
–
0.728 [0.692, 0.762]
twin of the cued-readout task
300
twin control
–
0.623 [0.570, 0.677]
twin of the header-cued table task
300
twin control
–
0.997 [0.990, 1.000]
Appendix
Table C.1: The independently constructed grounding tasks: the 1,200-pair counterfactual suite (three tasks with distinct roles, reported separately) and its independently regenerated twin (fresh seeds, no iteration). Untrained pair accuracy (both members correct) for the 3B and 7B models; 7B levels with 95% intervals. Validation: Qwen2.5-VL-72B-Instruct given question-blind captions reaches at most 0.054; every image-removed condition scores exactly 0.000, with all responses collapsing to one answer; 60 of 60 audited pairs were confirmed by human review; Gemma-3 and InternVL3-9B also score exactly 0.000 without the image.
experiment
backbone, vision encoder
training images
reward
training data
steps; evaluated at
seeds
runs
Geometry3K access conditions (§ 2.1 , § 3 )
3B, frozen
Real / Gray / None / Caption , one condition per run
answer correctness
Geometry3K, filtered (1,288 items)
100
3
3
7B access pair (§ 2.1 , § 3 )
7B, frozen
Real ; Gray
answer correctness
Geometry3K
100
1
1
long-horizon runs (§ 3 )
3B, trainable
real images
0.5⋅ accuracy +0.5⋅ format
Geometry3K, unfiltered
400, in segments; evaluated at 100, 150, 200, 300, 400
2
2
resolvable training, standard reward (§ 6 )
7B, frozen
real images
0.5⋅ accuracy +0.5⋅ format
constructed corpus, 2,880 items (discovery and probe, both twins)
seeds 0 and 1: 100, in segments (seed 0 evaluated at 10, 20, 30, 50, 75, 100; seed 1 at 30 and 100). Seeds 2 and 3: 30
4
5 (two of seed 0)
Gray -trained control (§ 6.2 )
7B, frozen
gray canvases
as above
same corpus
30; evaluated at 10, 20, 30 (seed 0) or 30 (seeds 1–3)
4
4
named-target ablation (§ 6.1 ; Table G.6 )
7B, frozen
real images
as above
same scene programs, with the find-and-bind (L2) question in place of the discovery question; probe unchanged
30
3
3
Appendix
Table D.1: Training recipes. Every run uses GRPO with 5 rollouts per prompt, learning rate 10−6 and KL coefficient 0.01. In every recipe the verifier scores each response’s extracted answer against its gold answer and, where noted, its format; no image-dependent auxiliary loss or pairwise training reward is added. Constructed-corpus runs (§§ 6 – 7 ): rollout batch 240, global batch 48. Geometry3K, ViRL39K and long-horizon runs: rollout batch 512, global batch 128 (the long-horizon runs on 4 GPUs with tensor parallelism 2). The last two columns count, per condition, the independent seeds and the training runs: an independent seed draws its own shuffled data order and rollout sampling; a run beyond the seed count repeats one configuration. The dose response of § 7.1 has four points, the three mixtures listed here and the standard run at step 30 ( f=1 ).
condition
seed
acc. real
acc. gray
acc. none
acc. caption
gain real
gain gray
gain none
gain caption
share
base model
–
0.175
0.090
0.068
0.210
–
–
–
–
–
Real
1
0.423
0.106
0.108
0.311
+ 0.248
+ 0.017
+ 0.040
+ 0.102
–
2
0.419
0.112
0.098
0.301
+ 0.245
+ 0.022
+ 0.030
+ 0.091
–
3
0.398
0.110
0.106
0.326
+ 0.223
+ 0.020
+ 0.038
+ 0.117
–
Caption
1
0.361
0.118
0.112
0.318
+ 0.186
+ 0.028
+ 0.043
+ 0.108
0.752
2
0.346
0.108
0.106
0.291
+ 0.171
+ 0.018
+ 0.038
+ 0.082
0.701
Appendix
Table D.2: Table 1 per seed, with the untrained and trained models scored by the same answer matcher: accuracy of each condition’s step-100 model under the four test conditions (601 Geometry3K test items), the gain over the untrained model under the same test condition, and the recovery share (real-tested gain ÷ Real ’s real-tested gain). Intervals are not computed for the three-seed means and shares; the per-seed values show their spread.
task
pairs
untrained 7B
Real Δ [95% CI]
Gray Δ [95% CI]
grounding task
600
0.768
+ 0.035 [0.013, 0.058]
+ 0.012 [ − 0.010, 0.033]
twin of the grounding task
600
0.728
+ 0.038 [0.017, 0.062]
+ 0.013 [ − 0.010, 0.037]
cued-readout task
300
0.673
+ 0.030 [ − 0.007, 0.067]
+ 0.017 [ − 0.013, 0.047]
twin of the cued-readout task
300
0.623
+ 0.013 [ − 0.030, 0.060]
+ 0.013 [ − 0.023, 0.050]
header-cued table task
300
0.993
0.000 [ − 0.010, 0.010]
− 0.003 [ − 0.010, 0.000]
twin of the header-cued table task
300
0.997
0.000 [ − 0.010, 0.010]
− 0.007 [ − 0.017, 0.000]
Appendix
Table D.3: The ordinary 7B access pair of § 3 on the independently constructed grounding suite and its twin (Geometry3K training, step 100): untrained pair accuracy and the change under Real and Gray training, with 95% paired-item bootstrap intervals (2,000 draws). The interval excludes zero for Real on the grounding task and on its twin, and includes zero in every other cell.
trained with
step
real image: correct [95% CI]
gain over untrained [95% CI]
gray canvas: correct [95% CI]
untrained 7B
–
141/601 = 0.235 [0.201, 0.270]
–
48/601 = 0.080 [0.058, 0.103]
Real
100
290/601 = 0.483 [0.443, 0.521]
+ 0.248 [0.203, 0.291] (149/601)
75/601 = 0.125 [0.100, 0.151]
Gray
100
257/601 = 0.428 [0.388, 0.466]
+ 0.193 [0.153, 0.235] (116/601)
79/601 = 0.131 [0.103, 0.160]
Appendix
Table D.4: The 7B access pair behind the share in Table 1 (Geometry3K training; 601 Geometry3K test items; the answer matcher of Table 1 ): correct answers with real test images and with a gray canvas, with 95% item-bootstrap intervals, and the gain over the untrained model with real images, with its item-paired interval (5,000 draws). The recovery share is Gray ’s gain divided by Real ’s: 116/149 = 0.779 [0.642, 0.921].
questions
n
untrained
Real gain
Caption gain
None gain
Gray gain
all questions
601
0.175
+ 0.238 [0.202, 0.275]
+ 0.171 [0.136, 0.207]
+ 0.137 [0.103, 0.171]
+ 0.125 [0.092, 0.158]
extractable
497
0.211
+ 0.200 [0.160, 0.240]
+ 0.140 [0.100, 0.179]
+ 0.107 [0.069, 0.145]
+ 0.101 [0.064, 0.139]
not extractable
104
0.000
+ 0.423 [0.343, 0.503]
+ 0.324 [0.250, 0.401]
+ 0.279 [0.212, 0.349]
+ 0.237 [0.176, 0.301]
Appendix
Table D.5: The blind gains of Table 1 by whether the untrained 3B model’s real-image answer can be extracted (extractable: 497 of 601 questions): the untrained model’s accuracy, the gain over it with real test images, averaged over the three seeds per question, and each condition’s share of Real ’s gain, with 95% intervals from a 10,000-draw bootstrap over questions. The three runs of a condition answer the same 601 questions, so questions, not the 1,803 responses, are resampled.
grounding loss under prolonged training (§ 3 ; Table E.1 )
2
2
Table E.1 , per run
grounding pairs
discovery on the confirmatory set (§ 6.1 ; Tables G.2 , G.6 )
4
5
Table G.2 (seed 0, per run); Table G.6 (seeds 1–3; seed 1 also at step 100)
scene programs (seed 0); seeds, t -based (Table G.7 )
Appendix
Table D.6: Run basis of each reported result. Seeds counts the independent seeds of a condition and runs its training runs (Table D.1 ); where runs exceed seeds, the further run repeats one configuration. The fourth column names the table that prints the individual seeds or runs, and the last the unit that each result’s 95% intervals resample with the trained models held fixed.
condition
seed
step 1: overall / format / accuracy
step 30: overall / format / accuracy
mean accuracy, steps 19–30
Real
1
0.5975 / 0.8658 / 0.3292
0.9858 / 1.0000 / 0.9717
0.975
Real
2
0.5900 / 0.8875 / 0.2925
0.9863 / 1.0000 / 0.9725
0.982
Real
3
0.6095 / 0.8825 / 0.3365
0.9930 / 0.9992 / 0.9867
0.970
Gray -trained
1
0.4350 / 0.8683 / 0.0017
0.5175 / 1.0000 / 0.0350
0.033
Gray -trained
2
0.4413 / 0.8800 / 0.0025
0.5238 / 0.9992 / 0.0483
0.041
Gray -trained
3
0.4425 / 0.8800 / 0.0050
0.5104 / 1.0000 / 0.0208
0.024
Appendix
Table D.7: Training rewards of seeds 1–3 (Table G.6 ): the overall reward of Eq. ( 2 ) with its format and accuracy components at steps 1 and 30, and the mean accuracy component over steps 19–30. The Gray -trained control earns format reward from the start and almost no accuracy reward at any point; the Real , named-target and shortcut-accessible runs approach the ceiling of both components by step 30.
seed
step
benchmark [95% CI]
grounding [95% CI]
Δ vs base [95% CI]
1
0
0.175 [0.145, 0.206]
0.455 [0.415, 0.495]
–
100
0.431 [0.393, 0.471]
0.477 [0.437, 0.517]
+ 0.022 [ − 0.008, 0.052]
150
0.463 [0.424, 0.504]
0.465 [0.425, 0.505]
+ 0.010 [ − 0.020, 0.040]
200
0.483 [0.444, 0.524] (peak)
0.450 [0.412, 0.490]
− 0.005 [ − 0.035, 0.025]
300
0.464 [0.424, 0.506]
0.440 [0.400, 0.480]
− 0.015 [ − 0.047, 0.017]
400
0.436 [0.398, 0.474]
0.407 [0.367, 0.447]
− 0.048 [ − 0.082, − 0.017]
Appendix
Table E.1: The two long-horizon runs of § 3 (Qwen2.5-VL-3B, trainable encoder, real images, unfiltered Geometry3K): benchmark accuracy (correct answers out of 601) with 95% item-bootstrap intervals, and grounding-task pair accuracy with its interval and its change from the untrained model (the 600 pairs, each its own scene, resampled with the same 10,000 draws at every checkpoint). Peaks are marked.
run
degraded pairs
pairs gained
wrong members
seed 1
51
24
52
seed 2
49
22
53
seed 3
45
23
46
Appendix
Table E.2: Shared errors across the three Gray -trained runs of § 3 (grounding task, 600 pairs). The sets are defined with the earlier answer-matcher revision (Appendix I ). Top: per run, the degraded set (pairs the untrained model solves and the run fails), pairs gained, and wrong members (individual pair members answered incorrectly within the degraded set). Bottom: overlap of the degraded sets (Jaccard; intersection/union) against a 10,000-draw permutation null of sets of the same sizes ( p=10−4 for every comparison), and the rate of identical wrong answers on the wrong members the runs share.
run
step
8-pt: level [95% CI]; Δ [95% CI]
12-pt
20-pt
probe 8 / 12 / 20-pt
untrained 7B
0
0.660 [0.580, 0.740]
0.575 [0.485, 0.660]
0.470 [0.385, 0.555]
0.940 / 0.910 / 0.840
run 1
30
0.965 [0.935, 0.990] Δ+ 0.305 [0.230, 0.380]
0.890 [0.830, 0.945] Δ+ 0.315 [0.230, 0.400]
0.800 [0.730, 0.865] Δ+ 0.330 [0.255, 0.410]
1.000 / 0.975 / 0.975
100
200/200 (100 programs) Δ+ 0.340 [0.260, 0.420]
0.940 [0.895, 0.980] Δ+ 0.365 [0.280, 0.455]
0.910 [0.860, 0.950] Δ+ 0.440 [0.360, 0.525]
1.000 / 0.990 / 0.995
run 2
30
0.960 [0.925, 0.985] Δ+ 0.300 [0.225, 0.380]
0.860 [0.795, 0.920] Δ+ 0.285 [0.205, 0.370]
0.795 [0.730, 0.860] Δ+ 0.325 [0.245, 0.405]
1.000 / 0.960 / 0.960
100
0.995 [0.985, 1.000] Δ+ 0.335 [0.255, 0.415]
0.950 [0.915, 0.980] Δ+ 0.375 [0.290, 0.465]
0.905 [0.860, 0.945] Δ+ 0.435 [0.350, 0.520]
0.995 / 0.985 / 0.990
Appendix
Table G.1: Development-set acquisition (Figure 4 (a)): discovery (L3) member accuracy with real images at steps 0, 30 and 100 for both runs, by density, and its change from the untrained model, with 95% intervals from resampling scene programs (Appendix I ); last column, the identification probe with real images at 8 / 12 / 20 points. With the test image replaced by a gray canvas, discovery and the probe score 0 of 200 at every one of the six checkpoints of both runs.
real images
gray canvas
run
step
density
level [95% CI]
Δ vs untrained [95% CI]
level [95% CI]
criteria met
untrained 7B
0
8-pt
0.585 [0.500, 0.665]
–
0/200 (100 programs)
4 of 5
untrained 7B
0
12-pt
0.560 [0.470, 0.645]
–
0/200 (100 programs)
4 of 5
untrained 7B
0
20-pt
0.425 [0.335, 0.515]
–
0/200 (100 programs)
5 of 5
Real , run 1
30
8-pt
0.935 [0.895, 0.970]
+ 0.350 [0.275, 0.430]
0/200 (100 programs)
4 of 5
Real , run 1
30
12-pt
0.860 [0.805, 0.910]
+ 0.300 [0.225, 0.380]
0/200 (100 programs)
4 of 5
Appendix
Table G.2: All confirmatory-set cells (150 new scene programs per density, generated after the evaluation criteria of § 5.3 were set): discovery member accuracy with real images and its change from the untrained model, and accuracy with the test image replaced by a gray canvas, with 95% intervals from resampling scene programs; a cell with 0 of 200 correct is given as its count and number of scene programs. The last column gives how many of the five criteria the evaluation cell met before training; it describes the cell, not a run’s outcome. The 20-point cell met all five and supplies the confirmatory result of § 6.1 ; the 8- and 12-point cells did not meet the caption criterion (Appendix B ).
gray-canvas count
set
density
task
Real Δ [95% CI]
control Δ [95% CI]
τvis [95% CI]
Real
control
dev
8-pt
L3
+ 0.305 [0.230, 0.380]
+ 0.030 [ − 0.005, 0.065]
+ 0.275 [0.205, 0.350]
0/200
11/200
dev
8-pt
probe
+ 0.060 [0.025, 0.105]
+ 0.005 [ − 0.020, 0.035]
+ 0.055 [0.020, 0.095]
0/200
0/200
dev
12-pt
L3
+ 0.315 [0.230, 0.400]
− 0.035 [ − 0.080, 0.005]
+ 0.350 [0.265, 0.435]
0/200
9/200
dev
12-pt
probe
+ 0.065 [0.025, 0.110]
− 0.005 [ − 0.035, 0.020]
+ 0.070 [0.030, 0.120]
0/200
0/200
dev
20-pt
L3
+ 0.330 [0.255, 0.410]
− 0.030 [ − 0.070, 0.010]
+ 0.360 [0.275, 0.445]
0/200
8/200
Appendix
Table G.3: Attribution at matched step 30: the Real gain and the Gray -trained control’s change (both evaluated with real images) and their difference τvis (Eq. ( F.1 )), on the development set (dev) and replicated on the confirmatory set (conf); task: discovery (L3) or the identification probe, with each model’s gray-canvas count at the same checkpoint (correct/total). 95% intervals from resampling scene programs, with the same draws for both models.
run
step
grounding task (600 pairs)
twin (600)
cued-readout task (300)
header-cued table (300)
untrained 7B
0
0.768 [0.733, 0.802]
0.728 [0.692, 0.762]
0.673 [0.620, 0.727]
0.993 [0.983, 1.000]
run 1
30
0.930 [0.908, 0.950] Δ+ 0.162 [0.130, 0.193]
0.900 Δ+ 0.172
0.680 [0.627, 0.733] Δ+ 0.007 [ − 0.023, 0.037]
0.997 [0.990, 1.000] Δ+ 0.003 [0.000, 0.010]
100
0.948 [0.930, 0.965] Δ+ 0.180 [0.147, 0.215]
0.937 [0.917, 0.955] Δ+ 0.208 [0.175, 0.243]
0.707 [0.657, 0.757] Δ+ 0.033 [0.000, 0.070]
0.997 [0.990, 1.000] Δ+ 0.003 [0.000, 0.010]
run 2
30
0.935 [0.915, 0.955] Δ+ 0.167 [0.137, 0.198]
0.905 Δ+ 0.177
0.690 [0.640, 0.743] Δ+ 0.017 [ − 0.013, 0.047]
0.997 [0.990, 1.000] Δ+ 0.003 [0.000, 0.010]
100
0.962 [0.947, 0.977] Δ+ 0.193 [0.162, 0.227]
–
0.727 [0.677, 0.777] Δ+ 0.053 [0.020, 0.087]
0.997 [0.990, 1.000] Δ+ 0.003 [0.000, 0.010]
Appendix
Table G.4: Transfer to the independently constructed grounding tasks (Figure 4 (c)) at steps 0, 30 and 100 for both runs: pair accuracy with 95% intervals and, beneath, the paired change from the untrained model (5,000-draw item bootstrap). Changes are computed from unrounded accuracies, so they can differ by 0.001 from the difference of the rounded levels (as do the labels of Figure 4 (c)). Step-30 twin levels are from the evaluation of Table G.6 ; their changes are differences of rounded levels and carry no interval.
task
untrained
Real run 1, 30
Real run 2, 30
Gray -trained control, 30
Real run 1, 100
L1 cued readout
0.610 [0.525, 0.695]
0.820 [0.755, 0.880] Δ+ 0.210 [0.140, 0.285]
0.810 [0.745, 0.870] Δ+ 0.200 [0.130, 0.270]
0.640 [0.555, 0.720] Δ+ 0.030 [ − 0.010, 0.075]
0.890 [0.835, 0.935] Δ+ 0.280 [0.200, 0.360]
L2 find-and-bind
0.620 [0.530, 0.705]
0.900 [0.850, 0.945] Δ+ 0.280 [0.200, 0.360]
0.875 [0.815, 0.930] Δ+ 0.255 [0.180, 0.330]
0.680 [0.600, 0.760] Δ+ 0.060 [0.020, 0.105]
0.950 [0.915, 0.980] Δ+ 0.330 [0.250, 0.415]
L3 discovery
0.425 [0.335, 0.515]
0.740 [0.665, 0.815] Δ+ 0.315 [0.235, 0.400]
0.720 [0.645, 0.795] Δ+ 0.295 [0.215, 0.375]
0.385 [0.300, 0.475] Δ− 0.040 [ − 0.085, 0.000]
0.875 [0.815, 0.930] Δ+ 0.450 [0.365, 0.540]
identification probe
0.825 [0.755, 0.890]
0.975 [0.940, 1.000] Δ+ 0.150 [0.090, 0.215]
0.975 [0.940, 1.000] Δ+ 0.150 [0.090, 0.215]
0.850 [0.785, 0.905] Δ+ 0.025 [0.000, 0.050]
0.985 [0.960, 1.000] Δ+ 0.160 [0.100, 0.225]
Appendix
Table G.5: The same-scene hierarchy on the 20-point confirmatory scenes, a density never trained (100 scene programs in the top block): member accuracy with real images with its 95% interval over scene programs, its change from the untrained model, and, for pairs whose answer changes between the twins, the number of scene programs with both members correct. Top: the four tasks on the target-stable and invariance pairs, the population of the discovery result (200 members). Bottom: pair roles separately (100 members each); the cued levels have no target-switch pairs. L1 and L2 ask the same question, and L1 adds only the arrow; L1, L2 and the 20-point density were never trained, while the probe was.
seed
condition
discovery, confirmatory
discovery, untouched
grounding task
twin
target-switch, conf.
target-switch, untouched
find-and-bind
–
untrained 7B
0.425 (85/200)
0.471 (628/1,332)
0.768
0.728
13/50
0.318
0.620
0
Real , run 1
0.740
0.791
0.930
0.900
30/50
0.586
0.900
0
Real , run 2 (same seed)
0.720
0.770
0.935
0.905
27/50
0.562
0.875
0
Gray -trained
0.385
0.440
0.770
0.730
12/50
0.252
0.680
1
Real
0.760 (152/200)
0.792 (1,055/1,332)
0.907
0.885
30/50
0.583
0.915
1
Real , step 100
0.895 (179/200)
0.900 (1,199/1,332)
0.942
0.925
–
–
0.960
Appendix
Table G.6: Seed replication (§§ 6.1 – 6.3 ): seeds 1–3 of the Real , Gray -trained and named-target conditions at step 30, and the Real run of seed 1 continued to step 100, beside seed 0 (Tables G.2 – G.5 ). Discovery member accuracy on the 20-point confirmatory set (200 members) and on the untouched set (1,332 members); pair accuracy on the grounding task and its twin; target-switch discovery with both members correct on the confirmatory (of 50 pairs) and untouched (of 333) sets; find-and-bind (200 members) on the confirmatory scenes. Counts are given where recorded.
quantity
endpoint
seeds
per-seed values
mean
seed SD
range
seed-level 95% interval
Real level
discovery, confirmatory
0–3
0.740, 0.760, 0.730, 0.755
0.746
0.014
0.030
–
Real − Gray
discovery, confirmatory
0–3
+ 0.355, + 0.345, + 0.380, + 0.335
+ 0.354
0.019
0.045
[ + 0.323, + 0.384]
Real − Gray
discovery, untouched
0–3
+ 0.351, + 0.341, + 0.355, + 0.309
+ 0.339
0.021
0.046
[ + 0.306, + 0.372]
Real − untrained
discovery, confirmatory
0–3
+ 0.315, + 0.335, + 0.305, + 0.330
+ 0.321
0.014
–
–
Real − untrained
discovery, untouched
0–3
+ 0.320, + 0.321, + 0.297, + 0.291
+ 0.307
0.015
–
–
Gray − untrained
discovery, confirmatory
0–3
− 0.040, − 0.010, − 0.075, − 0.005
− 0.033
0.032
–
–
Appendix
Table G.7: Levels and contrasts over seeds at step 30 (seed 0 uses Real run 1): per-seed values, their mean and sample standard deviation, and, for the contrasts used as seed-level claims in §§ 6.1 – 6.3 , the range and the t -based 95% interval over seeds (Appendix I ).
subset
members
untrained
Real
Gray -trained
probe correct
165
0.485 (80/165)
0.764 (126/165)
0.436 (72/165)
probe wrong
35
0.143 (5/35)
0.629 (22/35)
0.143 (5/35)
all
200
0.425 (85/200)
0.740 (148/200)
0.385 (77/200)
Appendix
Table G.8: Step-30 discovery on the 20-point confirmatory set (200 members; seed 0, Real run 1 and the Gray -trained control), by whether the untrained model’s identification probe for the same member is correct.
seed
test input
discovery, confirmatory
discovery, untouched
grounding task
gray test images
1
image only
0.550 (110/200)
0.589 (784/1,332)
0.820 (492/600)
0/200
2
image only
0.515 (103/200)
0.553 (737/1,332)
0.802 (481/600)
0/200
3
image only
0.570 (114/200)
0.595 (793/1,332)
0.812 (487/600)
0/200
1
image and caption
0.645 (129/200)
0.669 (891/1,332)
–
–
2
image and caption
0.610 (122/200)
0.646 (861/1,332)
–
–
3
image and caption
0.655 (131/200)
0.682 (908/1,332)
–
–
Appendix
Table G.9: The shortcut-accessible twin at step 30, seeds 1–3, tested with the image alone (the paper’s evaluation) and with the image and its caption (the training input): discovery member accuracy on the confirmatory (200) and untouched (1,332) sets, pair accuracy on the grounding task (600) and discovery with gray test images. Real and Gray -trained seeds 1–3 and the untrained model are in Table G.6 .
quantity
endpoint
per-seed values (1, 2, 3)
mean
seed SD
seed-level 95% interval
Real reference
shortcut level
discovery, confirmatory
0.550, 0.515, 0.570
0.545
0.028
[0.476, 0.614]
0.760, 0.730, 0.755
shortcut − untrained
discovery, confirmatory
+ 0.125, + 0.090, + 0.145
+ 0.120
0.028
[ + 0.051, + 0.189]
+ 0.335, + 0.305, + 0.330
Real − shortcut
discovery, confirmatory
+ 0.210, + 0.215, + 0.185
+ 0.203
0.016
[ + 0.163, + 0.243]
Real − Gray : + 0.345, + 0.380, + 0.335
shortcut level
discovery, untouched
0.589, 0.553, 0.595
0.579
0.023
[0.523, 0.635]
0.792, 0.769, 0.762
Real − shortcut
discovery, untouched
+ 0.203, + 0.215, + 0.167
+ 0.195
0.025
[ + 0.132, + 0.258]
Real − Gray : + 0.341, + 0.355, + 0.309
shortcut − untrained
grounding task
+ 0.052, + 0.033, + 0.043
+ 0.043
0.009
[ + 0.020, + 0.066]
+ 0.139, + 0.154, + 0.125
Appendix
Table G.10: Contrasts over seeds for the shortcut-accessible twin (image only), in the format of Table G.7 ; Real values from Table G.7 for comparison.
f
m(f)
m(f)
in-band
8-pt: level; gain [95% CI]
12-pt
20-pt (never trained)
grounding [95% CI]
0
0.126
0.094
0.752
0.740 + 0.080 [0.030, 0.135]
0.570 − 0.005 [ − 0.035, 0.030]
0.460 − 0.010 [ − 0.040, 0.025]
0.797 [0.763, 0.828]
31
0.165
0.229
0.900
0.935 + 0.275 [0.200, 0.355]
0.805 + 0.230 [0.155, 0.305]
0.745 + 0.275 [0.205, 0.350]
0.908 [0.885, 0.930]
32
0.199
0.240
0.926
0.950 + 0.290 [0.215, 0.370]
0.850 + 0.275 [0.195, 0.360]
0.740 + 0.270 [0.200, 0.345]
0.927 [0.905, 0.947]
1
0.249
0.251
–
0.965 + 0.305 [0.230, 0.380]
0.890 + 0.315 [0.230, 0.400]
0.800 + 0.330 [0.255, 0.410]
0.930 [0.908, 0.950]
Appendix
Table H.1: Dose response at step 30 by constructed fraction f (same 7B starting model, reward, recipe and 7,200 prompt presentations in all four conditions): audited mass m(f) (Eq. ( 3 )), standardized mass m(f) (Appendix J.3 ) and the share of audit-joined training rows with 0<bi<1 (in-band); per density, discovery accuracy with real images and its gain over the untrained model with a 95% interval from resampling scene programs; and grounding-task pair accuracy. With a gray canvas, discovery accuracy is 0 of 200 in every cell of every mixture.
endpoint
slope on m(f) [95% CI]
slope on m(f) [95% CI]
8-pt discovery
1.670 [1.187, 2.190]
1.441 [1.005, 1.907]
12-pt discovery
2.446 [1.813, 3.088]
1.942 [1.432, 2.471]
20-pt discovery (never trained)
2.482 [1.886, 3.083]
2.079 [1.565, 2.596]
grounding task
1.009 [0.787, 1.233]
0.861 [0.669, 1.054]
Appendix
Table H.2: Audit association, equal-exposure and direct comparisons for the mixtures. Top: ordinary least-squares slope (with intercept, over the four conditions) of step-30 held-out accuracy on the audited mass m(f) and on the standardized mass m(f) (Appendix J.3 ); 95% intervals resample scene programs (grounding pairs for the grounding task) with the audited masses held fixed (Appendix I ). Middle: equal constructed-prompt exposure. Discovery accuracy of a mixture at step 30 and of the fully constructed run at the step where it had seen the same number of constructed prompt presentations (five responses each), with their difference and its 95% interval. The two runs of each comparison differ in total presentations, so these comparisons match constructed exposure, not total training. Bottom: direct comparisons of the mixtures at step 30.
discovery, real images
identification probe
discovery
model
step
density
level [95% CI]
Δ vs untrained [95% CI]
real
gray canvas
gray canvas
grounding task [95% CI]
3B
0
8-pt
0.330 [0.255, 0.405]
–
0.705
0.000
0.080
0.455 [0.415, 0.495]
12-pt
0.260 [0.190, 0.340]
–
0.680
0.000
0.025
20-pt
0.245 [0.175, 0.320]
–
0.630
0.000
0.055
10
8-pt
0.475 [0.395, 0.555]
+ 0.145 [0.065, 0.225]
0.875
0.000
0.080
–
12-pt
0.425 [0.340, 0.510]
+ 0.165 [0.085, 0.245]
0.755
0.000
0.025
Appendix
Table H.3: Capacity (Figure 5 (b)): the 3B and 7B models trained on the same constructed corpus, per checkpoint and density (development set): discovery with real images and its change from the same model untrained, with 95% intervals from resampling scene programs; the identification probe with real images and with a gray canvas; discovery with a gray canvas; and grounding-task pair accuracy where evaluated. With a gray canvas the probe is exactly zero for both models at every checkpoint, while the 3B model’s discovery accuracy is small and nonzero at every checkpoint, including before training: it answers with a fixed coordinate that is sometimes correct.
benchmark
model
with image
no image
c
ρblind
MathVista testmini
untrained 7B
0.682 (682/1,000)
0.366 (366/1,000)
0.179
0.372
MathVista testmini
f=1 , step 30
0.532 (532/1,000)
0.258 (258/1,000)
0.179
0.224
MathVista testmini
f=31 , step 30
0.696 (696/1,000)
0.367 (367/1,000)
0.179
0.364
BLINK
untrained 7B
0.536 (1,018/1,901)
0.416 (791/1,901)
0.377
0.247
BLINK
f=1 , step 30
0.457 (869/1,901)
0.363 (690/1,901)
0.377
− 0.175
BLINK
f=31 , step 30
0.547 (1,039/1,901)
0.420 (798/1,901)
0.377
0.252
Appendix
Table H.4: The untrained 7B model and the step-30 f=1 and f=31 checkpoints on MathVista testmini (1,000 items, 539 of them multiple-choice) and BLINK (1,901 multiple-choice items), with and without the image, with the chance reference c and the chance-corrected blind retention ρblind of § 2.2 . Retention is reported unclipped; without the image the f=1 model answers BLINK below chance, so its retention is negative.
condition
seed
rows
step-1 accuracy
step-1 format
mixed, step 1
nonzero advantage, step 1
mixed, steps 1–5
nonzero advantage, steps 1–5
all correct, step 1
all wrong, step 1
Real
1
discovery
0.223
0.857
0.633
0.808
0.645
0.698
0.008
0.358
Real
1
probe
0.435
0.875
0.892
0.933
0.612
0.622
0.025
0.083
Real
2
discovery
0.178
0.883
0.575
0.800
0.632
0.690
0.008
0.417
Real
2
probe
0.407
0.892
0.875
0.942
0.598
0.610
0.017
0.108
Real
3
discovery
0.245
0.875
0.667
0.842
0.655
0.718
0.008
0.325
Real
3
probe
0.428
0.890
0.892
0.933
0.622
0.637
0.025
0.083
Appendix
Table J.1: Reward variation within response groups at the start of training, for the runs of Table G.6 , by training-row type (discovery, the identification probe, or find-and-bind for the named-target corpus). Columns: mean accuracy and format components at step 1; the fraction of groups containing both a correct and an incorrect response (“mixed”, the event of Eq. ( J.2 )); the fraction of groups with any within-group reward variation (“nonzero advantage”); the same two fractions averaged over steps 1–5; and the step-1 fractions of all-correct and all-wrong groups.
Figure K.1: Eight of the nine task families: coordinate scenes (top) and other formats (bottom).
component
scale
constructed evaluations
a 1,200-pair suite and its regenerated 1,200-pair twin; discovery scenes at 8, 12 and 20 points, with 450 confirmatory scene programs; an untouched replication set of 333 scene programs per pair role (Table G.6 ); 160 premise groups (an untrained nearest-point relation task); 100 label-swap diagnostic pairs
constructed training data
2,880 discovery and probe prompts; a second corpus of 3,000 pairs and 300 answer-preserving twins
Reinforcement learning with verifiable rewards (RLVR) improves vision-language models (VLMs) by optimizing outcome rewards derived from final answers. However, such outcome-only rewards do not tell the model which image regions justify an answer. For questions that require visual grounding, these rewards cannot distinguish responses supported by relevant visual evidence from those produced by language-prior shortcuts or lucky guesses. We introduce EASE (Evidence-Anchored Spatial Attention), which augments multimodal RLVR with visual-evidence process supervision. EASE converts annotated evidence regions into a smoothed visual-token target and uses it to guide response-to-image attention during RL training, but only on high-reward trajectories. The annotations are used solely as privileged training labels, while inference requires only the original image and question. Across Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3-VL-8B, EASE raises average scores over DAPO by 2.5 to 3.1 points on perception, hallucination, visual math, and multimodal reasoning benchmarks. Diagnostics and ablations show that EASE better aligns visual attention with annotated evidence regions.
Ruina Hu, Chen Wang, Lai Wei +5
1Harbin Institute of Technology · 2Zhongguancun Academy · 4Nankai University +4
Reinforcement Learning with Verifiable Rewards (RLVR) makes Multimodal Large Language Models more accurate, but the gains are brittle: simply paraphrasing a question or changing the prompt template can degrade them, which challenges reliable deployment in high-stakes scenarios like medical VQA. We trace this to two issues of the standard RL objective. First, the binary verifier conflates format with content, so the reward signal cannot tell a wrong answer apart from a misformatted one. Second, the training distribution covers only a thin slice of the real-world prompts that the model might meet at deployment, so policies that perform well on the training distribution can behave differently under unseen prompts during test. Both failures call for a robust post-training method that helps the policy cover a broader distribution of semantically equivalent prompts, and we identify two measures that help achieve this objective: separating format from semantics in the reward, and applying policy invariance across perturbed prompts with equivalent semantics. We therefore propose Prompt-Invariant RLVR (PIRL), consisting of a dynamic trinary reward and a consistency regularizer based on an embedding-space adversary. Under stress testing, PIRL's average accuracy on benchmarks drops by only ≤1%, where GRPO drops ~3%. On dynamic evaluation, PIRL also achieves the smallest performance drop.
Pengfei Zhou, Zhiwei Tang, Xiaopeng Peng +11
1National University of Singapore, HPC-AI Lab · 2DAMO Academy, Alibaba Group · 3Hupan Lab +7
Vision-Language Models (VLMs) frequently suffer from visual perception errors and hallucinations that compromise answer accuracy in complex reasoning tasks. Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising solution by optimizing policies using answer correctness signals. Despite their effectiveness, prevailing RLVR methods face two critical limitations. First, much of the sampling budget is wasted on trajectories doomed to fail due to early visual description errors. Second, sparse rewards cannot distinguish whether failures stem from visual perception or reasoning stages. We introduce MIRL, a decoupled framework that addresses both limitations by leveraging mutual information (MI) between generated descriptions and visual inputs as a cheap pre-screening signal. This enables intelligent budget allocation toward high-potential trajectories via forking, while decoupled training provides independent MI-based rewards for visual perception optimization, resolving reward blindness. Experiments on six vision-language reasoning benchmarks demonstrate that MIRL achieves 70.22% average accuracy and successfully surpasses the performance of sampling 16 complete trajectories using only 10 pre-samples with top-6 selection (25% fewer complete trajectories). Our code is available at: https://anonymous.4open.science/r/mirl-main/.
Yin Zhang, Jiaxuan Zhao, Zonghan Wu +5
School of Mathematics, Tianjin University, Tianjin, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Shanghai Advanced Institute of Finance (SAIFS), East China Normal University, Shanghai, China +4