Same Reward, Different Skills: When Multimodal RL Learns to Look

Haocun Ye; Xinlong Jiang; Qile Chen; Bingyu Wang; Teng Zhang; Shubai Chen; Tingyu Wu; Zhenkun Zheng

cs.LGOct 1, 2026

Same Reward, Different Skills: When Multimodal RL Learns to Look

Authors: Haocun Ye, Xinlong Jiang, Qile Chen, Bingyu Wang, Teng Zhang, Shubai Chen, Tingyu Wu, Zhenkun Zheng, +1 more

Organizations: University of the Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences · Independent Researcher

Abstract

Reinforcement learning with verifiable rewards (RLVR) improves vision-language benchmark scores even without visual information during training. With images at test, blind-trained models recover roughly half of the real-image gain at 3B and nearly four fifths at 7B. Prolonged real-image training can erode grounding while benchmark gains persist. Both findings expose the same gap: an image in the prompt is not an image in the learning signal. Our design rule, visual resolvability, asks that visual evidence be necessary for a correct answer and that the task remain learnable. We test it on counterfactual coordinate scenes in which the question stays fixed and the target is never named, so a correct answer requires finding the target in the image. With standard GRPO and correctness-and-format rewards, a 7B model raises its accuracy at finding the target (discovery) from 0.425 to 0.875 on held-out scenes denser than any it trained on, and it improves on question types it never trained on. Two controls locate the source of the gain. Replacing test images with gray canvases drops discovery to zero; training on gray canvases instead, at matched step 30 and in each of four seeds, yields essentially none of the gain even when the model is then tested with real images. The learned skill carries over to grounding tasks built independently of the training corpus. A caption that answers the training question, added to the same images, reward and budget, cuts the gain by nearly two thirds. Changing what reward requires changes what RL learns.

Figures & tables

Figure 1: Same learning rule, different lessons. Left: shortcut-accessible tasks allow reward without visual evidence; gains in the reported 3B comparisons mainly favor cued readout. Right: constructed tasks demand visual evidence and relation-based target identification; discovery, cued readout, and named-target grounding improve. The hierarchy distinguishes supplied locations, supplied identities, and relation-defined targets. GRPO is shared across the study; rewards stay fixed within matched comparisons.

trained without visual information
	Evaluated with the real image		Test image removed
Trained with	gain over base	share of Real ’s gain	gray canvas	no image
base model (accuracy)	0.175	—	0.090	0.068
Real	$+$ 0.238	1 (reference)	$+$ 0.019	$+$ 0.036
Caption	$+$ 0.171	0.72	$+$ 0.024	$+$ 0.037
None	$+$ 0.137	0.57	$+$ 0.015	$+$ 0.044

Table 1: Blind gains. Training without visual information recovers more than half of Real ’s benchmark gain when real images are supplied at test; with the test image removed, no condition gains more than 0.05. Base row: accuracy; other cells: gain over that base under the same evaluation; shares: fraction of Real ’s gain (three-seed pooled). All cells use the same answer matcher.

Figure 2: Benchmark gains are not visual acquisition. (a) Gains by operation: readout and grounding for Geometry3K-trained 3B models (three-seed means), discovery for ViRL39K-trained models (two-seed means; 7B: Geometry3K-trained pair). (b) Benchmark accuracy and grounding pair accuracy under prolonged Real training with a trainable encoder. (c) Grounding pairs lost by three Gray -trained runs, and their wrong answers. All evaluations use real images; error bars are 95% intervals and shading marks the grounding equivalence band.

Figure 3: Measure and shape. (a) Audited visual benefit against initial success, constructed corpus and ViRL39K. (b) The K8 scene and its counterfactual twin. (c) Its target-cue hierarchy.

Figure 4: The regime flips. (a) Discovery member accuracy on the development set (inset: confirmatory 20-point result). (b) Step-30 discovery gains under Real and Gray training, evaluated with real images. (c) Step-100 change in pair accuracy. Error bars: 95% intervals over scene programs in (a, b) and over pairs in (c).

Figure 5: Composition changes the gain; model size changes the trajectory. (a) Step-30 discovery gain of each training mixture against the constructed fraction

f

. (b) Discovery accuracy of 3B and 7B models trained on the constructed corpus. Error bars: 95% intervals over scene programs.

Appendix figures & tables33 assets

Supplementary material from the paper’s appendix.

Appendix

benchmark (format)	$n$	with image	blind	chance	naive retention	corrected retention
BLINK (MC, pooled)	1901	0.493	0.409	0.377	0.829 [0.781, 0.879]	0.271 [0.089, 0.454]
MathVerse (MC, pooled)	2180	0.465	0.394	0.260	0.848 [0.807, 0.889]	0.655 [0.568, 0.743]
MathVerse (free-form)	1755	0.055	0.019	0.000	0.340 [0.240, 0.458]	0.340 [0.240, 0.458]
MMVP (MC, $k=2$ )	300	0.660	0.500	0.500	0.758 [0.668, 0.854]	0.000 [ $-$ 0.417, 0.342]
MMMU dev+val (MC, pooled)	988	0.506	0.413	0.263	0.816 [0.764, 0.873]	0.617 [0.512, 0.729]
MMMU dev+val (free-form)	62	0.097	0.048	0.000	0.500 [0.000, 1.000]	0.500 [0.000, 1.000]

Appendix

Table A.1: Visual-necessity audit of the untrained Qwen2.5-VL-3B model on seven benchmarks: accuracy with the image and with the image removed (blind), the chance level (mean of the item-specific

1/k

for multiple choice; 0 for free-form answers), naive retention (blind

\div

with-image) and chance-corrected retention

(\text{blind}-\text{chance})/(\text{with-image}-\text{chance})

, with 95% item-paired bootstrap intervals (10,000 draws). Mixed-format benchmarks are reported by format; HallusionBench is reported under both plausible chance levels.

model	pool	$n$	with image	blind	chance	naive retention	corrected retention
Gemma-3	ViRL39K free-form	2789	0.430	0.312	0.000	0.727 [0.690, 0.765]	0.727 [0.690, 0.765]
Gemma-3	ViRL39K multiple choice	1215	0.135	0.086	0.268	0.634 [0.533, 0.744]	undefined
InternVL3-9B	ViRL39K free-form	2789	0.269	0.130	0.000	0.485 [0.439, 0.533]	0.485 [0.439, 0.533]
InternVL3-9B	ViRL39K multiple choice	1215	0.294	0.205	0.268	0.697 [0.619, 0.782]	$-$ 2.439 [ $-$ 17.961, 6.505]

Appendix

Table A.2: The same audit for two other model families on a 4,096-item ViRL39K sample (the image is removed rather than replaced): with-image and blind accuracy, the chance level, and naive and chance-corrected retention with 95% item-paired bootstrap intervals. Corrected retention is defined only when with-image accuracy exceeds the chance level (§ 2.2 ). On the multiple-choice pool Gemma-3 scores below chance with the image, so its ratio is undefined; InternVL3-9B exceeds chance by only 0.026, so its ratio is defined but unstable and is reported unclipped. Without the image, both models score 0.000 pair accuracy on the grounding task and its twin (§ 2.2 ).

level	question	visual cue	gold (twin a)	gold (twin b)
L1 (arrow cue; position given)	Point K8 has the smallest $y$ -coordinate. What is the $x$ -coordinate of point K8?	offset arrow at K8	$-1$	$6$
L2 (target named; identity given)	Point K8 has the smallest $y$ -coordinate. What is the $x$ -coordinate of point K8?	none	$-1$	$6$
L3 (discover $+$ ground $+$ read; nothing given)	Consider the point with the smallest $y$ -coordinate. What is its $x$ -coordinate?	none	$-1$	$6$
probe (identification only)	Which labeled point has the smallest $y$ -coordinate?	none	K8	K8

Appendix

Table B.1: The three levels and the identification probe on one development scene (eight labeled points; the K8 scene of Figure 3 (b)): the question templates, the cue each level supplies, and the gold answers on the two twins, which differ in one fact (K8 moves from

(-1,-1)

(6,-1)

). L2 and the probe use the L3 image; L1 adds only the offset arrow.

model	discovery 8-pt	12-pt	20-pt	probe 8-pt	12-pt	20-pt
3B, ViRL39K-trained (step 100); two seeds
untrained 3B	0.330	0.260	0.245	0.705	0.680	0.630
Real , two-seed mean	0.383	0.338	0.330	0.728	0.700	0.610
seeds 1 / 2	0.380 / 0.385	0.330 / 0.345	0.330 / 0.330	0.725 / 0.730	0.710 / 0.690	0.620 / 0.600
Caption , two-seed mean	0.370	0.323	0.340	0.728	0.715	0.635
seeds 1 / 2	0.365 / 0.375	0.325 / 0.320	0.345 / 0.335	0.715 / 0.740	0.710 / 0.720	0.610 / 0.660

Appendix

Table B.2: Discovery on the coordinate development set (member accuracy with real images) for the models trained with ordinary rewards: the ViRL39K-trained 3B conditions (two seeds; step 100) and the Geometry3K-trained 7B pair (step 100), with the untrained models. Discovery (L3) and the identification probe at 8, 12, and 20 points. Two-seed means are computed from the per-seed counts; intervals are not reported for these values.

condition	cued readout: gain [95% CI]	find-and-bind	header-cued table
untrained 3B (pair accuracy)	0.320	0.455	0.867
Real	$+$ 0.143 [0.102, 0.188]	$+$ 0.018 [ $-$ 0.007, 0.044]	$+$ 0.019 [ $-$ 0.002, 0.042]
seeds 1 / 2 / 3	$+$ 0.157 / $+$ 0.127 / $+$ 0.147	$+$ 0.012 / $+$ 0.022 / $+$ 0.022	$+$ 0.030 / $+$ 0.013 / $+$ 0.013
Caption	$+$ 0.107 [0.066, 0.149]	$+$ 0.006 [ $-$ 0.018, 0.030]	$+$ 0.021 [ $-$ 0.001, 0.044]
seeds 1 / 2 / 3	$+$ 0.130 / $+$ 0.113 / $+$ 0.077	$+$ 0.003 / 0.000 / $+$ 0.013	$+$ 0.023 / $+$ 0.027 / $+$ 0.013
None	$+$ 0.108 [0.070, 0.148]	$-$ 0.013 [ $-$ 0.039, 0.013]	$+$ 0.023 [0.000, 0.049]

Appendix

Table B.3: Task-layer gains of the Geometry3K-trained 3B conditions at step 100 (pair accuracy; three-seed means with 95% paired-item intervals; per-seed gains beneath). The equivalence test for the grounding task is in Appendix I .

set	density	discovery (L3)	accuracy	limit ( $\tfrac{1}{2}$ L3)	outcome	(5) artifact screen	criteria met
			(4) caption-only accuracy
development	8-pt	0.660	0.580	0.330	not met	met	4 of 5
	12-pt	0.575	0.445	0.288	not met	not met: 0.557 in 1 of 3 seeds	3 of 5
	20-pt	0.470	0.185	0.235	met	not met: 0.565 in 1 of 3 seeds	4 of 5
confirmatory	8-pt	0.585	0.570	0.293	not met	met	4 of 5
	12-pt	0.560	0.410	0.280	not met	met	4 of 5

Appendix

Table B.4: Outcome of the five evaluation criteria of § 5.3 on the untrained 7B model, for every density of the development set (used to track learning) and of the separate confirmatory set: (1) discovery accuracy between 0.20 and 0.80; (2) discovery at least 0.05 below both cued levels; (3) exactly zero accuracy on discovery and its probe with a gray canvas and with no image; (4) caption-only accuracy at most half of with-image discovery accuracy; (5) the artifact screen of Appendix B : out-of-fold

\max(\mathrm{AUC},1-\mathrm{AUC})

of four classifiers at most 0.55, and the upper end of the unfolded interval at most 0.62, at each of three cross-validation seeds. Criteria 1–3 are met in every cell. Only the 20-point confirmatory cell meets all five; it supplies the confirmatory result.

task	pairs	role	untrained 3B	untrained 7B [95% CI]
grounding task (named-point coordinate pairs)	600	find-and-bind (primary)	0.455	0.768 [0.733, 0.802]
header-cued table task	300	cued control, high baseline	0.867	0.993 [0.983, 1.000]
cued-readout task (marked-plot pairs)	300	cued readout, location marked	0.320	0.673 [0.620, 0.727]
twin of the grounding task	600	regenerated twin	–	0.728 [0.692, 0.762]
twin of the cued-readout task	300	twin control	–	0.623 [0.570, 0.677]
twin of the header-cued table task	300	twin control	–	0.997 [0.990, 1.000]

Appendix

Table C.1: The independently constructed grounding tasks: the 1,200-pair counterfactual suite (three tasks with distinct roles, reported separately) and its independently regenerated twin (fresh seeds, no iteration). Untrained pair accuracy (both members correct) for the 3B and 7B models; 7B levels with 95% intervals. Validation: Qwen2.5-VL-72B-Instruct given question-blind captions reaches at most 0.054; every image-removed condition scores exactly 0.000, with all responses collapsing to one answer; 60 of 60 audited pairs were confirmed by human review; Gemma-3 and InternVL3-9B also score exactly 0.000 without the image.

experiment	backbone, vision encoder	training images	reward	training data	steps; evaluated at	seeds	runs
Geometry3K access conditions (§ 2.1 , § 3 )	3B, frozen	Real / Gray / None / Caption , one condition per run	answer correctness	Geometry3K, filtered (1,288 items)	100	3	3
7B access pair (§ 2.1 , § 3 )	7B, frozen	Real ; Gray	answer correctness	Geometry3K	100	1	1
long-horizon runs (§ 3 )	3B, trainable	real images	$0.5\cdot$ accuracy $+\;0.5\cdot$ format	Geometry3K, unfiltered	400, in segments; evaluated at 100, 150, 200, 300, 400	2	2
resolvable training, standard reward (§ 6 )	7B, frozen	real images	$0.5\cdot$ accuracy $+\;0.5\cdot$ format	constructed corpus, 2,880 items (discovery and probe, both twins)	seeds 0 and 1: 100, in segments (seed 0 evaluated at 10, 20, 30, 50, 75, 100; seed 1 at 30 and 100). Seeds 2 and 3: 30	4	5 (two of seed 0)
Gray -trained control (§ 6.2 )	7B, frozen	gray canvases	as above	same corpus	30; evaluated at 10, 20, 30 (seed 0) or 30 (seeds 1–3)	4	4
named-target ablation (§ 6.1 ; Table G.6 )	7B, frozen	real images	as above	same scene programs, with the find-and-bind (L2) question in place of the discovery question; probe unchanged	30	3	3

Appendix

Table D.1: Training recipes. Every run uses GRPO with 5 rollouts per prompt, learning rate

10^{-6}

and KL coefficient 0.01. In every recipe the verifier scores each response’s extracted answer against its gold answer and, where noted, its format; no image-dependent auxiliary loss or pairwise training reward is added. Constructed-corpus runs (§§ 6 – 7 ): rollout batch 240, global batch 48. Geometry3K, ViRL39K and long-horizon runs: rollout batch 512, global batch 128 (the long-horizon runs on 4 GPUs with tensor parallelism 2). The last two columns count, per condition, the independent seeds and the training runs: an independent seed draws its own shuffled data order and rollout sampling; a run beyond the seed count repeats one configuration. The dose response of § 7.1 has four points, the three mixtures listed here and the standard run at step 30 (

f=1

condition	seed	acc. real	acc. gray	acc. none	acc. caption	gain real	gain gray	gain none	gain caption	share
base model	–	0.175	0.090	0.068	0.210	–	–	–	–	–
Real	1	0.423	0.106	0.108	0.311	$+$ 0.248	$+$ 0.017	$+$ 0.040	$+$ 0.102	–
	2	0.419	0.112	0.098	0.301	$+$ 0.245	$+$ 0.022	$+$ 0.030	$+$ 0.091	–
	3	0.398	0.110	0.106	0.326	$+$ 0.223	$+$ 0.020	$+$ 0.038	$+$ 0.117	–
Caption	1	0.361	0.118	0.112	0.318	$+$ 0.186	$+$ 0.028	$+$ 0.043	$+$ 0.108	0.752
	2	0.346	0.108	0.106	0.291	$+$ 0.171	$+$ 0.018	$+$ 0.038	$+$ 0.082	0.701

Appendix

Table D.2: Table 1 per seed, with the untrained and trained models scored by the same answer matcher: accuracy of each condition’s step-100 model under the four test conditions (601 Geometry3K test items), the gain over the untrained model under the same test condition, and the recovery share (real-tested gain

\div

Real ’s real-tested gain). Intervals are not computed for the three-seed means and shares; the per-seed values show their spread.

task	pairs	untrained 7B	Real $\Delta$ [95% CI]	Gray $\Delta$ [95% CI]
grounding task	600	0.768	$+$ 0.035 [0.013, 0.058]	$+$ 0.012 [ $-$ 0.010, 0.033]
twin of the grounding task	600	0.728	$+$ 0.038 [0.017, 0.062]	$+$ 0.013 [ $-$ 0.010, 0.037]
cued-readout task	300	0.673	$+$ 0.030 [ $-$ 0.007, 0.067]	$+$ 0.017 [ $-$ 0.013, 0.047]
twin of the cued-readout task	300	0.623	$+$ 0.013 [ $-$ 0.030, 0.060]	$+$ 0.013 [ $-$ 0.023, 0.050]
header-cued table task	300	0.993	0.000 [ $-$ 0.010, 0.010]	$-$ 0.003 [ $-$ 0.010, 0.000]
twin of the header-cued table task	300	0.997	0.000 [ $-$ 0.010, 0.010]	$-$ 0.007 [ $-$ 0.017, 0.000]

Appendix

Table D.3: The ordinary 7B access pair of § 3 on the independently constructed grounding suite and its twin (Geometry3K training, step 100): untrained pair accuracy and the change under Real and Gray training, with 95% paired-item bootstrap intervals (2,000 draws). The interval excludes zero for Real on the grounding task and on its twin, and includes zero in every other cell.

trained with	step	real image: correct [95% CI]	gain over untrained [95% CI]	gray canvas: correct [95% CI]
untrained 7B	–	141/601 = 0.235 [0.201, 0.270]	–	48/601 = 0.080 [0.058, 0.103]
Real	100	290/601 = 0.483 [0.443, 0.521]	$+$ 0.248 [0.203, 0.291] (149/601)	75/601 = 0.125 [0.100, 0.151]
Gray	100	257/601 = 0.428 [0.388, 0.466]	$+$ 0.193 [0.153, 0.235] (116/601)	79/601 = 0.131 [0.103, 0.160]

Appendix

Table D.4: The 7B access pair behind the share in Table 1 (Geometry3K training; 601 Geometry3K test items; the answer matcher of Table 1 ): correct answers with real test images and with a gray canvas, with 95% item-bootstrap intervals, and the gain over the untrained model with real images, with its item-paired interval (5,000 draws). The recovery share is Gray ’s gain divided by Real ’s: 116/149 = 0.779 [0.642, 0.921].

questions	$n$	untrained	Real gain	Caption gain	None gain	Gray gain
all questions	601	0.175	$+$ 0.238 [0.202, 0.275]	$+$ 0.171 [0.136, 0.207]	$+$ 0.137 [0.103, 0.171]	$+$ 0.125 [0.092, 0.158]
extractable	497	0.211	$+$ 0.200 [0.160, 0.240]	$+$ 0.140 [0.100, 0.179]	$+$ 0.107 [0.069, 0.145]	$+$ 0.101 [0.064, 0.139]
not extractable	104	0.000	$+$ 0.423 [0.343, 0.503]	$+$ 0.324 [0.250, 0.401]	$+$ 0.279 [0.212, 0.349]	$+$ 0.237 [0.176, 0.301]

Appendix

Table D.5: The blind gains of Table 1 by whether the untrained 3B model’s real-image answer can be extracted (extractable: 497 of 601 questions): the untrained model’s accuracy, the gain over it with real test images, averaged over the three seeds per question, and each condition’s share of Real ’s gain, with 95% intervals from a 10,000-draw bootstrap over questions. The three runs of a condition answer the same 601 questions, so questions, not the 1,803 responses, are resampled.

result (section; table)	seeds	runs	individual seeds or runs	95% intervals resample
blind recovery shares, 3B (§ 2.1 ; Table 1 )	3	3	Table D.2 , per seed	point estimates over seeds
Gray recovery share, 7B (§ 2.1 ; Table D.4 )	1	1	–	benchmark items
operation-level gains, 3B (§ 2.3 ; Table B.3 )	3	3	Table B.3 , per seed	grounding pairs
discovery, ViRL39K-trained 3B (§ 2.3 ; Table B.2 )	2	2	Table B.2 , per seed	point estimates over seeds
grounding loss under prolonged training (§ 3 ; Table E.1 )	2	2	Table E.1 , per run	grounding pairs
discovery on the confirmatory set (§ 6.1 ; Tables G.2 , G.6 )	4	5	Table G.2 (seed 0, per run); Table G.6 (seeds 1–3; seed 1 also at step 100)	scene programs (seed 0); seeds, $t$ -based (Table G.7 )

Appendix

Table D.6: Run basis of each reported result. Seeds counts the independent seeds of a condition and runs its training runs (Table D.1 ); where runs exceed seeds, the further run repeats one configuration. The fourth column names the table that prints the individual seeds or runs, and the last the unit that each result’s 95% intervals resample with the trained models held fixed.

condition	seed	step 1: overall / format / accuracy	step 30: overall / format / accuracy	mean accuracy, steps 19–30
Real	1	0.5975 / 0.8658 / 0.3292	0.9858 / 1.0000 / 0.9717	0.975
Real	2	0.5900 / 0.8875 / 0.2925	0.9863 / 1.0000 / 0.9725	0.982
Real	3	0.6095 / 0.8825 / 0.3365	0.9930 / 0.9992 / 0.9867	0.970
Gray -trained	1	0.4350 / 0.8683 / 0.0017	0.5175 / 1.0000 / 0.0350	0.033
Gray -trained	2	0.4413 / 0.8800 / 0.0025	0.5238 / 0.9992 / 0.0483	0.041
Gray -trained	3	0.4425 / 0.8800 / 0.0050	0.5104 / 1.0000 / 0.0208	0.024

Appendix

Table D.7: Training rewards of seeds 1–3 (Table G.6 ): the overall reward of Eq. ( 2 ) with its format and accuracy components at steps 1 and 30, and the mean accuracy component over steps 19–30. The Gray -trained control earns format reward from the start and almost no accuracy reward at any point; the Real , named-target and shortcut-accessible runs approach the ceiling of both components by step 30.

seed	step	benchmark [95% CI]	grounding [95% CI]	$\Delta$ vs base [95% CI]
1	0	0.175 [0.145, 0.206]	0.455 [0.415, 0.495]	–
	100	0.431 [0.393, 0.471]	0.477 [0.437, 0.517]	$+$ 0.022 [ $-$ 0.008, 0.052]
	150	0.463 [0.424, 0.504]	0.465 [0.425, 0.505]	$+$ 0.010 [ $-$ 0.020, 0.040]
	200	0.483 [0.444, 0.524] (peak)	0.450 [0.412, 0.490]	$-$ 0.005 [ $-$ 0.035, 0.025]
	300	0.464 [0.424, 0.506]	0.440 [0.400, 0.480]	$-$ 0.015 [ $-$ 0.047, 0.017]
	400	0.436 [0.398, 0.474]	0.407 [0.367, 0.447]	$-$ 0.048 [ $-$ 0.082, $-$ 0.017]

Appendix

Table E.1: The two long-horizon runs of § 3 (Qwen2.5-VL-3B, trainable encoder, real images, unfiltered Geometry3K): benchmark accuracy (correct answers out of 601) with 95% item-bootstrap intervals, and grounding-task pair accuracy with its interval and its change from the untrained model (the 600 pairs, each its own scene, resampled with the same 10,000 draws at every checkpoint). Peaks are marked.

run	degraded pairs	pairs gained	wrong members
seed 1	51	24	52
seed 2	49	22	53
seed 3	45	23	46

Appendix

Table E.2: Shared errors across the three Gray -trained runs of § 3 (grounding task, 600 pairs). The sets are defined with the earlier answer-matcher revision (Appendix I ). Top: per run, the degraded set (pairs the untrained model solves and the run fails), pairs gained, and wrong members (individual pair members answered incorrectly within the degraded set). Bottom: overlap of the degraded sets (Jaccard; intersection/union) against a 10,000-draw permutation null of sets of the same sizes (

p=10^{-4}

for every comparison), and the rate of identical wrong answers on the wrong members the runs share.

run	step	8-pt: level [95% CI]; $\Delta$ [95% CI]	12-pt	20-pt	probe 8 / 12 / 20-pt
untrained 7B	0	0.660 [0.580, 0.740]	0.575 [0.485, 0.660]	0.470 [0.385, 0.555]	0.940 / 0.910 / 0.840
run 1	30	0.965 [0.935, 0.990] $\Delta$ $+$ 0.305 [0.230, 0.380]	0.890 [0.830, 0.945] $\Delta$ $+$ 0.315 [0.230, 0.400]	0.800 [0.730, 0.865] $\Delta$ $+$ 0.330 [0.255, 0.410]	1.000 / 0.975 / 0.975
	100	200/200 (100 programs) $\Delta$ $+$ 0.340 [0.260, 0.420]	0.940 [0.895, 0.980] $\Delta$ $+$ 0.365 [0.280, 0.455]	0.910 [0.860, 0.950] $\Delta$ $+$ 0.440 [0.360, 0.525]	1.000 / 0.990 / 0.995
run 2	30	0.960 [0.925, 0.985] $\Delta$ $+$ 0.300 [0.225, 0.380]	0.860 [0.795, 0.920] $\Delta$ $+$ 0.285 [0.205, 0.370]	0.795 [0.730, 0.860] $\Delta$ $+$ 0.325 [0.245, 0.405]	1.000 / 0.960 / 0.960
	100	0.995 [0.985, 1.000] $\Delta$ $+$ 0.335 [0.255, 0.415]	0.950 [0.915, 0.980] $\Delta$ $+$ 0.375 [0.290, 0.465]	0.905 [0.860, 0.945] $\Delta$ $+$ 0.435 [0.350, 0.520]	0.995 / 0.985 / 0.990

Appendix

Table G.1: Development-set acquisition (Figure 4 (a)): discovery (L3) member accuracy with real images at steps 0, 30 and 100 for both runs, by density, and its change from the untrained model, with 95% intervals from resampling scene programs (Appendix I ); last column, the identification probe with real images at 8 / 12 / 20 points. With the test image replaced by a gray canvas, discovery and the probe score 0 of 200 at every one of the six checkpoints of both runs.

run	step	density	level [95% CI]	$\Delta$ vs untrained [95% CI]	level [95% CI]	criteria met
			real images		gray canvas
untrained 7B	0	8-pt	0.585 [0.500, 0.665]	–	0/200 (100 programs)	4 of 5
untrained 7B	0	12-pt	0.560 [0.470, 0.645]	–	0/200 (100 programs)	4 of 5
untrained 7B	0	20-pt	0.425 [0.335, 0.515]	–	0/200 (100 programs)	5 of 5
Real , run 1	30	8-pt	0.935 [0.895, 0.970]	$+$ 0.350 [0.275, 0.430]	0/200 (100 programs)	4 of 5
Real , run 1	30	12-pt	0.860 [0.805, 0.910]	$+$ 0.300 [0.225, 0.380]	0/200 (100 programs)	4 of 5

Appendix

Table G.2: All confirmatory-set cells (150 new scene programs per density, generated after the evaluation criteria of § 5.3 were set): discovery member accuracy with real images and its change from the untrained model, and accuracy with the test image replaced by a gray canvas, with 95% intervals from resampling scene programs; a cell with 0 of 200 correct is given as its count and number of scene programs. The last column gives how many of the five criteria the evaluation cell met before training; it describes the cell, not a run’s outcome. The 20-point cell met all five and supplies the confirmatory result of § 6.1 ; the 8- and 12-point cells did not meet the caption criterion (Appendix B ).

set	density	task	Real $\Delta$ [95% CI]	control $\Delta$ [95% CI]	$\tau_{\mathrm{vis}}$ [95% CI]	Real	control
						gray-canvas count
dev	8-pt	L3	$+$ 0.305 [0.230, 0.380]	$+$ 0.030 [ $-$ 0.005, 0.065]	$+$ 0.275 [0.205, 0.350]	0/200	11/200
dev	8-pt	probe	$+$ 0.060 [0.025, 0.105]	$+$ 0.005 [ $-$ 0.020, 0.035]	$+$ 0.055 [0.020, 0.095]	0/200	0/200
dev	12-pt	L3	$+$ 0.315 [0.230, 0.400]	$-$ 0.035 [ $-$ 0.080, 0.005]	$+$ 0.350 [0.265, 0.435]	0/200	9/200
dev	12-pt	probe	$+$ 0.065 [0.025, 0.110]	$-$ 0.005 [ $-$ 0.035, 0.020]	$+$ 0.070 [0.030, 0.120]	0/200	0/200
dev	20-pt	L3	$+$ 0.330 [0.255, 0.410]	$-$ 0.030 [ $-$ 0.070, 0.010]	$+$ 0.360 [0.275, 0.445]	0/200	8/200

Appendix

Table G.3: Attribution at matched step 30: the Real gain and the Gray -trained control’s change (both evaluated with real images) and their difference

\tau_{\mathrm{vis}}

(Eq. ( F.1 )), on the development set (dev) and replicated on the confirmatory set (conf); task: discovery (L3) or the identification probe, with each model’s gray-canvas count at the same checkpoint (correct/total). 95% intervals from resampling scene programs, with the same draws for both models.

run	step	grounding task (600 pairs)	twin (600)	cued-readout task (300)	header-cued table (300)
untrained 7B	0	0.768 [0.733, 0.802]	0.728 [0.692, 0.762]	0.673 [0.620, 0.727]	0.993 [0.983, 1.000]
run 1	30	0.930 [0.908, 0.950] $\Delta$ $+$ 0.162 [0.130, 0.193]	0.900 $\Delta$ $+$ 0.172	0.680 [0.627, 0.733] $\Delta$ $+$ 0.007 [ $-$ 0.023, 0.037]	0.997 [0.990, 1.000] $\Delta$ $+$ 0.003 [0.000, 0.010]
	100	0.948 [0.930, 0.965] $\Delta$ $+$ 0.180 [0.147, 0.215]	0.937 [0.917, 0.955] $\Delta$ $+$ 0.208 [0.175, 0.243]	0.707 [0.657, 0.757] $\Delta$ $+$ 0.033 [0.000, 0.070]	0.997 [0.990, 1.000] $\Delta$ $+$ 0.003 [0.000, 0.010]
run 2	30	0.935 [0.915, 0.955] $\Delta$ $+$ 0.167 [0.137, 0.198]	0.905 $\Delta$ $+$ 0.177	0.690 [0.640, 0.743] $\Delta$ $+$ 0.017 [ $-$ 0.013, 0.047]	0.997 [0.990, 1.000] $\Delta$ $+$ 0.003 [0.000, 0.010]
	100	0.962 [0.947, 0.977] $\Delta$ $+$ 0.193 [0.162, 0.227]	–	0.727 [0.677, 0.777] $\Delta$ $+$ 0.053 [0.020, 0.087]	0.997 [0.990, 1.000] $\Delta$ $+$ 0.003 [0.000, 0.010]

Appendix

Table G.4: Transfer to the independently constructed grounding tasks (Figure 4 (c)) at steps 0, 30 and 100 for both runs: pair accuracy with 95% intervals and, beneath, the paired change from the untrained model (5,000-draw item bootstrap). Changes are computed from unrounded accuracies, so they can differ by 0.001 from the difference of the rounded levels (as do the labels of Figure 4 (c)). Step-30 twin levels are from the evaluation of Table G.6 ; their changes are differences of rounded levels and carry no interval.

task	untrained	Real run 1, 30	Real run 2, 30	Gray -trained control, 30	Real run 1, 100
L1 cued readout	0.610 [0.525, 0.695]	0.820 [0.755, 0.880] $\Delta$ $+$ 0.210 [0.140, 0.285]	0.810 [0.745, 0.870] $\Delta$ $+$ 0.200 [0.130, 0.270]	0.640 [0.555, 0.720] $\Delta$ $+$ 0.030 [ $-$ 0.010, 0.075]	0.890 [0.835, 0.935] $\Delta$ $+$ 0.280 [0.200, 0.360]
L2 find-and-bind	0.620 [0.530, 0.705]	0.900 [0.850, 0.945] $\Delta$ $+$ 0.280 [0.200, 0.360]	0.875 [0.815, 0.930] $\Delta$ $+$ 0.255 [0.180, 0.330]	0.680 [0.600, 0.760] $\Delta$ $+$ 0.060 [0.020, 0.105]	0.950 [0.915, 0.980] $\Delta$ $+$ 0.330 [0.250, 0.415]
L3 discovery	0.425 [0.335, 0.515]	0.740 [0.665, 0.815] $\Delta$ $+$ 0.315 [0.235, 0.400]	0.720 [0.645, 0.795] $\Delta$ $+$ 0.295 [0.215, 0.375]	0.385 [0.300, 0.475] $\Delta$ $-$ 0.040 [ $-$ 0.085, 0.000]	0.875 [0.815, 0.930] $\Delta$ $+$ 0.450 [0.365, 0.540]
identification probe	0.825 [0.755, 0.890]	0.975 [0.940, 1.000] $\Delta$ $+$ 0.150 [0.090, 0.215]	0.975 [0.940, 1.000] $\Delta$ $+$ 0.150 [0.090, 0.215]	0.850 [0.785, 0.905] $\Delta$ $+$ 0.025 [0.000, 0.050]	0.985 [0.960, 1.000] $\Delta$ $+$ 0.160 [0.100, 0.225]

Appendix

Table G.5: The same-scene hierarchy on the 20-point confirmatory scenes, a density never trained (100 scene programs in the top block): member accuracy with real images with its 95% interval over scene programs, its change from the untrained model, and, for pairs whose answer changes between the twins, the number of scene programs with both members correct. Top: the four tasks on the target-stable and invariance pairs, the population of the discovery result (200 members). Bottom: pair roles separately (100 members each); the cued levels have no target-switch pairs. L1 and L2 ask the same question, and L1 adds only the arrow; L1, L2 and the 20-point density were never trained, while the probe was.

seed	condition	discovery, confirmatory	discovery, untouched	grounding task	twin	target-switch, conf.	target-switch, untouched	find-and-bind
–	untrained 7B	0.425 (85/200)	0.471 (628/1,332)	0.768	0.728	13/50	0.318	0.620
0	Real , run 1	0.740	0.791	0.930	0.900	30/50	0.586	0.900
0	Real , run 2 (same seed)	0.720	0.770	0.935	0.905	27/50	0.562	0.875
0	Gray -trained	0.385	0.440	0.770	0.730	12/50	0.252	0.680
1	Real	0.760 (152/200)	0.792 (1,055/1,332)	0.907	0.885	30/50	0.583	0.915
1	Real , step 100	0.895 (179/200)	0.900 (1,199/1,332)	0.942	0.925	–	–	0.960

Appendix

Table G.6: Seed replication (§§ 6.1 – 6.3 ): seeds 1–3 of the Real , Gray -trained and named-target conditions at step 30, and the Real run of seed 1 continued to step 100, beside seed 0 (Tables G.2 – G.5 ). Discovery member accuracy on the 20-point confirmatory set (200 members) and on the untouched set (1,332 members); pair accuracy on the grounding task and its twin; target-switch discovery with both members correct on the confirmatory (of 50 pairs) and untouched (of 333) sets; find-and-bind (200 members) on the confirmatory scenes. Counts are given where recorded.

quantity	endpoint	seeds	per-seed values	mean	seed SD	range	seed-level 95% interval
Real level	discovery, confirmatory	0–3	0.740, 0.760, 0.730, 0.755	0.746	0.014	0.030	–
Real $-$ Gray	discovery, confirmatory	0–3	$+$ 0.355, $+$ 0.345, $+$ 0.380, $+$ 0.335	$+$ 0.354	0.019	0.045	[ $+$ 0.323, $+$ 0.384]
Real $-$ Gray	discovery, untouched	0–3	$+$ 0.351, $+$ 0.341, $+$ 0.355, $+$ 0.309	$+$ 0.339	0.021	0.046	[ $+$ 0.306, $+$ 0.372]
Real $-$ untrained	discovery, confirmatory	0–3	$+$ 0.315, $+$ 0.335, $+$ 0.305, $+$ 0.330	$+$ 0.321	0.014	–	–
Real $-$ untrained	discovery, untouched	0–3	$+$ 0.320, $+$ 0.321, $+$ 0.297, $+$ 0.291	$+$ 0.307	0.015	–	–
Gray $-$ untrained	discovery, confirmatory	0–3	$-$ 0.040, $-$ 0.010, $-$ 0.075, $-$ 0.005	$-$ 0.033	0.032	–	–

Appendix

Table G.7: Levels and contrasts over seeds at step 30 (seed 0 uses Real run 1): per-seed values, their mean and sample standard deviation, and, for the contrasts used as seed-level claims in §§ 6.1 – 6.3 , the range and the

t

-based 95% interval over seeds (Appendix I ).

subset	members	untrained	Real	Gray -trained
probe correct	165	0.485 (80/165)	0.764 (126/165)	0.436 (72/165)
probe wrong	35	0.143 (5/35)	0.629 (22/35)	0.143 (5/35)
all	200	0.425 (85/200)	0.740 (148/200)	0.385 (77/200)

Appendix

Table G.8: Step-30 discovery on the 20-point confirmatory set (200 members; seed 0, Real run 1 and the Gray -trained control), by whether the untrained model’s identification probe for the same member is correct.

seed	test input	discovery, confirmatory	discovery, untouched	grounding task	gray test images
1	image only	0.550 (110/200)	0.589 (784/1,332)	0.820 (492/600)	0/200
2	image only	0.515 (103/200)	0.553 (737/1,332)	0.802 (481/600)	0/200
3	image only	0.570 (114/200)	0.595 (793/1,332)	0.812 (487/600)	0/200
1	image and caption	0.645 (129/200)	0.669 (891/1,332)	–	–
2	image and caption	0.610 (122/200)	0.646 (861/1,332)	–	–
3	image and caption	0.655 (131/200)	0.682 (908/1,332)	–	–

Appendix

Table G.9: The shortcut-accessible twin at step 30, seeds 1–3, tested with the image alone (the paper’s evaluation) and with the image and its caption (the training input): discovery member accuracy on the confirmatory (200) and untouched (1,332) sets, pair accuracy on the grounding task (600) and discovery with gray test images. Real and Gray -trained seeds 1–3 and the untrained model are in Table G.6 .

quantity	endpoint	per-seed values (1, 2, 3)	mean	seed SD	seed-level 95% interval	Real reference
shortcut level	discovery, confirmatory	0.550, 0.515, 0.570	0.545	0.028	[0.476, 0.614]	0.760, 0.730, 0.755
shortcut $-$ untrained	discovery, confirmatory	$+$ 0.125, $+$ 0.090, $+$ 0.145	$+$ 0.120	0.028	[ $+$ 0.051, $+$ 0.189]	$+$ 0.335, $+$ 0.305, $+$ 0.330
Real $-$ shortcut	discovery, confirmatory	$+$ 0.210, $+$ 0.215, $+$ 0.185	$+$ 0.203	0.016	[ $+$ 0.163, $+$ 0.243]	Real $-$ Gray : $+$ 0.345, $+$ 0.380, $+$ 0.335
shortcut level	discovery, untouched	0.589, 0.553, 0.595	0.579	0.023	[0.523, 0.635]	0.792, 0.769, 0.762
Real $-$ shortcut	discovery, untouched	$+$ 0.203, $+$ 0.215, $+$ 0.167	$+$ 0.195	0.025	[ $+$ 0.132, $+$ 0.258]	Real $-$ Gray : $+$ 0.341, $+$ 0.355, $+$ 0.309
shortcut $-$ untrained	grounding task	$+$ 0.052, $+$ 0.033, $+$ 0.043	$+$ 0.043	0.009	[ $+$ 0.020, $+$ 0.066]	$+$ 0.139, $+$ 0.154, $+$ 0.125

Appendix

Table G.10: Contrasts over seeds for the shortcut-accessible twin (image only), in the format of Table G.7 ; Real values from Table G.7 for comparison.

$f$	$m(f)$	$\widetilde{m}(f)$	in-band	8-pt: level; gain [95% CI]	12-pt	20-pt (never trained)	grounding [95% CI]
0	0.126	0.094	0.752	0.740 $+$ 0.080 [0.030, 0.135]	0.570 $-$ 0.005 [ $-$ 0.035, 0.030]	0.460 $-$ 0.010 [ $-$ 0.040, 0.025]	0.797 [0.763, 0.828]
$\tfrac{1}{3}$	0.165	0.229	0.900	0.935 $+$ 0.275 [0.200, 0.355]	0.805 $+$ 0.230 [0.155, 0.305]	0.745 $+$ 0.275 [0.205, 0.350]	0.908 [0.885, 0.930]
$\tfrac{2}{3}$	0.199	0.240	0.926	0.950 $+$ 0.290 [0.215, 0.370]	0.850 $+$ 0.275 [0.195, 0.360]	0.740 $+$ 0.270 [0.200, 0.345]	0.927 [0.905, 0.947]
1	0.249	0.251	–	0.965 $+$ 0.305 [0.230, 0.380]	0.890 $+$ 0.315 [0.230, 0.400]	0.800 $+$ 0.330 [0.255, 0.410]	0.930 [0.908, 0.950]

Appendix

Table H.1: Dose response at step 30 by constructed fraction

f

(same 7B starting model, reward, recipe and 7,200 prompt presentations in all four conditions): audited mass

m(f)

(Eq. ( 3 )), standardized mass

\widetilde{m}(f)

(Appendix J.3 ) and the share of audit-joined training rows with

0<b_{i}<1

(in-band); per density, discovery accuracy with real images and its gain over the untrained model with a 95% interval from resampling scene programs; and grounding-task pair accuracy. With a gray canvas, discovery accuracy is 0 of 200 in every cell of every mixture.

endpoint	slope on $m(f)$ [95% CI]	slope on $\widetilde{m}(f)$ [95% CI]
8-pt discovery	1.670 [1.187, 2.190]	1.441 [1.005, 1.907]
12-pt discovery	2.446 [1.813, 3.088]	1.942 [1.432, 2.471]
20-pt discovery (never trained)	2.482 [1.886, 3.083]	2.079 [1.565, 2.596]
grounding task	1.009 [0.787, 1.233]	0.861 [0.669, 1.054]

Appendix

Table H.2: Audit association, equal-exposure and direct comparisons for the mixtures. Top: ordinary least-squares slope (with intercept, over the four conditions) of step-30 held-out accuracy on the audited mass

m(f)

and on the standardized mass

\widetilde{m}(f)

(Appendix J.3 ); 95% intervals resample scene programs (grounding pairs for the grounding task) with the audited masses held fixed (Appendix I ). Middle: equal constructed-prompt exposure. Discovery accuracy of a mixture at step 30 and of the fully constructed run at the step where it had seen the same number of constructed prompt presentations (five responses each), with their difference and its 95% interval. The two runs of each comparison differ in total presentations, so these comparisons match constructed exposure, not total training. Bottom: direct comparisons of the mixtures at step 30.

model	step	density	level [95% CI]	$\Delta$ vs untrained [95% CI]	real	gray canvas	gray canvas	grounding task [95% CI]
			discovery, real images		identification probe		discovery
3B	0	8-pt	0.330 [0.255, 0.405]	–	0.705	0.000	0.080	0.455 [0.415, 0.495]
		12-pt	0.260 [0.190, 0.340]	–	0.680	0.000	0.025
		20-pt	0.245 [0.175, 0.320]	–	0.630	0.000	0.055
	10	8-pt	0.475 [0.395, 0.555]	$+$ 0.145 [0.065, 0.225]	0.875	0.000	0.080	–
		12-pt	0.425 [0.340, 0.510]	$+$ 0.165 [0.085, 0.245]	0.755	0.000	0.025

Appendix

Table H.3: Capacity (Figure 5 (b)): the 3B and 7B models trained on the same constructed corpus, per checkpoint and density (development set): discovery with real images and its change from the same model untrained, with 95% intervals from resampling scene programs; the identification probe with real images and with a gray canvas; discovery with a gray canvas; and grounding-task pair accuracy where evaluated. With a gray canvas the probe is exactly zero for both models at every checkpoint, while the 3B model’s discovery accuracy is small and nonzero at every checkpoint, including before training: it answers with a fixed coordinate that is sometimes correct.

benchmark	model	with image	no image	$c$	$\rho_{\mathrm{blind}}$
MathVista testmini	untrained 7B	0.682 (682/1,000)	0.366 (366/1,000)	0.179	0.372
MathVista testmini	$f=1$ , step 30	0.532 (532/1,000)	0.258 (258/1,000)	0.179	0.224
MathVista testmini	$f=\tfrac{1}{3}$ , step 30	0.696 (696/1,000)	0.367 (367/1,000)	0.179	0.364
BLINK	untrained 7B	0.536 (1,018/1,901)	0.416 (791/1,901)	0.377	0.247
BLINK	$f=1$ , step 30	0.457 (869/1,901)	0.363 (690/1,901)	0.377	$-$ 0.175
BLINK	$f=\tfrac{1}{3}$ , step 30	0.547 (1,039/1,901)	0.420 (798/1,901)	0.377	0.252

Appendix

Table H.4: The untrained 7B model and the step-30

f=1

and

f=\tfrac{1}{3}

checkpoints on MathVista testmini (1,000 items, 539 of them multiple-choice) and BLINK (1,901 multiple-choice items), with and without the image, with the chance reference

c

and the chance-corrected blind retention

\rho_{\mathrm{blind}}

of § 2.2 . Retention is reported unclipped; without the image the

f=1

model answers BLINK below chance, so its retention is negative.

condition	seed	rows	step-1 accuracy	step-1 format	mixed, step 1	nonzero advantage, step 1	mixed, steps 1–5	nonzero advantage, steps 1–5	all correct, step 1	all wrong, step 1
Real	1	discovery	0.223	0.857	0.633	0.808	0.645	0.698	0.008	0.358
Real	1	probe	0.435	0.875	0.892	0.933	0.612	0.622	0.025	0.083
Real	2	discovery	0.178	0.883	0.575	0.800	0.632	0.690	0.008	0.417
Real	2	probe	0.407	0.892	0.875	0.942	0.598	0.610	0.017	0.108
Real	3	discovery	0.245	0.875	0.667	0.842	0.655	0.718	0.008	0.325
Real	3	probe	0.428	0.890	0.892	0.933	0.622	0.637	0.025	0.083

Appendix

Table J.1: Reward variation within response groups at the start of training, for the runs of Table G.6 , by training-row type (discovery, the identification probe, or find-and-bind for the named-target corpus). Columns: mean accuracy and format components at step 1; the fraction of groups containing both a correct and an incorrect response (“mixed”, the event of Eq. ( J.2 )); the fraction of groups with any within-group reward variation (“nonzero advantage”); the same two fractions averaged over steps 1–5; and the step-1 fractions of all-correct and all-wrong groups.

Figure K.1: Eight of the nine task families: coordinate scenes (top) and other formats (bottom).

component	scale
constructed evaluations	a 1,200-pair suite and its regenerated 1,200-pair twin; discovery scenes at 8, 12 and 20 points, with 450 confirmatory scene programs; an untouched replication set of 333 scene programs per pair role (Table G.6 ); 160 premise groups (an untrained nearest-point relation task); 100 label-swap diagnostic pairs
constructed training data	2,880 discovery and probe prompts; a second corpus of 3,000 pairs and 300 answer-preserving twins
public image corpora	Geometry3K (1,288 training, 601 test items); ViRL39K (23,542 training, 4,239 held-out items)
public benchmarks	seven, more than 10,000 items, audited with and without the image at 3B and 7B (3B: Table A.1 )
models	Qwen2.5-VL-3B and 7B trained; Gemma-3 and InternVL3-9B audited (Table A.2 )
training runs	over thirty GRPO runs of 30 to 400 steps, one to five runs per condition (main-text runs: Table D.1 )

Appendix

Table K.1: The experimental program in numbers.

Explore similar work

Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR
May 29, 2026Ruina Hu, Chen Wang, Lai Wei +5Multimodal Reasoning Benchmarks Spatial Supervision
1Harbin Institute of Technology · 2Zhongguancun Academy · 4Nankai University +4
Improving Generalization Robustness of Multimodal RLVR
Aug 9, 2026Pengfei Zhou, Zhiwei Tang, Xiaopeng Peng +11Reinforcement Learning With Verifiable Reward Verifiable Rewards
1National University of Singapore, HPC-AI Lab · 2DAMO Academy, Alibaba Group · 3Hupan Lab +7
MIRL: Mutual Information-Guided Reinforcement Learning for Vision-Language Models
May 2, 2026Yin Zhang, Jiaxuan Zhao, Zonghan Wu +5Recent Vision-Language Models Reinforcement Learning With Verifiable Reward
School of Mathematics, Tianjin University, Tianjin, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Shanghai Advanced Institute of Finance (SAIFS), East China Normal University, Shanghai, China +4

May 29, 2026, cs.CVJ/K move · Enter open · S save

Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR

Ruina Hu, Chen Wang, Lai Wei, Jionghao Bai+4

1Harbin Institute of Technology · 2Zhongguancun Academy · 4Nankai University+4

Multimodal Reasoning Benchmarks Spatial Supervision Reinforcement Learning With Verifiable Reward Recent Vision-Language Models Verifiable Rewards

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards (RLVR) improves vision-language models (VLMs) by optimizing outcome rewards derived from final answers. However, such outcome-only rewards do not tell the model which image regions justify an answer. For questions that require visual grounding, these rewards cannot distinguish responses supported by relevant visual evidence from those produced by language-prior shortcuts or lucky guesses. We introduce EASE (Evidence-Anchored Spatial Attention), which augments multimodal RLVR with visual-evidence process supervision. EASE converts annotated evidence regions into a smoothed visual-token target and uses it to guide response-to-image attention during RL training, but only on high-reward trajectories. The annotations are used solely as privileged training labels, while inference requires only the original image and question. Across Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3-VL-8B, EASE raises average scores over DAPO by 2.5 to 3.1 points on perception, hallucination, visual math, and multimodal reasoning benchmarks. Diagnostics and ablations show that EASE better aligns visual attention with annotated evidence regions.

Abstract

Explore similar work

Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR

Improving Generalization Robustness of Multimodal RLVR

MIRL: Mutual Information-Guided Reinforcement Learning for Vision-Language Models