Vision-language verifiers judge whether an image faithfully matches a textual description, yet existing training pipelines typically rely on static supervision from stronger external judges or curated hard-negative data. Such supervision requires external generation or annotation, is teacher-bounded, and is difficult to adapt as model failures evolve. We introduce PROVE (Perturb-and-Repair Optimization for Vision-Language Evaluation), a self-play framework that trains verifiers through a closed perturb-verify-repair-verify loop. Starting from a known-correct image-caption pair, a Mutator introduces a typed perturbation over objects, attributes, counts, or relations. The verifier then judges the perturbed caption and provides textual feedback, which a Repairer uses to recover a faithful caption for re-verification. These two verification rounds produce structured rewards for informative mismatches and visually grounded repairs. Since perturbation, verification, and repair are role-conditioned behaviors of the same evolving multimodal backbone, updating the shared parameters also improves the verifier itself. Experiments show that PROVE improves fine-grained compositional verification across model scales and families and strengthens visual-detail robustness on the reference backbone, with limited, benchmark-dependent changes on broader multimodal tasks.
Figures & tables
Figure 1: Motivation of PROVE . (a) Fine-grained image-text verification requires detecting localized object, attribute, count, and relation mismatches. (b) Existing verifier training relies on static external supervision, while (c) PROVE creates adaptive supervision through a perturb–verify–repair–verify self-play loop.
Figure 2: Overview of PROVE . A single Qwen backbone is reused under three role prompts: Mutator, Verifier, and Repairer. Given a clean image-caption pair, the Mutator produces a localized caption perturbation, e.g., changing an attribute from “red” to “blue”, to create a hard image-text mismatch. The Verifier first judges the perturbed caption and returns a decision-feedback pair (y1,r1) that diagnoses the mismatch. Conditioned on this feedback, the Repairer generates a revised caption, which is then checked by the same Verifier to obtain (y2,r2) . Rewards from the two verification rounds train the shared model with DrGRPO, so perturbation, verification, and repair co-evolve within one multimodal backbone.
Model
ARO
VL-CheckList
ColorSwap
FG-OVD
Overall
Rel
Attr
Avg
Attr
Obj
Rel
Avg
ITT
Avg
Avg
Contextual comparison: discriminative VLMs
CLIP [ 32 ]
59.00
62.00
60.50
67.93
82.83
64.19
71.65
35.67
47.33
53.79
NegCLIP [ 44 ]
81.00
71.00
76.00
72.24
87.00
71.39
76.87
35.67
41.69
57.56
GNM [ 33 ]
65.00
65.00
65.00
70.15
85.91
64.10
73.38
13.00
38.79
47.54
PANeg [ 2 ]
65.07
67.94
66.51
76.51
88.13
69.90
78.17
17.67
44.98
51.83
Table 1: Comparison with compositional understanding methods. We report all available sub-metrics on ARO, VL-CheckList, ColorSwap, and FG-OVD. “–” indicates that the benchmark was not evaluated. Best and second-best results are shown in bold and underline . PROVE achieves the best overall average among methods with complete benchmark coverage and consistently improves over the Qwen3.5-2B backbone across all benchmark groups.
Model
Wino GS
MMVP Pair
MMBench
Avg
Discriminative VLMs
CLIP [ 32 ]
8.00
15.55
–
–
NegCLIP [ 44 ]
8.00
26.67
–
–
PANeg [ 2 ]
6.75
21.48
–
–
DAC-SAM [ 7 ]
6.25
23.70
–
–
DAC-LLM [ 7 ]
3.25
28.14
–
–
Table 2: Comparison on broader vision-language benchmarks. PROVE achieves the best average and improves the Qwen3.5-2B backbone on all three benchmarks, indicating transfer beyond the core compositional suites. Broader task changes are benchmark-dependent (Appendix C.3 ).
Method / variant
Comp. Avg.
Broader Avg.
Qwen3.5-2B
83.22
58.77
Fixed typed negatives
85.02
59.68
External-judge distillation
86.29
60.72
Verifier self-training
85.66
60.21
No Repairer or second verification
84.24
59.31
Repairer without verifier feedback
85.17
59.84
Table 3: Matched supervision and core loop ablations. All training variants use Qwen3.5-2B, 48k source pairs, and the same target-model optimization budget and evaluation protocol. Complete ablations are in Appendix B .
Figure 3: Self-play dynamics and axis-level gains. (a) The Repairer steadily improves in repair success and structural alignment, while the verifier pass rate rises as repairs catch up with the current Mutator distribution; the dashed line marks the lower finite-group maximum-variance reference at 1/3 (Appendix F ). (b) Each panel compares Qwen3.5-2B with PROVE on an aggregated visual-semantic capability; Total is the overall average in Table 1 . The largest gains appear on attribute and relation verification, matching the typed perturbation axes used during self-play.
Comp. Avg.
Broader Avg.
Backbone
Base
PROVE
Base
PROVE
Qwen3.5-2B
83.22
86.85
58.77
61.03
Qwen3.5-4B
85.15
88.71
60.92
63.26
InternVL3-2B
82.19
85.34
58.16
60.99
Table 4: Transfer across model scale and family. The same perturbation and reward formulation improves both evaluation averages.
Figure 4: Qualitative verifier examples. Each panel compares the base Qwen3.5-2B model with PROVE on a fine-grained verification case. The examples cover relation, object existence, attribute, and counting errors. PROVE better localizes the relevant visual evidence and avoids accepting captions or answers that miss the targeted mismatch.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Backbone
Qwen3.5-2B
Training data
48k image-caption pairs
Validation data
5k image-caption pairs
max prompt length / response length
4096 / 1024
rollout batch size
48
rollout size n
3
Appendix
Table 5: Training hyperparameters for the Qwen3.5-2B reference run.
Benchmark
Base
PROVE @450
PROVE @600
PROVE @750
Winoground (2-AFC)
69.50
74.50
74.75
75.62
ARO macro
73.57
78.01
78.27
78.66
SugarCrepe micro
93.76
94.24
94.28
94.17
VL-CheckList 2-axis avg
79.80
83.58
83.87
84.20
POPE F1
89.57
89.52
89.58
89.66
MMVP Pair Acc.
56.00
59.33
57.33
60.00
Appendix
Table 6: Extended checkpoint results for Qwen3.5-2B. PROVE @450, PROVE @600, and PROVE @750 denote checkpoints at steps 450, 600, and 750, respectively.
Metric
Base
PROVE @450
PROVE @600
PROVE @750
Attribute axis
76.75
80.72
81.13
81.46
Object axis
91.20
–
–
94.14
Relation axis
82.85
86.45
86.60
86.95
3-axis average
83.60
–
–
87.52
Appendix
Table 7: VL-CheckList axis-level results on Qwen3.5-2B.
Role
Component
Purpose
Mutator
Rexpose(y1)
Expose verifier blind spots
Mutator
Rhard(y2)
Prefer perturbations that remain challenging after repair
Mutator
CM(aM)
Penalize overly wide edits
Mutator
invalid-action penalty
Fixed penalty when the editor rejects the action
Repairer
Raccept(y2)
Reward repaired captions accepted by the verifier
Repairer
Rflip(y1,y2)
Reward a change in the verifier judgment
Appendix
Table 8: Verifier-guided reward components in PROVE .
Method
Supervision generation
Target training
Total GPU-hours
Generated tokens/source
Fixed typed negatives
4.8
43.2
48.0
46
External-judge distillation
27.6
45.3
72.9
214
PROVE
Online
88.0
88.0
1,126
Appendix
Table 9: Complete pipeline compute costs. Generation and training costs are GPU-hours. Online generation for PROVE is included in its 88.0 GPU-hours.
Training variant
Comp. Avg.
Broader Avg.
Qwen3.5-2B
83.22
58.77
Fixed typed negatives
85.02
59.68
No Repairer or second verification
84.24
59.31
Repairer without verifier feedback
85.17
59.84
GRPO instead of DrGRPO
86.42
60.71
Without Mutator reward
85.03
59.76
Appendix
Table 10: Complete component, optimizer, and reward ablations. All variants use the same source data, backbone initialization, target-model optimization budget, and evaluation protocol; the GRPO row changes only the optimization variant.
Role configuration
Comp. Avg.
Broader Avg.
Shared backbone, role prompts
86.85
61.03
Shared backbone, role-specific adapters
86.41
60.78
Frozen internal Verifier
85.12
59.81
Frozen external Verifier
86.07
60.54
Appendix
Table 11: Role configuration and verifier-update ablations.
External audit
Base
PROVE
Fixed-set verifier accuracy
75.8
83.6
Valid localized perturbations
82.4
92.1
Correct verifier judgments
71.5
84.7
Faithful minimal repairs
55.2
73.4
Fully valid trajectories
40.8
64.2
Appendix
Table 12: Complete external audit. All entries are percentages. Fixed-set verifier accuracy and trajectory-quality rates are separate audit measurements.
Checkpoint
Self-play pass rate
Comp. Avg.
Broader Avg.
Base
36.0
83.22
58.77
Step 150
39.3
84.08
59.18
Step 300
43.2
85.14
59.73
Step 450
46.3
85.97
60.41
Step 600
56.6
86.48
60.16
Step 750
63.3
86.85
61.03
Appendix
Table 13: Internal self-play pass rate and external benchmark averages across training. The pass rate is in percent; benchmark averages use the main-paper definitions.
Figure 5: External benchmark performance and internal self-play pass rate. Each marker is one reported checkpoint; connecting lines guide the eye. Gains in the left panel are measured against the base checkpoint. The right panel reports internal pass rate separately, preserving the distinction between independent accuracy and role agreement.
Benchmark
Base
PROVE (mean ± std)
Change
ViVerBench
55.16
53.12±0.58
−2.04
MMMU
40.78
40.39±0.47
−0.39
MathVista
73.80
72.94±0.52
−0.86
MMBench
71.31
72.26±0.31
+0.95
POPE
89.57
89.64±0.09
+0.07
Core compositional average
83.22
86.73±0.27
+3.51
Appendix
Table 14: Results across three independent Qwen3.5-2B training runs. Change is the mean PROVE result minus the base score.
Held-out type
Base
Leave-one-out PROVE
Full PROVE
Retained gain
Object
86.1
87.2
88.2
52.4%
Attribute
78.1
81.1
84.1
50.0%
Relation
76.3
79.0
81.9
48.2%
Appendix
Table 15: Leave-one-type-out transfer on the three reported axes. The held-out axis is absent from the training action space.
Curriculum
Comp. Avg.
Broader Avg.
Repair success
One edit
86.85
61.03
62.7
Mixture of one/two edits
86.06
61.18
57.4
Two edits
85.72
60.31
48.6
Appendix
Table 16: Perturbation curriculum comparison. Repair success is a percentage.
Caption quality
Fraction
Fully faithful
90.1%
Minor unsupported detail or ambiguity
8.2%
Material image-caption mismatch
1.7%
Appendix
Table 17: Independent audit of a random subset of the positive-caption pool.
Training pool
Comp. Avg.
Broader Avg.
Original 48k
86.85
61.03
Filtered positives
87.08
61.15
Appendix
Table 18: Training after filtering low-confidence positive captions.
Weight
Value
Term
λ1
1.0
Perturbation not rejected by the first verification
Visual outcomes are increasingly central to multimodal large language models, making reliable and fine-grained verification essential for scaling generalist foundation models. In this work, we investigate multimodal meta-verification, which leverages verifier-generated rationales rather than decision-only signals, and explore how to effectively incorporate meta-verification feedback into multimodal verifier training. We identify two key findings. First, symbolic verifier outputs (e.g., bounding boxes) outperform textual explanations as meta-verification rationales, enabling efficient rule-based reinforcement learning rewards while avoiding reliance on model-based rewards from auxiliary judge models. Second, decoupling reinforcement learning objectives for binary judgment and meta-verification substantially outperforms joint reward optimization, due to intrinsic differences in output structure and learning dynamics. Based on these insights, we train OmniVerifier-M1, a generalist visual verifier leveraging symbolic meta-verification and decoupled reinforcement learning. OmniVerifier-M1 provides robust verification and fine-grained error localization, and further enables M1-TTS, a verifier-driven agentic generation system achieving dynamic region-level self-correction. This approach paves the way for more reliable, interpretable, and fine-grained multimodal verification, supporting safer and more controllable foundation model deployment.
Xinchen Zhang, Bowei Liu, Jiale Liu +7
Tsinghua University · Pennsylvania State University · University of Southern California +2
Vision-language models (VLMs) have achieved strong performance in multimodal reasoning, yet they remain prone to generating plausible but incorrect answers. Self-verification offers a practical way to improve answer reliability without relying on external judges, but existing methods typically depend on a single verification criterion or fixed prompt, resulting in incomplete and unstable reliability estimates. We first systematically analyze how verifier capability and prompt design affect verification performance. Our findings show that stronger verifiers provide more reliable judgments, while verification performance is highly sensitive to prompt choice, with no single prompt consistently dominating across tasks. Guided by these findings, we propose MOTIVE, a Multi-View Self-VerificatiOn wiTh RelIability-Guided SelectiVE Rethinking framework for reliable multimodal reasoning. MOTIVE evaluates each candidate answer from complementary verification perspectives and learns a correctness-aligned reliability score through correctness-grounded multi-view verification learning. During inference, this score governs an accept-or-rethink decision, allowing reliable answers to be returned directly while uncertain ones trigger history-guided rethinking. Extensive experiments across diverse multimodal benchmarks and VLM backbones demonstrate that MOTIVE consistently outperforms strong self-verification and self-correction baselines. Further results show that reliable verification improves accept-or-rethink decisions and reduces unnecessary reasoning turns, enabling more reliable and efficient self-verification without an external judge.
Ziquan Zhu, Hanruo Zhu, Si-Yuan Lu +10
University of Leicester · Nanjing University of Posts and Telecommunications · University of North Carolina at Chapel Hill +5
Achieving human-like reasoning in Vision-Language Models (VLMs) remains a long-standing challenge. Recent approaches leverage Chain-of-Thought (CoT) rationales generated by human annotators or proprietary models, which are costly and difficult to scale. Self-training offers a promising alternative but often suffers from visual hallucinations and language shortcuts because rationales are filtered only by answer correctness without verifying visual perception. We propose a perception-verified self-training framework that enforces visually grounded reasoning. Our method employs a CoT template (caption-reasoning-conclusion) that disentangles perception from reasoning, enabling independent verification of visual understanding. To compensate for the absence of ground-truth captions, we introduce PerceptEval, an unsupervised method that evaluates caption quality based on its alignment with visual and textual elements in the image. Using caption verification together with answer correctness, we partition the data into easy, medium, and hard subsets and design a two-stage curriculum learning strategy. Stage 1 trains on easy samples, while Stage 2 enhances medium samples by regenerating reasoning conditioned on verified captions and retaining only those with correct conclusions. This ensures training is performed exclusively on perceptually grounded reasoning, reducing hallucinations and language shortcuts. Extensive experiments across diverse domains and models demonstrate improvements of up to 16% over standard self-training baselines, showing that our framework provides a scalable and cost-effective solution for advancing multimodal reasoning without manually annotated CoT rationales.
Sourabh Sharma, Sonam Gupta, Sadbhawna
Malaviya National Institute of Technology Jaipur, India · IBM Software Innovation Lab, India