Perturb, Repair, Verify: Self-Play Vision-Language Verifiers for Compositional Understanding

Zijun Huang; Yingjun Du; Cees G. M. Snoek

cs.CVNov 3, 2026

Perturb, Repair, Verify: Self-Play Vision-Language Verifiers for Compositional Understanding

Authors: Zijun Huang, Yingjun Du, Cees G. M. Snoek

Organizations: University of Southern California · VisLab, University of Amsterdam

Abstract

Vision-language verifiers judge whether an image faithfully matches a textual description, yet existing training pipelines typically rely on static supervision from stronger external judges or curated hard-negative data. Such supervision requires external generation or annotation, is teacher-bounded, and is difficult to adapt as model failures evolve. We introduce PROVE (Perturb-and-Repair Optimization for Vision-Language Evaluation), a self-play framework that trains verifiers through a closed perturb-verify-repair-verify loop. Starting from a known-correct image-caption pair, a Mutator introduces a typed perturbation over objects, attributes, counts, or relations. The verifier then judges the perturbed caption and provides textual feedback, which a Repairer uses to recover a faithful caption for re-verification. These two verification rounds produce structured rewards for informative mismatches and visually grounded repairs. Since perturbation, verification, and repair are role-conditioned behaviors of the same evolving multimodal backbone, updating the shared parameters also improves the verifier itself. Experiments show that PROVE improves fine-grained compositional verification across model scales and families and strengthens visual-detail robustness on the reference backbone, with limited, benchmark-dependent changes on broader multimodal tasks.

Figures & tables

Panel of Figure 1 — Figure 1: Motivation of PROVE . (a) Fine-grained image-text verification requires detecting localized object, attribute, count, and relation mismatches. (b) Existing verifier training relies on static external supervision, while (c) PROVE creates adaptive supervision through a perturb–verify–repair–verify self-play loop.

Figure 2: Overview of PROVE . A single Qwen backbone is reused under three role prompts: Mutator, Verifier, and Repairer. Given a clean image-caption pair, the Mutator produces a localized caption perturbation, e.g., changing an attribute from “red” to “blue”, to create a hard image-text mismatch. The Verifier first judges the perturbed caption and returns a decision-feedback pair $(y_{1},r_{1})$ that diagnoses the mismatch. Conditioned on this feedback, the Repairer generates a revised caption, which is then checked by the same Verifier to obtain $(y_{2},r_{2})$ . Rewards from the two verification rounds train the shared model with DrGRPO, so perturbation, verification, and repair co-evolve within one multimodal backbone.

Model	ARO			VL-CheckList				ColorSwap	FG-OVD	Overall
	Rel	Attr	Avg	Attr	Obj	Rel	Avg	ITT	Avg	Avg
Contextual comparison: discriminative VLMs
CLIP [ 32 ]	59.00	62.00	60.50	67.93	82.83	64.19	71.65	35.67	47.33	53.79
NegCLIP [ 44 ]	81.00	71.00	76.00	72.24	87.00	71.39	76.87	35.67	41.69	57.56
GNM [ 33 ]	65.00	65.00	65.00	70.15	85.91	64.10	73.38	13.00	38.79	47.54
PANeg [ 2 ]	65.07	67.94	66.51	76.51	88.13	69.90	78.17	17.67	44.98	51.83

Table 1: Comparison with compositional understanding methods. We report all available sub-metrics on ARO, VL-CheckList, ColorSwap, and FG-OVD. “–” indicates that the benchmark was not evaluated. Best and second-best results are shown in bold and underline . PROVE achieves the best overall average among methods with complete benchmark coverage and consistently improves over the Qwen3.5-2B backbone across all benchmark groups.

Model	Wino GS	MMVP Pair	MMBench	Avg
Discriminative VLMs
CLIP [ 32 ]	8.00	15.55	–	–
NegCLIP [ 44 ]	8.00	26.67	–	–
PANeg [ 2 ]	6.75	21.48	–	–
DAC-SAM [ 7 ]	6.25	23.70	–	–
DAC-LLM [ 7 ]	3.25	28.14	–	–

Table 2: Comparison on broader vision-language benchmarks. PROVE achieves the best average and improves the Qwen3.5-2B backbone on all three benchmarks, indicating transfer beyond the core compositional suites. Broader task changes are benchmark-dependent (Appendix C.3 ).

Method / variant	Comp. Avg.	Broader Avg.
Qwen3.5-2B	83.22	58.77
Fixed typed negatives	85.02	59.68
External-judge distillation	86.29	60.72
Verifier self-training	85.66	60.21
No Repairer or second verification	84.24	59.31
Repairer without verifier feedback	85.17	59.84

Table 3: Matched supervision and core loop ablations. All training variants use Qwen3.5-2B, 48k source pairs, and the same target-model optimization budget and evaluation protocol. Complete ablations are in Appendix B .

Figure 3: Self-play dynamics and axis-level gains. (a) The Repairer steadily improves in repair success and structural alignment, while the verifier pass rate rises as repairs catch up with the current Mutator distribution; the dashed line marks the lower finite-group maximum-variance reference at

1/3

(Appendix F ). (b) Each panel compares Qwen3.5-2B with PROVE on an aggregated visual-semantic capability; Total is the overall average in Table 1 . The largest gains appear on attribute and relation verification, matching the typed perturbation axes used during self-play.

	Comp. Avg.		Broader Avg.
Backbone	Base	PROVE	Base	PROVE
Qwen3.5-2B	83.22	86.85	58.77	61.03
Qwen3.5-4B	85.15	88.71	60.92	63.26
InternVL3-2B	82.19	85.34	58.16	60.99

Table 4: Transfer across model scale and family. The same perturbation and reward formulation improves both evaluation averages.

Figure 4: Qualitative verifier examples. Each panel compares the base Qwen3.5-2B model with PROVE on a fine-grained verification case. The examples cover relation, object existence, attribute, and counting errors. PROVE better localizes the relevant visual evidence and avoids accepting captions or answers that miss the targeted mismatch.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Backbone	Qwen3.5-2B
Training data	48k image-caption pairs
Validation data	5k image-caption pairs
max prompt length / response length	4096 / 1024
rollout batch size	48
rollout size $n$	3

Appendix

Table 5: Training hyperparameters for the Qwen3.5-2B reference run.

Benchmark	Base	PROVE @450	PROVE @600	PROVE @750
Winoground (2-AFC)	69.50	74.50	74.75	75.62
ARO macro	73.57	78.01	78.27	78.66
SugarCrepe micro	93.76	94.24	94.28	94.17
VL-CheckList 2-axis avg	79.80	83.58	83.87	84.20
POPE F1	89.57	89.52	89.58	89.66
MMVP Pair Acc.	56.00	59.33	57.33	60.00

Appendix

Table 6: Extended checkpoint results for Qwen3.5-2B. PROVE @450, PROVE @600, and PROVE @750 denote checkpoints at steps 450, 600, and 750, respectively.

Metric	Base	PROVE @450	PROVE @600	PROVE @750
Attribute axis	76.75	80.72	81.13	81.46
Object axis	91.20	–	–	94.14
Relation axis	82.85	86.45	86.60	86.95
3-axis average	83.60	–	–	87.52

Appendix

Table 7: VL-CheckList axis-level results on Qwen3.5-2B.

Role	Component	Purpose
Mutator	$R_{\mathrm{expose}}(y_{1})$	Expose verifier blind spots
Mutator	$R_{\mathrm{hard}}(y_{2})$	Prefer perturbations that remain challenging after repair
Mutator	$C_{M}(a_{M})$	Penalize overly wide edits
Mutator	invalid-action penalty	Fixed penalty when the editor rejects the action
Repairer	$R_{\mathrm{accept}}(y_{2})$	Reward repaired captions accepted by the verifier
Repairer	$R_{\mathrm{flip}}(y_{1},y_{2})$	Reward a change in the verifier judgment

Appendix

Table 8: Verifier-guided reward components in PROVE .

Method	Supervision generation	Target training	Total GPU-hours	Generated tokens/source
Fixed typed negatives	4.8	43.2	48.0	46
External-judge distillation	27.6	45.3	72.9	214
PROVE	Online	88.0	88.0	1,126

Appendix

Table 9: Complete pipeline compute costs. Generation and training costs are GPU-hours. Online generation for PROVE is included in its 88.0 GPU-hours.

Training variant	Comp. Avg.	Broader Avg.
Qwen3.5-2B	83.22	58.77
Fixed typed negatives	85.02	59.68
No Repairer or second verification	84.24	59.31
Repairer without verifier feedback	85.17	59.84
GRPO instead of DrGRPO	86.42	60.71
Without Mutator reward	85.03	59.76

Appendix

Table 10: Complete component, optimizer, and reward ablations. All variants use the same source data, backbone initialization, target-model optimization budget, and evaluation protocol; the GRPO row changes only the optimization variant.

Role configuration	Comp. Avg.	Broader Avg.
Shared backbone, role prompts	86.85	61.03
Shared backbone, role-specific adapters	86.41	60.78
Frozen internal Verifier	85.12	59.81
Frozen external Verifier	86.07	60.54

Appendix

Table 11: Role configuration and verifier-update ablations.

External audit	Base	PROVE
Fixed-set verifier accuracy	75.8	83.6
Valid localized perturbations	82.4	92.1
Correct verifier judgments	71.5	84.7
Faithful minimal repairs	55.2	73.4
Fully valid trajectories	40.8	64.2

Appendix

Table 12: Complete external audit. All entries are percentages. Fixed-set verifier accuracy and trajectory-quality rates are separate audit measurements.

Checkpoint	Self-play pass rate	Comp. Avg.	Broader Avg.
Base	36.0	83.22	58.77
Step 150	39.3	84.08	59.18
Step 300	43.2	85.14	59.73
Step 450	46.3	85.97	60.41
Step 600	56.6	86.48	60.16
Step 750	63.3	86.85	61.03

Appendix

Table 13: Internal self-play pass rate and external benchmark averages across training. The pass rate is in percent; benchmark averages use the main-paper definitions.

Figure 5: External benchmark performance and internal self-play pass rate. Each marker is one reported checkpoint; connecting lines guide the eye. Gains in the left panel are measured against the base checkpoint. The right panel reports internal pass rate separately, preserving the distinction between independent accuracy and role agreement.

Benchmark	Base	PROVE (mean $\pm$ std)	Change
ViVerBench	55.16	$53.12\pm 0.58$	$-2.04$
MMMU	40.78	$40.39\pm 0.47$	$-0.39$
MathVista	73.80	$72.94\pm 0.52$	$-0.86$
MMBench	71.31	$72.26\pm 0.31$	$+0.95$
POPE	89.57	$89.64\pm 0.09$	$+0.07$
Core compositional average	83.22	$86.73\pm 0.27$	$+3.51$

Appendix

Table 14: Results across three independent Qwen3.5-2B training runs. Change is the mean PROVE result minus the base score.

Held-out type	Base	Leave-one-out PROVE	Full PROVE	Retained gain
Object	86.1	87.2	88.2	52.4%
Attribute	78.1	81.1	84.1	50.0%
Relation	76.3	79.0	81.9	48.2%

Appendix

Table 15: Leave-one-type-out transfer on the three reported axes. The held-out axis is absent from the training action space.

Curriculum	Comp. Avg.	Broader Avg.	Repair success
One edit	86.85	61.03	62.7
Mixture of one/two edits	86.06	61.18	57.4
Two edits	85.72	60.31	48.6

Appendix

Table 16: Perturbation curriculum comparison. Repair success is a percentage.

Caption quality	Fraction
Fully faithful	90.1%
Minor unsupported detail or ambiguity	8.2%
Material image-caption mismatch	1.7%

Appendix

Table 17: Independent audit of a random subset of the positive-caption pool.

Training pool	Comp. Avg.	Broader Avg.
Original 48k	86.85	61.03
Filtered positives	87.08	61.15

Appendix

Table 18: Training after filtering low-confidence positive captions.

Weight	Value	Term
$\lambda_{1}$	1.0	Perturbation not rejected by the first verification
$\lambda_{2}$	0.5	Repaired caption still judged unfaithful
$\lambda_{3}$	0.3	Mutator edit cost $C_{M}$
$\beta_{1}$	0.8	Repair accepted by the second verification
$\beta_{2}$	0.7	Judgment flip between the two verifications
$\beta_{3}$	0.3	Repairer rewriting cost $C_{R}$

Appendix

Table 19: Reward weights in Equations 10 and 12 .

Compositional Reasoning in VLMs

See all 37 papers

OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration
May 27, 2026Xinchen Zhang, Bowei Liu, Jiale Liu +7Multimodal Large Language Models Reinforcement Learning with Verifiable Rewards
Tsinghua University · Pennsylvania State University · University of Southern California +2
When to Rethink: Learning Multi-Perspective Self-Verification for Vision-Language Models
Oct 4, 2026Ziquan Zhu, Hanruo Zhu, Si-Yuan Lu +10Self-Correction in VLMs VLM Reasoning
University of Leicester · Nanjing University of Posts and Telecommunications · University of North Carolina at Chapel Hill +5
Improving Reasoning in Vision-Language Models via Perception Verified Self-Training
Jun 20, 2026Sourabh Sharma, Sonam Gupta, SadbhawnaVLM Reasoning VLM Hallucination Mitigation
Malaviya National Institute of Technology Jaipur, India · IBM Software Innovation Lab, India

May 27, 2026J/K move · Enter open · S save

OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration

Xinchen Zhang, Bowei Liu, Jiale Liu, Chufan Shi+6

Tsinghua University · Pennsylvania State University · University of Southern California+2

Multimodal Large Language Models Reinforcement Learning with Verifiable Rewards Multimodal Foundation Models Multimodal Model Evaluation

Open paper AskPDF arXiv

Visual outcomes are increasingly central to multimodal large language models, making reliable and fine-grained verification essential for scaling generalist foundation models. In this work, we investigate multimodal meta-verification, which leverages verifier-generated rationales rather than decision-only signals, and explore how to effectively incorporate meta-verification feedback into multimodal verifier training. We identify two key findings. First, symbolic verifier outputs (e.g., bounding boxes) outperform textual explanations as meta-verification rationales, enabling efficient rule-based reinforcement learning rewards while avoiding reliance on model-based rewards from auxiliary judge models. Second, decoupling reinforcement learning objectives for binary judgment and meta-verification substantially outperforms joint reward optimization, due to intrinsic differences in output structure and learning dynamics. Based on these insights, we train OmniVerifier-M1, a generalist visual verifier leveraging symbolic meta-verification and decoupled reinforcement learning. OmniVerifier-M1 provides robust verification and fine-grained error localization, and further enables M1-TTS, a verifier-driven agentic generation system achieving dynamic region-level self-correction. This approach paves the way for more reliable, interpretable, and fine-grained multimodal verification, supporting safer and more controllable foundation model deployment.

Abstract

Compositional Reasoning in VLMs

Explore similar work

OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration

When to Rethink: Learning Multi-Perspective Self-Verification for Vision-Language Models

Improving Reasoning in Vision-Language Models via Perception Verified Self-Training