VIEScore2: Unified Image Evaluation with Spatially Grounded Explanations

Xianda Du; Max Ku; Weiming Ren; Zhi Rui Tam; Chunlin Ren; Ping Nie; Min-Hung Chen; Wenhu Chen

cs.CVOct 1, 2026

VIEScore2: Unified Image Evaluation with Spatially Grounded Explanations

Authors: Xianda Du, Max Ku, Weiming Ren, Zhi Rui Tam, Chunlin Ren, Ping Nie, Min-Hung Chen, Wenhu Chen

Organizations: University of Waterloo · National Taiwan University · Nanyang Technological University · NVIDIA

Abstract

Existing synthetic image evaluators typically provide only a scalar quality score and do not identify the image regions that support it. We introduce VIEScore2, a unified evaluator for image generation and editing tasks with optional conditioning images. VIEScore2 represents an image as an N x N grid and jointly predicts quality scores and defect locations in a single model pass. Its text-native grid representation provides a common interface for heterogeneous spatial supervision and enables directly verifiable post-training objectives. We train on 38K examples spanning score-only, localization-only, and joint supervision across generation and editing tasks. Starting from supervised fine-tuning, we further apply GRPO to improve defect localization using rewards that combine cell-level Dice overlap, score accuracy, and output-format validity. A parameter-free parser converts the structured predictions into readable explanations. On the primary suite, VIEScore2 achieves an overall-score SRCC of 0.601, compared with 0.491 for Gemini-3-Flash, the strongest zero-shot general-purpose VLM baseline under matched inputs. For defect localization, VIEScore2 outperforms both general-purpose VLMs and specialized spatial evaluators on three of six benchmarks in per-image grid IoU and ranks among the top three on five, including datasets beyond its training sources.

Figures & tables

Figure 1: An evaluation example with VIEScore2. The model predicts perceptual quality (PQ), semantic consistency (SC), and a $16\times 16$ defect grid, where red and amber denote visual artifacts and semantic misalignments. A parameter-free parser converts the predictions into natural language.

Figure 2: Overview of VIEScore2: data preparation, SFT and GRPO training, and inference. GRPO combines Dice, score, and format rewards. The parameter-free parser converts predicted scores and defect grids into explanations. GT denotes ground truth.

	Training sources				Additional datasets
	RichHF		PAL4VST		AbHuman		HAD		SynthScars		SDG-30K
Method	IoU	$F_{1}$	IoU	$F_{1}$	IoU	$F_{1}$	IoU	$F_{1}$	IoU	$F_{1}$	IoU	$F_{1}$
Qwen3-VL-8B	0.022	0.137	0.026	0.124	0.024	0.108	0.044	0.111	0.026	0.117	0.114	0.269
GPT-5.6-terra †	0.096	0.218	0.118	0.239	0.171	0.217	0.148	0.251	0.162	0.281	0.058	0.217
GPT-5.6-sol †	0.156	0.274	0.141	0.248	0.217	0.282	0.211	0.298	0.186	0.290	0.078	0.258
Gemini-3-Flash †	0.047	0.082	0.048	0.068	0.057	0.079	0.075	0.119	0.062	0.101	0.031	0.068

Table 1: Defect localization on six test sets using a shared

16\times 16

grid. Methods use their supported inputs. IoU averages per-image overlap;

F_{1}

pools cell counts. † API models use fixed prompts and temperature

0

; § LEGION uses its released intermediate checkpoint.

Figure 3: Selected defect-localization examples across six benchmarks. Red cells denote predictions and green outlines denote ground-truth defect regions. Numbers report per-image grid IoU. Additional qualitative examples and failure cases are provided in Appendix F .

Method	All	RichHF	EvalMuse	TIG	TIE	SRIG	SRIE	MRIG	MRIE
Qwen3-VL-8B judge ‡	0.373	0.456	0.473	0.570	0.066	0.128	0.393	0.537	$-0.290$
GPT-5.6-terra †	0.402	0.440	0.567	0.640	0.493	0.258	0.110	0.523	0.539
GPT-5.6-sol †	0.437	0.470	0.655	0.623	0.590	0.275	0.253	0.539	0.684
Gemini-3-Flash †	0.491	0.515	0.535	0.531	0.686	0.356	0.404	0.418	0.677
VIEScore2	0.601	0.692	0.803	0.589	0.429	0.375	0.403	0.509	0.417

Table 2: Overall-score SRCC on the primary suite (

900

examples) with matched

\chi_{K}

inputs. † API correlations use successfully parsed scores. ‡ Qwen3-VL-8B uses our evaluation instructions without fine-tuning.

Model	Grid IoU	$F_{1}$	PQ SRCC	SC SRCC
Qwen3-VL-8B	0.070	0.273	0.181	0.352
GPT-5.6-terra †	0.112	0.247	0.357	0.508
VIEScore2	0.324	0.506	0.558	0.564

Table 3: Joint evaluation with matched

\chi_{K}

inputs. Localization uses

1{,}100

examples; PQ/SC SRCC uses

700

. Grid IoU excludes empty ground-truth grids. † GPT-5.6-terra returns valid scores for

554/700

examples.

Figure 4: Grid resolution versus annotation fidelity, output length, and trained localization. Annotation fidelity compares grids with native masks. Shading marks

N=16

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Scores	Localization	Raw supervision
ImagenWorld	PQ, SC	single grid	ratings, masks
RichHF	PQ, SC	artifact and misalignment grids	ratings, heatmaps
PAL4VST	–	single grid	masks
EvalMuse	overall	–	alignment ratings
COCO	PQ, SC	single grid	clean constraint

Appendix

Table 4: Supervision and requested output fields by source. “–” indicates an omitted field.

Figure 5: RichHF annotations and their $16\times 16$ grid targets. Red and blue denote artifact and misalignment; “!” marks high annotation coverage.

Source	Train	Validation	Primary eval.	Conditioning images
RichHF	13,920	1,518	400	no
EvalMuse	9,000	1,000	200	no
PAL4VST	8,177	908	300	no
ImagenWorld editing/reference	3,642	415	250	yes
ImagenWorld TIG	1,165	121	50	no
COCO	2,218	272	100	no

Appendix

Table 5: Data splits by source. Counts are records, which may share an image.

Primary source	Total	Localization	Overall score	PQ/SC
RichHF	400	400	400	400
PAL4VST	300	300	0	0
EvalMuse	200	0	200	0
ImagenWorld (six tasks)	300	300	300	300
COCO	100	100	0	0
Total	1,300	1,100	900	700

Appendix

Table 6: Examples available for each metric in the primary suite.

Dataset	Evaluation subset	$n$
PAL4VST	Full test set	1,405
RichHF	Test split after threshold tuning	840
EvalMuse	External score subset	2,000
AbHuman	$2$ K validation subset	1,407
HAD	Validation subset	709
SynthScars	Official test split ( $1{,}000$ images)	951

Appendix

Table 7: External evaluation subsets. AbHuman, HAD, SynthScars, and SDG-30K report localization on non-empty ground-truth grids.

Method	Spatial output	Threshold	Setting source
RAHF	Two heatmaps	0.06	RichHF ( $100$ )
ImageDoctor	Two heatmaps	0.03	RichHF ( $100$ )
PAL	Artifact mask	0.5	Released default
SegFormer-b0	Defect probabilities	0.75	Validation
LEGION	Artifact mask	0.5 cell fraction	Released rule
SDG	Defect boxes	–	Released output

Appendix

Table 8: Spatial conversion settings, fixed across benchmarks.

Method	Scores	Spatial output	Text feedback	GRPO	Cond.
VIEScore ( Ku et al., 2024a )	✓	–	Rationale	–	✓
RAHF ( Liang et al., 2024 )	✓	Heatmaps	–	–	–
ImageDoctor ( Guo et al., 2026 )	✓	Heatmap	Reasoning	✓	–
PAL ( Zhang et al., 2023 )	–	Mask	–	–	–
LEGION ( Kang et al., 2025 )	–	Mask	Explanation	–	–
FGA-BLIP2 ( Han et al., 2026 )	✓	–	–	–	–

Appendix

Table 9: Evaluator output interfaces. Cond. denotes support for conditioning images; “–” denotes an unsupported output or training setting.

Model	$Precision$	$Recall$	$F_{1}$	$\operatorname{IoU}^{\mathrm{grid}}_{p}$	PQ SRCC	SC SRCC
Qwen3-VL-8B	0.211	0.388	0.273	0.070	0.181	0.352
GPT-5.6-terra †	0.257	0.239	0.247	0.112	0.357	0.508
VIEScore2 ( $-\mathrm{GRPO}$ )	0.415	0.545	0.471	0.296	0.580	0.532
VIEScore2	0.466	0.552	0.506	0.324	0.558	0.564
SegFormer-b0 (matched loc.)	0.433	0.573	0.493	0.272	–	–

Appendix

Table 10: Full primary-suite results: localization on

1{,}100

examples and PQ/SC SRCC on

700

. Rankings compare the

\chi_{K}

VLMs; SegFormer uses only the generated image. † GPT-5.6-terra returns valid scores for

554/700

examples.

Task/source	$F_{1}$	$\rho_{\mathrm{o}}$	Task/source	$F_{1}$	$\rho_{\mathrm{o}}$
TIG	0.572	0.589	MRIG	0.545	0.509
TIE	0.692	0.429	MRIE	0.496	0.417
SRIG	0.512	0.375	RichHF	0.465	0.692
SRIE	0.412	0.403	PAL4VST	0.376	–
			EvalMuse	–	0.803

Appendix

Table 11: Primary-suite results by task and source.

\rho_{\mathrm{o}}

denotes overall-score SRCC; each ImagenWorld task has

50

examples.

Method	All	RichHF	EvalMuse	TIG	TIE	SRIG	SRIE	MRIG	MRIE
ImageReward	0.239	0.227	0.595	0.107	$-0.278$	0.195	0.151	0.035	0.263
PickScore	0.233	0.263	0.441	0.412	$-0.143$	0.288	0.184	0.391	0.407
HPSv2	0.166	0.182	0.473	0.209	0.034	0.181	0.177	0.486	0.406
VQAScore	0.224	0.238	0.440	0.110	0.081	0.145	0.085	0.105	0.140
FGA-BLIP2	0.423	0.452	0.918	0.422	$-0.141$	0.255	0.151	$-0.062$	0.233
ImageDoctor	0.411	0.714	0.280	0.459	0.160	0.271	0.355	0.479	0.299

Appendix

Table 12: Overall-score SRCC with the generated image and prompt only (

\chi_{0}

). Rankings compare methods within this input setting.

Task	$n$	$\rho_{\mathrm{o}}[\chi_{K}]$	$\rho_{\mathrm{o}}[\chi_{0}]$	$F_{1}[\chi_{K}]$	$F_{1}[\chi_{0}]$	$\Delta F_{1}$
TIE	50	0.429	0.403	0.692	0.683	$+0.009$
SRIG	50	0.375	0.378	0.512	0.503	$+0.009$
SRIE	50	0.403	0.265	0.412	0.370	$+0.042$
MRIG	50	0.509	0.470	0.545	0.522	$+0.023$
MRIE	50	0.417	0.509	0.496	0.458	$+0.039$
All	250	0.451	0.420	0.536	0.514	$+0.022$

Appendix

Table 13: Paired comparison with (

\chi_{K}

) and without (

\chi_{0}

) conditioning images. Changes are computed before rounding.

Figure 6: Per-generator SRCC of VIEScore2 on the EvalMuse

2

K subset.

EvalMuse slice	$n$	VIEScore2 SRCC	FGA-BLIP2 SRCC
Primary suite	200	0.803	0.918
External 2K	2,000	0.773	0.910

Appendix

Table 14: Score correlation on two EvalMuse subsets.

Method	Input	T2I	Edit
ImageReward	$\chi_{0}$	0.530	0.559
PickScore	$\chi_{0}$	0.574	0.574
HPSv2	$\chi_{0}$	0.546	0.541
VQAScore	$\chi_{0}$	0.537	0.554
ImageDoctor	$\chi_{0}$	0.546	0.525
Q-Align	$\chi_{0}$	0.513	0.523

Appendix

Table 15: MMRB2 preference accuracy (

1{,}000

pairs per task). Rankings are separate for

\chi_{0}

and

\chi_{K}

. Ties and unparseable pairs receive

0.5

Figure 7: Localization before and after GRPO.

\Delta

is the final model minus VIEScore2 (

-\mathrm{GRPO}

), computed from the displayed values.

Variant	Grid IoU	$F_{1}$	Overall SRCC
VIEScore2 ( $-\mathrm{GRPO}$ )	0.296	0.471	0.596
VIEScore2 ( $-r_{\mathrm{score}},-r_{\mathrm{format}}$ )	0.320	0.481	0.601
VIEScore2 ( $-r_{\mathrm{score}}$ )	0.323	0.496	0.596
VIEScore2	0.324	0.506	0.601

Appendix

Table 16: Reward ablation on the primary suite with matched initialization, data, sampling budget, and optimization steps.

Run	$F_{1}$	$\operatorname{IoU}^{\mathrm{grid}}_{p}$	$\Delta F_{1}$	$\Delta\operatorname{IoU}^{\mathrm{grid}}_{p}$
VIEScore2 ( $-\mathrm{GRPO}$ ) (epoch 4)	0.471	0.296	–	–
VIEScore2 ( $-\mathrm{GRPO}$ ) (epoch 5)	0.472	0.297	$+0.001$	$+0.001$
VIEScore2 (seed 42)	0.506	0.324	$+0.034$	$+0.028$
VIEScore2 (seed 2)	0.483	0.320	$+0.012$	$+0.024$
VIEScore2 (seed 3)	0.493	0.325	$+0.022$	$+0.029$

Appendix

Table 17: Results across three GRPO runs and additional SFT. Changes are relative to SFT epoch 4 and computed before rounding.

Policy	$P$	$R$	$F_{1}$	$\operatorname{IoU}^{\mathrm{grid}}_{p}$	$\mathrm{FP}_{\mathrm{PAL}}/40$
VIEScore2 ( $-\mathrm{GRPO}$ )	0.374	0.548	0.445	0.303	14
GRPO $\beta=0.5$	0.452	0.450	0.451	0.300	19
GRPO $\beta=1$	0.411	0.534	0.464	0.322	26
GRPO $\beta=2$	0.334	0.695	0.451	0.310	27
GRPO Tversky $\alpha{=}0.7$	0.446	0.494	0.469	0.313	21

Appendix

Table 18: Development-set precision–recall trade-offs.

\mathrm{FP}_{\mathrm{PAL}}/40

counts false alarms on

40

clean PAL4VST images.

Model	Artifact $F_{1}$	Misalign. $F_{1}$	Coverage $F_{1}$	COCO FP/100	PAL FP/115
VIEScore2 ( $-\mathrm{GRPO}$ )	0.488	0.165	0.330	0	63
VIEScore2	0.493	0.198	0.297	0	85
SegFormer-b0	–	–	–	34	62

Appendix

Table 19: Per-category localization on RichHF (

400

examples) and clean-image false alarms. Coverage evaluates “!” marks.

Output form	$P$	$R$	$F_{1}$	$\operatorname{IoU}^{\mathrm{grid}}_{p}$
Dense bitmap	0.580	0.099	0.169	0.116
Point list	0.250	0.446	0.320	0.206
Bounding boxes	0.354	0.671	0.463	0.280
Sparse cells	0.298	0.596	0.397	0.246

Appendix

Table 20: Localization representations under matched training data and compute.

	Representation ceiling (model-free)			Serialized target		Trained model
$N$	pixel-F1	pixel-IoU	GT vanished	tokens (mean)	tokens (p95)	F1@ $N$	pixel-IoU p
4	0.219	0.178	68.4%	7	35	—	—
8	0.477	0.374	24.1%	27	116	0.425	0.192
12	0.657	0.535	9.8%	66	250	0.398	0.235
16	0.731	0.615	6.4%	120	471	0.415	0.231
24	0.813	0.712	2.8%	273	1,089	0.388	0.209

Appendix

Table 21: Grid resolution, annotation fidelity, and output length. Rankings apply to trained-model results.

Figure 8: Localization before and after GRPO. Orange cells mark defects; values are per-image grid IoU. VIEScore2 ( $-\mathrm{GRPO}$ ) uses the SFT checkpoint; epochs 4 and 5 give identical predictions. VIEScore2 denotes the final model after GRPO.

Figure 9: A RichHF example with annotations, predictions, raw output, and the parameter-free parser’s explanation.

Figure 11: SRIE, MRIG, and MRIE examples.

Figure 12: Failure cases: lost small defects, missed misalignment, false alarms, and an incorrect prediction with its explanation.

Explore similar work

Where, What, Why, and Importance: Structured Defect Grounding for Text-to-Image Feedback
Jun 4, 2026Huaisong Zhang, Hao Yu, Yuxuan Zhang +7Text-To-Image Diffusion Alignment
Tsinghua University · University of British Columbia · Vector Institute +2
Seeing Isn't Believing: Uncovering Blind Spots in Evaluator Vision-Language Models
Apr 23, 2026Mohammed Safi Ur Rahman Khan, Sanjay Suryanarayanan, Tushar Anand +1Large Language Model Evaluation Blind Spots
♠Nilekani Centre at AI4Bharat ♢Indian Institute of Technology Madras · ♠Nilekani Centre at AI4Bharat · ♣BITS Pilani, Hyderabad
SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images
Jun 10, 2026Xiaoxiao Sun, Ruotian Zhang, Junzhe Huang +2Ai-Generated Image Detection Textvqa
Zhejiang University · Stanford University

Jun 4, 2026, cs.CVJ/K move · Enter open · S save

Where, What, Why, and Importance: Structured Defect Grounding for Text-to-Image Feedback

Huaisong Zhang, Hao Yu, Yuxuan Zhang, Jiahe Wang+6

Tsinghua University · University of British Columbia · Vector Institute+2

Text-To-Image Diffusion Alignment Defect Contextual Grounding Feedback

Open paper AskPDF arXiv

Despite generating increasingly photorealistic images, text-to-image (T2I) models still exhibit localized, subtle, and structurally complex failures. Diagnosing these failures requires instance-level feedback that answers where a defect occurs, what type it is, why it is defective, and its importance to overall image quality. While recent dense-feedback methods move beyond scalar supervision, their heatmap-centric representations still formulate diagnosis as pixel-field regression, making it difficult to localize variable-cardinality defects and bind semantic reasons to individual failures. To address this representation bottleneck, we propose Structured Defect Grounding (SDG), which casts T2I diagnosis as structured set prediction by modeling each defect as a (location, type, reason, importance) tuple. To make this formulation trainable and measurable, we introduce SDG-30K, a 30K-image dataset with box-grounded annotations across four modern T2I generators, together with a dedicated evaluation protocol, SDG-Eval. Building on this structured representation, we further present a diagnosis-to-alignment framework in which a Vision-Language Model (VLM) serves as the SDG detector, and BoxFlow-GRPO converts predicted defect sets into box-derived, importance-weighted spatial rewards for diffusion model alignment. Extensive experiments show that our SDG detector outperforms leading proprietary VLMs on structured defect grounding, while SDG-guided rewards consistently improve T2I alignment and support localized image refinement. These results establish SDG as a unified, instance-level interface for diagnosing, evaluating, and enhancing modern generative models.

Abstract

Explore similar work

Where, What, Why, and Importance: Structured Defect Grounding for Text-to-Image Feedback

Seeing Isn't Believing: Uncovering Blind Spots in Evaluator Vision-Language Models

SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images