An Evolutionary Agentic Approach for Open-ended Image Quality Perception

Zhenchen Tang; Bo Peng; Zichuan Wang; Songlin Yang; Leilei Cao; Fengjie Zhu; Jing Dong

cs.CVSep 19, 2026

An Evolutionary Agentic Approach for Open-ended Image Quality Perception

Authors: Zhenchen Tang, Bo Peng, Zichuan Wang, Songlin Yang, Leilei Cao, Fengjie Zhu, Jing Dong

Organizations: New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China · Transsion, Shenzhen, China

Abstract

Generative models are rapidly expanding image quality assessment (IQA) beyond traditional fidelity factors to emerging dimensions such as physical plausibility and text-rendering correctness. However, existing IQA models rely on fixed definitions and heavy supervision, making them difficult to extend to open-ended perceptual dimensions. We identify holistic bias as an important limitation: when scoring an unseen dimension, models reuse generic quality priors, leading to scoring errors and rank inversion. To address this, we propose PACE (Perceptual Agentic Collaborative Evolution), a training-free multi-agent framework that formulates open-ended IQA as explicit protocol construction. Given a target dimension, PACE uses collaborative agents to construct an evaluation protocol composed of verifiable Visual Question Answering (VQA) probes, grounding evaluation in concrete visual evidence rather than holistic impressions. The resulting protocol is calibrated using only four human-annotated images per dimension, while a dual-track scoring mechanism aligns model perception with human scoring scales. Across traditional IQA, structural fidelity, context-aware aesthetics, and newly defined open-ended dimensions, PACE consistently improves its MLLM backbone, achieving competitive performance across diverse IQA settings, and reduces the Holistic Override Rate (HOR) from 44.4% to 8.6%.

Figures & tables

Figure 1: Comparison between traditional MLLM scoring and PACE. Direct MLLM scoring is affected by holistic bias, while PACE decomposes the target dimension (e.g., tactile satisfaction) into verifiable visual probes and produces evidence-based, human-aligned scores.

Figure 2: Overview of the PACE pipeline and dual-track scoring. (Left) The Planning Agent decomposes the target dimension into observable sub-dimensions, while the Visualizer and Critic construct and refine VQA probes. (Right) A Generator produces HITL calibration anchors. During scoring, Track 1 evaluates the image using the resulting evaluation protocol, while Track 2 aligns the prediction with the human-rated anchors. The two tracks are fused to produce the final score.

Category	Methods	KonIQ	SPAQ	KADID	LIVE-Wild	AGIQA-3K	CSIQ
Handcrafted	NIQE	0.533 / 0.530	0.679 / 0.664	0.468 / 0.405	0.493 / 0.449	0.560 / 0.533	0.718 / 0.628
Handcrafted	BRISQUE	0.225 / 0.226	0.490 / 0.406	0.429 / 0.356	0.361 / 0.313	0.541 / 0.497	0.740 / 0.556
Non-MLLM Deep-learning	NIMA	0.896 / 0.859	0.838 / 0.856	0.532 / 0.535	0.814 / 0.771	0.715 / 0.654	0.695 / 0.649
	HyperIQA	0.917 / 0.906	0.791 / 0.788	0.506 / 0.468	0.772 / 0.749	0.702 / 0.640	0.752 / 0.717
	DBCNN	0.884 / 0.875	0.812 / 0.806	0.497 / 0.484	0.773 / 0.755	0.730 / 0.641	0.586 / 0.572
	MUSIQ	0.924 / 0.929	0.868 / 0.863	0.575 / 0.556	0.789 / 0.830	0.722 / 0.630	0.771 / 0.710

Table 1: Performance comparison on traditional and in-the-wild IQA benchmarks (Tier 1). Results are evaluated by PLCC / SRCC. The best results are highlighted in bold .

Methods	HandEval	PSNR-SR	GAN-based SR	Denoising	SR Full	PIPAL
CLIP-IQA	0.403 / 0.423	0.382 / 0.325	0.359 / 0.354	0.325 / 0.326	0.250 / 0.240	0.315 / 0.305
PickScore	0.000 / 0.002	0.120 / -0.105	0.271 / -0.217	0.102 / 0.048	0.173 / -0.134	0.058 / -0.013
ImageReward	0.282 / 0.279	0.244 / 0.129	0.232 / 0.189	0.231 / 0.182	0.160 / 0.132	0.163 / 0.130
HPSv2	0.010 / 0.059	0.275 / 0.260	0.351 / 0.317	0.250 / 0.226	0.144 / 0.107	0.189 / 0.167
Q-Align	0.521 / 0.528	0.438 / 0.430	0.515 / 0.494	0.426 / 0.389	0.407 / 0.408	0.403 / 0.419
Qwen2.5-VL	0.503 / 0.501	0.471 / 0.442	0.425 / 0.390	0.346 / 0.304	0.416 / 0.351	0.390 / 0.357

Table 2: Evaluation on Structural Integrity (Tier 2). The results highlight the performance on complex biological and algorithmic hallucinations, including HandEval and PIPAL subtypes.

Methods	City	Nature	Sea	Yellow	Flower
CLIP-IQA	0.210 / -0.120	0.152 / 0.008	0.217 / -0.153	0.054 / 0.007	0.120 / 0.009
PickScore	0.251 / 0.163	0.233 / 0.224	0.151 / 0.141	0.237 / 0.233	0.125 / 0.068
ImageReward	0.217 / 0.140	0.220 / 0.165	0.115 / 0.062	0.268 / 0.219	0.131 / 0.049
HPSv2	0.259 / 0.242	0.152 / 0.128	0.195 / 0.141	0.213 / 0.197	0.031 / 0.026
Q-Align	0.304 / 0.285	0.118 / 0.129	0.170 / 0.110	0.228 / 0.202	0.157 / 0.086
Qwen2.5-VL	0.391 / 0.370	0.295 / 0.290	0.249 / 0.206	0.213 / 0.225	0.144 / 0.111

Table 3: Evaluation on Context-Aware Aesthetics (Tier 3). Performance across 5 representative scenes from the TAD66k dataset, demonstrating PACE’s adaptability to diverse semantic contexts.

Methods	Typo	Light	Tactile	Dream	Portrait	Cine	OVERALL
CLIP-IQA	-0.142 / -0.240	0.128 / 0.051	0.095 / 0.090	0.430 / 0.363	0.423 / 0.416	0.624 / 0.637	0.260 / 0.219
PickScore	-0.073 / -0.028	0.043 / 0.015	0.008 / -0.002	0.675 / 0.595	0.225 / 0.159	0.611 / 0.649	0.248 / 0.231
ImageReward	-0.104 / -0.176	-0.462 / -0.205	0.071 / 0.116	0.746 / 0.661	-0.294 / -0.216	0.736 / 0.714	0.115 / 0.149
HPSv2	-0.013 / 0.062	-0.257 / -0.161	0.388 / 0.437	0.697 / 0.725	-0.010 / -0.038	0.617 / 0.609	0.237 / 0.272
Q-Align	0.461 / 0.608	-0.187 / -0.211	0.312 / 0.386	0.009 / 0.155	-0.058 / 0.125	0.487 / 0.519	0.170 / 0.264
Qwen2.5-VL	0.339 / 0.480	-0.088 / -0.057	0.604 / 0.644	0.874 / 0.758	0.535 / 0.468	0.694 / 0.710	0.493 / 0.501

Table 4: Zero-Training-Data Discovery of Open-Ended Dimensions (Tier 4). Evaluation of 6 novel and subjective perceptual dimensions. Most baseline models experience polarity inversion (negative SRCC) due to holistic bias, whereas PACE maintains highly aligned continuous scoring.

Methods / Config	KonIQ	SPAQ	KADID	LIVE-Wild	AGIQA-3K	CSIQ
Qwen2.5-VL (Q-Align)	0.737 / 0.692	0.855 / 0.860	0.576 / 0.522	0.625 / 0.615	0.813 / 0.744	0.724 / 0.679
Track 2 Only	0.826 / 0.792	0.881 / 0.886	0.646 / 0.607	0.793 / 0.760	0.814 / 0.759	0.680 / 0.617
Without Critic	0.836 / 0.792	0.888 / 0.880	0.761 / 0.756	0.792 / 0.751	0.807 / 0.742	0.705 / 0.663
Track 1 Only	0.798 / 0.738	0.868 / 0.860	0.779 / 0.775	0.763 / 0.709	0.778 / 0.710	0.729 / 0.695
Ours ( $\omega_{1}=0.3,\omega_{2}=0.7$ )	0.843 / 0.795	0.895 / 0.889	0.768 / 0.761	0.811 / 0.770	0.824 / 0.765	0.723 / 0.680
Ours ( $\omega_{1}=0.7,\omega_{2}=0.3$ )	0.829 / 0.768	0.886 / 0.877	0.783 / 0.780	0.793 / 0.744	0.807 / 0.739	0.737 / 0.703

Table 5: Ablation studies of PACE components. We analyze the effects of the dual-track scoring mechanism and multi-agent protocol construction.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: The overall architecture of the PACE framework. Operating on a dual-process cognitive paradigm, PACE routes traditional dimensions to a fast thinking process via an expert toolkit. For open-ended dimensions, it triggers the slow thinking pipeline, where the Planning, Visualizer, Critic, and Generator agents collaborate to evolve VQA protocols and synthesize reference images. The knowledge is then solidified into a RAG memory bank for future rapid inference.

Method	Typo	Light	Tactile	Dream	Portrait	Cine	Overall
PACE (Slow)	18.82	25.86	23.89	23.85	21.56	22.32	22.72
PACE (Fast)	0.72	0.83	0.61	0.70	0.59	0.59	0.67
Qwen2.5-VL	0.12	0.12	0.12	0.12	0.12	0.12	0.12
Q-Align	0.13	0.13	0.11	0.10	0.11	0.11	0.11

Appendix

Table 6: Inference latency on the six Tier-4 open-ended dimensions. Values are wall-clock seconds per image; lower is better. PACE (Slow) includes protocol evolution, while PACE (Fast) reuses the stored protocol.

Dimension	ICC(2, $k$ )	Cronbach’s $\alpha$	MOS [95% CI]
Text Rendering Fidelity	0.911	0.919	3.08 [2.71, 3.46]
Lighting Consistency	0.675	0.677	3.69 [3.42, 3.93]
Tactile Satisfaction	0.868	0.872	2.32 [1.96, 2.71]
Surrealist Coherence	0.781	0.826	3.01 [2.69, 3.31]
Human Vitality	0.919	0.918	3.03 [2.62, 3.44]
Cinematic Narrative	0.652	0.655	3.33 [3.08, 3.59]

Appendix

Table 7: Inter-annotator reliability on the Tier-4 open-ended dimensions. ICC(2,

k

) measures the reliability of the averaged ratings from six annotators. The 95% confidence interval is reported for MOS.

Figure 4: Quantitative Evidence of Holistic Bias across All 6 Open-Ended Dimensions. The red shaded area represents the Holistic Bias Zone where human GT

\leq 3.0

but the model predicts

\geq 4.0

. (a) Q-Align frequently falls into this zone with a 44.4% HOR. (b) PACE substantially reduces this bias, achieving an 8.6% HOR.

Figure 5: Holistic Bias on Prominent Dimensions ( Tactile Satisfaction & Human Vitality ). On highly specific and challenging dimensions, direct MLLM prompting (Q-Align) suffers a catastrophic HOR of 58.8%, showing strong interference from holistic aesthetic priors during task-specific reasoning. PACE maintains robust evidence-based scoring (2.9% HOR).

Dimension	#GT-low	Q-Align $\uparrow$	Q-Scorer $\uparrow$	DeQA-Score $\uparrow$	PACE on $d$ $\downarrow$	GT on $d$ $\downarrow$
Typo (Text Rendering)	15	3.13	2.99	3.21	2.61	2.09
Light (Lighting Consistency)	7	3.72	2.96	3.54	3.26	2.55
Tactile (Tactile Satisfaction)	19	4.01	3.07	3.67	2.66	1.61
Dream (Surrealist Coherence)	12	4.20	3.30	3.97	2.69	2.11
Portrait (Human Vitality)	15	4.25	3.32	3.98	2.68	2.09
Cine (Cinematic Narrative)	13	3.36	2.68	3.14	2.80	2.54

Appendix

Table 8: Traditional quality of GT-low images on Tier 4 (holistic quality vs. specific quality). For the 81 images that have poor human ratings on the target dimension

d

, we report the mean predictions of three holistic IQA models (Q-Align / Q-Scorer / DeQA-Score) alongside PACE’s predictions and human GT. Values are on the standard

[1,5]

scale.

Backbone / Method	KonIQ	SPAQ	KADID	LIVE-Wild	AGIQA-3K	CSIQ
mPLUG-Owl2 (baseline)	0.4290 / 0.3647	0.7041 / 0.6775	0.4402 / 0.4343	0.5250 / 0.4430	0.5951 / 0.5826	0.5328 / 0.3585
mPLUG-Owl2 + PACE	0.5723 / 0.5265	0.7616 / 0.7466	0.4677 / 0.4344	0.6326 / 0.5684	0.5505 / 0.5453	0.6317 / 0.5209
$\Delta$ (Absolute)	+0.143 / +0.162	+0.058 / +0.069	+0.028 / +0.000	+0.108 / +0.125	-0.045 / -0.037	+0.099 / +0.162

Appendix

Table 9: Performance comparison of PACE using mPLUG-Owl2 backbone (Tier 1). Values are PLCC / SRCC.

Figure 6: Qualitative case studies of PACE on two open-ended dimensions. For each example, PACE first decomposes the target concept into observable sub-dimensions and constructs a Critic-filtered VQA protocol. Track 1 evaluates the target using the resulting protocol, while Track 2 positions it relative to the human-rated high-quality and low-quality anchors. The two continuous perception coefficients, $\alpha_{1}$ and $\alpha_{2}$ , are computed from logit-weighted probability distributions and fused to obtain the final score. Red crosses indicate rejected protocol components.

Explore similar work

Bridging Adversarial and Collaborative Learning for AI-Generated Image Quality Assessment
Aug 25, 2026Baoliang Chen, Qing Lin, Sijie MaiImage Quality Assessment Image Generation Evaluation
Department of Computer Science, South China Normal University, China · Department of Data Science and Analytics, Hong Kong University of Science and Technology (Guangzhou)
IQA-T1: Tool-based Visual Evidence Reasoning for Image Quality Assessment
Jul 14, 2026Jinjian Wu, Jiaqi Tang, Wei Wei +5Tool Use in VLMs VLM Reasoning
School of Computer Science, Northwestern Polytechnical University, Xi’an, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
Tool-IQA: Augmenting Image Quality Assessment with Simple Tools
Jun 15, 2026Guanyi Qin, Junjie Zhang, Chunming He +4Tool Use in VLMs Image Quality Assessment
1National University of Singapore · 2OPPO Research Institute · 3Nanyang Technical University +3

Aug 25, 2026J/K move · Enter open · S save

Bridging Adversarial and Collaborative Learning for AI-Generated Image Quality Assessment

Baoliang Chen, Qing Lin, Sijie Mai

Department of Computer Science, South China Normal University, China · Department of Data Science and Analytics, Hong Kong University of Science and Technology (Guangzhou)

Image Quality Assessment Image Generation Evaluation T2I Generation Evaluation

Open paper AskPDF arXiv

AI-generated image quality assessment (AIGIQA) requires jointly reasoning about perceptual fidelity and prompt alignment, two quality dimensions that are often treated as independent in existing AIGIQA models. However, by re-examining human ratings, we uncover a previously overlooked phenomenon: the two dimensions are interdependent and exhibit both competitive and cooperative interactions during human rating. This observation suggests that a unified model should neither collapse the two dimensions nor rigidly separate them, but rather adaptively negotiate their interplay. Motivated by this insight, we introduce an interaction-aware learning framework that models perception-alignment relations through adversarial and collaborative inference pathways. Instead of designing a rigid dual-branch architecture, our method employs a gated interaction module that dynamically routes features according to the inferred relationship between the two dimensions. Task-aware prompts further modulate the gating behaviour, enabling the model to switch between competition and cooperation when necessary. Experiments across multiple AIGIQA benchmarks demonstrate that our approach not only achieves state-of-the-art accuracy but also yields interpretable interaction patterns, offering a more faithful approximation of human judgment. The codes are available at https://github.com/LQAMEI/ACL-IQA.

Abstract

Explore similar work

Bridging Adversarial and Collaborative Learning for AI-Generated Image Quality Assessment

IQA-T1: Tool-based Visual Evidence Reasoning for Image Quality Assessment

Tool-IQA: Augmenting Image Quality Assessment with Simple Tools