Organizations: New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China · Transsion, Shenzhen, China
Generative models are rapidly expanding image quality assessment (IQA) beyond traditional fidelity factors to emerging dimensions such as physical plausibility and text-rendering correctness. However, existing IQA models rely on fixed definitions and heavy supervision, making them difficult to extend to open-ended perceptual dimensions. We identify holistic bias as an important limitation: when scoring an unseen dimension, models reuse generic quality priors, leading to scoring errors and rank inversion. To address this, we propose PACE (Perceptual Agentic Collaborative Evolution), a training-free multi-agent framework that formulates open-ended IQA as explicit protocol construction. Given a target dimension, PACE uses collaborative agents to construct an evaluation protocol composed of verifiable Visual Question Answering (VQA) probes, grounding evaluation in concrete visual evidence rather than holistic impressions. The resulting protocol is calibrated using only four human-annotated images per dimension, while a dual-track scoring mechanism aligns model perception with human scoring scales. Across traditional IQA, structural fidelity, context-aware aesthetics, and newly defined open-ended dimensions, PACE consistently improves its MLLM backbone, achieving competitive performance across diverse IQA settings, and reduces the Holistic Override Rate (HOR) from 44.4% to 8.6%.
Figures & tables
Figure 1: Comparison between traditional MLLM scoring and PACE. Direct MLLM scoring is affected by holistic bias, while PACE decomposes the target dimension (e.g., tactile satisfaction) into verifiable visual probes and produces evidence-based, human-aligned scores.
Figure 2: Overview of the PACE pipeline and dual-track scoring. (Left) The Planning Agent decomposes the target dimension into observable sub-dimensions, while the Visualizer and Critic construct and refine VQA probes. (Right) A Generator produces HITL calibration anchors. During scoring, Track 1 evaluates the image using the resulting evaluation protocol, while Track 2 aligns the prediction with the human-rated anchors. The two tracks are fused to produce the final score.
Category
Methods
KonIQ
SPAQ
KADID
LIVE-Wild
AGIQA-3K
CSIQ
Handcrafted
NIQE
0.533 / 0.530
0.679 / 0.664
0.468 / 0.405
0.493 / 0.449
0.560 / 0.533
0.718 / 0.628
BRISQUE
0.225 / 0.226
0.490 / 0.406
0.429 / 0.356
0.361 / 0.313
0.541 / 0.497
0.740 / 0.556
Non-MLLM Deep-learning
NIMA
0.896 / 0.859
0.838 / 0.856
0.532 / 0.535
0.814 / 0.771
0.715 / 0.654
0.695 / 0.649
HyperIQA
0.917 / 0.906
0.791 / 0.788
0.506 / 0.468
0.772 / 0.749
0.702 / 0.640
0.752 / 0.717
DBCNN
0.884 / 0.875
0.812 / 0.806
0.497 / 0.484
0.773 / 0.755
0.730 / 0.641
0.586 / 0.572
MUSIQ
0.924 / 0.929
0.868 / 0.863
0.575 / 0.556
0.789 / 0.830
0.722 / 0.630
0.771 / 0.710
Table 1: Performance comparison on traditional and in-the-wild IQA benchmarks (Tier 1). Results are evaluated by PLCC / SRCC. The best results are highlighted in bold .
Methods
HandEval
PSNR-SR
GAN-based SR
Denoising
SR Full
PIPAL
CLIP-IQA
0.403 / 0.423
0.382 / 0.325
0.359 / 0.354
0.325 / 0.326
0.250 / 0.240
0.315 / 0.305
PickScore
0.000 / 0.002
0.120 / -0.105
0.271 / -0.217
0.102 / 0.048
0.173 / -0.134
0.058 / -0.013
ImageReward
0.282 / 0.279
0.244 / 0.129
0.232 / 0.189
0.231 / 0.182
0.160 / 0.132
0.163 / 0.130
HPSv2
0.010 / 0.059
0.275 / 0.260
0.351 / 0.317
0.250 / 0.226
0.144 / 0.107
0.189 / 0.167
Q-Align
0.521 / 0.528
0.438 / 0.430
0.515 / 0.494
0.426 / 0.389
0.407 / 0.408
0.403 / 0.419
Qwen2.5-VL
0.503 / 0.501
0.471 / 0.442
0.425 / 0.390
0.346 / 0.304
0.416 / 0.351
0.390 / 0.357
Table 2: Evaluation on Structural Integrity (Tier 2). The results highlight the performance on complex biological and algorithmic hallucinations, including HandEval and PIPAL subtypes.
Methods
City
Nature
Sea
Yellow
Flower
CLIP-IQA
0.210 / -0.120
0.152 / 0.008
0.217 / -0.153
0.054 / 0.007
0.120 / 0.009
PickScore
0.251 / 0.163
0.233 / 0.224
0.151 / 0.141
0.237 / 0.233
0.125 / 0.068
ImageReward
0.217 / 0.140
0.220 / 0.165
0.115 / 0.062
0.268 / 0.219
0.131 / 0.049
HPSv2
0.259 / 0.242
0.152 / 0.128
0.195 / 0.141
0.213 / 0.197
0.031 / 0.026
Q-Align
0.304 / 0.285
0.118 / 0.129
0.170 / 0.110
0.228 / 0.202
0.157 / 0.086
Qwen2.5-VL
0.391 / 0.370
0.295 / 0.290
0.249 / 0.206
0.213 / 0.225
0.144 / 0.111
Table 3: Evaluation on Context-Aware Aesthetics (Tier 3). Performance across 5 representative scenes from the TAD66k dataset, demonstrating PACE’s adaptability to diverse semantic contexts.
Methods
Typo
Light
Tactile
Dream
Portrait
Cine
OVERALL
CLIP-IQA
-0.142 / -0.240
0.128 / 0.051
0.095 / 0.090
0.430 / 0.363
0.423 / 0.416
0.624 / 0.637
0.260 / 0.219
PickScore
-0.073 / -0.028
0.043 / 0.015
0.008 / -0.002
0.675 / 0.595
0.225 / 0.159
0.611 / 0.649
0.248 / 0.231
ImageReward
-0.104 / -0.176
-0.462 / -0.205
0.071 / 0.116
0.746 / 0.661
-0.294 / -0.216
0.736 / 0.714
0.115 / 0.149
HPSv2
-0.013 / 0.062
-0.257 / -0.161
0.388 / 0.437
0.697 / 0.725
-0.010 / -0.038
0.617 / 0.609
0.237 / 0.272
Q-Align
0.461 / 0.608
-0.187 / -0.211
0.312 / 0.386
0.009 / 0.155
-0.058 / 0.125
0.487 / 0.519
0.170 / 0.264
Qwen2.5-VL
0.339 / 0.480
-0.088 / -0.057
0.604 / 0.644
0.874 / 0.758
0.535 / 0.468
0.694 / 0.710
0.493 / 0.501
Table 4: Zero-Training-Data Discovery of Open-Ended Dimensions (Tier 4). Evaluation of 6 novel and subjective perceptual dimensions. Most baseline models experience polarity inversion (negative SRCC) due to holistic bias, whereas PACE maintains highly aligned continuous scoring.
Methods / Config
KonIQ
SPAQ
KADID
LIVE-Wild
AGIQA-3K
CSIQ
Qwen2.5-VL (Q-Align)
0.737 / 0.692
0.855 / 0.860
0.576 / 0.522
0.625 / 0.615
0.813 / 0.744
0.724 / 0.679
Track 2 Only
0.826 / 0.792
0.881 / 0.886
0.646 / 0.607
0.793 / 0.760
0.814 / 0.759
0.680 / 0.617
Without Critic
0.836 / 0.792
0.888 / 0.880
0.761 / 0.756
0.792 / 0.751
0.807 / 0.742
0.705 / 0.663
Track 1 Only
0.798 / 0.738
0.868 / 0.860
0.779 / 0.775
0.763 / 0.709
0.778 / 0.710
0.729 / 0.695
Ours ( ω1=0.3,ω2=0.7 )
0.843 / 0.795
0.895 / 0.889
0.768 / 0.761
0.811 / 0.770
0.824 / 0.765
0.723 / 0.680
Ours ( ω1=0.7,ω2=0.3 )
0.829 / 0.768
0.886 / 0.877
0.783 / 0.780
0.793 / 0.744
0.807 / 0.739
0.737 / 0.703
Table 5: Ablation studies of PACE components. We analyze the effects of the dual-track scoring mechanism and multi-agent protocol construction.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: The overall architecture of the PACE framework. Operating on a dual-process cognitive paradigm, PACE routes traditional dimensions to a fast thinking process via an expert toolkit. For open-ended dimensions, it triggers the slow thinking pipeline, where the Planning, Visualizer, Critic, and Generator agents collaborate to evolve VQA protocols and synthesize reference images. The knowledge is then solidified into a RAG memory bank for future rapid inference.
Method
Typo
Light
Tactile
Dream
Portrait
Cine
Overall
PACE (Slow)
18.82
25.86
23.89
23.85
21.56
22.32
22.72
PACE (Fast)
0.72
0.83
0.61
0.70
0.59
0.59
0.67
Qwen2.5-VL
0.12
0.12
0.12
0.12
0.12
0.12
0.12
Q-Align
0.13
0.13
0.11
0.10
0.11
0.11
0.11
Appendix
Table 6: Inference latency on the six Tier-4 open-ended dimensions. Values are wall-clock seconds per image; lower is better. PACE (Slow) includes protocol evolution, while PACE (Fast) reuses the stored protocol.
Dimension
ICC(2, k )
Cronbach’s α
MOS [95% CI]
Text Rendering Fidelity
0.911
0.919
3.08 [2.71, 3.46]
Lighting Consistency
0.675
0.677
3.69 [3.42, 3.93]
Tactile Satisfaction
0.868
0.872
2.32 [1.96, 2.71]
Surrealist Coherence
0.781
0.826
3.01 [2.69, 3.31]
Human Vitality
0.919
0.918
3.03 [2.62, 3.44]
Cinematic Narrative
0.652
0.655
3.33 [3.08, 3.59]
Appendix
Table 7: Inter-annotator reliability on the Tier-4 open-ended dimensions. ICC(2, k ) measures the reliability of the averaged ratings from six annotators. The 95% confidence interval is reported for MOS.
Figure 4: Quantitative Evidence of Holistic Bias across All 6 Open-Ended Dimensions. The red shaded area represents the Holistic Bias Zone where human GT ≤3.0 but the model predicts ≥4.0 . (a) Q-Align frequently falls into this zone with a 44.4% HOR. (b) PACE substantially reduces this bias, achieving an 8.6% HOR.
Figure 5: Holistic Bias on Prominent Dimensions ( Tactile Satisfaction & Human Vitality ). On highly specific and challenging dimensions, direct MLLM prompting (Q-Align) suffers a catastrophic HOR of 58.8%, showing strong interference from holistic aesthetic priors during task-specific reasoning. PACE maintains robust evidence-based scoring (2.9% HOR).
Dimension
#GT-low
Q-Align ↑
Q-Scorer ↑
DeQA-Score ↑
PACE on d↓
GT on d↓
Typo (Text Rendering)
15
3.13
2.99
3.21
2.61
2.09
Light (Lighting Consistency)
7
3.72
2.96
3.54
3.26
2.55
Tactile (Tactile Satisfaction)
19
4.01
3.07
3.67
2.66
1.61
Dream (Surrealist Coherence)
12
4.20
3.30
3.97
2.69
2.11
Portrait (Human Vitality)
15
4.25
3.32
3.98
2.68
2.09
Cine (Cinematic Narrative)
13
3.36
2.68
3.14
2.80
2.54
Appendix
Table 8: Traditional quality of GT-low images on Tier 4 (holistic quality vs. specific quality). For the 81 images that have poor human ratings on the target dimension d , we report the mean predictions of three holistic IQA models (Q-Align / Q-Scorer / DeQA-Score) alongside PACE’s predictions and human GT. Values are on the standard [1,5] scale.
Backbone / Method
KonIQ
SPAQ
KADID
LIVE-Wild
AGIQA-3K
CSIQ
mPLUG-Owl2 (baseline)
0.4290 / 0.3647
0.7041 / 0.6775
0.4402 / 0.4343
0.5250 / 0.4430
0.5951 / 0.5826
0.5328 / 0.3585
mPLUG-Owl2 + PACE
0.5723 / 0.5265
0.7616 / 0.7466
0.4677 / 0.4344
0.6326 / 0.5684
0.5505 / 0.5453
0.6317 / 0.5209
Δ (Absolute)
+0.143 / +0.162
+0.058 / +0.069
+0.028 / +0.000
+0.108 / +0.125
-0.045 / -0.037
+0.099 / +0.162
Appendix
Table 9: Performance comparison of PACE using mPLUG-Owl2 backbone (Tier 1). Values are PLCC / SRCC.
Figure 6: Qualitative case studies of PACE on two open-ended dimensions. For each example, PACE first decomposes the target concept into observable sub-dimensions and constructs a Critic-filtered VQA protocol. Track 1 evaluates the target using the resulting protocol, while Track 2 positions it relative to the human-rated high-quality and low-quality anchors. The two continuous perception coefficients, α1 and α2 , are computed from logit-weighted probability distributions and fused to obtain the final score. Red crosses indicate rejected protocol components.
AI-generated image quality assessment (AIGIQA) requires jointly reasoning about perceptual fidelity and prompt alignment, two quality dimensions that are often treated as independent in existing AIGIQA models. However, by re-examining human ratings, we uncover a previously overlooked phenomenon: the two dimensions are interdependent and exhibit both competitive and cooperative interactions during human rating. This observation suggests that a unified model should neither collapse the two dimensions nor rigidly separate them, but rather adaptively negotiate their interplay. Motivated by this insight, we introduce an interaction-aware learning framework that models perception-alignment relations through adversarial and collaborative inference pathways. Instead of designing a rigid dual-branch architecture, our method employs a gated interaction module that dynamically routes features according to the inferred relationship between the two dimensions. Task-aware prompts further modulate the gating behaviour, enabling the model to switch between competition and cooperation when necessary. Experiments across multiple AIGIQA benchmarks demonstrate that our approach not only achieves state-of-the-art accuracy but also yields interpretable interaction patterns, offering a more faithful approximation of human judgment. The codes are available at https://github.com/LQAMEI/ACL-IQA.
Baoliang Chen, Qing Lin, Sijie Mai
Department of Computer Science, South China Normal University, China · Department of Data Science and Analytics, Hong Kong University of Science and Technology (Guangzhou)
Image Quality Assessment (IQA) in open-world environments remains challenging due to limited generalization and interpretability. Recent approaches based on multimodal large language models (MLLMs) introduce textual reasoning for quality prediction, yet their judgments rely heavily on semantically biased internal representations, making them insensitive to low-level perceptual degradations. We propose IQA-T1, a tool-based visual evidence reasoning framework that augments MLLM reasoning with explicit perceptual observations. During inference, the model autonomously invokes specialized analysis tools to generate structured visual evidence, such as noise residual maps, gradient statistics, and frequency spectra, which are progressively integrated into the reasoning process. To support this paradigm, we construct Q-Tool, a dataset containing 11k multimodal reasoning chains grounded in tool-generated evidence. Extensive experiments on seven IQA benchmarks show that IQA-T1 achieves the best overall performance across datasets while producing interpretable and evidence-grounded quality assessments. Code and dataset are available at https://github.com/zibuyu-02/IQA-T1.
Jinjian Wu, Jiaqi Tang, Wei Wei +5
School of Computer Science, Northwestern Polytechnical University, Xi’an, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
Vision-Language Models (VLMs) have been increasingly adopted for Image Quality Assessment (IQA). However, current methods typically employ a static one-shot scoring paradigm, despite the fact that humans assess image quality through dynamic visual inspection, e.g., selectively adjusting views to verify details and subtle artifacts. Specifically, relying solely on a single-pass observation introduces two primary limitations: first, perceiving the image only at a global scale restricts the assessment of finer local details; second, the original intensity distribution of the image may overwhelm the visibility, leading to insufficient inspection of image quality. To address these issues, we propose Tool-IQA, shifting the assessment mechanism from passive scoring to a tool-augmented workflow. In particular, we equip VLMs with simple yet effective view tools: a Magnifier to inspect local details, and a Gamma Corrector to uncover visibility and hidden artifacts. The assessment follows a structured pipeline that consists of an initial observation with rubric notes, a tool-augmented in-depth inspection, and a final quantification for calibrated quality score. Furthermore, to ensure efficient and purposeful tool callings, we introduce a batch-aware training strategy to reward tool interactions that can yield positive contributions rather than simply encouraging usage. Experiments on a variety of IQA benchmarks demonstrate that, with effective tool calling and calibrated assessment, our proposed Tool-IQA significantly outperforms existing state-of-the-art models, e.g., it achieves a PLCC of 0.854 on the challenging CLIVE dataset.
Guanyi Qin, Junjie Zhang, Chunming He +4
1National University of Singapore · 2OPPO Research Institute · 3Nanyang Technical University +3