Organizations: MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China · Department of Networked Intelligence, Pengcheng Laboratory, Shenzhen, China · Department of Computer Science and Shenzhen Research Institute, City University of Hong Kong, Kowloon, Hong Kong
Contemporary no-reference image quality assessment (NR-IQA) models can effectively quantify perceived image quality, often achieving strong correlations with human perceptual scores on standard IQA benchmarks. Yet, limited efforts have been devoted to treating NR-IQA models as natural image priors for real-world image enhancement, and consequently comparing them from a perceptual optimization standpoint. In this work, we show -- for the first time -- that NR-IQA models can be plugged into the maximum a posteriori (MAP) estimation framework for image enhancement. This is achieved by performing gradient ascent in the diffusion latent space rather than in the raw pixel domain, leveraging a pretrained differentiable and bijective diffusion process. Likely, different NR-IQA models lead to different enhanced outputs, which in turn provides a new computational means of comparing them. Unlike conventional correlation-based measures, our comparison method offers complementary insights into the respective strengths and weaknesses of the competing NR-IQA models in perceptual optimization scenarios. Additionally, we aim to improve the best-performing NR-IQA model in diffusion latent MAP estimation by incorporating the advantages of other top-performing methods. The resulting model delivers noticeably better results in enhancing real-world images afflicted by unknown and complex distortions, all preserving a high degree of image fidelity.
Figures & tables
Fig. 1: (a) Distorted image used as the initial estimate. (b) Enhancement output obtained by directly maximizing a state-of-the-art NR-IQA model, LIQE [ 11 ] (see Eq. ( 1 )). (c) Enhancement output obtained by MAP estimation (see Eq. ( 2 )), where a reduced-resolution Euclidean distance serves as the fidelity term and the LIQE score [ 11 ] defines the energy-based prior. (d) Enhancement output obtained by diffusion latent MAP estimation (see Eq. ( 17 )). Each image is accompanied by its LIQE-predicted quality score, which ranges from zero to a maximum of five, with higher values indicating better predicted quality.
Fig. 2: Optimized images via MAP estimation (see Eq. ( 2 )) for a range of λ values.
Fig. 3: (a) System diagram of diffusion latent MAP estimation. (b) NR-IQA model comparison and improvement enabled by the proposed diffusion latent MAP estimation framework.
Fig. 4: Representative photographic images used as inputs to the proposed enhancement framework.
NR-IQA Model
Training Set
# Parameters (in Millions)
SRCC Rank (SRCC)
MAP Rank
Δ Rank
NIQE [ 24 ]
–
0.001
8 (0.706)
8
0
CLIPIQA+ [ 40 ]
KonIQ-10k
101.349
2 (0.855)
6
-4
PaQ-2-PiQ [ 33 ]
FLIVE
11.704
5 (0.827)
7
-2
HyperIQA [ 89 ]
KonIQ-10k
27.375
7 (0.776)
4
3
MUSIQ [ 18 ]
KonIQ-10k
27.125
3 (0.853)
2
1
DBCNN [ 27 ]
KonIQ-10k
15.311
6 (0.789)
5
1
TABLE I: Global ranking of eight NR-IQA models, where lower ranks indicate better performance. “Multiple” denotes training on the combined LIVE, CSIQ, BID, CLIVE, KonIQ-10k, and KADID-10k datasets. We define Δ Rank = SRCC Rank − MAP Rank; thus, a positive value indicates a better rank under diffusion latent MAP estimation.
Fig. 5: Global ranking scores of eight NR-IQA models under diffusion latent MAP estimation, where higher scores indicate better performance. Models enclosed by the same colored box are statistically indistinguishable according to a two-tailed z -test at the 5% significance level.
Fig. 6: Visual comparison of MAP-refined images associated with different pairs of NR-IQA models.
Fig. 7: Visual comparison of MAP-refined images using vanilla and ControlNet-augmented Stable Diffusion 2.0.
Fig. 8: (a) Initial image with mixed, realistic degradations. (b) Enhancement output by direct diffusion latent optimization of LIQE [ 11 ] (equivalent to diffusion latent MAP estimation with λ=∞ ), which results in an overall color appearance that undesirably deviates from (a). (c) Enhancement output by the proposed diffusion latent MAP estimation using the same LIQE [ 11 ] .
Fig. 9: Visual comparison of MAP-refined images guided by MS-LIQE and its three predecessor models: LIQE, MUSIQ, and UNIQUE.
Fig. 10: Visual examples of the proposed post-enhancement method applied to task-specific enhancement outputs. From left to right, the columns show two super-resolution (SR) examples, one low-light image enhancement (LLIE) example, and one face restoration (FR) example.
Method
Super Resolution on Real60
Super Resolution on DPED
Face Restoration
MANIQA
Q-Align
VQ-R1
MANIQA
Q-Align
VQ-R1
MANIQA
Q-Align
VQ-R1
SeeSR
0.655 / 0.681
4.46 / 4.69
4.65 / 4.74
0.586 / 0.635
3.00 / 3.62
3.29 / 3.79
0.633 / 0.697
3.96 / 4.50
4.42 / 4.68
SUPIR
0.655 / 0.671
4.46 / 4.77
4.69 / 4.77
0.574 / 0.621
2.70 / 3.40
3.06 / 3.44
0.610 / 0.686
3.68 / 4.46
4.26 / 4.63
StableSR
0.641 / 0.668
3.89 / 4.53
4.51 / 4.73
0.536 / 0.604
2.43 / 3.22
2.72 / 3.56
0.608 / 0.671
3.60 / 4.38
4.11 / 4.60
PiSA-SR
0.688 / 0.703
4.40 / 4.71
4.70 / 4.81
0.617 / 0.658
2.85 / 3.44
3.22 / 3.73
0.663 / 0.701
4.11 / 4.65
4.51 / 4.76
TABLE II: Quantitative comparison of the baseline and post-enhanced outputs on super-resolution and face restoration. Each entry reports baseline / post-enhancement, and higher values indicate better performance for all metrics
Method
MANIQA
Q-Align
VQ-R1
QuadPrior
0.634 / 0.683
3.71 / 4.25
4.12 / 4.44
HVI-CIDNet
0.642 / 0.664
3.64 / 4.18
4.01 / 4.37
RetinexFormer
0.570 / 0.628
3.25 / 3.95
3.63 / 4.03
ISP-UNet
0.643 / 0.678
3.81 / 4.29
4.18 / 4.43
TABLE III: Quantitative comparison of the baseline and post-enhanced outputs on low-light image enhancement
Task
Baseline
Preference (%)
Super-Resolution on Real60
PiSA-SR
85.6
Super-Resolution on DPED
SeeSR
79.6
Face Restoration
PiSA-SR
88.0
Low-light Enhancement
ISP-UNet
76.8
TABLE IV: Percentage of trials in which the post-enhanced output is preferred to the strongest competing baseline for three tasks
Current no-reference image quality assessment (NR-IQA) models for enhanced images often struggle to generalize, as they tend to overfit to the distinct patterns of specific enhancement algorithms rather than evaluating genuine perceptual quality. To address this issue, we propose a preference-guided debiasing framework for no-reference enhancement image quality assessment (EIQA). Specifically, we first learn a continuous enhancement-preference embedding space using supervised contrastive learning, where images generated by similar enhancement styles are encouraged to have closer representations. Based on this, we further estimate the enhancement-induced nuisance component contained in the raw quality representation and remove it before quality regression. In this way, the model is guided to focus on algorithm-invariant perceptual quality cues instead of enhancement-specific visual fingerprints. To facilitate stable optimization, we adopt a two-stage training strategy that first learns the enhancement-preference space and then performs debiased quality prediction. Extensive experiments on public EIQA benchmarks demonstrate that the proposed method effectively mitigates algorithm-induced representation bias and achieves superior robustness and cross-algorithm generalization compared with existing approaches.
Shiqi Gao, Kang Fu, Zitong Xu +4
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
Over the past decades, numerous Image Quality Assessment (IQA) models have emerged, aiming to predict the perceptual quality of images. However, individual models are often biased toward certain types of image content or distortions, depending on the design principle and process. An intuitive idea is to harness the strengths and mitigate the weaknesses of each IQA model, by fusing the scores of multiple models into a stronger one. Here we make one of the first attempts to seek an optimal solution for the idea and propose a general framework for unsupervised IQA score fusion using deep Maximum a Posteriori (MAP) estimation. The proposed model conducts fine-grained uncertainty estimation at the score level to increase the accuracy and reduce the uncertainty in fused predictions. Comprehensive experiments demonstrate the superiority of the proposed model over individual IQA models and other fusion methods. It also exhibits an interesting capability of rejecting ``bad" models in the fusion process.
Zhongling Wang, Raymond Zhou, Shahrukh Athar +2
University of Waterloo, Canada · McMaster University, Canada
No-Reference Image Quality Assessment (NR-IQA) aims to estimate perceptual quality without access to a reference image of pristine quality. Learning an NR-IQA model faces a fundamental bottleneck: its need for a large number of costly human perceptual labels. We propose SHAMISA, a non-contrastive self-supervised framework that learns from unlabeled distorted images by leveraging explicitly structured relational supervision. Unlike prior methods that impose rigid, binary similarity constraints, SHAMISA introduces implicit structural associations, defined as soft, controllable relations that are both distortion-aware and content-sensitive, inferred from synthetic metadata and intrinsic feature structure. A key innovation is our compositional distortion engine, which generates an uncountable family of degradations from continuous parameter spaces, grouped so that only one distortion factor varies at a time. This enables fine-grained control over representational similarity during training: images with shared distortion patterns are pulled together in the embedding space, while severity variations produce structured, predictable shifts. We integrate these insights via dual-source relation graphs that encode both known degradation profiles and emergent structural affinities to guide the learning process throughout training. A convolutional encoder is trained under this supervision and then frozen for inference, with quality prediction performed by a linear regressor on its features. Extensive experiments on synthetic, authentic, and cross-dataset NR-IQA benchmarks demonstrate that SHAMISA achieves strong overall performance with improved cross-dataset generalization and robustness, all without human quality annotations or contrastive losses.
Mahdi Naseri, Zhou Wang
Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada