MCIQA-2K: A Multi-Dimensional Dataset and No-Reference Quality Assessment Benchmark for Colorized Images

Yunkai Zhuang; Qihang Yan; Zicheng Zhang; Guangtao Zhai

cs.CVSep 13, 2026

MCIQA-2K: A Multi-Dimensional Dataset and No-Reference Quality Assessment Benchmark for Colorized Images

Authors: Yunkai Zhuang, Qihang Yan, Zicheng Zhang, Guangtao Zhai

Organizations: School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China · Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China · School of Electronics, Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

Abstract

Image colorization is an inherently ill-posed task, since a single grayscale image may correspond to multiple plausible colorized results. Consequently, conventional full-reference image quality assessment (IQA) metrics fail to accurately reflect human perceptual preferences for colorized images. In this paper, we present MCIQA-2K, a large-scale multi-dimensional benchmark specifically designed for no-reference quality assessment of colorized images. We construct a dataset containing 2,000 colorized images generated by five representative colorization models, together with human annotations across three perceptual dimensions: color smearing, semantic color misalignment, and global naturalness. Building upon the proposed benchmark, we further introduce MCIQA, a dedicated multi-branch NR-IQA framework for colorized images. Extensive experiments demonstrate that MCIQA significantly outperforms existing full-reference and no-reference IQA methods on the proposed benchmark, while also exhibiting competitive generalization capability on several widely-used IQA datasets. The dataset and code are publicly available at https://github.com/ARBEZ-ZEBRA/MCIQA.

Figures & tables

Figure 1: Example image samples of MCIQA-2K dataset.

Figure 2: Illustration of MOS distribution across all samples.

Figure 3: Color smearing, where colors bleed across object boundaries, resulting in spatial inconsistency.

Figure 4: Semantic color misalignment, where objects are assigned implausible colors that contradict real-world semantics.

Figure 5: Illustration of the architecture of MCIQA.

Metric	MCIQA-2K-CS		MCIQA-2K-SCM		MCIQA-2K-GN
Metric	SRCC	PLCC	SRCC	PLCC	SRCC	PLCC
PSNR $\uparrow$	0.6131	0.6229	0.3710	0.4037	0.4644	0.4797
SSIM $\uparrow$	-0.0268	0.0136	-0.1332	-0.0682	0.0639	0.0240
LPIPS $\downarrow$	-0.5559	-0.5735	-0.4847	-0.5205	-0.5122	-0.5164
DISTS $\downarrow$	-0.3527	-0.3749	-0.2686	-0.2938	-0.3508	-0.3650
BRISQUE $\downarrow$	0.0062	0.0200	-0.0101	0.0248	0.0563	0.0591

Table 1: Performance comparison of different metrics on the MCIQA-2K dataset based on the value of SRCC and PLCC. The best results are highlighted in red . The second best results are highlighted in blue . The reference images of all FR metrics are the original RGB images from COCO test set [ 20 ] .

\uparrow

means 1 is the best and -1 is the worst.

\downarrow

means -1 is the best and 1 is the worst. The value of each IQA on each dataset is the average result of 3 experiments with different random seeds.

Metric	KONIQ-10K		BIQ2021		AGIQA-1K		AGIQA-3K
Metric	SRCC	PLCC	SRCC	PLCC	SRCC	PLCC	SRCC	PLCC
BRISQUE $\downarrow$	-0.1797	-0.1603	-0.4502	-0.4674	-0.4958	-0.5387	-0.4728	-0.5007
NIQE $\downarrow$	-0.3107	-0.2330	-0.4366	-0.4735	-0.5594	-0.5806	-0.5075	-0.4847
CNN-IQA $\uparrow$	0.7199	0.7667	0.5067	0.5531	0.1461	0.3136	0.6006	0.7724
DBCNN $\uparrow$	0.9021	0.9198	0.6743	0.7270	0.4502	0.4620	0.6398	0.7483
WaDIQaM $\uparrow$	0.7824	0.8188	0.5797	0.6407	0.2675	0.3609	0.5705	0.7254

Table 2: Performance comparison of different metrics on the different datasets based on the value of SRCC and PLCC. The value of each IQA on each dataset is the average result of 3 experiments with different random seeds.

Explore similar work

SHAMISA: SHAped Modeling of Implicit Structural Associations for Self-supervised No-Reference Image Quality Assessment
Mar 14, 2026Mahdi Naseri, Zhou WangImage Quality Assessment Self-Supervised Learning
Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada
Semantic Color Naturalness Breaker: Preventing Illegitimate Colorization via Content-Aware Color Priors
Jul 20, 2026Yuki Nii, Futa Waseda, Ching-Chun Chang +1Adversarial Examples
The University of Tokyo, Japan · National Institute of Informatics, Japan
Image Quality Assessment: Exploring Quality Awareness via Memory-driven Distortion Patterns Matching
Feb 20, 2026Xuting Lan, Mingliang Zhou, Xuekai Wei +5Image Quality Assessment

Mar 14, 2026J/K move · Enter open · S save

SHAMISA: SHAped Modeling of Implicit Structural Associations for Self-supervised No-Reference Image Quality Assessment

Mahdi Naseri, Zhou Wang

Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada

Image Quality Assessment Self-Supervised Learning Self-Supervised Visual Representation Learning

Open paper AskPDF arXiv

No-Reference Image Quality Assessment (NR-IQA) aims to estimate perceptual quality without access to a reference image of pristine quality. Learning an NR-IQA model faces a fundamental bottleneck: its need for a large number of costly human perceptual labels. We propose SHAMISA, a non-contrastive self-supervised framework that learns from unlabeled distorted images by leveraging explicitly structured relational supervision. Unlike prior methods that impose rigid, binary similarity constraints, SHAMISA introduces implicit structural associations, defined as soft, controllable relations that are both distortion-aware and content-sensitive, inferred from synthetic metadata and intrinsic feature structure. A key innovation is our compositional distortion engine, which generates an uncountable family of degradations from continuous parameter spaces, grouped so that only one distortion factor varies at a time. This enables fine-grained control over representational similarity during training: images with shared distortion patterns are pulled together in the embedding space, while severity variations produce structured, predictable shifts. We integrate these insights via dual-source relation graphs that encode both known degradation profiles and emergent structural affinities to guide the learning process throughout training. A convolutional encoder is trained under this supervision and then frozen for inference, with quality prediction performed by a linear regressor on its features. Extensive experiments on synthetic, authentic, and cross-dataset NR-IQA benchmarks demonstrate that SHAMISA achieves strong overall performance with improved cross-dataset generalization and robustness, all without human quality annotations or contrastive losses.

Abstract

Explore similar work

SHAMISA: SHAped Modeling of Implicit Structural Associations for Self-supervised No-Reference Image Quality Assessment

Semantic Color Naturalness Breaker: Preventing Illegitimate Colorization via Content-Aware Color Priors

Image Quality Assessment: Exploring Quality Awareness via Memory-driven Distortion Patterns Matching