Fine-grained Image Aesthetic Assessment: Learning Discriminative Scores from Relative Ranks

Zhichao Yang; Jianjie Wang; Zhixianhe Zhang; Pangu Xie; Xiangfei Sheng; Pengfei Chen; Leida Li

cs.CVMar 4, 2026

Fine-grained Image Aesthetic Assessment: Learning Discriminative Scores from Relative Ranks

Authors: Zhichao Yang, Jianjie Wang, Zhixianhe Zhang, Pangu Xie, Xiangfei Sheng, Pengfei Chen, Leida Li

Organizations: School of Artificial Intelligence, Xidian University · State Key Laboratory of EMIM, Xidian University

Abstract

Image aesthetic assessment (IAA) has extensive applications in content creation, album management, and recommendation systems, etc. In such applications, it is commonly needed to pick out the most aesthetically pleasing image from a series of images with subtle aesthetic variations, a topic we refer to as fine-grained IAA. Unfortunately, state-of-the-art IAA models are typically designed for coarse-grained evaluation, where images with notable aesthetic differences are evaluated independently on an absolute scale. These models are inherently limited in discriminating fine-grained aesthetic differences. To address the dilemma, we contribute FGAesthetics, a fine-grained IAA database with 32,217 images organized into 10,028 series, which are sourced from diverse categories including Natural, AIGC, and Cropping. Annotations are collected via pairwise comparisons within each series. We also devise Series Refinement and Rank Calibration to ensure the reliability of data and labels. Based on FGAesthetics, we further propose FGAesQ, a novel IAA framework that learns discriminative aesthetic scores from relative ranks through Difference-preserved Tokenization (DiffToken), Comparative Text-assisted Alignment (CTAlign), and Rank-aware Regression (RankReg). FGAesQ enables accurate aesthetic assessment in fine-grained scenarios while still maintains competitive performance in coarse-grained evaluation. Extensive experiments and comparisons demonstrate the superiority of the proposed method.

Figures & tables

Dataset	Year	Num. Img	Data Type	Img Source	Label Type	Evaluation Concern
CUHK-PQ [ 25 ]	2011	17,673	Single	Natural	Classification	Aesthetic Classification
AVA [ 26 ]	2012	255,530	Single	Natural	MOS	Aesthetic Scoring
AADB [ 21 ]	2016	10,000	Single	Natural	MOS	Aesthetic Attributes
TAD66K [ 11 ]	2022	66,327	Single	Natural/Art	MOS	Theme-specific Aesthetics
BAID [ 48 ]	2023	60,337	Single	Art	MOS	Artistic Aesthetics
ICAA17K [ 12 ]	2023	17,726	Single	Natural	MOS	Color Aesthetics

Table 1: Comparison between the proposed FGAesthetics and representative IAA datasets.

Figure 2: Overview of the construction pipeline for FGAesthetics . The pipeline consists of three stages: (a) Data Collection. Visually similar photo series are collected from three distinct sources: Natural, AIGC, and Cropping. (b) Series Refinement. Noisy series data undergo rigorous filtering using a Metric-MLLMs-Human refinement protocol. (c) Rank Calibration. Pairwise comparisons are annotated within each series, excluding data that cannot be aesthetically distinguished to obtain calibrated aesthetic rankings.

Figure 3: Statistical analysis of FGAesthetics. (a) Image and series count distribution across Natural, AIGC, and Cropping. (b) Within-series similarity distributions measured by LPIPS (low-level) [ 51 ] , DreamSim (mid-level) [ 9 ] , and CLIPScore (high-level) [ 13 ] across three sources. Note that LPIPS and DreamSim are subtracted from 1, ensuring consistent polarity with CLIPScore, where higher values indicate greater similarity.

Figure 4: Overall pipeline of the proposed FGAesQ . FGAesQ learns discriminative aesthetic scores from relative ranks through three modules: (a) Difference-preserved Tokenization (DiffToken) selectively maintains difference regions at their original resolution while downscaling others. (b) Comparative Text-assisted Alignment (CTAlign) achieves distinctive aesthetic visual representations. (c) Rank-aware Regression (RankReg) rectifies the coarse-grained score regression with fine-grained aesthetic rankings.

Methods	Params	Natural				AIGC				Cropping
		Pair-level		Series-level		Pair-level		Series-level		Pair-level		Series-level
		Acc	F1	s-Acc	s-SRCC	Acc	F1	s-Acc	s-SRCC	Acc	F1	s-Acc	s-SRCC
NIMA (TIP 2018) [ 36 ]	54.3M	0.589	0.588	0.493	0.225	0.566	0.565	0.379	0.137	0.655	0.649	0.272	0.312
MLSP (CVPR 2019) [ 15 ]	114 M	0.608	0.607	0.531	0.255	0.604	0.602	0.414	0.182	0.710	0.710	0.396	0.430
MUSIQ (ICCV 2021) [ 17 ]	78.6M	0.607	0.606	0.487	0.233	0.535	0.532	0.344	0.054	0.731	0.731	0.375	0.495
TANet (IJCAI 2022) [ 11 ]	13.9M	0.629	0.628	0.539	0.281	0.591	0.590	0.447	0.195	0.641	0.641	0.244	0.295

Table 2: Performance comparison between the proposed FGAesQ and state-of-the-art IAA methods on FGAesthetics. Results are reported across all three image sources (Natural, AIGC, and Cropping) for both pair-level and series-level evaluations. Model parameter counts are also provided. The top two results are highlighted in bold and underlined , respectively.

Methods	AVA		FGAesthetics
	SRCC	PLCC	Pair	Series
NIMA (TIP 2018) [ 36 ]	0.633	0.647	0.602	0.303
MUSIQ (ICCV 2021) [ 17 ]	0.726	0.738	0.624	0.331
VILA (CVPR 2023) [ 18 ]	0.774	0.774	0.659	0.399
Charm (CVPR 2025) [ 3 ]	0.777	0.779	0.665	0.419
NIMA (Fine-tuned) [ 36 ]	0.330 ( -0.303)	0.313 ( -0.334)	0.649 ( +0.047)	0.350 ( +0.047)

Table 3: Balance between coarse-grained (AVA) and fine-grained (FGAesthetics) evaluations. Metrics shown are ( Acc , F1 )/2 for Pair and ( s-Acc , s-SRCC )/2 for Series. Best in bold.

Methods	ICAA17K		AADB		TAD66K
	SRCC	PLCC	SRCC	PLCC	SRCC	PLCC
NIMA [ 36 ]	0.629	0.629	0.365	0.371	0.392	0.370
MUSIQ [ 17 ]	0.581	0.578	0.331	0.343	0.314	0.332
VILA [ 18 ]	0.652	0.663	0.548	0.561	0.413	0.443
UNIAA [ 53 ]	0.472	0.390	-	-	0.261	0.260
Charm [ 3 ]	0.651	0.661	0.510	0.528	0.411	0.441

Table 4: Cross-dataset validation on three IAA benchmarks: ICAA17K [ 12 ] , AADB [ 21 ] , and TAD66K [ 11 ] . Results for UNIAA [ 53 ] on AADB are omitted as data is used in training.

Methods	Coarse-grained		Fine-grained
	SRCC	PLCC	Pair	Series
w/o Fine	0.713	0.726	0.578	0.364
w/o Coarse	0.031	0.050	0.565	0.299
Coarse \| Fine	0.200	0.214	0.637	0.380
w/o DiffToken	0.751	0.760	0.666	0.423
w/o CTAlign	0.770	0.780	0.747	0.581

Table 5: Ablation study on training strategies and model components. ‘w/o’ is removal of the specified training or component. Metrics are ( Acc , F1 )/2 for Pair and ( s-Acc , s-SRCC )/2 for Series.

Figure 5: Visualization of evaluation results on three test series. Images arranged left-to-right in decreasing aesthetic quality. Red boxes and text indicate the best aesthetics.

Sources	Collected Img Count	After Series Refinement	After Rank Calibration	Series Count	Pair Count	Avg. Pairs per Series
SPS [ 5 ]	15,543	12,522	12,522	4,755	12,558	2.64
LSVQ [ 49 ]	2630	565	556	150	636	4.24
Pick-a-pic [ 20 ]	13,917	3,751	3,719	1,685	2,115	1.26
Q-Eval-100K [ 52 ]	11,430	418	372	180	196	1.09
Midjourney [ 14 ]	11,352	2,815	2,599	909	2,124	2.34
NIGHTS [ 9 ]	9,822	1,944	1,895	647	1,590	2.46

Table 6: Source Distribution and Statistics of FGAesthetics

Figure 6: Detailed Procedure of Metrics Filtering. (a) Generic measures (average of SSIM and SIFT) exclude outliers within visually similar photo series. (b) Domain-specific measures (IoU for cropping series, T2I alignment scores for AIGC series) effectively identify nearly identical and dissimilar images.

Figure 7: Annotation platform for Human Qualification. The platform displays all images from a single series on one page, enabling annotators to identify and filter outliers.

Figure 8: Annotation platform for Pairwise Comparison. The platform displays two images side-by-side for aesthetic comparison. Annotators make a forced-choice selection from three options: ’A is better’, ’B is better’, or ’Equally favored’.

Figure 9: Word clouds visualizing most frequent rationales for MLLM-generated comparative textual descriptions.

Backbone	Natural				AIGC				Cropping				Fine-grained		Coarse-grained
	Acc	F1	s-Acc	s-SRCC	Acc	F1	s-Acc	s-SRCC	Acc	F1	s-Acc	s-SRCC	Pair	Series	SRCC	PLCC
ViT-B/16	0.779	0.779	0.753	0.729	0.709	0.707	0.561	0.482	0.774	0.773	0.488	0.590	0.753	0.600	0.770	0.781
ViT-B/32	0.711	0.711	0.634	0.503	0.653	0.651	0.445	0.301	0.740	0.740	0.491	0.548	0.701	0.487	0.747	0.760

Table 7: Backbone ablation results on fine-grained and coarse-grained IAA tasks. For fine-grained evaluation on FGAesthetics, we report pair-level local discrimination using Acc and F1 , and series-level ranking quality using s-Acc and s-SRCC across three source categories (Natural, AIGC, Cropping). Pair and Series represent category-averaged values of ( Acc + F1 )/2 and ( s-Acc + s-SRCC )/2, respectively. For coarse-grained evaluation, we report SRCC and PLCC on AVA [ 26 ] .

Backbone	Natural				AIGC				Cropping				Fine-grained		Coarse-grained
	Acc	F1	s-Acc	s-SRCC	Acc	F1	s-Acc	s-SRCC	Acc	F1	s-Acc	s-SRCC	Pair	Series	SRCC	PLCC
FGAesQ	0.779	0.779	0.753	0.729	0.709	0.707	0.561	0.482	0.774	0.773	0.488	0.590	0.753	0.600	0.770	0.781
w/o Natural	0.745	0.745	0.710	0.630	0.704	0.703	0.569	0.455	0.771	0.771	0.488	0.583	0.740	0.573	0.770	0.782
w/o AIGC	0.767	0.766	0.720	0.675	0.687	0.685	0.542	0.425	0.765	0.765	0.482	0.577	0.739	0.570	0.776	0.786
w/o Cropping	0.765	0.764	0.727	0.678	0.699	0.698	0.552	0.438	0.755	0.755	0.462	0.558	0.739	0.569	0.772	0.783

Table 8: Out-of-Distribution generalization performance. OOD generalization is evaluated by training FGAesQ with one source category excluded and testing on fine-grained (all three categories) and coarse-grained (AVA [ 26 ] ) benchmarks.

Figure 10: Ablation study on DiffToken configuration. Performance is evaluated across different difference localization patch sizes and percentile thresholds for identifying aesthetics-decisive regions. Fine-grained metrics show pair- and series-averaged performance across Natural, AIGC, and Cropping categories. Coarse-grained metrics report SRCC and PLCC on AVA [ 26 ] .

Figure 11: Additional visualization examples of FGAesQ on test series from Natural, AIGC, and Cropping categories.

Explore similar work

Beyond Absolute Scores: Relative Edit-induced Difference for Generalizable Image Aesthetic Assessment
Jun 4, 2026Qifei Jia, Xintong Yao, Yasen Zhang +7Personalized Image Aesthetics Assessment Image Quality Assessment
Xiaomi Corporation, Beijing, China
Preferences Order, Ratings Anchor: From Fused Expert Aesthetic Ground Truth to Self-Distillation
May 19, 2026Yuanpei Zhao, Jie Lin, Chao Zhang +5Pairwise Preference Learning Personalized Image Aesthetics Assessment
1Sichuan University · 2NetEase Fuxi AI Lab · †Work done during an internship at NetEase Fuxi AI Lab. +1
From Concepts to Judgments: Interpretable Image Aesthetic Assessment
Mar 18, 2026Xiao-Chang Liu, Johan WagemansConcept-Based Explanations Interpretable ML
Research Unit Brain and Cognition, KU Leuven Leuven, Belgium

Jun 4, 2026J/K move · Enter open · S save

Beyond Absolute Scores: Relative Edit-induced Difference for Generalizable Image Aesthetic Assessment

Qifei Jia, Xintong Yao, Yasen Zhang, Minghao Li+6

Xiaomi Corporation, Beijing, China

Personalized Image Aesthetics Assessment Image Quality Assessment

Open paper AskPDF arXiv

Traditional Image Aesthetic Assessment (IAA) methods mainly rely on regressing absolute Mean Opinion Scores (MOS). However, such a paradigm overlooks the inherently dynamic nature of human aesthetic perception, which relies on subconscious comparison against implicit visual references. Consequently, the lack of causal reasoning regarding aesthetic differences prevents models from learning generalizable aesthetic principles, thus limiting their generalization across diverse scenarios. In this work, we rethink the IAA task and propose Relative Edit-induced Difference Aesthetic learning (RED-Aes), a novel framework that leverages controllable image editing models to simulate the human aesthetic reasoning process. Instead of fitting absolute score distributions, RED-Aes explicitly learns the visual factors that drive aesthetic changes. To support this paradigm, we construct the RED-20k dataset, which comprises editing-based image pairs, quantitative aesthetic differences, and Chain-of-Thought (CoT) reasoning. Furthermore, we introduce a three-stage training strategy guided by a relative ranking consistency reward, optimizing the model solely via relative supervision. Extensive experiments demonstrate that RED-Aes achieves state-of-the-art performance on multiple public benchmarks, exhibiting superior generalization capabilities.

Abstract

Explore similar work

Beyond Absolute Scores: Relative Edit-induced Difference for Generalizable Image Aesthetic Assessment

Preferences Order, Ratings Anchor: From Fused Expert Aesthetic Ground Truth to Self-Distillation

From Concepts to Judgments: Interpretable Image Aesthetic Assessment