Organizations: Harbin Institute of Technology, Shenzhen · Independent Researcher · College of Computer Science and Aritificial Intelligence, Fudan University · Institute for Artificial Intelligence, Peking University · Peng Cheng Laboratory
Recent advances in text-to-image (T2I) generation have greatly improved visual quality, yet producing images that appear visually authentic to real-world photography remains challenging. This is partly due to biases in existing evaluation paradigms: human ratings and preference-trained metrics often favor visually vivid images with exaggerated saturation and contrast, which make generations often too vivid to be real even when prompted for realistic-style images. To address this issue, we present Color Fidelity Dataset (CFD) and Color Fidelity Metric (CFM) for objective evaluation of color fidelity in realistic-style generations. CFD contains over 1.3M real and synthetic images with ordered levels of color realism, while CFM employs a multimodal encoder to learn perceptual color fidelity. In addition, we propose a training-free Color Fidelity Refinement (CFR) that adaptively modulates spatial-temporal guidance scale in generation, thereby enhancing color authenticity. Together, CFD supports CFM for assessment, whose learned attention further guides CFR to refine T2I fidelity, forming a progressive framework for assessing and improving color fidelity in realistic-style T2I generation. The dataset and code are available at https://github.com/ZhengyaoFang/CFM.
Figures & tables
Figure 1: A. Challenges in existing T2I generation and evaluation. (1) Statistical analysis shows that when prompted to produce realistic-style outputs, most T2I models generate images with higher saturation and contrast than real-world photographs. (2) A controlled saturation-scaling experiment on real photographs reveals that existing evaluation models exhibit a strong bias toward highly saturated images. The vertical axis shows the normalized score difference with respect to the reference scale (=1.0). B. Definition of color fidelity and the target of our CFM. Color fidelity measures how closely generated images preserve the natural color distribution of real-world photography, which serves as the learning objective of our proposed Color Fidelity Metric (CFM).
Figure 2: Overview of the Color Fidelity Dataset. We first apply IQA filtering process to obtain about 190K high-quality real-world images across 12 categories. Through automatic caption generation and guidance-controlled image synthesis , we further construct 1.33M images exhibiting ordered levels of color fidelity, and divide them into training and testing splits.
Figure 3: Framework of CFM and training pipeline. CFM employs Qwen2-VL as a multimodal feature backbone and an MLP projection head to map features into scalar fidelity scores. It is trained on the CFD-Training with group-wise, order-preserving samples, optimized by the softrank loss within each group.
Figure 4: Benchmark results of color fidelity evaluation across different text-to-image models. Left: category-wise CFM scores across 12 categories. Right: average CFM scores obtained by each model using our benchmark.
Model
CFD-SynPairs
CFD-Real&Syn
Traditional image quality measures
MUSIQ [ 13 ]
53.4
21.5
MANIQA [ 35 ]
54.5
24.0
CLIPIQA [ 29 ]
44.7
22.7
LIQE [ 37 ]
50.1
23.6
Generative aesthetic scoring methods
Table 1: Color fidelity prediction accuracy (%) on CFD-Test.
Figure 5: Examples of evaluation bias between existing metrics and ours. Existing metrics perform less favorably due to their training bias toward vivid, high-contrast images, whereas our proposed CFM provides a more accurate assessment of color realism, assigning higher scores to images with naturally balanced and authentic colors.
Metric
Spearman ( ρ ) ↑
Pearson ( r ) ↑
Kendall ( τ ) ↑
ImageReward [ 34 ]
62.8
63.5
49.2
PickScore [ 14 ]
71.2
72.1
56.5
MPS [ 36 ]
66.7
78.5
52.8
HPSv3 [ 18 ]
74.4
76.0
62.8
Ours
84.9
85.4
71.4
Table 2: Correlation of CFM scores on the CFD-Human.
Figure 6: Qualitative examples of different CFR variants. ‘Origin’ denotes the original image before applying CFR, while ‘CFR’ represents the results enhanced by the corresponding model-based CFR. The bottom-right value in each image indicates its CFM score, where a higher score reflects better color fidelity.
Model
Setting
FID ↓
CLIPScore ↑
Δ Sat. ↓
CFM ↑
SD3.5 [ 6 ]
−
13.3
28.2
0.15
4.9
CFR HPSv3
13.2
28.1
0.11
5.6
CFR CFM
13.1
28.2
0.07
6.9
PixArt- Σ [ 3 ]
−
16.5
27.2
0.09
4.4
CFR HPSv3
16.4
27.4
0.05
5.1
CFR CFM
16.4
27.5
0.02
6.4
Table 3: Quantitative results of CFR.
Variant
CFD-Test Acc. (%)
CFD-Human Corr.
SynPairs
Real&Syn
Spearman ↑
Kendall ↑
Pairwise loss
76.2
74.8
78.3
66.1
Visual-only
77.1
74.3
78.9
67.0
Ours
83.6
80.1
84.9
71.4
Table 4: Ablation on CFM components.
Setting
FID ↓
CLIPScore ↑
Δ Sat. ↓
CFM ↑
Baseline ( s0 )
13.3
28.2
0.15
4.9
Temporal-only
18.0
25.9
0.18
-1.3
Spatial-only
13.2
28.2
0.12
6.8
Full (ours)
13.0
28.2
0.07
6.9
Table 5: Ablation on CFR: Spatial–temporal modulation.
Figure 7: Qualitative comparison of different CFR variants on SD3.5. The temporal-only variant causes semantic inconsistency and severely distorted colors, while the spatial-only variant mitigates over-saturation but yields suboptimal detail preservation. The full spatial–temporal CFR achieves balanced and natural color rendering with the highest CFM scores.
Table 7: Category distribution of the Color Fidelity Dataset (CFD).
Figure 10: Visualization examples of CFR. The CFM score for each image is shown in the lower-right corner.
Figure 11: Visualization examples of CFD.
Figure 12: Visualization examples of CFM scores on realistic-style generations produced by different T2I models. The CFM score for each image is shown in the lower-right corner.
Figure 13: Visualization examples of CFM scores on realistic-style generations produced by different T2I models. The CFM score for each image is shown in the lower-right corner.