Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generation

Dian Xie; Shitong Shao; Lichen Bai; Zikai Zhou; Bojun Cheng; Shuo Yang; Jun Wu; Zeke Xie

cs.CVFeb 26, 2026

Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generation

Authors: Dian Xie, Shitong Shao, Lichen Bai, Zikai Zhou, Bojun Cheng, Shuo Yang, Jun Wu, Zeke Xie

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Harbin Institute of Technology (Shenzhen) · Cogniser Information Technology

Abstract

Classifier-free guidance (CFG) has helped diffusion models achieve great conditional generation in various fields. Recently, more diffusion guidance methods have emerged with improved generation quality and human preference. However, can these emerging diffusion guidance methods really achieve solid and significant improvements? In this paper, we rethink recent progress on diffusion guidance. Our work mainly consists of four contributions. First, we reveal a critical evaluation pitfall that common human preference models exhibit a strong bias towards large guidance scales. Simply increasing the CFG scale can easily improve quantitative evaluation scores due to strong semantic alignment, even if image quality is severely damaged (e.g., oversaturation and artifacts). Second, we introduce a novel guidance-aware evaluation (GA-Eval) framework that employs effective guidance scale calibration to enable fair comparison between current guidance methods and CFG by identifying the effects orthogonal and parallel to CFG effects. Third, motivated by the evaluation pitfall, we design Transcendent Diffusion Guidance (TDG) method that can significantly improve human preference scores in the conventional evaluation framework but actually does not work in practice. Fourth, in extensive experiments, we empirically evaluate recent eight diffusion guidance methods within the conventional evaluation framework and the proposed GA-Eval framework. Notably, simply increasing the CFG scales can compete with most studied diffusion guidance methods, while all methods suffer severely from winning rate degradation over standard CFG. Our work would strongly motivate the community to rethink the evaluation paradigm and future directions of this field.

Figures & tables

Panel of Figure 1 — Figure 1: The HPS v2 scores of generated images under different CFG scales $\omega\in\{5.5,10,15,20\}$ , respectively. Model: Stable Diffusion-XL. HPSv2 exhibit a strong bias to large CFG scales in a wide range, even if generation quality starts to degrade due to too strong guidance.

Figure 2: Visual comparison of different methods with their corresponding results under effective guidance scale $\omega^{\mathrm{e}}$ . The e-CFG method, namely standard CFG with effective CFG scales calibrated by GA-Eval, can easily achieve performance improvement comparable to most recent diffusion guidance or sampling methods.

Figure 3: Different evaluation metrics under different guidance scales. Model: Stable Diffusion-XL. Dataset: Pick-a-Pic. Except AES and PickScore, other metrics would give higher ratings on images generated with a larger guidance scale within

\omega\in[5.5,20]

Figure 4: Comparison of classifier-free guidance Ho and Salimans (2021) and transcendent diffusion guidance (TDG).

Figure 5: The winning rate

\eta^{\text{e-CFG}}

compared to effective CFG and their degradation

\Delta\eta

of different methods on HPD dataset. Left:

\eta^{\text{e-CFG}}

. Right:

\Delta\eta

. Among all methods, applying

\omega^{\mathrm{e}}

has minor influence to APG. Meanwhile, AES demonstrated negative

\Delta\eta

on many methods, which means AES would give worse ratings to images generated with a large guidance scale.

Method	HPS v2	ImageReward	AES	PickScore	CLIPScore	Average $\eta$	$\omega^{\mathrm{e}}$
Pick-a-Pic	( $\eta^{\text{CFG}}/\eta^{\text{e-CFG}}/\Delta\eta$ )					( $\eta^{\text{CFG}}$ / $\eta^{\text{e-CFG}}$ )
Z-Sampling	90% / 74% / 16%	75% / 63% / 12%	65% / 76% /-11%	72% / 73% / -1%	64% / 59% / 5%	73% / 69%	13.51
CFG++	82% / 68% / 14%	70% / 58% / 12%	39% / 53% /-14%	56% / 57% / -1%	60% / 54% / 6%	61% / 58%	8.91
SAG	75% / 59% / 16%	45% / 34% / 11%	64% / 66% / -2%	62% / 56% / 6%	53% / 44% / 9%	60% / 52%	8.14
TDG	62% / 48% / 14%	61% / 46% / 15%	49% / 42% / 7%	51% / 51% / 0%	63% / 53% / 10%	57% / 48%	8.27
SEG	60% / 56% / 4%	59% / 49% / 10%	57% / 56% / 1%	35% / 33% / 2%	47% / 42% / 5%	52% / 47%	6.10

Table 1: The winning rates

\eta^{\text{CFG}},\eta^{\text{e-CFG}}

and its degradation

\Delta\eta

of different methods on various datasets. Model: Stable Diffusion-XL (

\omega=5.5

). We sort these methods based on average

\eta^{\text{e-CFG}}

, labeling those greater than 55% in blue and the others in red . Most of methods have

\omega^{\mathrm{e}}>\omega

, and suffering from severe winning rate degradation. Only recent Z-Sampling can still maintain high winning rates

\eta^{\text{e-CFG}}

after setting CFG scale as

\omega^{\mathrm{e}}

, while some methods even cannot compete with effective CFG.

Method	Single object ( $\uparrow$ )	Two object ( $\uparrow$ )	Counting ( $\uparrow$ )	Colors ( $\uparrow$ )	Position ( $\uparrow$ )	Color attribution ( $\uparrow$ )	Overall ( $\uparrow$ )	$\omega^{\mathrm{e}}$
CFG	98.91%	75.63%	37.66%	85.11%	10.87%	26.88%	55.84%	5.5
TDG	100.00%	76.77%	53.75%	87.23%	11.00%	21.00%	58.29%	8.28
e-CFG	100.00%	73.74%	47.50%	86.17%	9.00%	27.00%	57.23%	8.28
Z-Sampling	100.00%	75.76%	46.25%	84.04%	17.00%	20.00%	57.18%	16.05
e-CFG	98.75%	77.78%	36.25%	85.11%	16.00%	25.00%	56.48%	16.05
CFG++	100.00%	81.82%	46.25%	85.11%	16.00%	20.00%	58.20%	8.81

Table 2: The quantitative results of different methods on GenEval. Model: Stable Diffusion-XL (

\omega=5.5

). We labeled methods surpassed effective CFG with blue , and others with red . Increasing the CFG scale can also significantly increase the GenEval scores.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: The qualitative results of TDG.

Figure 7: Comparison of HPS v2 Score of TDG and CFG at different timesteps. Model: SD-3.5.

Figure 8: TDG vs. CFG in manifold space. Compared to CFG, TDG enlarges the search space to a hyperplane.

Figure 9: TDG hyperparameter grid search heatmap, with HPSv2 winning rate compared to CFG as the criteria. Model: SD-XL.

Figure 10: Different mask ratio of prompt tokens in TDG. Model: SD-XL.

Panel of Figure 11 — Figure 11: The HPS v2 scores of generated images under different CFG scales $\omega\in\{5.5,10,15,20\}$ , respectively. Model: Stable Diffusion-XL. HPSv2 exhibit a strong bias to large CFG scales in a wide range, even if generation quality starts to degrade due to too strong guidance.

Method	HPS v2	ImageReward	AES	PickScore	CLIPScore	Average $\eta$	$\omega^{\mathrm{e}}$
Pick-a-Pic	( $\eta^{\text{CFG}}/\eta^{\text{e-CFG}}/\Delta\eta$ )					( $\eta^{\text{CFG}}$ / $\eta^{\text{e-CFG}}$ )
PAG	53% / 53% / 0%	44% / 43% / 1%	53% / 57% / -4%	36% / 43% / -7%	44% / 41% / 3%	46% / 47%	5.33
TDG	61% / 48% / 13%	50% / 51% / -1%	45% / 51% / -6%	38% / 47% / -9%	53% / 47% / 6%	49% / 49%	6.62
Z-Sampling	40% / 36% / 4%	41% / 54% /-13%	46% / 49% / -3%	31% / 37% / -6%	53% / 58% / -5%	42% / 47%	9.57
DrawBench	( $\eta^{\text{CFG}}/\eta^{\text{e-CFG}}/\Delta\eta$ )					( $\eta^{\text{CFG}}$ / $\eta^{\text{e-CFG}}$ )
PAG	51% / 50% / 2%	44% / 40% / 3%	61% / 64% / -3%	42% / 40% / 2%	39% / 42% / -3%	48% / 47%	5.41

Appendix

Table 3: The winning rate and its degradation of different methods on various datasets. Model: Stable Diffusion-3.5 (

\omega=4.5

Method	HPS v2	ImageReward	AES	PickScore	CLIPScore	Average $\eta$	$\omega^{\mathrm{e}}$
Pick-a-Pic	( $\eta^{\text{CFG}}/\eta^{\text{e-CFG}}/\Delta\eta$ )					( $\eta^{\text{CFG}}$ / $\eta^{\text{e-CFG}}$ )
Z-Sampling	90% / 60% / 30%	80% / 62% / 18%	70% / 63% / 7%	84% / 73% / 11%	71% / 46% / 25%	79% / 61%	11.80
CFG++	82% / 62% / 20%	73% / 58% / 15%	58% / 55% / 3%	65% / 49% / 16%	57% / 55% / 2%	67% / 56%	9.09
FreeU	64% / 61% / 3%	73% / 67% / 6%	63% / 57% / 6%	52% / 45% / 7%	50% / 47% / 3%	60% / 55%	6.57
PAG	48% / 45% / 3%	49% / 41% / 8%	54% / 55% / -1%	49% / 45% / 4%	46% / 44% / 2%	49% / 46%	5.62
APG	11% / 31% /-20%	21% / 43% /-22%	49% / 53% / -4%	26% / 34% / -8%	45% / 58% /-13%	30% / 44%	11.19

Appendix

Table 4: The winning rates and the degradation of different methods on various datasets. Model: Stable Diffusion-2.1 (

\omega=5.5

Dataset	Method	HPS v2 ( $\uparrow$ )	ImageReward ( $\uparrow$ )	AES ( $\uparrow$ )	PickScore ( $\uparrow$ )	CLIPScore ( $\uparrow$ )
Pick-a-Pic	CFG	28.51	0.845	6.112	22.50	31.01
	APG	28.70	0.937	5.995	22.53	32.28
	e-CFG	29.30	1.032	5.992	22.61	32.23
	CFG++	29.21	1.025	6.091	22.60	31.46
	e-CFG	29.05	0.990	6.103	22.64	31.39
	FreeU	28.88	0.929	6.156	22.11	30.01

Appendix

Table 5: Quantitative results of different methods. Model: Stable Diffusion-XL (

\omega=5.5

Dataset	Method	HPS v2 ( $\uparrow$ )	ImageReward ( $\uparrow$ )	AES ( $\uparrow$ )	PickScore ( $\uparrow$ )	CLIPScore ( $\uparrow$ )
Pick-a-Pic	CFG	26.25	-0.353	5.658	20.14	27.08
	APG	26.03	-0.441	5.675	20.05	31.97
	e-CFG	26.00	-0.420	5.537	20.17	30.29
	CFG++	27.13	0.045	5.751	20.43	27.72
	e-CFG	26.99	-0.030	5.731	20.45	27.49
	FreeU	26.93	-0.004	5.759	20.34	27.08

Appendix

Table 6: Quantitative results of different methods. Model: Stable Diffusion-2.1 (

\omega=5.5

Dataset	Method	HPS v2 ( $\uparrow$ )	ImageReward ( $\uparrow$ )	AES ( $\uparrow$ )	PickScore ( $\uparrow$ )	CLIPScore ( $\uparrow$ )
Pick-a-Pic	CFG	28.43	1.004	5.921	21.94	28.72
	PAG	28.49	0.918	5.994	21.77	28.19
	e-CFG	28.54	1.053	5.925	21.96	28.92
	TDG	28.56	0.951	5.890	21.82	28.57
	e-CFG	28.56	1.016	5.921	21.87	28.83
	Z-Sampling	28.30	0.980	5.853	21.63	28.74

Appendix

Table 7: Quantitative results of different methods. Model: Stable Diffusion-3.5 (

\omega=4.5

Method	FID ( $\downarrow$ )	HPS v2 ( $\uparrow$ )	AES ( $\uparrow$ )	$\omega^{\mathrm{e}}$
CFG	13.56	28.73	5.558	5.5
Z-Sampling	17.32	29.56	5.666	10.81
e-CFG	14.21	29.22	5.550	10.81
CFG++	17.27	29.46	5.643	8.99
e-CFG	16.07	29.28	5.609	8.99

Appendix

Table 8: Quantitative results of different methods on COCO-30K. Model: Stable Diffusion-XL (

\omega=5.5

Method	FID ( $\downarrow$ )	IS ( $\uparrow$ )	AES ( $\uparrow$ )	$\omega^{\mathrm{e}}$
CFG	21.22	474.9	4.911	4
Z-Sampling	25.43	471.7	4.853	9.17
e-CFG	24.04	465.1	4.834	9.17
CFG++	24.33	480.1	4.950	8.01
e-CFG	25.55	480.0	4.791	8.01

Appendix

Table 9: Quantitative results of different methods on ImageNet-50K. Model: DiT-XL/2 (

\omega=4

Method	HPS v3	OneIG
Pick-a-Pic	( $\eta^{\text{CFG}}/\eta^{\text{e-CFG}}/\Delta\eta$ )
PAG	71% / 68% / 3%	44% / 44% / 0%
CFG++	55% / 54% / 1%	36% / 26% / 10%
Z-Sampling	79% / 67% / 12%	54% / 55% / -1%
APG	44% / 39% / 5%	24% / 26% / -2%
FreeU	39% / 35% / 4%	26% / 21% / 5%

Appendix

Table 10: The winning rate and its degradation of different methods on Pick-a-Pic datasets. Model: Stable Diffusion-XL (

\omega=5.5

Dataset	Model	HPS v3 ( $\uparrow$ )	UnifiedReward (to CFG)	UnifiedReward (to e-CFG)
Pick-a-Pic	CFG	6.837	/	/
	PAG	7.781	0.16	0.16
	e-CFG	7.341	/	/
	CFG++	7.648	0.1	-0.02
	e-CFG	7.612	/	/
	Z-Sampling	7.776	0.38	0.4

Appendix

Table 11: Quantitative results of different methods on Pick-a-Pic dataset. Model: Stable Diffusion-XL (

\omega=5.5

Guidance Scale	Alignment ( $\uparrow$ )	Diversity ( $\uparrow$ )	Reasoning ( $\uparrow$ )	Style ( $\uparrow$ )	Text ( $\uparrow$ )
5.5	.5638	.4399	.1932	.3325	.0128
6.0	.5738	.4357	.1910	.3328	.0178
6.5	.5771	.4294	.1970	.3374	.0170
7.0	.5820	.4242	.2000	.3420	.0164
7.5	.5837	.4225	.2076	.3363	.0181
8.0	.5936	.4169	.2048	.3366	.0166

Appendix

Table 12: Quantitative results of SD-XL on OneIG-Bench with different guidance scales.

Figure 12: Reward evaluation on images with different saturation, each color represents images with different saturation. 20 images were randomly picked in MS-COCO dataset with changed saturation. Model: SD-XL.

Figure 13: Reward evaluation on images with different saturation, each color represents images generated by a prompt. 20 prompts were randomly selected in Pick-a-Pic, DrawBench, and HPD, and corresponding images were generated with different guidance scale. Model: SD-XL.

Panel of Figure 14 — Figure 14: Effective guidance scale value over timestep. Model: SD-XL. Dataset: Pick-a-Pic. Note that except Z-Sampling, CFG++, and APG, which modified the noise scheduling process, other methods use EulerDiscreteScheduler by default.

Figure 15: Qualitative Results of Z-Sampling.

Figure 16: Qualitative Results of CFG++.

Figure 20: Qualitative Results of FreeU.

Explore similar work

Aug 17, 2026

Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models

Inference-time quality-enhancement methods are an effective and widely adopted means of improving diffusion models without expensive retraining. We study a family of training-free techniques conceptually rooted in Classifier-Free Guidance (CFG), most of which were originally proposed on older U-Net diffusion models and validated using metrics that assess image quality in isolation, without accounting for compositional alignment or semantic correspondence between the generated image and its associated text prompt. We re-evaluate eight such methods on two open-weight rectified-flow transformers under a fixed per-model protocol and three compositional-alignment benchmarks. No method consistently improves on CFG across the measured criteria. APG obtains several nominal best scores, but the corresponding gains often remain within the estimated evaluation uncertainty. Attention-perturbation methods provide isolated gains on SD3.5 Medium and more frequent degradations on FLUX.2 [klein] 4B Base, while CFG remains a competitive lower-cost baseline.

Artem Sergievskii, Artyom Turevich, Sergey Kastryulin

HSE University · Yandex, HSE University

Diffusion Model Guidance Classifier-Free Guidance Text-Image Alignment Attention Control in Diffusion Models

Ask PDF

Mar 11, 2026

Guiding Diffusion Models with Semantically Degraded Conditions

Classifier-Free Guidance (CFG) is a cornerstone of modern text-to-image models, yet its reliance on a semantically vacuous null prompt (

\varnothing

) generates a guidance signal prone to geometric entanglement. This is a key factor limiting its precision, leading to well-documented failures in complex compositional tasks. We propose Condition-Degradation Guidance (CDG), a novel paradigm that replaces the null prompt with a strategically degraded condition,

\boldsymbol{c}_{\text{deg}}

. This reframes guidance from a coarse "good vs. null" contrast to a more refined "good vs. almost good" discrimination, thereby compelling the model to capture fine-grained semantic distinctions. We find that tokens in transformer text encoders split into two functional roles: content tokens encoding object semantics, and context-aggregating tokens capturing global context. By selectively degrading only the former, CDG constructs

\boldsymbol{c}_{\text{deg}}

without external models or training. Validated across diverse architectures including Stable Diffusion 3, FLUX, and Qwen-Image, CDG markedly improves compositional accuracy and text-image alignment. As a lightweight, plug-and-play module, it achieves this with negligible computational overhead. Our work challenges the reliance on static, information-sparse negative samples and establishes a new principle for diffusion guidance: the construction of adaptive, semantically-aware negative samples is critical to achieving precise semantic control. Code is available at https://github.com/Ming-321/Classifier-Degradation-Guidance.

Shilong Han, Yuming Zhang, Hongxia Wang

College of Science, National University of Defense Technology

Diffusion Model Guidance T2I Diffusion Models Classifier-Free Guidance Compositional T2I Generation

Ask PDF

Aug 14, 2026

Adversarial Learning of Classifier-Free Guidance Schedules

Modern text-to-image diffusion models rely on classifier-free guidance (CFG) to achieve high image fidelity and text alignment. However, CFG typically applies a static, global scale across all timesteps, samples, and conditions -- a choice that is generally suboptimal and can introduce artifacts, as different states may benefit from different levels of guidance. While time-varying schedules are known to improve quality, designing them by hand is non-trivial and application-dependent. In this paper, we learn the guidance schedule as a function of diffusion time, conditioning and the current noisy sample, in order to better align sampled images with the text prompt. We frame this as a density ratio estimation problem: a discriminator is trained to estimate the time-dependent log-density ratio between the true and guided marginal distributions, while a lightweight generator network predicts the optimal, state-dependent guidance scale. Empirically, our approach outperforms both heuristic CFG schedules and prior methods for learning dynamic guidance on text-to-image generation benchmarks.

Ashwini Pokle, Alexandre Galashov, Arnaud Doucet +2

Google DeepMind

Classifier-Free Guidance T2I Diffusion Models Diffusion Timestep Scheduling

Ask PDF