Adversarial Learning of Classifier-Free Guidance Schedules

Ashwini Pokle; Alexandre Galashov; Arnaud Doucet; Mauricio Delbracio; Valentin De Bortoli

cs.LGAug 14, 2026

Adversarial Learning of Classifier-Free Guidance Schedules

Authors: Ashwini Pokle, Alexandre Galashov, Arnaud Doucet, Mauricio Delbracio, Valentin De Bortoli

Organizations: Google DeepMind

Abstract

Modern text-to-image diffusion models rely on classifier-free guidance (CFG) to achieve high image fidelity and text alignment. However, CFG typically applies a static, global scale across all timesteps, samples, and conditions -- a choice that is generally suboptimal and can introduce artifacts, as different states may benefit from different levels of guidance. While time-varying schedules are known to improve quality, designing them by hand is non-trivial and application-dependent. In this paper, we learn the guidance schedule as a function of diffusion time, conditioning and the current noisy sample, in order to better align sampled images with the text prompt. We frame this as a density ratio estimation problem: a discriminator is trained to estimate the time-dependent log-density ratio between the true and guided marginal distributions, while a lightweight generator network predicts the optimal, state-dependent guidance scale. Empirically, our approach outperforms both heuristic CFG schedules and prior methods for learning dynamic guidance on text-to-image generation benchmarks.

Figures & tables

Figure 1: Marginal consistency via GAN. We draw two independent samples, $x_{1}^{1}$ and $x_{1}^{2}$ , from the data distribution. First sample $x_{1}^{1}$ is noised to the time $s$ and then denoised with guidance to the time $t$ which denotes our generated path. Second sample $x_{1}^{2}$ is noised to the time $t$ through forward process and denotes the real samples. The discriminator is then trained to discriminate between these two.

\displaystyle\phi\leftarrow\phi-\alpha_{D}\nabla\mathcal{L}_{D}(\phi)

Algorithm 1 Algorithm to learn parameter of CFG with Flow Matching model.

Method	Guidance Weight	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
	Baselines
Unguided	$\omega=0$	27.65	0.2742	4.79	0.2547	0.1997
Constant	$\omega=7.5$	29.73	0.3044	5.29	0.2817	0.2186
LIG	$\omega(t)=12.5,t\in[0.1,0.9]$	25.57	0.3032	5.30	0.2812	0.2171
CLG	$\omega=15.0$	26.38	0.3038	5.26	0.2807	0.2175
MMD + SC	$\bm{\omega}(x_{s},s,t,c)$	28.38	0.30419	5.2647	0.28129	0.21813

Table 1: MS COCO

512\times 512

. Performance of different methods on a small, MMDiT-XS model. We highlight the best metric in bold .

Method	Guidance Weight	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
	Baselines
Unguided	$\omega=0$	24.3	0.279	4.84	0.2588	0.2024
Constant	$\omega=7.5$	31.11	0.3063	5.31	0.285	0.2214
LIG	$\omega(t)=15.0,t\in[0.1,0.9]$	24.95	0.3042	5.31	0.2843	0.2194
CLG	$\omega=14.0$	27.14	0.3055	5.27	0.2838	0.2203
MMD + SC	$\bm{\omega}(x_{s},s,t,c)$	28.54873	0.30674	5.29430	0.28471	0.22078

Table 2: MS COCO

512\times 512

. Performance of different methods on a large, MMDiT-S model. We highlight the best metric in bold .

Figure 2: Learned guidance weights on MS COCO $512\times 512$ trained with Algorithm 1 on MMDiT-S. Please refer to Figures 3 and 4 for the corresponding images.

Conditioning	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
MMDiT-XS, $\bm{\omega}(s,t,c)$	26.91	0.30368	5.27	0.28112	0.21778
MMDiT-XS, $\bm{\omega}(x_{s},s,t,c)$	31.31	0.30483	5.32	0.2829	0.2186
MMDiT-S, $\bm{\omega}(s,t,c)$	28	0.3063	5.28	0.2841	0.2206
MMDiT-S, $\bm{\omega}(x_{s},s,t,c)$	31.7	0.3068	5.34	0.28564	0.2214

Table 3: MS COCO

512\times 512

. Impact of conditioning information on empirical performance of our method. In bold we highlight the best metric among each model size.

$\bm{\delta}$	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
$0.01$	31.2	0.30476	5.32	0.28273	0.21888
$0.10$	31.31	0.30484	5.32	0.28288	0.21862
$0.20$	30.3	0.30486	5.31	0.28263	0.21850

Table 4: Impact of

\delta

variations on MMDiT-XS performance.

Panel of Figure 3 — Figure 3: T2I Results on MS-COCO with MMDiT-S . (left-to-right) We provide results of images generated from the given text prompt with CFG $\omega=7.5$ , Clamp linear guidance (CLG), Limited Interval guidance (LIG), MMD guidance, and our method. Qualitatively, our method (rightmost) exhibits better adherence to text prompts, precisely capturing nuanced details such as the dog’s interaction with a book of maps in (a), the temporal dichotomy of the split room in (b), and the specific winter-field context in (c) that baselines simplify or omit.

Panel of Figure 4 — Figure 4: T2I Results on MS-COCO with MMDiT-S . (left-to-right) We provide results of images generated from the given text prompt with CFG $\omega=7.5$ , CLG, LIG, MMD and our method.

Panel of Figure 5 — Figure 5: T2I Results on MS-COCO with MMDiT-S . (left-to-right) We provide results of images generated from the given text prompt with CFG $\omega=7.5$ , CLG, LIG, MMD and our method.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Guidance Weight	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
	Baselines
Unguided	$\omega=0$	27.65	0.2742	4.79	0.2547	0.1997
Constant	$\omega=7.5$	29.73	0.3044	5.29	0.2817	0.2186
LIG	$\omega(t)=12.5\in[0.1,0.9]$	25.57	0.3032	5.30	0.2812	0.2171
CLG	$\omega=15.0$	26.38	0.3038	5.26	0.2807	0.2175
MMD + SC	$\bm{\omega}(s,t,c)^{\dagger}$	28.39	0.30388	5.25	0.28	0.21795

Appendix

Table 5: MS COCO

512\times 512

. Performance of different methods on a small, MMDiT-XS model. We highlight the best metric in bold .

\dagger

indicates that we follow the setup from Galashov et al. (2026) .

Method	Guidance Weight	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
	Baselines
Unguided	$\omega=0$	24.3	0.279	4.84	0.2588	0.2024
Constant	$\omega=7.5$	31.11	0.3063	5.31	0.285	0.2214
LIG	$\omega(t)=15.0,t\in[0.1,0.9]$	24.95	0.3042	5.31	0.2843	0.2194
CLG	$\omega=14.0$	27.14	0.3055	5.27	0.2838	0.2203
MMD + SC	$\bm{\omega}(s,t,c)^{\dagger}$	28.5	0.30615	5.27	0.28351	0.2204

Appendix

Table 6: MS COCO

512\times 512

. Performance of different methods on a large, MMDiT-S model. We highlight the best metric in bold .

\dagger

indicates that we follow the setup from Galashov et al. (2026) .

Guidance Weight	Interval	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	Pickscore $\uparrow$
$7.5$	$[0.1,0.9]$	23.97	0.3018	5.26	0.2795	0.2166
$7.5$	$[0.1,0.8]$	24.00	0.3018	5.26	0.2795	0.2166
$7.5$	$[0.2,0.9]$	19.65	0.2969	5.17	0.2734	0.2124
$7.5$	$[0.2,0.8]$	19.66	0.297	5.17	0.2735	0.2124
$10.0$	$[0.1,0.9]$	24.70	0.3024	5.29	0.2805	0.217
$10.0$	$[0.1,0.8]$	24.83	0.2977	5.29	0.2806	0.217

Appendix

Table 7: MS COCO

512\times 512

. Performance of limited interval guidance ( Kynkäänniemi et al., 2024 ) on the MMDiT-XS model with

128

sampling steps. We highlight the best metrics in bold .

Guidance Weight $\bm{\omega_{1}}$	$\bm{c_{1}}$	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	Pickscore $\uparrow$
$7.5$	$1$	22.37	0.3012	5.17	0.2769	0.2167
$7.5$	$2$	23.74	0.302	5.18	0.2778	0.2161
$7.5$	$4$	26.82	0.303	5.23	0.2798	0.2174
$10.0$	$1$	23.15	0.3022	5.21	0.2782	0.2165
$10.0$	$2$	24.24	0.2067	5.21	0.2788	0.2166
$10.0$	$4$	26.55	0.3033	5.24	0.2801	0.2174

Appendix

Table 8: MS COCO

512\times 512

. Performance of clamp-linear guidance schedule

w_{t}=\text{max}(c_{1},\frac{2w_{1}t}{T})

( Wang et al., 2024 ) on the MMDiT-XS model with

128

sampling steps. We highlight the best metrics in bold .

$\bm{\delta}$	Reward scale $\eta$	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	Pick Score $\uparrow$
$0.01$	2.50	31.54	0.3049	5.3287	0.2830	0.2188
$0.10$	2.50	31.15	0.3047	5.3145	0.2826	0.2188
$0.20$	2.50	30.88	0.3049	5.3190	0.2828	0.2187
$0.30$	2.50	31.16	0.3049	5.3207	0.2829	0.2189
$0.01$	0.25	31.24	0.3048	5.3179	0.2827	0.2189
$0.10$	0.25	31.31	0.3048	5.3236	0.2829	0.2186

Appendix

Table 9: Performance metrics for MMDiT-XS for different values of

\delta

values in time distribution

p(s,t)

and different CLIP reward scale

\eta

Sampling steps	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
$128$	31.70	0.3068	5.34	0.28564	0.2214
$64$	30.55	0.3068	5.32	0.2853	0.2212
$32$	28.45	0.3062	5.31	0.2835	0.2196

Appendix

Table 10: Impact of number of sampling steps on the performance of our GAN-based guidance scheduler. We use MMDiT-S backbone.

Architecture	$x_{s}$ input type	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
Convolutional	$x_{s}$ latents	26.38	0.3041	5.26	0.2807	0.2170
MLP	$x_{s}$ statistics	31.31	0.3048	5.32	0.2829	0.2186

Appendix

Table 11: Impact of generator architecture and choice of inputs ( i.e., raw

x_{s}

vs statistics) with MMDiT-XS backbone.

Independent noise	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
True	31.31	0.3048	5.32	0.2829	0.2186
False	31.41	0.3048	5.32	0.2827	0.2186

Appendix

Table 12: Impact of how the noise sampled on the performance of our GAN-based guidance scheduler. We use MMDiT-XS backbone.

Independent $\bm{(x_{1},c)}$	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
✓	31.41	0.3048	5.32	0.2927	0.2186
✗	31.31	0.3048	5.32	0.2829	0.2186

Appendix

Table 13: Impact of choosing independent image-conditioning pairs on performance of our method with MMDiT-XS backbone.

CLIP reward	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
✓	31.41	0.3048	5.32	0.2927	0.2186
✗	31.11	0.3047	5.31	0.2825	0.2184

Appendix

Table 14: Impact of choosing independent image-conditioning pairs on performance of our method with MMDiT-XS backbone.

$\bm{\alpha}$	$\bm{\beta}$	$\bm{\delta}$	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	HPSv2 $\uparrow$	PickScore $\uparrow$
1	1	0.01	30.85	0.3048	5.32	0.2827	0.21847
		0.10	30.79	0.3038	5.31	0.2819	0.21786
		0.20	31.52	0.3048	5.325	0.2828	0.21861
1	2	0.01	31.87	0.3046	5.32	0.2826	0.21869
		0.10	32.00	0.3048	5.33	0.2829	0.21882
		0.20	31.54	0.3048	5.32	0.2827	0.21851

Appendix

Table 15: Beta-trapezoid time distribution ablation on MMDiT-XS. We vary the Beta distribution parameters

(\alpha,\beta)

and minimum step size

\delta

. When

\alpha=\beta=1

, the distribution reduces to uniform sampling over the triangular support

\mathcal{T}

. The last 3 rows correspond to the distribution

p(s,t)

from Section 3.2 .

Config	MMDiT-XS	MMDiT-S
Discriminator Training
Batch size	128	128
Optimizer	Adam	Adam
Learning rate	1e-5	1e-5
$(\beta_{1},\beta_{2})$	(0.5, 0.99)	(0.5, 0.99)
Warmup steps	$5000$	5000

Appendix

Table 16: Training and Sampling Details. Hyperparameter settings for MMDiT-XS, MMDiT-S.

Panel of Figure 6 — Figure 6: T2I Results on MS-COCO with MMDiT-XS . (left-to-right) We provide results of images generated from the given text prompt with CFG $\omega=7.5$ , CLG, LIG, MMD and our method.

Panel of Figure 7 — Figure 7: Variation with seeds: T2I Results on MS-COCO with MMDiT-S . (left-to-right) We provide results of images generated from the given text prompt with different seeds (0,1,2,3) with our method.

Explore similar work

Information-Theoretic Classifier-Free Guidance with Adaptive Schedule Optimization
Jun 23, 2026Haobo Chen, Xiangxiang Xu, Yuheng BuDiffusion Model Guidance Classifier-Free Guidance
Department of Computer Science University of California, Santa Barbara · Department of Computer Science University of Rochester
Analytic Distribution of Classifier-Free Guidance for Schedule Design
Jul 22, 2026Enze Jiang, Zheng MaClassifier-Free Guidance Diffusion Sampling
School of Mathematical Sciences Shanghai Jiao Tong University Shanghai, China
DICE: Distilling Classifier-Free Guidance into Text Embeddings
Feb 6, 2025Zhenyu Zhou, Defang Chen, Can Wang +2Diffusion Model Alignment Diffusion Model Guidance
Zhejiang University, State Key Laboratory of Blockchain and Data Security · University at Buffalo, State University of New York

Jun 23, 2026J/K move · Enter open · S save

Information-Theoretic Classifier-Free Guidance with Adaptive Schedule Optimization

Haobo Chen, Xiangxiang Xu, Yuheng Bu

Department of Computer Science University of California, Santa Barbara · Department of Computer Science University of Rochester

Diffusion Model Guidance Classifier-Free Guidance Diffusion-Based Image Generation

Open paper AskPDF arXiv

Diffusion models have achieved strong performance in image, text-to-image, and video generation, where conditional generation is often controlled by classifier-free guidance (CFG). CFG improves condition consistency by increasing a guidance weight, but stronger guidance typically reduces diversity and distributional coverage. It remains unclear how this consistency-coverage trade-off should be controlled across the reverse trajectory, since the distribution induced by CFG is not simply the fixed-time tilted distribution given by the guided score field. To address this issue, we propose an information-theoretic framework for CFG schedule optimization. Our approach uses a clean endpoint reference to specify the desired consistency-coverage trade-off, while optimizing the actual distribution induced by the guided sampler toward this reference. We derive trajectory-level formulas to estimate the objective from samples and score evaluations, avoiding explicit density estimation. On ImageNet-512 with EDM-XXL and COCO with SD-XL, the learned schedules achieve competitive or improved trade-offs over constant guidance and allocate guidance selectively across noise levels.

Abstract

Explore similar work

Information-Theoretic Classifier-Free Guidance with Adaptive Schedule Optimization

Analytic Distribution of Classifier-Free Guidance for Schedule Design

DICE: Distilling Classifier-Free Guidance into Text Embeddings