Opening the Black Box of Classifier-Free Guidance via Information Bottleneck
Authors: Jiayang Gao, Tianyi Zheng, Jiayang Zou, Fengxiang Yang, Luyao Fan, Lv Tang, Bo Li, Jia Wang
Organizations: Shanghai Jiao Tong University, Shanghai, China · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China · Stanford University, California, USA · Michael Smith Laboratories, University of British Columbia
Classifier-free guidance (CFG) is a standard technique for improving conditional generation in diffusion models, yet its activation schedule is usually selected heuristically. In practice, most existing CFG-based methods rely on a pre-defined activation policy throughout inference, while recent dynamic variants often depend on hand-designed schedules without a principled theoretical explanation of when conditional information is most useful. In order to understand how the informational relevance between conditioning inputs and the score function evolves over the course of generation, we explore when guidance should be activated from an information-theoretic perspective. In this work, we formulate the diffusion trajectory as a time-dependent information bottleneck and show that the effectiveness of conditional information is not uniform throughout sampling, but becomes significantly stronger after a critical stage characterized by a Fisher-information balance condition. Motivated by this insight, we propose a training-free dynamic CFG strategy for inference. Rather than prescribing a fixed schedule in advance, our method constructs a stochastic process from conditional and unconditional score statistics and activates guidance when the estimated guidance gain crosses a prescribed threshold. This yields an adaptive activation rule that aligns condition injection with the underlying generation dynamics, while also recovering interval-based guidance as a limiting special case. Extensive experiments across diverse architectures and tasks, demonstrate that our method is effective, robust, and broadly applicable. Beyond empirical improvements, our framework provides an interpretation of dynamic guidance methods and a principled foundation for inference-time control in conditional diffusion models.
Figures & tables
Figure 1: Sensitivity of the solution t∗ to the hyperparameter in the original and simplified formulations. The original parameterization ( red Δt ) exhibits a much flatter logβ – t relationship than the rescaled version ( blue Δt ), indicating higher sensitivity to perturbations. Notably, for a fixed perturbation in logβ , the resulting deviation in t ( Δt ) is significantly larger in the original formulation. Experiments are conducted on SiT-XL/2.
Figure 2: Overview of our adaptive CFG activation pipeline. Starting from noise, the diffusion model jointly evaluates conditional and unconditional scores at each reverse step, computes a guidance-gain statistic ΔJ(t) , and activates classifier-free guidance only after the stopping criterion is met. The illustration uses a DiT/SiT-style diffusion backbone.
Table 3
Model
Approach
FID ↓
HPS v2.1
CLIP ↑
Aesthetic ↑
paintings ↑
photo ↑
concept-art ↑
anime ↑
average ↑
SD 3 (1024, 28 steps)
CFG( ω=7.5,B=4 )
25.98
30.99
28.39
30.63
31.31
30.33
31.83
5.44
Interval( ω=7.5,th=0.7T )
–
27.66
24.76
27.57
28.55
27.13
31.41
5.21
Ours( ω=7.5,logβ∗=10.0 )
24.88
31.12
28.57
30.98
31.51
30.55
31.89
5.45
FLUX.1 (256, 50 steps)
CFG( ω=1.5,B=64 )
30.93
29.77
27.18
29.21
30.80
29.24
31.10
5.46
Ours( ω=1.5,logβ∗=8.0 )
30.36
29.87
27.38
29.26
30.94
29.36
30.93
5.48
Table 3: Quantitative comparison on text-to-image tasks. Quantitative results on SD3 and FLUX.1 under different guidance configurations. We report FID, HPSv2.1, CLIP Score, and Aesthetic Score where available. For FLUX.1 at 1024×1024 , we additionally evaluate single-sample inference ( B=1 ). “–” denotes unavailable results.
Table 5
Figure 3: Qualitative Comparison. Visual results on text-to-image tasks using Stable Diffusion 3 and Flux.1. The red text in the prompts highlights specific details that the baseline fails to generate.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Probability density function (PDF) of the stopping time τ . The histogram represents the empirical distribution of sampled τ values. The solid curve is the fitted Weibull PDF, and the dashed curve is the Kernel Density Estimation (KDE) curve.
ImageNet B=48
Model-256, 50k samples, 250 inference timesteps
FID ↓
IS ↑
sFID ↓
SiT-XL/2 (REPA)( ω=1.35 , SDE)
1.80
284.0
4.5
SiT-XL/2 (REPA, Interval) ( ω=1.8,tl=0,th=0.7 )
1.42
305.7
4.7
SiT-XL/2 (REPA) ( ω=1.8,tl=0,th=0.68 )
1.43
299.3
4.7
SiT-XL/2 (REPA) ( ω=1.9,tl=0,th=0.68 )
1.42
310.4
4.7
SiT-XL/2 (REPA) + Ours ( ω=1.9,logβ∗=5.9 )
1.41
306.6
4.7
Appendix
Table 7: Comparison of fixed and stochastic stopping strategies on ImageNet 256. We evaluate a deterministic variant that uses the empirical mean stopping time ( th=τˉ≈0.68 ) against Interval Guidance and our proposed stochastic stopping method. While the mean-based threshold improves over the baseline, it remains inferior to the stochastic strategy, highlighting the importance of modeling the variability of stopping time.
Table 8: Ablation comparison on logβ∗ . Comparison of different evaluation metrics on Class-Conditional ImageNet datasets with different logβ∗ .
Figure 5: FID-IS trajectory under varying logβ∗ based on Table 8 . Each point corresponds to a specific logβ∗ value along the trajectory. As logβ∗ increases, IS improves while FID exhibits a U-shaped trend, reflecting the quality-diversity trade-off in generation.
Figure 6: Additional visual results on different models.
Diffusion models have achieved strong performance in image, text-to-image, and video generation, where conditional generation is often controlled by classifier-free guidance (CFG). CFG improves condition consistency by increasing a guidance weight, but stronger guidance typically reduces diversity and distributional coverage. It remains unclear how this consistency-coverage trade-off should be controlled across the reverse trajectory, since the distribution induced by CFG is not simply the fixed-time tilted distribution given by the guided score field. To address this issue, we propose an information-theoretic framework for CFG schedule optimization. Our approach uses a clean endpoint reference to specify the desired consistency-coverage trade-off, while optimizing the actual distribution induced by the guided sampler toward this reference. We derive trajectory-level formulas to estimate the objective from samples and score evaluations, avoiding explicit density estimation. On ImageNet-512 with EDM-XXL and COCO with SD-XL, the learned schedules achieve competitive or improved trade-offs over constant guidance and allocate guidance selectively across noise levels.
Haobo Chen, Xiangxiang Xu, Yuheng Bu
Department of Computer Science University of California, Santa Barbara · Department of Computer Science University of Rochester
Classifier-Free Guidance (CFG) is a cornerstone of modern conditional diffusion models, yet its reliance on the fixed or heuristic dynamic guidance weight is predominantly empirical and overlooks the inherent dynamics of the diffusion process. In this paper, we provide a rigorous theoretical analysis of the Classifier-Free Guidance. Specifically, we establish strict upper bounds on the score discrepancy between conditional and unconditional distributions at different timesteps based on the diffusion process. This finding explains the limitations of fixed-weight strategies and establishes a principled foundation for time-dependent guidance. Motivated by this insight, we introduce \textbf{Control Classifier-Free Guidance (C2FG)}, a novel, training-free, and plug-in method that aligns the guidance strength with the diffusion dynamics via an exponential decay control function. Extensive experiments demonstrate that C2FG is effective and broadly applicable across diverse generative tasks, while also exhibiting orthogonality to existing strategies.
Jiayang Gao, Tianyi Zheng, Jiayang Zou +9
Shanghai Jiao Tong University, Shanghai, China · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China
Classifier-free guidance (CFG) is the default mechanism for conditional generation in diffusion models, but the distribution sampled by its deterministic guided dynamics is not captured by the usual product-distribution heuristic p0ωq01−ω. We analyze CFG through the probability flow ODE and derive exact analytic path-integral representations of the induced distributions for both constant and time-dependent guidance. The resulting formulas show that CFG modifies pt0 by an exponential path-integral correction, and that a time-dependent schedule enters this correction through the weight ω(t)−1. This characterization explains how score discrepancies accumulate along sampling trajectories and motivates Distribution-Guided CFG (DG-CFG), a schedule that balances timestep contributions while accounting for signal strength and low-noise score-error amplification. A toy model with analytic scores closely verifies the predicted distributions. Across Stable Diffusion1.5, Stable Diffusion2.1, and Stable DiffusionXL, DG-CFG yields a stronger diversity--fidelity trade-off and robustly mitigates the saturation and quality degradation caused by strong constant or heuristic guidance. Complete NFE experiments on Stable Diffusion1.5 and Stable Diffusion~2.1 confirm that these gains persist across sampling budgets, while fixed-quality experiments on both backbones show that DG-CFG reaches target metrics with fewer sampling steps.
Enze Jiang, Zheng Ma
School of Mathematical Sciences Shanghai Jiao Tong University Shanghai, China