Classifier-free guidance (CFG) is a standard component of conditional diffusion models, and guided generation performance has become the de facto measure of state-of-the-art models. Recent work on latent diffusion models seeks to learn latent spaces that are easier to model with diffusion. However, under CFG, generation additionally depends on how conditions are arranged in the latent space. We study this arrangement under discrete class conditions, termed class geometry, and investigate what class geometry allows a latent space to work well with CFG. Under a Gaussian class model, we show that shifting variance from intra- to inter-class at fixed total variance makes classes more distinguishable under noise, and derive a noisy posterior objective whose per-sample descent direction is proportional to the oracle CFG correction. Based on the analysis, we propose Prototype Regularization (PR), a lightweight surrogate for the intractable noisy posterior objective, which applies a cosine prototype classifier to spatially pooled latents. With a controlled training budget (20 epochs for VAE, 80 epochs for DiT), adding PR to an LDM baseline reduces guided gFID on ImageNet from 3.31 to 2.28, surpassing a DINOv3-aligned VA-VAE (2.44) with negligible overhead. We further demonstrate that PR generalizes to pseudo-labels derived from clustering, continuous text-embedding conditions, and video latents. Together, our findings highlight class geometry as a critical factor in guided generation and take a first step toward CFG-compatible representation learning. Our project page is available at: https://maps-research.github.io/cfg-geometry/.
Figures & tables
Figure 1: Simulations under the Gaussian class model with K=1000 classes and latent dimensions d=8192 (Eq. 4 ). (a) Oracle CFG correction strength along time t ( t=1 is clean), for varying inter-class variance fraction α . The correction vanishes once the target-class posterior concentrates. (b) Monte Carlo average of the analytic target-class posterior under forward corruption. Larger α yields earlier posterior concentration. (c) Posterior losses (normalized by logK ) versus α , on clean-only and uniform-time corrupted latents. Both posterior losses are non-increasing in α .
Method
Stage 1
Stage 2
Reconstruction
Geometry
Speed
w/o CFG
w/ CFG
rFID ↓
PSNR ↑
LPIPS ↓
Purity ↑
Steps/s ↑
gFID ↓
gFID ↓
sFID ↓
IS ↑
Precision ↑
Recall ↑
LDM
0.308
27.07
0.101
0.020
2.16
13.09
3.31
4.68
237.73
0.859
0.504
VA-VAE (DINOv3-B)
0.280
27.09
0.099
0.053
1.65
9.76
2.64
4.94
236.02
0.828
0.557
VA-VAE (DINOv3-L)
0.275
27.06
0.099
0.071
1.61
9.01
2.44
4.83
242.17
0.816
0.564
LDM + PR (Ours)
0.277
26.95
0.101
0.279
2.13
11.59
2.28
4.87
269.85
0.808
0.585
Table 1: Results of class-conditional generation on ImageNet. All quality metrics are evaluated with 50K samples. Purity measures the fraction of each latent’s k=10 nearest neighbors that share its class, averaged over 50K normalized validation latents. Best results are highlighted in bold .
Figure 2: (a) Energy decomposition of per-channel normalized latents into global mean ∥μ∥2 , inter-class variance Ec∥μc−μ∥2 , and intra-class variance E∥z−μc∥2 . (b) Average target-class posterior E[qc(zt)] on noised validation latents (solid) and CFG sampling trajectories (dotted), where qc(zt) is computed analytically from the empirical distribution of ImageNet training latents under Gaussian forward corruption. (c) gFID (10K samples) versus guidance scale ω ; diamonds mark the minima of the displayed curves ( ω⋆ ). PR allocates more energy to inter-class variance, makes the target class identifiable earlier and more confidently, and reaches a lower best gFID at a larger ω⋆ .
Base Latent
Center Construction
Center Shape
gFID@50K ↓
ω⋆
w/o CFG
w/ CFG
LDM + PR
Native
–
11.59
2.28
2.5
LDM
Native
–
13.09
3.31
2.0
LDM
LDM (Rescaled)
32×16×16
12.58
2.89
2.0
LDM
PR
32×16×16
12.79
2.63
3.5
LDM
PR
32×1×1
12.65
2.52
4.0
Table 2: Center transplantation on LDM latents ( z′=z−μcLDM+μcRef ). Rescaled LDM centers match the inter-class energy of full PR centers. Random centers match the pooled PR energy. Spatially constant centers are broadcast over the spatial grid. ω⋆ denotes the best CFG scale.
Stage 1 (LeanVAE)
Stage 2 (Latte-XL/2)
w/ CFG
w/o CFG
FVD ↓
Accuracy (%) ↑
FVD ↓
Accuracy (%) ↑
Official
Official
167.0
92.9
340.2
87.2
Official
Fine-Tuned
161.4
98.5
302.0
92.7
Fine-Tuned (Recon Only)
Fine-Tuned
152.7
99.0
287.3
93.0
Fine-Tuned (PR, Ours)
Fine-Tuned
140.1
98.9
275.7
93.0
Table 4: Generalization to video latents on UCF-101 ( Soomro et al., 2012 ) using LeanVAE-16 ( Cheng and Yuan, 2025 ) and Latte-XL/2 ( Ma et al., 2025 ) . Official denotes the VAE and Latte-XL/2 checkpoints provided by LeanVAE, used without further training. We report FVD and action classification accuracy with and without CFG. The best results are in bold .
Method
w/ CFG
w/o CFG
gFID Train ↓
gFID Val ↓
CLIP ↑
gFID Train ↓
gFID Val ↓
CLIP ↑
LDM
3.15
2.90
0.2775
10.38
7.52
0.2577
VA-VAE (DINOv3-L)
2.53
2.67
0.2814
8.02
5.65
0.2620
Cosine Alignment (CLIP-L)
2.93
3.04
0.2799
9.52
6.84
0.2590
Cosine Alignment (DINOv3-L)
2.86
2.78
0.2784
9.35
6.74
0.2585
LDM + PR (GT Labels)
2.51
2.35
0.2811
9.63
6.87
0.2570
Table 5: ImageNet text-to-image generation using different Stage-1 latent representations. All metrics use 50K images generated from held-out validation captions. All alignment to CLIP-L adopts its text embeddings. CLIP denotes mean cosine similarity of CLIP embeddings calculated from text captions and generated images. All cluster labels use K=1000 . The best results are in bold .
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: UMAP visualization of full and spatially pooled latents from LDM, VA-VAE, and PR (ours). We encode 50K validation images and plot latents from 50 selected classes, colored by class label. Latents are normalized per dimension before UMAP. PR produces noticeably tighter and better separated class clusters. Zoom in for the best view. Better view with colors.
Figure 4: Noisy marginals of zt at several t∈(0,1) for toy Gaussian mixtures with fixed total variance but different allocations between inter-class and intra-class variance. Background colors mark the Maximum A Posteriori (MAP) decision regions argmaxcqc(zt) , with opacity proportional to the confidence maxcqc(zt) . With larger inter-class variance, classes become separable at smaller t , i.e., at noisier levels, and the confident MAP regions align more closely with the true class supports.
Figure 5: Oracle velocity fields on toy Gaussian mixtures with fixed total variance. From top to bottom, the fraction of variance allocated to class means increases. For each space, we show the conditional velocity vc⋆ (conditioned on the red class), the unconditional velocity vu⋆ , and the correction gc⋆=vc⋆−vu⋆ at t∈{0.2,0.5,0.8} . Gray contours mark the 95% density region of zt , and dots mark the class means of zt . With clearer class geometry, the conditional and correction velocities point more consistently toward the target class mean, and the correction vanishes where the posterior concentrates on the target class. Zoom in for the best view. Better view with colors.
Configuration
Full Budget
Reduced Budget
Data and Architecture
Dataset
ImageNet, 256×256
GPUs per Run
8
2
Stage 1 Architecture
KL-VAE, downsampling factor 16
VAE Base Channels / Multipliers
128 / (1,1,2,2,4)
VAE Residual Blocks / Attention Resolution
2 per level / 16×16
Appendix
Table 6: Training and evaluation configurations for ImageNet experiments. Full-budget experiments use 8 GPUs; reduced-budget ablations use 2 GPUs. Both budgets use the same architectures and evaluation protocol, with experiment-specific CFG search ranges listed below.
Method
Stage 1
Stage 2
rFID ↓
PSNR ↑
LPIPS ↓
gFID ↓
gFID ↓
w/ CFG
w/o CFG
LDM
0.285
27.13
0.100
4.56
18.79
LDM + PR (Cosine Head, Clean-only Latent)
0.279
26.75
0.103
3.24
17.11
LDM + PR (Cosine Head, Uniform-time Noisy Latent)
0.457
26.83
0.110
14.02
41.63
LDM + PR (Linear Head, Clean-only Latent)
0.319
26.64
0.108
21.55
64.18
Appendix
Table 7: Ablation on the classifier type in PR (reduced compute budget). We vary the classifier head (cosine vs. linear) and the latent on which PR is applied (clean latents only vs. noisy latents at uniformly sampled timesteps). The cosine head on clean-only latents achieves the best rFID and gFID, both with and without CFG. Applying PR on uniform-time noisy latents over-regularizes the latent space: the top PCA component degenerates into a near-constant channel that fails to capture spatial variations (i.e., over-smoothed latents). The linear head, optimized with additional weight decay, collapses to a trivial solution that arbitrarily inflates the latent norm to enlarge the logits.
Method
Stage 1
Stage 2
rFID ↓
PSNR ↑
LPIPS ↓
gFID ↓
gFID ↓
w/ CFG
w/o CFG
LDM
0.285
27.13
0.100
4.56
18.79
LDM + PR ( 1×1 Prototypes)
0.279
26.75
0.103
3.24
17.11
LDM + PR ( 2×2 Prototypes)
0.312
26.41
0.109
3.19
20.66
LDM + PR ( 4×4 Prototypes)
0.307
26.40
0.109
4.04
22.53
Appendix
Table 8: Ablation on prototype shape (reduced compute budget). All variants use the default PR setting (cosine prototype classifier on clean-only latents) and differ only in the output shape of spatial average pooling applied before the classifier, where 16×16 corresponds to no pooling. Overall, prototypes with lower resolutions perform best: 1×1 prototypes (i.e., global average pooling) achieve the best rFID and gFID without CFG, and 2×2 prototypes achieve the best gFID with CFG, while further increasing prototype resolution consistently degrades generation quality. This suggests that reducing the spatial resolution of prototypes benefits generation, likely because it avoids imposing class-level layout constraints on the latent space.
Method
Stage 1
Stage 2
rFID ↓
PSNR ↑
LPIPS ↓
gFID ↓
gFID ↓
w/ CFG
w/o CFG
LDM
0.285
27.13
0.100
4.56
18.79
LDM + PR ( τ=0.03 )
0.299
26.76
0.104
3.27
17.61
LDM + PR ( τ=0.1 )
0.279
26.75
0.103
3.24
17.11
LDM + PR ( τ=0.3 )
0.291
26.76
0.103
3.14
16.24
Appendix
Table 9: Ablation on the temperature of PR (reduced compute budget). τ scales the cosine-similarity logits of the prototype classifier, where a smaller τ yields a sharper softmax. The default τ=0.1 was fixed prior to this ablation and is used throughout the main text. All PR variants improve over LDM both with and without CFG, and the differences among τ values are small compared with the gain from adding PR, indicating that τ=0.1 is a reasonable default. Adjusting τ (e.g., τ=0.3 here) may bring further improvements, though the best value is likely setting-dependent.
Labels / Clusters for PR
Stage 1
Stage 2
rFID ↓
PSNR ↑
LPIPS ↓
gFID ↓
gFID ↓
w/ CFG
w/o CFG
DINOv3-B K=64
0.264
28.09
0.096
3.01
14.07
DINOv3-B K=256
0.254
28.00
0.095
3.04
14.99
DINOv3-B K=1000
0.294
27.59
0.101
2.99
17.60
DINOv3-B K=4096
0.263
27.99
0.095
3.27
17.93
Appendix
Table 10: Ablation on DINO pseudo-label granularity (reduced compute budget). This group of experiments uses a separate hyper-parameter setup, so the results are comparable within this table but not with other tables. Reconstruction is largely insensitive to the choice of targets. Unguided gFID degrades as K increases, while guided gFID remains comparable across K except for overly fine clusters. The nested variant (i.e., partitioning each of the 64 clusters into 16 sub-clusters) achieves the best generation among unsupervised targets (best unguided gFID and guided gFID close to GT labels), showing the feasibility of hierarchical PR.
Method
Stage 1
Stage 2
rFID ↓
PSNR ↑
LPIPS ↓
gFID ↓
gFID ↓
w/ CFG
w/o CFG
LDM
0.248
28.21
0.095
4.25
18.85
LDM + PR
0.263
27.82
0.098
2.87
14.60
VAVAE (DINOv3-L)
0.229
28.27
0.092
3.04
12.71
VAVAE (DINOv3-L) + PR
0.282
27.55
0.101
2.57
10.89
Appendix
Table 11: Composability of PR and VA-VAE. This group of experiments uses the same separate hyper-parameter setup as Tab. 10 , so the results are comparable with that table but not with the others. We apply PR on top of VA-VAE, which aligns to DINOv3-L features. Combining the two yields better generation both with and without CFG, at the cost of mildly degraded reconstruction, indicating that PR is complementary to external feature alignment for generation.
Classifier-Free Guidance (CFG) is essential for high-fidelity conditional generation in flow matching, yet it imposes significant computational overhead by requiring dual forward passes at each sampling step. In this work, we address this bottleneck by introducing P-Guide, a framework that achieves high-quality guidance through a single inference pass by modulating only the initial latent state. We further show that, under a first-order approximation, P-Guide is equivalent to CFG in the sense that it steers generation from the prior space, without requiring explicit velocity field extrapolation during sampling. We consider both homoscedastic and heteroscedastic priors, and find that jointly modeling the mean and variance enables adaptive loss attenuation and improved robustness to data uncertainty. Extensive experiments demonstrate that P-Guide reduces inference latency by approximately 50% while maintaining fidelity and prompt alignment competitive with standard dual-pass CFG baselines.
Xin Peng, Ang Gao
School of Physical Science and Technology, Beijing University of Posts and Telecommunications, Beijing, China
Diffusion models achieve strong generative performance but often rely on classifier-free guidance (CFG), an inference-time intervention that modifies the sampling trajectory. Since standard denoising score matching (DSM) should recover the target distribution when the score is learned accurately, the practical necessity of CFG suggests deficiencies in learned conditional scores. We identify weak inter-class separation as one such limitation: samples generated from the same noise under different labels can retain similar class-agnostic structure. To address this deficiency, we introduce MCLR, a principled fine-tuning objective that explicitly maximizes inter-class likelihood-ratios, thereby strengthening class-specific structures under standard sampling. Across diverse models and datasets, MCLR substantially improves guidance-free conditional generation and narrows the gap to CFG. Theoretically, we prove that the CFG-guided score is the unique minimizer of a sample-adaptively weighted, ELBO-approximated MCLR objective, interpreting CFG as an implicit inference-time contrastive alignment procedure. We further compare MCLR with related contrastive objectives, including DPO, CCA, and CFM, clarifying their connections and key differences in an otherwise fragmented literature.
Xiang Li, Yixuan Jia, Xiao Li +3
University of Michigan · Michigan State University
Classifier-free guidance (CFG) is the default mechanism for conditional generation in diffusion models, but the distribution sampled by its deterministic guided dynamics is not captured by the usual product-distribution heuristic p0ωq01−ω. We analyze CFG through the probability flow ODE and derive exact analytic path-integral representations of the induced distributions for both constant and time-dependent guidance. The resulting formulas show that CFG modifies pt0 by an exponential path-integral correction, and that a time-dependent schedule enters this correction through the weight ω(t)−1. This characterization explains how score discrepancies accumulate along sampling trajectories and motivates Distribution-Guided CFG (DG-CFG), a schedule that balances timestep contributions while accounting for signal strength and low-noise score-error amplification. A toy model with analytic scores closely verifies the predicted distributions. Across Stable Diffusion1.5, Stable Diffusion2.1, and Stable DiffusionXL, DG-CFG yields a stronger diversity--fidelity trade-off and robustly mitigates the saturation and quality degradation caused by strong constant or heuristic guidance. Complete NFE experiments on Stable Diffusion1.5 and Stable Diffusion~2.1 confirm that these gains persist across sampling budgets, while fixed-quality experiments on both backbones show that DG-CFG reaches target metrics with fewer sampling steps.
Enze Jiang, Zheng Ma
School of Mathematical Sciences Shanghai Jiao Tong University Shanghai, China