Classifier-Free Guidance (CFG) with guidance scale γ is commonly motivated by the ω-powered conditional distribution obtained by setting ω=γ, but CFG does not, in general, sample from this distribution exactly. We study whether the powered target can instead be learned explicitly. We introduce Importance-Weighted Conditional Flow Matching (IW-CFM), which reweights conditional flow-matching examples using density ratios obtained from frozen conditional and unconditional flow models. With exact normalized weights, the resulting objective has the same gradient as flow matching on the powered target; in practice, we use self-normalized estimation and a regularized empirical approximation for high-dimensional data. On synthetic distributions, IW-CFM recovers analytical powered targets for which numerically matched CFG, γ=ω, exhibits a visible mismatch. On ImageNet, an amortized lightweight corrector achieves better alignment than fixed- and matched-guidance CFG with the corresponding weighted and filtered reference distributions. We additionally present an exploratory CLIP analysis of powered-density rankings on real ImageNet data. Aggregate shifts across selected prompt-specific contrasts coexist with heterogeneous classwise responses, including nonmonotonic trajectories. These results provide a practical route for studying and approximating model-defined powered conditional distributions.
Figures & tables
Figure 1: Synthetic target recovery and qualitative ImageNet generations. (a,b) On a synthetic dataset, IW-CFM samples match the 99.7% probability-mass contour of the analytical pω , whereas CFG with numerically matched guidance γ=ω shows a visible mismatch. (c) On ImageNet, IW-CFM is trained using regularized empirical weights; both the CFG baseline and IW-CFM use inference-time guidance γ=4 , while IW-CFM additionally varies the target powering parameter ω .
Figure 2: Guided practical evaluation against regularized ImageNet references indexed by the powering parameter ω . IW-CFM targets each displayed ω and uses inference-time guidance γ=1.5 . The CFG baselines use fixed guidance γ=1.5 . The plots show FID (left) and FD-DINOv2 (right). “Weighted” uses feature moments weighted by rωreg , while “Filtered” uses uniform moments over the numerically active weights.
Model and parameter
Reference ω
Precision
Recall
Density
Coverage
CFG ( γ=1.5 )
4.0
0.55
0.50
0.95
0.90
CFG ( γ=4.0 )
4.0
0.70
0.30
1.10
0.89
IW-CFM ( ω=4.0 )
4.0
0.69
0.60
1.15
0.98
CFG ( γ=1.5 )
8.0
0.56
0.51
0.93
0.90
CFG ( γ=8.0 )
8.0
0.66
0.22
1.02
0.86
IW-CFM ( ω=8.0 )
8.0
0.70
0.61
1.15
0.98
Table 1: Guided practical evaluation using PRDC ( Naeem et al., 2020 ) against filtered regularized ImageNet references indexed by ω . Each IW-CFM row targets the listed ω and uses inference-time guidance γ=1.5 ; CFG uses either fixed guidance γ=1.5 or matched guidance γ=ω . The best score in each reference group is bolded. The class-balanced reference contains 10 instances with numerically nonzero stored weights from each of the 1,000 classes.
Figure 3: Sensitivity to the number of log-density integration steps. For each step count, we recompute the ImageNet log-densities and weights and train a corrector from scratch on the same subset of 100 categories. All models are evaluated against references constructed from the 250-step estimates, using 50 generated samples per category. Solid and dashed lines show distances to the filtered and weighted references, respectively. Both FID and FD-DINOv2 show increasing sensitivity to density-estimation accuracy as ω grows. Lower is better.
Figure 4: Sensitivity to target-regularization parameters. Each column corresponds to a clipped upper fraction τ , and colors denote the temperature T . Every corrector is trained using 250-step log-density estimates and evaluated against weighted and filtered references constructed using its matching (T,τ) , with 50 generated samples per ImageNet category. Solid and dashed lines denote the filtered and weighted references, respectively. More aggressive clipping produces an easier but more strongly modified operational target, while the effect of temperature depends on the powering parameter. Lower is better.
Figure 5: Illustrative powered-density rankings of real images from the ImageNet “hammerhead shark” category at different values of ω .
Figure 6: Class-dependent changes in the association between powered-density rankings and selected CLIP attributes. Rankings use the unregularized powered log-density score, before temperature scaling or clipping. For each ImageNet class c and attribute contrast a , dc,a(ω) compares the top- and bottom-ranked 10%, and Δdc,a(ω) reports its change from ω=1 . The density fields summarize the trajectories of all 1,000 classes; darker regions contain more trajectories, and orange curves show the across-class means.
Appendix figures & tables32 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Comparison of the ω -powered distribution and matched-guidance CFG on a univariate Gaussian mixture. The ω -powered distribution (dashed) concentrates around the target-class mode. The CFG distribution (solid), evaluated at the numerically matched guidance scale γ=ω , shifts probability mass toward low-density regions as the matched values increase.
Figure 8: hammerhead shark
Figure 9: loggerhead turtle (lat. Caretta caretta)
Figure 10: ruddy turnstone (lat. Arenaria interpres)
Figure 11: backpack, back pack, knapsack, packsack, rucksack, haversack
Figure 12: balloon
Figure 13: plastic bag
Figure 14: recreational vehicle, RV, R.V.
Figure 15: running shoe
Figure 16: triumphal arch
Contrast
Positive Prompt
Negative Prompt
complex/simple
Complex photo.
Simple photo.
natural/synthetic
Natural photo.
Synthetic photo.
realism/CGI
An authentic, real-world photo of a { class_name} , captured by a camera.
A synthetic, computer-generated image of a { class_name} , a digital rendering.
texture
A photo of a { class_name} with realistic, detailed surface textures.
A photo of a { class_name} with an unnaturally smooth, waxy, or plastic-like texture.
Appendix
Table 2: Prompt pairs for the CLIP evaluation suite. During evaluation, { class_name} is replaced with the ImageNet class name. The complex/simple and natural/synthetic pairs are from Wang et al. (2023) ; the realism/CGI and texture pairs are inspired by its paired-prompt design.
Figure 17: Ranking stability of log-density evaluation. Spearman rank correlation ( ρ ) between rankings at each step count and the 250-step estimates. Both conditional and unconditional rankings reach ρ>0.99 by 125 steps.
Figure 18: Numerical convergence of log-densities. Hexbin plots comparing log-density estimates at N∈{2,5,25,50,125} steps with the 250-step estimates.
Batch
Mode
Forward (%)
Sampling (%)
Sampling + VAE (%)
1
Eager
+21.41±0.33
+21.49±0.12
+18.38±3.03
1
Compiled
+1.98±0.03
+2.91±0.02
+2.88±0.00
24
Eager
+0.58±0.07
+0.38±0.47
+0.84±0.40
24
Compiled
+0.12±0.11
+0.36±0.50
+0.18±0.12
Appendix
Table 3: Corrector runtime overhead relative to base SiT for 250-NFE Heun sampling. Central values are computed from the ratio of mean latencies; uncertainties are delta-method standard errors assuming independent base-model and corrector timing samples. The VAE decoder is always compiled.
Figure 19: Effect of weight regularization. Average number of numerically nonzero per-class weights after finite-precision normalization as a function of ω , for different temperatures T and clipping fractions τ . The highlighted curve is the configuration used in our experiments ( T=1.2 , τ=0.01 ). Error bars show the standard deviation across the 1,000 ImageNet categories. Horizontal-flip augmentation doubles the number of available instances in practice.
Animals (29 categories)
great white shark (2)
bald eagle (22)
axolotl (29)
loggerhead turtle (33)
Komodo dragon (48)
triceratops (51)
scorpion (71)
peacock (84)
platypus (103)
jellyfish (107)
chambered nautilus (117)
hermit crab (125)
flamingo (130)
king penguin (145)
killer whale (148)
Labrador retriever (208)
tabby cat (281)
tiger (292)
bee (309)
dragonfly (319)
monarch butterfly (323)
starfish (327)
zebra (340)
hippopotamus (344)
Arabian camel (354)
three-toed sloth (364)
chimpanzee (367)
African elephant (386)
pufferfish (397)
Appendix
Table 4: The 100 ImageNet categories used in the number of steps in log-densities ablation experiment. Numbers in parentheses denote the ImageNet class IDs.
Figure 20: Absolute classwise CLIP contrasts underlying the baseline-relative trajectories in Figure 6 . Positive dc,a(ω) indicates that the top-ranked images align more strongly with the positive prompt than the bottom-ranked images. The density fields show the classwise trajectories, and the orange curves show their across-class means.
Figure 21: In-sample oracle upper envelope for the additional IW-CFM–CFG comparison. Each panel shows ImageNet categories according to the paired CLIP score of CFG at γ=4 (x-axis) and the maximum score difference attained by IW-CFM at the same inference-time guidance after selecting its target ω on the evaluation samples (y-axis). Color intensity denotes the number of categories in each hexagonal bin, and the percentage is the fraction with a positive post-selection difference. Because selection and evaluation use the same samples, the figure is neither a held-out performance estimate nor evidence of statistical significance.
Figure 22: Three-Gaussian comparison. CFG with matched guidance γ=ω (left) departs from the analytical target contours, whereas IW-CFM targeting ω (right) remains close to them.
Figure 23: Fractal comparison. CFG with matched guidance γ=ω (left) departs from the analytical target contours, whereas IW-CFM targeting ω (right) remains close to them.
Figure 24: REG and FKC on a Gaussian mixture. Results from our implementations of REG ( Gao et al., 2025b ) (left) and FKC ( Skreta et al., 2025a ) (right), which follow the published algorithms and reuse components from released code where applicable. Under these configurations, neither sample distribution closely follows the analytical ω -powered contours.
Figure 25: Distributional alignment across ω . Extension of Figure 2 across the full range of target powering parameters. IW-CFM targets each ω and uses inference-time guidance γ=1.5 . CFG uses either fixed guidance γ=1.5 or matched guidance γ=ω . The log-scale y-axis shows the increase in FID and FD-DINOv2 for matched-guidance CFG at larger γ=ω , while IW-CFM remains closer to the regularized empirical reference.
Figure 26: Qualitative matched-guidance comparison on ImageNet. Columns correspond to target powers ω∈{4,8,16,32,64,128} . The top row shows standard CFG at the numerically matched guidance scale γ=ω ; the bottom row shows IW-CFM targeting the same ω while holding inference-time guidance fixed at γ=4 . In this golden retriever example, CFG remains recognizable through γ=16 but exhibits severe saturation and loss of coherent image structure for γ≥32 , whereas IW-CFM remains visually coherent across the evaluated target powers. This illustrative comparison complements the distributional alignment metrics in Figure 25 .
Model
Reference ω
FID
FD-DINOv2
Base SiT ( γ=1 )
4.0
19.97
324.02
IW-CFM ( ω=4.0 )
4.0
8.48
215.80
Base SiT ( γ=1 )
8.0
20.55
328.66
IW-CFM ( ω=8.0 )
8.0
8.90
221.41
Base SiT ( γ=1 )
16.0
21.41
339.18
IW-CFM ( ω=16.0 )
16.0
9.29
223.39
Appendix
Table 5: Unguided distribution-shift evaluation using Fréchet Distance to the filtered regularized ImageNet reference. All IW-CFM rows use γ=1 and differ only in the target powering parameter ω . IW-CFM outperforms the conditional Base SiT at every evaluated ω ; comparison with Figure 2 shows the additional benefit of guided inference.
Model
Reference ω
Precision
Recall
Density
Coverage
Base SiT ( γ=1 )
4.0
0.37
0.59
0.64
0.81
IW-CFM ( ω=4.0 )
4.0
0.51
0.68
0.87
0.93
Base SiT ( γ=1 )
8.0
0.37
0.60
0.62
0.80
IW-CFM ( ω=8.0 )
8.0
0.51
0.69
0.84
0.93
Base SiT ( γ=1 )
16.0
0.39
0.62
0.61
0.79
IW-CFM ( ω=16.0 )
16.0
0.51
0.70
0.83
0.92
Appendix
Table 6: Unguided distribution-shift evaluation using Precision, Recall, Density, and Coverage with respect to the filtered regularized ImageNet reference. All IW-CFM rows use γ=1 and differ only in the target powering parameter ω . IW-CFM is better aligned than the conditional Base SiT at every ω ; Table 1 reports the guided practical configuration.
Model
FID-50k
FD-DINOv2-50k
CFG ( γ=1.5 )
2.04
88.36
CFG ( γ=4.0 )
14.07
85.64
IW-CFM ( ω=4.0 )
8.33
157.81
CFG ( γ=8.0 )
16.32
137.56
IW-CFM ( ω=8.0 )
8.49
159.98
CFG ( γ=16.0 )
16.25
238.82
Appendix
Table 7: Fréchet Distance with respect to the category-saturated 50k subset of the ImageNet training set using two feature backbones. CFG rows are labeled by guidance scale γ ; IW-CFM rows are labeled by powering parameter ω and use inference-time guidance γ=1.5 . The best CFG configuration outperforms IW-CFM, although the gap narrows as ω increases.
Model
Precision
Recall
Density
Coverage
CFG ( γ=1.5 )
0.83
0.58
1.17
0.98
IW-CFM ( ω=4.0 )
0.78
0.49
1.05
0.94
IW-CFM ( ω=8.0 )
0.78
0.49
1.05
0.93
IW-CFM ( ω=16.0 )
0.79
0.48
1.07
0.94
IW-CFM ( ω=32.0 )
0.81
0.49
1.08
0.94
IW-CFM ( ω=64.0 )
0.81
0.52
1.08
0.95
Appendix
Table 8: Precision, Recall, Density, and Coverage with respect to the 10k-feature ImageNet evaluation batch from OpenAI (2021) . CFG is labeled by guidance scale γ ; IW-CFM is labeled by powering parameter ω and uses inference-time guidance γ=1.5 . As in Table 7 , the baseline has the best overlap with the reference, although the gap narrows as ω increases.
Model
FID-50k
FD-DINOv2-50k
Base SiT ( γ=1 )
9.55
246.24
IW-CFM ( ω=4.0 )
11.21
247.82
IW-CFM ( ω=8.0 )
11.53
252.24
IW-CFM ( ω=16.0 )
10.52
238.72
IW-CFM ( ω=32.0 )
7.64
204.28
IW-CFM ( ω=64.0 )
7.14
202.63
Appendix
Table 9: Fréchet Distance with respect to the category-saturated 50k subset of the ImageNet training set without inference-time guidance ( γ=1 ). IW-CFM at ω=64 achieves the best result for both feature backbones, outperforming the conditional Base SiT.
Figure 27: Nearest-neighbor retrieval test. Fraction of IW-CFM realizations satisfying 1.82d1<d2 across powering parameters ω and inference-time guidance scales γ∈{1.5,4} , where d1 and d2 are Euclidean distances to the first and second nearest neighbors in the deduplicated training set. The ratios remain low under this test, but do not by themselves establish generalization.
Figure 28: Additional qualitative ImageNet trajectories for four classes. Baseline CFG and IW-CFM both use γ=4 ; the IW-CFM columns vary the target powering parameter ω .
Figure 29: Additional qualitative ImageNet trajectories for four classes. Baseline CFG and IW-CFM both use γ=4 ; the IW-CFM columns vary the target powering parameter ω .
Figure 30: Additional qualitative trajectory for the triumphal arch class. Baseline CFG and IW-CFM both use γ=4 ; the IW-CFM columns vary the target powering parameter ω .
Classifier-free guidance (CFG) improves conditional generation in Flow Matching, but strong guidance can distort the generated distribution and reduce diversity. We provide a geometric account of this behavior by viewing Flow Matching as a time-varying gradient flow and characterizing how CFG reshapes its underlying potential. This view explains how stronger alignment can be accompanied by mean displacement and trajectory concentration, and motivates controlling guidance through the model-implied terminal posterior mean. We therefore propose Posterior-Mean-Capped CFG (PMC-CFG), a training-free, per-sample method that adaptively retains the strongest feasible guidance without additional network evaluations. Experiments on synthetic and large-scale image-generation benchmarks show that PMC-CFG limits guidance-induced distortion and concentration while improving the alignment--diversity trade-off, with particularly strong benefits when nominal guidance is large.
Jishen Peng, Zheng Ma
School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China · Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University, Shanghai, China · Qing Yuan Research Institute, Shanghai Jiao Tong University, Shanghai, China +1
Classifier-free Guidance (CFG) lets practitioners trade-off fidelity against diversity in Diffusion Models (DMs). The practicality of CFG is however hindered by DMs sampling cost. On the other hand, Consistency Models (CMs) generate images in one or a few steps, but existing guidance methods require knowledge distillation from a separate DM teacher, limiting CFG to Consistency Distillation (CD) methods. We propose Joint Flow Distribution Learning (JFDL), a lightweight alignment method enabling guidance in a pre-trained CM. With a pre-trained CM as an ordinary differential equation (ODE) solver, we verify with normality tests that the variance-exploding noise implied by the velocity fields from unconditional and conditional distributions is Gaussian. In practice, JFDL equips CMs with the familiar adjustable guidance knob, yielding guided images with similar characteristics to CFG. Applied to an original Consistency Trained (CT) CM that could only do conditional sampling, JFDL unlocks guided generation and reduces FID on both CIFAR-10 and ImageNet 64x64 datasets. This is the first time that CMs are able to receive effective guidance post-hoc without a DM teacher, thus, bridging a key gap in current methods for CMs.
Classifier-Free Guidance (CFG) is essential for high-fidelity conditional generation in flow matching, yet it imposes significant computational overhead by requiring dual forward passes at each sampling step. In this work, we address this bottleneck by introducing P-Guide, a framework that achieves high-quality guidance through a single inference pass by modulating only the initial latent state. We further show that, under a first-order approximation, P-Guide is equivalent to CFG in the sense that it steers generation from the prior space, without requiring explicit velocity field extrapolation during sampling. We consider both homoscedastic and heteroscedastic priors, and find that jointly modeling the mean and variance enables adaptive loss attenuation and improved robustness to data uncertainty. Extensive experiments demonstrate that P-Guide reduces inference latency by approximately 50% while maintaining fidelity and prompt alignment competitive with standard dual-pass CFG baselines.
Xin Peng, Ang Gao
School of Physical Science and Technology, Beijing University of Posts and Telecommunications, Beijing, China