Learning $ω$-Powered Conditional Distributions with Importance-Weighted Flow Matching

Maciej Falkiewicz; Frantzeska Lavda; Jason Ramapuram; Alexandros Kalousis

cs.LGOct 12, 2026

Learning $ω$ -Powered Conditional Distributions with Importance-Weighted Flow Matching

Authors: Maciej Falkiewicz, Frantzeska Lavda, Jason Ramapuram, Alexandros Kalousis

Organizations: Computer Science Department, University of Geneva · HES-SO/HEG Genéve · Google Deepmind (work done at Apple)

Abstract

Classifier-Free Guidance (CFG) with guidance scale $γ$ is commonly motivated by the $ω$ -powered conditional distribution obtained by setting $ω=γ$ , but CFG does not, in general, sample from this distribution exactly. We study whether the powered target can instead be learned explicitly. We introduce Importance-Weighted Conditional Flow Matching (IW-CFM), which reweights conditional flow-matching examples using density ratios obtained from frozen conditional and unconditional flow models. With exact normalized weights, the resulting objective has the same gradient as flow matching on the powered target; in practice, we use self-normalized estimation and a regularized empirical approximation for high-dimensional data. On synthetic distributions, IW-CFM recovers analytical powered targets for which numerically matched CFG, $γ=ω$ , exhibits a visible mismatch. On ImageNet, an amortized lightweight corrector achieves better alignment than fixed- and matched-guidance CFG with the corresponding weighted and filtered reference distributions. We additionally present an exploratory CLIP analysis of powered-density rankings on real ImageNet data. Aggregate shifts across selected prompt-specific contrasts coexist with heterogeneous classwise responses, including nonmonotonic trajectories. These results provide a practical route for studying and approximating model-defined powered conditional distributions.

Figures & tables

Panel of Figure 1 — Figure 1: Synthetic target recovery and qualitative ImageNet generations. (a,b) On a synthetic dataset, IW-CFM samples match the 99.7% probability-mass contour of the analytical $p_{\omega}$ , whereas CFG with numerically matched guidance $\gamma=\omega$ shows a visible mismatch. (c) On ImageNet, IW-CFM is trained using regularized empirical weights; both the CFG baseline and IW-CFM use inference-time guidance $\gamma=4$ , while IW-CFM additionally varies the target powering parameter $\omega$ .

Figure 2: Guided practical evaluation against regularized ImageNet references indexed by the powering parameter

\omega

. IW-CFM targets each displayed

\omega

and uses inference-time guidance

\gamma=1.5

. The CFG baselines use fixed guidance

\gamma=1.5

. The plots show FID (left) and FD-DINOv2 (right). “Weighted” uses feature moments weighted by

r_{\omega}^{\mathrm{reg}}

, while “Filtered” uses uniform moments over the numerically active weights.

Model and parameter	Reference $\omega$	Precision	Recall	Density	Coverage
CFG ( $\gamma=1.5$ )	4.0	0.55	0.50	0.95	0.90
CFG ( $\gamma=4.0$ )	4.0	0.70	0.30	1.10	0.89
IW-CFM ( $\omega=4.0$ )	4.0	0.69	0.60	1.15	0.98
CFG ( $\gamma=1.5$ )	8.0	0.56	0.51	0.93	0.90
CFG ( $\gamma=8.0$ )	8.0	0.66	0.22	1.02	0.86
IW-CFM ( $\omega=8.0$ )	8.0	0.70	0.61	1.15	0.98

Table 1: Guided practical evaluation using PRDC ( Naeem et al., 2020 ) against filtered regularized ImageNet references indexed by

\omega

. Each IW-CFM row targets the listed

\omega

and uses inference-time guidance

\gamma=1.5

; CFG uses either fixed guidance

\gamma=1.5

or matched guidance

\gamma=\omega

. The best score in each reference group is bolded. The class-balanced reference contains 10 instances with numerically nonzero stored weights from each of the 1,000 classes.

Figure 3: Sensitivity to the number of log-density integration steps. For each step count, we recompute the ImageNet log-densities and weights and train a corrector from scratch on the same subset of 100 categories. All models are evaluated against references constructed from the 250-step estimates, using 50 generated samples per category. Solid and dashed lines show distances to the filtered and weighted references, respectively. Both FID and FD-DINOv2 show increasing sensitivity to density-estimation accuracy as

\omega

grows. Lower is better.

Figure 4: Sensitivity to target-regularization parameters. Each column corresponds to a clipped upper fraction

\tau

, and colors denote the temperature

T

. Every corrector is trained using 250-step log-density estimates and evaluated against weighted and filtered references constructed using its matching

(T,\tau)

, with 50 generated samples per ImageNet category. Solid and dashed lines denote the filtered and weighted references, respectively. More aggressive clipping produces an easier but more strongly modified operational target, while the effect of temperature depends on the powering parameter. Lower is better.

Figure 5: Illustrative powered-density rankings of real images from the ImageNet “hammerhead shark” category at different values of $\omega$ .

Panel of Figure 6 — Figure 6: Class-dependent changes in the association between powered-density rankings and selected CLIP attributes. Rankings use the unregularized powered log-density score, before temperature scaling or clipping. For each ImageNet class $c$ and attribute contrast $a$ , $d_{c,a}(\omega)$ compares the top- and bottom-ranked 10%, and $\Delta d_{c,a}(\omega)$ reports its change from $\omega=1$ . The density fields summarize the trajectories of all 1,000 classes; darker regions contain more trajectories, and orange curves show the across-class means.

Appendix figures & tables32 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Comparison of the $\omega$ -powered distribution and matched-guidance CFG on a univariate Gaussian mixture. The $\omega$ -powered distribution (dashed) concentrates around the target-class mode. The CFG distribution (solid), evaluated at the numerically matched guidance scale $\gamma=\omega$ , shifts probability mass toward low-density regions as the matched values increase.

Figure 9: loggerhead turtle (lat. Caretta caretta)

Figure 10: ruddy turnstone (lat. Arenaria interpres)

Figure 11: backpack, back pack, knapsack, packsack, rucksack, haversack

Figure 14: recreational vehicle, RV, R.V.

Contrast	Positive Prompt	Negative Prompt
complex/simple	Complex photo.	Simple photo.
natural/synthetic	Natural photo.	Synthetic photo.
realism/CGI	An authentic, real-world photo of a { class_name} , captured by a camera.	A synthetic, computer-generated image of a { class_name} , a digital rendering.
texture	A photo of a { class_name} with realistic, detailed surface textures.	A photo of a { class_name} with an unnaturally smooth, waxy, or plastic-like texture.

Appendix

Table 2: Prompt pairs for the CLIP evaluation suite. During evaluation, { class_name} is replaced with the ImageNet class name. The complex/simple and natural/synthetic pairs are from Wang et al. (2023) ; the realism/CGI and texture pairs are inspired by its paired-prompt design.

Figure 17: Ranking stability of log-density evaluation. Spearman rank correlation (

\rho

) between rankings at each step count and the 250-step estimates. Both conditional and unconditional rankings reach

\rho>0.99

by 125 steps.

Panel of Figure 18 — Figure 18: Numerical convergence of log-densities. Hexbin plots comparing log-density estimates at $N\in\{2,5,25,50,125\}$ steps with the 250-step estimates.

Batch	Mode	Forward (%)	Sampling (%)	Sampling + VAE (%)
1	Eager	$+21.41\pm 0.33$	$+21.49\pm 0.12$	$+18.38\pm 3.03$
1	Compiled	$+1.98\pm 0.03$	$+2.91\pm 0.02$	$+2.88\pm 0.00$
24	Eager	$+0.58\pm 0.07$	$+0.38\pm 0.47$	$+0.84\pm 0.40$
24	Compiled	$+0.12\pm 0.11$	$+0.36\pm 0.50$	$+0.18\pm 0.12$

Appendix

Table 3: Corrector runtime overhead relative to base SiT for 250-NFE Heun sampling. Central values are computed from the ratio of mean latencies; uncertainties are delta-method standard errors assuming independent base-model and corrector timing samples. The VAE decoder is always compiled.

Figure 19: Effect of weight regularization. Average number of numerically nonzero per-class weights after finite-precision normalization as a function of

\omega

, for different temperatures

T

and clipping fractions

\tau

. The highlighted curve is the configuration used in our experiments (

T=1.2

\tau=0.01

). Error bars show the standard deviation across the 1,000 ImageNet categories. Horizontal-flip augmentation doubles the number of available instances in practice.

Animals (29 categories)
great white shark (2)	bald eagle (22)	axolotl (29)	loggerhead turtle (33)	Komodo dragon (48)
triceratops (51)	scorpion (71)	peacock (84)	platypus (103)	jellyfish (107)
chambered nautilus (117)	hermit crab (125)	flamingo (130)	king penguin (145)	killer whale (148)
Labrador retriever (208)	tabby cat (281)	tiger (292)	bee (309)	dragonfly (319)
monarch butterfly (323)	starfish (327)	zebra (340)	hippopotamus (344)	Arabian camel (354)
three-toed sloth (364)	chimpanzee (367)	African elephant (386)	pufferfish (397)

Appendix

Table 4: The 100 ImageNet categories used in the number of steps in log-densities ablation experiment. Numbers in parentheses denote the ImageNet class IDs.

Panel of Figure 20 — Figure 20: Absolute classwise CLIP contrasts underlying the baseline-relative trajectories in Figure 6 . Positive $d_{c,a}(\omega)$ indicates that the top-ranked images align more strongly with the positive prompt than the bottom-ranked images. The density fields show the classwise trajectories, and the orange curves show their across-class means.

Figure 21: In-sample oracle upper envelope for the additional IW-CFM–CFG comparison. Each panel shows ImageNet categories according to the paired CLIP score of CFG at

\gamma=4

(x-axis) and the maximum score difference attained by IW-CFM at the same inference-time guidance after selecting its target

\omega

on the evaluation samples (y-axis). Color intensity denotes the number of categories in each hexagonal bin, and the percentage is the fraction with a positive post-selection difference. Because selection and evaluation use the same samples, the figure is neither a held-out performance estimate nor evidence of statistical significance.

Panel of Figure 22 — Figure 22: Three-Gaussian comparison. CFG with matched guidance $\gamma=\omega$ (left) departs from the analytical target contours, whereas IW-CFM targeting $\omega$ (right) remains close to them.

Panel of Figure 23 — Figure 23: Fractal comparison. CFG with matched guidance $\gamma=\omega$ (left) departs from the analytical target contours, whereas IW-CFM targeting $\omega$ (right) remains close to them.

Panel of Figure 24 — Figure 24: REG and FKC on a Gaussian mixture. Results from our implementations of REG ( Gao et al., 2025b ) (left) and FKC ( Skreta et al., 2025a ) (right), which follow the published algorithms and reuse components from released code where applicable. Under these configurations, neither sample distribution closely follows the analytical $\omega$ -powered contours.

Figure 25: Distributional alignment across

\omega

. Extension of Figure 2 across the full range of target powering parameters. IW-CFM targets each

\omega

and uses inference-time guidance

\gamma=1.5

. CFG uses either fixed guidance

\gamma=1.5

or matched guidance

\gamma=\omega

. The log-scale y-axis shows the increase in FID and FD-DINOv2 for matched-guidance CFG at larger

\gamma=\omega

, while IW-CFM remains closer to the regularized empirical reference.

Figure 26: Qualitative matched-guidance comparison on ImageNet. Columns correspond to target powers $\omega\in\{4,8,16,32,64,128\}$ . The top row shows standard CFG at the numerically matched guidance scale $\gamma=\omega$ ; the bottom row shows IW-CFM targeting the same $\omega$ while holding inference-time guidance fixed at $\gamma=4$ . In this golden retriever example, CFG remains recognizable through $\gamma=16$ but exhibits severe saturation and loss of coherent image structure for $\gamma\geq 32$ , whereas IW-CFM remains visually coherent across the evaluated target powers. This illustrative comparison complements the distributional alignment metrics in Figure 25 .

Model	Reference $\omega$	FID	FD-DINOv2
Base SiT ( $\gamma=1$ )	4.0	19.97	324.02
IW-CFM ( $\omega=4.0$ )	4.0	8.48	215.80
Base SiT ( $\gamma=1$ )	8.0	20.55	328.66
IW-CFM ( $\omega=8.0$ )	8.0	8.90	221.41
Base SiT ( $\gamma=1$ )	16.0	21.41	339.18
IW-CFM ( $\omega=16.0$ )	16.0	9.29	223.39

Appendix

Table 5: Unguided distribution-shift evaluation using Fréchet Distance to the filtered regularized ImageNet reference. All IW-CFM rows use

\gamma=1

and differ only in the target powering parameter

\omega

. IW-CFM outperforms the conditional Base SiT at every evaluated

\omega

; comparison with Figure 2 shows the additional benefit of guided inference.

Model	Reference $\omega$	Precision	Recall	Density	Coverage
Base SiT ( $\gamma=1$ )	4.0	0.37	0.59	0.64	0.81
IW-CFM ( $\omega=4.0$ )	4.0	0.51	0.68	0.87	0.93
Base SiT ( $\gamma=1$ )	8.0	0.37	0.60	0.62	0.80
IW-CFM ( $\omega=8.0$ )	8.0	0.51	0.69	0.84	0.93
Base SiT ( $\gamma=1$ )	16.0	0.39	0.62	0.61	0.79
IW-CFM ( $\omega=16.0$ )	16.0	0.51	0.70	0.83	0.92

Appendix

Table 6: Unguided distribution-shift evaluation using Precision, Recall, Density, and Coverage with respect to the filtered regularized ImageNet reference. All IW-CFM rows use

\gamma=1

and differ only in the target powering parameter

\omega

. IW-CFM is better aligned than the conditional Base SiT at every

\omega

; Table 1 reports the guided practical configuration.

Model	FID-50k	FD-DINOv2-50k
CFG ( $\gamma=1.5$ )	2.04	88.36
CFG ( $\gamma=4.0$ )	14.07	85.64
IW-CFM ( $\omega=4.0$ )	8.33	157.81
CFG ( $\gamma=8.0$ )	16.32	137.56
IW-CFM ( $\omega=8.0$ )	8.49	159.98
CFG ( $\gamma=16.0$ )	16.25	238.82

Appendix

Table 7: Fréchet Distance with respect to the category-saturated 50k subset of the ImageNet training set using two feature backbones. CFG rows are labeled by guidance scale

\gamma

; IW-CFM rows are labeled by powering parameter

\omega

and use inference-time guidance

\gamma=1.5

. The best CFG configuration outperforms IW-CFM, although the gap narrows as

\omega

increases.

Model	Precision	Recall	Density	Coverage
CFG ( $\gamma=1.5$ )	0.83	0.58	1.17	0.98
IW-CFM ( $\omega=4.0$ )	0.78	0.49	1.05	0.94
IW-CFM ( $\omega=8.0$ )	0.78	0.49	1.05	0.93
IW-CFM ( $\omega=16.0$ )	0.79	0.48	1.07	0.94
IW-CFM ( $\omega=32.0$ )	0.81	0.49	1.08	0.94
IW-CFM ( $\omega=64.0$ )	0.81	0.52	1.08	0.95

Appendix

Table 8: Precision, Recall, Density, and Coverage with respect to the 10k-feature ImageNet evaluation batch from OpenAI (2021) . CFG is labeled by guidance scale

\gamma

; IW-CFM is labeled by powering parameter

\omega

and uses inference-time guidance

\gamma=1.5

. As in Table 7 , the baseline has the best overlap with the reference, although the gap narrows as

\omega

increases.

Model	FID-50k	FD-DINOv2-50k
Base SiT ( $\gamma=1$ )	9.55	246.24
IW-CFM ( $\omega=4.0$ )	11.21	247.82
IW-CFM ( $\omega=8.0$ )	11.53	252.24
IW-CFM ( $\omega=16.0$ )	10.52	238.72
IW-CFM ( $\omega=32.0$ )	7.64	204.28
IW-CFM ( $\omega=64.0$ )	7.14	202.63

Appendix

Table 9: Fréchet Distance with respect to the category-saturated 50k subset of the ImageNet training set without inference-time guidance (

\gamma=1

). IW-CFM at

\omega=64

achieves the best result for both feature backbones, outperforming the conditional Base SiT.

Figure 27: Nearest-neighbor retrieval test. Fraction of IW-CFM realizations satisfying

1.82d_{1}<d_{2}

across powering parameters

\omega

and inference-time guidance scales

\gamma\in\{1.5,4\}

, where

d_{1}

and

d_{2}

are Euclidean distances to the first and second nearest neighbors in the deduplicated training set. The ratios remain low under this test, but do not by themselves establish generalization.

Panel of Figure 28 — Figure 28: Additional qualitative ImageNet trajectories for four classes. Baseline CFG and IW-CFM both use $\gamma=4$ ; the IW-CFM columns vary the target powering parameter $\omega$ .

Panel of Figure 29 — Figure 29: Additional qualitative ImageNet trajectories for four classes. Baseline CFG and IW-CFM both use $\gamma=4$ ; the IW-CFM columns vary the target powering parameter $\omega$ .

Figure 30: Additional qualitative trajectory for the triumphal arch class. Baseline CFG and IW-CFM both use $\gamma=4$ ; the IW-CFM columns vary the target powering parameter $\omega$ .

Explore similar work

Classifier-Free Guidance in Flow Matching: Non-Autonomous Potentials, Overshoot, and Posterior-Mean Control
Sep 21, 2026Jishen Peng, Zheng MaFlow Matching Classifier-Free Guidance
School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China · Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University, Shanghai, China · Qing Yuan Research Institute, Shanghai Jiao Tong University, Shanghai, China +1
Post-Hoc Guidance for Consistency Models by Joint Flow Distribution Learning
Apr 10, 2026Chia-Hong Hsu, Randall BalestrieroConsistency Models Diffusion Model Guidance
Brown University
P-Guide: Parameter-Efficient Prior Steering for Single-Pass CFG Inference
May 7, 2026Xin Peng, Ang GaoFlow Matching Classifier-Free Guidance
School of Physical Science and Technology, Beijing University of Posts and Telecommunications, Beijing, China

Sep 21, 2026J/K move · Enter open · S save

Classifier-Free Guidance in Flow Matching: Non-Autonomous Potentials, Overshoot, and Posterior-Mean Control

Jishen Peng, Zheng Ma

School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China · Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University, Shanghai, China · Qing Yuan Research Institute, Shanghai Jiao Tong University, Shanghai, China+1

Flow Matching Classifier-Free Guidance Conditional Image Generation Flow-Based Generative Modeling

Open paper AskPDF arXiv

Classifier-free guidance (CFG) improves conditional generation in Flow Matching, but strong guidance can distort the generated distribution and reduce diversity. We provide a geometric account of this behavior by viewing Flow Matching as a time-varying gradient flow and characterizing how CFG reshapes its underlying potential. This view explains how stronger alignment can be accompanied by mean displacement and trajectory concentration, and motivates controlling guidance through the model-implied terminal posterior mean. We therefore propose Posterior-Mean-Capped CFG (PMC-CFG), a training-free, per-sample method that adaptively retains the strongest feasible guidance without additional network evaluations. Experiments on synthetic and large-scale image-generation benchmarks show that PMC-CFG limits guidance-induced distortion and concentration while improving the alignment--diversity trade-off, with particularly strong benefits when nominal guidance is large.

Abstract

Explore similar work

Classifier-Free Guidance in Flow Matching: Non-Autonomous Potentials, Overshoot, and Posterior-Mean Control

Post-Hoc Guidance for Consistency Models by Joint Flow Distribution Learning

P-Guide: Parameter-Efficient Prior Steering for Single-Pass CFG Inference