Correcting Guided Diffusion Trajectories with Spectral Alignment

Gihoon Kim; Taesup Kim

cs.CVOct 2, 2026

Correcting Guided Diffusion Trajectories with Spectral Alignment

Authors: Gihoon Kim, Taesup Kim

Organizations: Graduate School of Data Science, Seoul National University

Abstract

The practical success of conditional image generation hinges on fine-grained differences in condition alignment and visual fidelity. Classifier-free guidance (CFG) is central to this success, but its lack of an explicit criterion makes it difficult to assess whether the guided trajectory is progressing as intended. To address this gap, we show that spectral alignment provides a principled criterion for understanding guidance behavior and improving guided diffusion sampling through adaptive correction. Our analysis identifies the spectra of intermediate states as an indicator of consistency with the expected spectral evolution of the forward process. Based on this observation, we introduce Spectral Correction Guidance, a method that corrects deviations from an analytic reference spectrum during sampling. The proposed method is training-free and applicable across diffusion backbones and conditional generation tasks without modifying the underlying model. Experiments demonstrate consistent gains in preference-based metrics over baseline guidance methods in text-to-image generation and improved generation quality over CFG on ImageNet. These improvements persist across a range of guidance scales and with fewer denoising steps. Our analyses and ablations provide insight into guidance behavior and how the proposed method affects generation quality.

Figures & tables

Panel of Figure 1 — Figure 1: For the prompt “ a male surfer in black shorts carrying a white board ” on SD3.5 ( Esser et al., 2024 ) , sampling without CFG fails to form a reliable conditional image. CFG produces a more plausible image but introduces an artifact where the surfboard overlaps with the person. Our method better preserves the intended composition and improves visual coherence. It also yields the lowest spectral distance, which we analyze in Section 3 .

Panel of Figure 2 — Figure 2: Effect of guidance scale on spectral distance defined in Eq. 6 , FID, and IS. The x-axis is the guidance scale and the y-axis is the metric value. Guidance scale values are uniformly spaced for visual clarity. The red dashed line marks the default guidance scale.

Panel of Figure 4 — Figure 4: Qualitative and spectral comparison for the prompt “ a man holding a horse dressed like a knight ”. Generated images are included for visual reference, and the power spectra are shown over denoising timesteps, with radial frequency bins on the $x$ -axis and power on the $y$ -axis. The corrected spectrum $\tilde{S}_{t}(\omega)$ follows the analytic target $T_{t}(\omega)$ more closely than the baseline spectrum $S_{t}(\omega)$ , reducing artifacts and improving visual quality in the generated image.

Model	$w$	Method	HPSv3 $\uparrow$	ImageReward $\uparrow$	PickScore $\uparrow$	HPSv2 $\uparrow$	CLIP-T $\uparrow$	FID $\downarrow$	sFID $\downarrow$
SDXL	5.0	Ours	8.437	0.731	22.43	0.279	0.264	17.02	47.94
		CFG	8.096	0.700	22.41	0.274	0.265	17.62	49.27
		CFG++	7.962	0.684	22.36	0.273	0.265	17.46	49.11
		TV-CFG	6.952	0.514	22.03	0.263	0.264	18.94	50.14
		LF-CFG	7.045	0.566	21.95	0.267	0.262	18.00	52.75
PixArt- $\alpha$	4.5	Ours	9.957	0.876	22.63	0.299	0.260	27.95	66.35

Table 1: Quantitative comparison of guidance methods on text-to-image diffusion models, including HPSv3. Our method achieves the strongest preference-based performance across models. Among existing guidance strategies, standard CFG remains the most competitive baseline in most cases.

Panel of Figure 5 — Figure 5: Qualitative comparison of guidance methods across text-to-image diffusion models. Our method better preserves the intended composition and improves visual coherence.

Panel of Figure 6 — Figure 6: Qualitative comparison between CFG and our method on SD3.5 at a high guidance scale of $w=14$ . Our method corrects over-saturated and over-simplified outputs produced by CFG.

Panel of Figure 8 — Figure 8: Ablation on the correction strength $\lambda$ . Increasing $\lambda$ improves visual fidelity and prompt alignment up to a point, but excessive correction introduces artifacts.

Panel of Figure 9 — Figure 9: Ablation on the relaxation parameter $\rho$ . Smaller $\rho$ applies correction for longer during denoising, while larger $\rho$ relaxes it earlier. Intermediate relaxation yields the best balance between spectral correction and artifact reduction.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

\tilde{S}_{t-1}(\omega_{k})=S_{t-1}(\omega_{k})\left(\frac{T_{t-1}(\omega_{k})+\varepsilon}{S_{t-1}(\omega_{k})+\varepsilon}\right)^{\lambda},\quad k=1,\dots,B

Appendix

Algorithm 1 Spectral Correction Guidance

Model	Method	Peak VRAM (GB)	Time (s/image)
SDXL	Ours	12.70	15.442
SDXL	CFG	12.84	13.775
SD3.5	Ours	12.68	25.236
SD3.5	CFG	12.67	22.628

Appendix

Table 4: Computational cost of standard CFG and our method, measured over 100 images.

	Ours	CFG	Win Rate (Ours)
Visual Quality	595	405	59.50%
Prompt Alignment	588	412	58.80%

Appendix

Table 5: Human preference study comparing our method with CFG on SDXL. Each criterion receives 1,000 votes from 20 participants evaluating 50 prompts.

Method	HPSv3 $\uparrow$	ImageReward $\uparrow$	PickScore $\uparrow$	HPSv2 $\uparrow$	CLIP-T $\uparrow$	FID $\downarrow$	sFID $\downarrow$
Ours	$\mathbf{8.375\pm 0.045}$	$\mathbf{0.7368\pm 0.0088}$	$\mathbf{22.463\pm 0.021}$	$\mathbf{0.2800\pm 0.0004}$	$0.2651\pm 0.0004$	$\mathbf{66.572\pm 0.557}$	$344.398\pm 0.434$
CFG	$8.068\pm 0.040$	$0.7020\pm 0.0137$	$22.449\pm 0.014$	$0.2750\pm 0.0003$	$\mathbf{0.2660\pm 0.0005}$	$67.201\pm 0.504$	$\mathbf{343.793\pm 0.861}$

Appendix

Table 6: Multi-seed comparison of our method and CFG on SDXL using 1K prompts per seed. Results are reported as mean

\pm

standard deviation across 10 random seeds.

Method	HPSv3 $\uparrow$	ImageReward $\uparrow$	PickScore $\uparrow$	HPSv2 $\uparrow$	CLIP-T $\uparrow$	FID $\downarrow$	sFID $\downarrow$
Ours	8.437	0.738	22.499	0.280	0.265	66.818	345.093
Output-only correction	8.218	0.710	22.482	0.276	0.267	67.557	345.246
CFG	8.104	0.708	22.473	0.275	0.267	67.430	345.259

Appendix

Table 7: Comparison with output-only spectral correction on SDXL using 1K randomly sampled prompts.

Style	Method	HPSv3 $\uparrow$	ImageReward $\uparrow$	PickScore $\uparrow$	HPSv2 $\uparrow$	CLIP-T $\uparrow$
Anime illustration	Ours	8.065	1.045	23.357	0.317	0.337
Anime illustration	CFG	7.995	1.011	23.331	0.314	0.336
Oil painting	Ours	8.646	1.016	22.822	0.300	0.320
Oil painting	CFG	8.473	1.011	22.775	0.294	0.320

Appendix

Table 8: Comparison of our method and CFG on SDXL for stylized generation using 1K prompts per style.

Setting	Method	ImageReward $\uparrow$	PickScore $\uparrow$	HPSv2 $\uparrow$	CLIP-T $\uparrow$	FID $\downarrow$	sFID $\downarrow$
$w=2.5$	Ours	0.391	21.86	0.252	0.257	18.28	52.82
	CFG	0.362	21.83	0.249	0.257	18.76	52.26
	CFG++	0.329	21.77	0.247	0.257	19.30	52.75
	TV-CFG	0.104	21.43	0.235	0.253	24.85	56.01
	LF-CFG	0.022	21.23	0.232	0.248	26.42	60.18

Appendix

Table 9: Quantitative comparison under reduced guidance on SDXL. At half of the standard guidance scale (

w=2.5

), our method improves most metrics, while existing guidance correction methods provide limited benefit.

Panel of Figure 10 — Figure 10: Qualitative comparison under reduced guidance on SDXL. At half of the standard guidance scale ( $w=2.5$ ), our method better preserves prompt-relevant content and visual coherence, while existing guidance correction methods provide limited qualitative improvement.

Panel of Figure 11 — Figure 11: Qualitative comparison on SiT for class-conditional generation. Compared to CFG, our method yields clearer structures aligned with the target class and better preserves class-relevant visual attributes.

Panel of Figure 12 — Figure 12: Qualitative comparison on JiT for class-conditional generation. Our method improves image sharpness and fine visual details, producing more recognizable samples than CFG.

Figure 13: Power spectrum of $z_{0}$ from baseline diffusion models with fitted curves. Although the spectral scale varies across models, the spectra consistently follow a power-law statistical structure. The fit based on Eq. 7 achieves $R^{2}>0.99$ across models.

Panel of Figure 14 — Figure 14: Ablation on the number of radial bins for power spectrum estimation on SDXL. We use a relatively strong correction strength of $\lambda=0.5$ to better highlight the effect of binning.

Panel of Figure 15 — Figure 15: Effect of the relaxation factor $\rho$ on PickScore, FID, and sFID. The x-axis is the relaxation factor and the y-axis is the metric value. Overall, $\rho=0.5$ yields the most balanced performance across the three metrics.

Panel of Figure 16 — Figure 16: Additional qualitative results on SDXL.

Panel of Figure 17 — Figure 17: Additional qualitative results on SDXL.

Panel of Figure 18 — Figure 18: Additional qualitative results on PixArt- $\alpha$ .

Panel of Figure 19 — Figure 19: Additional qualitative results on PixArt- $\alpha$ .

Panel of Figure 20 — Figure 20: Additional qualitative results on SD3.5.

Panel of Figure 21 — Figure 21: Additional qualitative results on SD3.5.

Panel of Figure 22 — Figure 22: Additional qualitative results under high guidance on SDXL.

Panel of Figure 23 — Figure 23: Additional qualitative results under high guidance on SDXL.

Panel of Figure 24 — Figure 24: Additional qualitative results under high guidance on PixArt- $\alpha$ .

Panel of Figure 25 — Figure 25: Additional qualitative results under high guidance on PixArt- $\alpha$ .

Panel of Figure 26 — Figure 26: Additional qualitative results under high guidance on SD3.5.

Panel of Figure 27 — Figure 27: Additional qualitative results under high guidance on SD3.5.

Explore similar work

Spectral Guidance for Flexible and Efficient Control of Diffusion Models
May 27, 2026Gabriel Moreira, Manuel Marques, João Paulo Costeira +1Diffusion Model Guidance Diffusion Guidance
Institute for Systems and Robotics, Instituto Superior Técnico, Lisbon, Portugal · Carnegie Mellon University, Pittsburgh, PA, USA
Correction Training for Diffusion Models
Oct 9, 2026Donghoon Ahn, Runqian Wang, Alexei A. Efros +2Diffusion-Based Image Generation Diffusion Models
University of California, Berkeley · Reve
ContrastiveCFG: Guiding Diffusion Sampling by Contrasting Positive and Negative Concepts
Nov 26, 2024Jinho Chang, Changsun Lee, Hyungjin Chung +1Diffusion Model Guidance Classifier-Free Guidance
Graduate School of Artificial Intelligence, Korea Advanced Institute of Science and Technology, Seoul, South Korea · EverEx, Seoul, South Korea

May 27, 2026J/K move · Enter open · S save

Spectral Guidance for Flexible and Efficient Control of Diffusion Models

Gabriel Moreira, Manuel Marques, João Paulo Costeira, Chenyan Xiong

Institute for Systems and Robotics, Instituto Superior Técnico, Lisbon, Portugal · Carnegie Mellon University, Pittsburgh, PA, USA

Diffusion Model Guidance Diffusion Guidance Conditional Generative Modeling Diffusion Models

Open paper AskPDF arXiv

We introduce Spectral Guidance, a framework for controlling diffusion models by leveraging the intrinsic geometry of the generative process. As data is progressively corrupted by noise, only a small number of features remain informative for control. We characterize them as the singular functions of a conditional expectation operator and show that they can be learned via a self-supervised objective. Once recovered, this basis enables the projection of arbitrary guidance signals, such as labels, CLIP embeddings, or masks, directly onto the sampling trajectory. This approach allows for stable, high-fidelity control without retraining or denoiser backpropagation during sampling. Empirically, we improve conditional accuracy on CIFAR-10 by 37 percentage points over the strongest training-free baseline while offering

4\times

faster sampling. Moreover, the same representations that support label and CLIP guidance also enable spatial control, such as mask-based guidance, without auxiliary models. Finally, our framework reveals a phase transition in the generative process, pinpointing the optimal time window for effective guidance.

Abstract

Explore similar work

Spectral Guidance for Flexible and Efficient Control of Diffusion Models

Correction Training for Diffusion Models

ContrastiveCFG: Guiding Diffusion Sampling by Contrasting Positive and Negative Concepts