cs.LGMay 27, 2026

Spectral Guidance for Flexible and Efficient Control of Diffusion Models

Authors: Gabriel MoreiraManuel MarquesJoão Paulo CosteiraChenyan Xiong

Abstract

We introduce Spectral Guidance, a framework for controlling diffusion models by leveraging the intrinsic geometry of the generative process. As data is progressively corrupted by noise, only a small number of features remain informative for control. We characterize them as the singular functions of a conditional expectation operator and show that they can be learned via a self-supervised objective. Once recovered, this basis enables the projection of arbitrary guidance signals, such as labels, CLIP embeddings, or masks, directly onto the sampling trajectory. This approach allows for stable, high-fidelity control without retraining or denoiser backpropagation during sampling. Empirically, we improve conditional accuracy on CIFAR-10 by 37 percentage points over the strongest training-free baseline while offering 4×4\times faster sampling. Moreover, the same representations that support label and CLIP guidance also enable spatial control, such as mask-based guidance, without auxiliary models. Finally, our framework reveals a phase transition in the generative process, pinpointing the optimal time window for effective guidance.

Explore similar work

CardsList
  1. Towards More General Control of Diffusion Models Using Jeffrey Guidance

    Jun 11, 2026Raphaël Razafindralambo, Rémy Sun, Frédéric Precioso +2

  2. General and Efficient Steering of Diffusion Models

    Feb 11, 2026Qingsong Wang, Mikhail Belkin, Yusu WangActivation Steering