CFG-Ctrl: Control-Based Classifier-Free Diffusion Guidance

Hanyang Wang; Yiyang Liu; Jiawei Chi; Fangfu Liu; Ran Xue; Yueqi Duan

cs.CVMar 3, 2026

CFG-Ctrl: Control-Based Classifier-Free Diffusion Guidance

Authors: Hanyang Wang, Yiyang Liu, Jiawei Chi, Fangfu Liu, Ran Xue, Yueqi Duan

Organizations: Tsinghua University

Abstract

Classifier-Free Guidance (CFG) has emerged as a central approach for enhancing semantic alignment in flow-based diffusion models. In this paper, we explore a unified framework called CFG-Ctrl, which reinterprets CFG as a control applied to the first-order continuous-time generative flow, using the conditional-unconditional discrepancy as an error signal to adjust the velocity field. From this perspective, we summarize vanilla CFG as a proportional controller (P-control) with fixed gain, and typical follow-up variants develop extended control-law designs derived from it. However, existing methods mainly rely on linear control, inherently leading to instability, overshooting, and degraded semantic fidelity especially on large guidance scales. To address this, we introduce Sliding Mode Control CFG (SMC-CFG), which enforces the generative flow toward a rapidly convergent sliding manifold. Specifically, we define an exponential sliding mode surface over the semantic prediction error and introduce a switching control term to establish nonlinear feedback-guided correction. Moreover, we provide a Lyapunov stability analysis to theoretically support finite-time convergence. Experiments across text-to-image generation models including Stable Diffusion 3.5, Flux, and Qwen-Image demonstrate that SMC-CFG outperforms standard CFG in semantic alignment and enhances robustness across a wide range of guidance scales. Project Page: https://hanyang-21.github.io/CFG-Ctrl

Figures & tables

Figure 1: Phase diagram in the $\mathbf{e}$ - $\dot{\mathbf{e}}$ plane. We schematically illustrate the convergence patterns of CFG and the proposed SMC-CFG. Left : CFG’s ideal linear convergence trajectory and the strong oscillatory divergence under high guidance scales. Right : the proposed SMC-CFG, through a switching-forcing mechanism, drives the system states toward the sliding mode surface governed by parameter $\lambda$ , achieving robust and rapid convergence.

Method	Gain $K_{t}$	Operator $\Pi_{t}$	Error $\mathbf{e}(t)$	Control Interpretation
CFG [ 16 ]	$w$	$I$	$\Delta\mathbf{v}_{\theta}(t)$	Proportional control
Weight Scheduler [ 49 ]	$w(t)$	$I$	$\Delta\mathbf{v}_{\theta}(t)$	Time-varying gain scheduling
APG [ 39 ]	$w\begin{bmatrix}I\;\;\eta I\end{bmatrix}$	$\begin{bmatrix}I-P_{t}\\ P_{t}\end{bmatrix},P_{t}=\frac{\mathbf{v}_{\theta}(\mathbf{c})\mathbf{v}_{\theta}(\mathbf{c})^{\top}}{\|\mathbf{v}_{\theta}(\mathbf{c})\|^{2}}$	$\Delta\mathbf{v}_{\theta}(t)$	Projection-based Feedback Control
CFG-Zero ⋆ [ 9 ]	$\begin{bmatrix}wI\;\;\frac{s_{t}}{1-s_{t}}I\end{bmatrix}$ , $s_{t}=\frac{\mathbf{v}_{\theta}(\mathbf{c})^{\top}\mathbf{v}_{\theta}(\mathbf{\varnothing})}{\|\mathbf{v}_{\theta}(\mathbf{\varnothing})\|^{2}}$	$\begin{bmatrix}I-P_{t}\\ P_{t}\end{bmatrix},P_{t}=\frac{\mathbf{v}_{\theta}(\mathbf{\varnothing})\mathbf{v}_{\theta}(\mathbf{\varnothing})^{\top}}{\|\mathbf{v}_{\theta}(\mathbf{\varnothing})\|^{2}}$	$\Delta\mathbf{v}_{\theta}(t)$	Projection-based Feedback Control
Rectified-CFG++ [ 41 ]	$\begin{bmatrix}I\;\;\alpha(t)I\end{bmatrix},\alpha(t)=\lambda_{max}(1-t)^{\gamma}$	$I$	$\begin{bmatrix}\Delta\mathbf{v}_{\theta}(t)\\ \Delta\mathbf{v}_{\theta}(t-\frac{\Delta t}{2})\end{bmatrix}$	Model Predictive Control
SMC-CFG	$w$	$I$	$\Delta\mathbf{v}_{\theta}(t)-k\cdot\text{sign}(\mathbf{s}_{t})$	Sliding Mode Control

Table 1: Typical CFG variants under CFG-Ctrl formulation. We summarize the key components of various methods under the control formulation, along with their corresponding types of control interpretations.

Guidance	FID $\downarrow$	CLIP $\uparrow$	Aesthetic $\uparrow$	ImageReward $\uparrow$	PickScore $\uparrow$	HPSv2 $\uparrow$	HPSv2.1 $\uparrow$	MPS $\uparrow$
SD3.5 [ 8 ]	41.725	0.3399	5.4256	0.3591	0.2124	0.2710	0.2372	6.5554
w/ CFG [ 16 ]	21.421	0.3681	5.5875	0.8889	0.2190	0.2930	0.2842	7.2476
w/ CFG-Zero ⋆ [ 9 ]	20.317	0.3691	5.6124	0.9312	0.2195	0.2942	0.2862	7.0430
w/ Rect-CFG++ [ 41 ]	20.550	0.3655	5.5663	0.7097	0.2173	0.2887	0.2748	6.7854
w/ SMC-CFG	20.044	0.3694	5.5790	0.9486	0.2211	0.2945	0.2875	7.5719
Flux-dev [ 22 ]	52.598	0.3272	5.4568	0.2572	0.2137	0.2650	0.2280	5.9592

Table 2: Quantitative evaluation of CFG methods. Lower (

\downarrow

) FID and higher (

\uparrow

) CLIP, Aesthetic, ImageReward, PickScore, HPSv2, HPSv2.1 and MPS scores indicate better performance. Note that Qwen-Image preserves natural image statistics, yielding the lowest FID.

Figure 2: Qualitative results across different T2I models. We provide visual comparisons between CFG and our SMC-CFG across various models. SMC-CFG exhibits better performance in positional relationships, text generation, and detailed object representation.

Figure 3: Qualitative comparison with baseline methods. For challenging scenarios including relative positions, clothing styles, and human actions, baseline methods produce irrational outputs, while SMC-CFG preserves robust text consistency.

$\lambda$	$k$	FID $\downarrow$	CLIP $\uparrow$	Aesth $\uparrow$	ImgRwd $\uparrow$
3	0.1	26.193	0.3698	5.7064	1.0174
4	0.1	26.006	0.3701	5.7098	1.0219
5	0.1	25.951	0.3709	5.7128	1.0248
6	0.1	26.143	0.3703	5.7071	1.0228
5	0.1	25.951	0.3709	5.7128	1.0248
5	0.4	26.143	0.3719	5.7218	1.0504

Table 3: Ablation study on hyperparameter

\lambda

and

k

. We conduct ablation across various hyperparameter settings in four metrics: FID, CLIP, Aesthetic (Aesth), and ImageReward (ImgRwd), respectively measuring generation quality, semantic alignment, aesthetic level, and human preference.

Figure 4: Visual comparison between CFG (top) and SMC-CFG (bottom) across different CFG scales.

Notation	Meaning	Value
$\mathbf{x}_{t}$	Latent state at time $t$ during generative flow sampling.	$\mathbf{x}_{0}\sim\mathcal{N}(0,\mathbf{I})$ (initial state)
$\mathbf{v}_{\theta}(\mathbf{x}_{t},t,\varnothing)$	Unconditional velocity field, obtained by dropping the condition $\mathbf{c}$ .	/
$\mathbf{v}_{\theta}(\mathbf{x}_{t},t,\mathbf{c})$	Conditional velocity field, incorporating the input condition $\mathbf{c}$ .	/
$\hat{\mathbf{v}}_{\theta}(\mathbf{x}_{t},t,\mathbf{c})$	Guided velocity field, combined via guidance.	/
$w$	CFG guidance scale.	$w\geq 1$
$\mathbf{e}(t)$	Semantic error signal.	$\mathbf{v}_{\theta}(\mathbf{x}_{t},t,\mathbf{c})-\mathbf{v}_{\theta}(\mathbf{x}_{t},t,\varnothing)$

Table 4: Notation table.

Model	Color $\uparrow$	Shape $\uparrow$	Texture $\uparrow$	Spatial $\uparrow$
SD3.5 [ 8 ]	0.6790	0.5915	0.7243	0.1625
w/ SMC-CFG	0.7461	0.6009	0.7406	0.2563
Flux-dev [ 22 ]	0.8172	0.5751	0.7432	0.2708
w/ SMC-CFG	0.8216	0.6199	0.7901	0.2939
Qwen-Image [ 50 ]	0.7747	0.5621	0.6747	0.2968
w/ SMC-CFG	0.8191	0.5934	0.7421	0.4085

Table 5: Quantitative evaluation on T2I-CompBench.

Method	VQAScore (GenAI-Bench)
Method	Base $\uparrow$	Advance $\uparrow$	Overall $\uparrow$
Base (w/o CFG)	0.79	0.64	0.70
w/ CFG	0.83	0.64	0.72
w/ CFG-Zero ⋆	0.88	0.66	0.75
w/ Rect-CFG++	0.87	0.64	0.73
w/ SMC-CFG	0.89	0.68	0.77

Table 6: Compositional alignment evaluation on SD3.5.

Method	Total Score	Quality Score	Semantic Score	Color	Human Action	Subject Consistency
CFG	0.5594	0.6581	0.4607	0.9087	0.5313	0.9450
SMC-CFG	0.5839	0.6747	0.4931	0.9818	0.6000	0.9609

Table 7: Video comparison on Wan2.2-TI2V-5B.

Resolution	Guidance	Memory (GB)	FLOPs (G)	Runtime (s)
512×512	CFG	31.99	1203370.06	23.84
512×512	SMC-CFG	31.99	1203370.07	23.97
1024×1024	CFG	33.59	3590870.89	44.78
1024×1024	SMC-CFG	33.59	3590870.93	45.09

Table 8: Computational cost and inference time comparison of standard CFG and SMC-CFG.

Figure 5: Qualitative results under various hyperparameters.

Figure 6: Performance curves of different methods under varying CFG scales.

Figure 7: Addtional visual comparison between CFG (left) and SMC-CFG (right) in SD3.5.

Figure 8: Addtional visual comparison between CFG (left) and SMC-CFG (right) in Flux-dev.

Figure 9: Addtional visual comparison between CFG (left) and SMC-CFG (right) in Qwen-Image.

Figure 10: Additional video comparisons between CFG (above) and SMC-CFG (below) in Wan2.2-TI2V-5B.

Explore similar work

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control
Oct 5, 2026Max Collins, Dasith de Silva Edirimuni, Jordan Vice +2Diffusion Model Guidance Classifier-Free Guidance
School of Physics, Maths and Computing The University of Western Australia Perth, WA 6009
Classifier-Free Guidance in Flow Matching: Non-Autonomous Potentials, Overshoot, and Posterior-Mean Control
Sep 21, 2026Jishen Peng, Zheng MaFlow Matching Classifier-Free Guidance
School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China · Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University, Shanghai, China · Qing Yuan Research Institute, Shanghai Jiao Tong University, Shanghai, China +1
C $^2$ FG: Control Classifier-Free Guidance via Score Discrepancy Analysis
Mar 9, 2026Jiayang Gao, Tianyi Zheng, Jiayang Zou +9Classifier-Free Guidance Inference-Time Guidance
Shanghai Jiao Tong University, Shanghai, China · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China

Oct 5, 2026J/K move · Enter open · S save

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control

Max Collins, Dasith de Silva Edirimuni, Jordan Vice, Tim French+1

School of Physics, Maths and Computing The University of Western Australia Perth, WA 6009

Diffusion Model Guidance Classifier-Free Guidance Stochastic Optimal Control Diffusion Sampling

Open paper AskPDF arXiv

Diffusion models generate samples by learning to reverse a fixed corruption process, and classifier-free guidance (CFG) is the standard mechanism for conditioning this process on a desired class or prompt. CFG can be applied at varying guidance strengths, and while higher strengths improve image quality and conditional alignment, too high a guidance strength can degrade image quality and diversity. Furthermore, CFG violates principled diffusion sampling dynamics, and existing explanations for why it works despite the violation disagree on the underlying theory or do not extend to deterministic samplers used in practice. We address both these issues. We first frame CFG sampling as a continuous-time optimal control problem, treating the sampling trajectory as a sequence of controls chosen to maximise the probability of the desired condition. Solving the resulting Hamilton--Jacobi--Bellman equation shows that CFG is recovered under specific path costs when using an unconstrained control set. We argue this lack of constraint is responsible for CFG's failure at high guidance strengths, since it permits the sampling path to move arbitrarily far from the current image estimate. To fix this, we propose curvature-aware CFG (CACFG), which constrains the control set to a hypersphere informed by the Gaussian regularisation used when training variational autoencoders. We show that the control inputs produced by CFG sampling routinely violate this bound, and that across diffusion models, datasets, and guidance schedules, CACFG achieves superior generative quality at mid-to-high guidance strengths with a less severe quality-diversity tradeoff than regular CFG.

Abstract

Explore similar work

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control

Classifier-Free Guidance in Flow Matching: Non-Autonomous Potentials, Overshoot, and Posterior-Mean Control

C2^22FG: Control Classifier-Free Guidance via Score Discrepancy Analysis

C $^2$ FG: Control Classifier-Free Guidance via Score Discrepancy Analysis