Manifold-Optimal Guidance: A Unified Riemannian Control View of Diffusion Guidance

Zexi Jia; Pengcheng Luo; Zhengyao Fang; Jinchao Zhang; Jie Zhou

cs.CVMar 12, 2026

Manifold-Optimal Guidance: A Unified Riemannian Control View of Diffusion Guidance

Authors: Zexi Jia, Pengcheng Luo, Zhengyao Fang, Jinchao Zhang, Jie Zhou

Organizations: WeChat AI, Tencent Inc., China.

Abstract

Classifier-Free Guidance (CFG) serves as the de facto control mechanism for conditional diffusion, yet high guidance scales notoriously induce oversaturation, texture artifacts, and structural collapse. We attribute this failure to a geometric mismatch: standard CFG performs Euclidean extrapolation in ambient space, inadvertently driving sampling trajectories off the high-density data manifold. To resolve this, we present Manifold-Optimal Guidance (MOG), a framework that reformulates guidance as a local optimal control problem. MOG yields a closed-form, geometry-aware Riemannian update that corrects off-manifold drift without requiring retraining. Leveraging this perspective, we further introduce Auto-MOG, a dynamic energy-balancing schedule that adaptively calibrates guidance strength, effectively eliminating the need for manual hyperparameter tuning. Extensive validation demonstrates that MOG yields superior fidelity and alignment compared to baselines, with virtually no added computational overhead.

Figures & tables

Figure 1: Toy spiral manifold illustration of guidance geometry. Left: trajectories on a high-density spiral tube. Standard CFG (red) takes a Euclidean shortcut that leaves the manifold. A projected update (orange) stays closer to the manifold but progresses more slowly. MOG (blue) follows a geometry-aware direction that remains near the manifold while moving efficiently toward the condition. Top right: distance to the manifold over steps, highlighting off-manifold deviation under CFG. Bottom right: conditional energy over steps, showing that MOG reduces energy rapidly without leaving the high-density region.

Figure 2: Qualitative comparison across guidance methods. All images are generated with Stable Diffusion XL at guidance scale 15. Standard CFG manifests oversaturation and harsh textures. CFG++ yields washed-out results with low contrast. APG introduces a hazy appearance. In contrast, Auto-MOG achieves the most realistic results.

Model	Guidance	FID $\downarrow$	Pre. $\uparrow$	Rec. $\uparrow$	Satu. $\downarrow$	Contrast $\downarrow$	HPSv2 $\uparrow$	CLIP $\uparrow$
DiT-XL/2 (ImageNet-256, $w=4$ )	CFG	19.14	0.92	0.35	0.37	0.25	28.10	31.00
	CFG++	12.45	0.90	0.45	0.34	0.23	28.05	31.02
	APG	9.34	0.89	0.56	0.30	0.20	28.00	30.98
	LF-CFG	10.12	0.91	0.52	0.32	0.21	28.08	31.01
	MOG-Score	9.05	0.92	0.57	0.29	0.19	28.20	31.10
	Auto-MOG	8.78	0.91	0.58	0.31	0.20	28.25	31.12

Table 1: Quantitative comparison across diverse benchmarks. DiT-XL/2 and EDM2-XXL are evaluated on ImageNet 256

\times

256 ; all text-to-image models are evaluated on COCO 512

\times

512 .

Figure 3: User preference study comparing Auto-MOG against standard CFG. Top row: model-based evaluation. Bottom row: domain-based evaluation. Auto-MOG achieves consistent preference advantages, particularly in Color and Realism.

Method	FID $\downarrow$	HPSv2 $\uparrow$	CLIP $\uparrow$	AES $\uparrow$	Latency
CFG ( $\mathbf{M}{=}\mathbf{I}$ )	26.29	28.42	33.62	6.65	1.00 $\times$
MOG-Score	25.75	28.88	34.35	6.70	1.01 $\times$
MOG-Feat	25.62	29.16	34.20	6.72	1.06 $\times$
MOG-Score+Feat	25.45	29.24	34.43	6.73	1.08 $\times$
Auto-MOG	25.10	29.36	34.67	6.75	1.08 $\times$

Table 2: Ablation of Riemannian metric estimators on SD-XL.

$\rho$	FID $\downarrow$	CLIP $\uparrow$	HPSv2 $\uparrow$	Satu. $\downarrow$
1	26.20	33.70	28.45	0.27
5	25.85	34.30	28.82	0.20
10 (Def.)	25.75	34.35	28.88	0.19
20	25.80	34.15	28.80	0.19
50	26.50	32.10	27.95	0.18

Table 3: Sensitivity to anisotropy ratio

\rho

(MOG-Score).

Method	FID $\downarrow$	HPSv2 $\uparrow$	CLIP $\uparrow$
CFG ( $w{=}3$ )	27.10	27.90	32.50
CFG ( $w{=}7.5$ )	25.90	28.70	34.10
CFG ( $w{=}15$ )	26.29	28.42	33.62
Auto-MOG	25.10	29.36	34.67

Table 4: Auto-MOG vs. Tuned CFG on SD-XL.

Figure 4: Ablation on guidance scale and sampling steps.

Figure 5: Ablation on the Auto-MOG balance factor

\gamma

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Resolution	Sample Count	Notes
ImageNet (Class-Conditional)	$256{\times}256$	50,000	Val labels; ADM reference stats
MS-COCO (FID & CLIP)	$512{\times}512$	30,000	Random captions from COCO 2017 val
HPSv2	$512{\times}512$	3,200	Standard HPSv2 prompt set

Appendix

Table 5: Evaluation protocols and sample counts used in our experiments.

Category	Parameter	Value
Metric (MOG-Score)	$\lambda_{\top}$	$1.0$
Metric (MOG-Score)	$\lambda_{\perp}$	$10.0$
Metric (MOG-Score)	$\rho=\lambda_{\perp}/\lambda_{\top}$	$10$
Auto-MOG	Balance factor $\gamma$	$1.0$
Stability	$\epsilon$ in $n_{t}$ normalization	$10^{-5}$
Stability	$\epsilon$ in Auto-MOG denominator	$10^{-6}$

Appendix

Table 6: Key hyperparameters (consistent with the main paper).

Model	Guidance Scale / Setting
DiT-XL/2 (ImageNet-256)	$w=4$
EDM2-XXL (ImageNet-256)	$w=2$
SD-2.1 (COCO-512)	$w=10$
SD-XL (COCO-512)	$w=15$
SD-3.5 (COCO-512)	$w=4.5$
FLUX.1 (COCO-512)	$w=4.0$

Appendix

Table 7: Guidance scales used in the main-paper comparisons for each model.

Metric	Auto-MOG Wins	Total Trials	Win Rate	95% C.I.	Fleiss’ $\kappa$	p-value
Overall Quality	1,835	2,960	62.0%	[60.2%, 63.8%]	0.42	$<0.001$
Color Fidelity	1,823	2,960	61.6%	[59.8%, 63.4%]	0.45	$<0.001$
Realism	1,794	2,960	60.6%	[58.8%, 62.4%]	0.48	$<0.001$
Texture Detail	1,732	2,960	58.5%	[56.7%, 60.3%]	0.38	$<0.001$

Appendix

Table 8: Aggregated User Study Results (Auto-MOG vs. Standard CFG). Data pools all 2,960 trials across 4 models. Win rates represent the mean preference for Auto-MOG. High statistical significance is observed for all metrics.

Comparison	Model Architecture	AutoMOG Win	Tie / Similar	Baseline Win
vs. APG	SDXL (Latent)	61.0%	11.0%	28.0%
	SD3.5 (Transformer)	53.0%	18.0%	29.0%
	DiT-XL/2 (Pixel)	56.0%	14.0%	30.0%
	Weighted Average	56.7%	14.3%	29.0%
vs. CFG++	SDXL (Latent)	64.0%	9.0%	27.0%
	SD3.5 (Transformer)	56.0%	13.0%	31.0%

Appendix

Table 9: Detailed User Preference Rates by Architecture. We report the win rates of AutoMOG against SOTA baselines across three distinct diffusion architectures. Note the higher variance on SDXL, indicating that AutoMOG’s geometric correction is particularly effective for high-dimensional latent diffusion models.

Method	Setting / Parameter	FID $\downarrow$	HPSv2 $\uparrow$	CLIP $\uparrow$
Standard CFG	Scale $w=1$	29.10	26.50	30.80
Standard CFG	Scale $w=5$	26.05	28.55	33.95
Standard CFG	Scale $w=10$	26.15	28.50	33.85
Standard CFG	Scale $w=15$	26.29	28.42	33.62
CFG (Linear)	Linear Decay ( $15\to 1$ )	25.80	28.65	33.90
AutoMOG (Ours)	Adaptive Energy Balance	25.10	29.36	34.67

Appendix

Table 10: Ablation on Guidance Strength and Scheduling. Comparison performed on SD-XL. Standard CFG shows a clear trade-off: weak guidance (

w=1

) yields poor alignment, while strong guidance (

w=15

) destroys fidelity. Linear Decay improves upon static scales but still falls short of AutoMOG , which achieves the best Pareto frontier across all metrics.

Ratio $\rho$	Interpretation	FID $\downarrow$	HPSv2 $\uparrow$	CLIP $\uparrow$	Saturation $\downarrow$
$\rho=1$	Isotropic (Standard CFG)	26.20	28.45	33.70	0.27
$\rho=5$	Mild Correction	25.85	28.82	34.30	0.20
$\rho=10$	Default (Ours)	25.75	28.88	34.35	0.19
$\rho=20$	Strong Correction	25.80	28.80	34.15	0.19
$\rho=50$	Over-Constrained	26.50	27.95	32.10	0.18

Appendix

Table 11: Sensitivity Analysis of Anisotropy Ratio

\rho

. Evaluated on SD-XL (

w=15

\rho\approx 10

yields the best trade-off.

Figure 6: Correction of Saturation Artifacts on SD 3.5 ( $w=4.5$ ). At the officially recommended guidance scale, standard CFG (odd columns) frequently produces oversharpened edges, saturated color distributions, and unnatural contrast enhancement. Auto MOG (even columns, $\beta=1$ ) effectively corrects these off manifold deviations, restoring natural skin tones, balanced highlight rolloff, and realistic lighting gradients throughout the scene.

Figure 7: Detail Enhancement on Flux.1 ( $w=3.0$ ). Even at the lower guidance scales appropriate for Rectified Flow architectures, standard CFG (odd columns) can induce texture degradation and geometric oversimplification. Auto MOG (even columns, $\beta=1$ ) substantially improves material fidelity, surface detail preservation, and structural coherence, demonstrating consistent effectiveness across fundamentally different generative paradigms.

Explore similar work

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control
Oct 5, 2026Max Collins, Dasith de Silva Edirimuni, Jordan Vice +2Diffusion Model Guidance Classifier-Free Guidance
School of Physics, Maths and Computing The University of Western Australia Perth, WA 6009
C $^2$ FG: Control Classifier-Free Guidance via Score Discrepancy Analysis
Mar 9, 2026Jiayang Gao, Tianyi Zheng, Jiayang Zou +9Classifier-Free Guidance Inference-Time Guidance
Shanghai Jiao Tong University, Shanghai, China · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China
Guidance Breaks the Fitted Operator: A Terminal-Fitted Repair for Classifier-Free Guidance
Jul 8, 2026Shiheng ZhangClassifier-Free Guidance Diffusion Sampling
University of Washington

Oct 5, 2026J/K move · Enter open · S save

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control

Max Collins, Dasith de Silva Edirimuni, Jordan Vice, Tim French+1

School of Physics, Maths and Computing The University of Western Australia Perth, WA 6009

Diffusion Model Guidance Classifier-Free Guidance Stochastic Optimal Control Diffusion Sampling

Open paper AskPDF arXiv

Diffusion models generate samples by learning to reverse a fixed corruption process, and classifier-free guidance (CFG) is the standard mechanism for conditioning this process on a desired class or prompt. CFG can be applied at varying guidance strengths, and while higher strengths improve image quality and conditional alignment, too high a guidance strength can degrade image quality and diversity. Furthermore, CFG violates principled diffusion sampling dynamics, and existing explanations for why it works despite the violation disagree on the underlying theory or do not extend to deterministic samplers used in practice. We address both these issues. We first frame CFG sampling as a continuous-time optimal control problem, treating the sampling trajectory as a sequence of controls chosen to maximise the probability of the desired condition. Solving the resulting Hamilton--Jacobi--Bellman equation shows that CFG is recovered under specific path costs when using an unconstrained control set. We argue this lack of constraint is responsible for CFG's failure at high guidance strengths, since it permits the sampling path to move arbitrarily far from the current image estimate. To fix this, we propose curvature-aware CFG (CACFG), which constrains the control set to a hypersphere informed by the Gaussian regularisation used when training variational autoencoders. We show that the control inputs produced by CFG sampling routinely violate this bound, and that across diffusion models, datasets, and guidance schedules, CACFG achieves superior generative quality at mid-to-high guidance strengths with a less severe quality-diversity tradeoff than regular CFG.

Abstract

Explore similar work

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control

C2^22FG: Control Classifier-Free Guidance via Score Discrepancy Analysis

Guidance Breaks the Fitted Operator: A Terminal-Fitted Repair for Classifier-Free Guidance

C $^2$ FG: Control Classifier-Free Guidance via Score Discrepancy Analysis