Geometry-Aware Attention Guidance for Diffusion Models via Modern Hopfield Dynamics

Kwanyoung Kim

cs.LGMar 3, 2026

Geometry-Aware Attention Guidance for Diffusion Models via Modern Hopfield Dynamics

Authors: Kwanyoung Kim

Organizations: Department of AI Convergence, GIST

Abstract

Classifier-Free Guidance (CFG) improves sample quality in diffusion models, but its dual-pass inference and reliance on null-condition training limit its use in few-step regimes. Attention-space guidance has emerged as a complementary paradigm that addresses this gap, yet why prior sparse-vs-dense attention guidance works remains elusive. We address this by analyzing attention extrapolation through Modern Hopfield dynamics, proving two directional properties of the sparse-dense discrepancy under shared conditioning that together certify it as a directionally consistent acceleration signal. Building on this, we propose Geometry-Aware Attention Guidance (GAG), a training-free, plug-and-play extrapolation rule that decomposes the discrepancy into parallel and orthogonal components relative to the retrieval direction, amplifying the convergence-aligned component while suppressing off-manifold noise; stability follows from a weak contraction property. We further provide an interpretation of this extrapolation as first-order Anderson Acceleration in attention space, offering a unified perspective on attention extrapolation methods. GAG is a universal method that generalizes across architectures (UNet, MMDiT) and sampling regimes (multi-step, few-step), consistently improving generation quality on diverse backbones, including FLUX.1, the recent FLUX.2, and Qwen-Image, with minimal computational overhead.

Figures & tables

Figure 1: Effect of Ours (GAG) on guidance methods (CFG [ 17 ] , APG [ 46 ] ; top), step-distilled models (Hyper-SDXL [ 42 ] , DMD2 [ 63 ] ; middle), and MMDiT backbones (FLUX.2 [ 28 ] , Qwen-Image [ 3 ] ; bottom). This training-free, plug-and-play module improves text–image alignment and visual details with negligible overhead.

Figure 2: Conceptual comparison. (a) Score-space guidance [ 17 , 46 ] requires two forward passes per step; (b) attention-space guidance [ 23 , 8 ] , while single-pass, amplifies the full residual including off-direction noise; (c) GAG decomposes the residual and amplifies only the convergence-aligned $r^{\parallel}$ .

Figure 3: $\left\lVert r(\mathbf{x}_{t})\right\rVert$ along the denoising trajectory: small at high noise, growing with localization.

Figure 4: Geometric components. Orthogonal-only fails to recover structure; full residual shows interference; parallel-only ( $\zeta{=}0$ ) yields the best quality.

Method	NFE	GE $\uparrow$	FID $\downarrow$	CS $\uparrow$	IR $\uparrow$	PS $\uparrow$	HPSv2 $\uparrow$
CFG	50	0.547	20.22	26.46	0.606	22.31	0.266
+ PLADIS	50	0.597	15.81	26.99	0.801	22.31	0.268
+ GAG (Ours)	50	0.605	14.63	27.25	0.811	22.39	0.271
APG	50	0.557	18.53	26.78	0.702	22.36	0.269
+ PLADIS	50	0.608	14.51	27.36	0.808	21.92	0.261
+ GAG (Ours)	50	0.613	13.22	27.55	0.851	22.24	0.274

Table 1: Quantitative comparison on GenEval and MS-COCO with SDXL. Left: high-NFE regimes; right: low-NFE step-distilled models.

Method	NFE	GE $\uparrow$	CS $\uparrow$	IR $\uparrow$	PS $\uparrow$	HPSv2 $\uparrow$
FLUX.1-Schnell	4	0.671	26.28	1.059	22.62	0.295
+ PLADIS	4	0.713	26.45	1.066	22.30	0.295
+ GAG (Ours)	4	0.739	26.56	1.114	22.79	0.301
FLUX.1-Dev	28	0.666	25.71	1.093	23.00	30.42
+ PLADIS	28	0.691	25.80	1.115	23.03	30.54
+ GAG (Ours)	28	0.737	25.76	1.127	23.14	30.94

Table 2: (Left) Quantitative comparison on FLUX.1 (Schnell and Dev). (Right) Computational overhead (per-image wall-clock and peak memory) on SDXL+CFG and FLUX.1-Schnell.

Method	GE $\uparrow$
FLUX.2 (4B)	0.824
+ PLADIS	0.826
+ GAG	0.843
FLUX.2 (9B)	0.849
+ PLADIS	0.865
+ GAG	0.878

Table 3: FLUX.2 and Qwen-Image on GenEval.

Panel of Figure 5 — Figure 5: (a) Robustness across guidance scale $\lambda$ . (b) GAG composed with NAG degrades, as predicted by Remark 2.1 . (c) $\zeta$ sweep: parallel-only ( $\zeta{=}0$ ) is best.

Method	Text Align.	Visual Quality
Baseline	$10.7\%$ $[7.9{-}13.8]$	$16.7\%$ $[13.1{-}20.5]$
PLADIS	$12.9\%$ $[9.8{-}16.2]$	$31.4\%$ $[27.1{-}36.0]$
GAG	$76.4\%$ $[72.1{-}80.5]$	$51.9\%$ $[47.1{-}56.7]$

Table 4: Three-way user study.

Figure 6: DDIM inversion-reconstruction.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Method	GE $\uparrow$	CS $\uparrow$	IR $\uparrow$
CFG	0.547	26.46	0.606
Attn-AA ( $m{=}1$ )	0.597	26.99	0.801
Attn-AA ( $m{=}2$ , approx.)	0.556	26.83	0.753

Appendix

Table 5:

m{=}1

vs.

m{=}2

AA in attention space (SDXL). An explicit

m=2

surrogate, constructed from a multi-

\alpha

history, does not improve over the implicit

m=1

form, consistent with the one-step MHN retrieval picture.

Method	CS $\uparrow$	IR $\uparrow$	PS $\uparrow$
CFG	26.46	0.606	22.31
GAG (w/o rescaling)	26.68	0.715	22.34
GAG (w/ rescaling)	27.25	0.811	22.39

Appendix

Table 6: Effect of the rescaling factor at high guidance scale (

\lambda=10

Method	SSIM $\uparrow$	LPIPS $\downarrow$
Dense ( $\alpha{=}1.0$ )	0.7461	0.2091
Sparse ( $\alpha{=}1.5$ )	0.7545	0.1900
GAG	0.7565	0.1838

Appendix

Table 7: DDIM inversion-reconstruction on SDXL: SSIM (higher is better) and LPIPS (lower is better), averaged over

200

MS-COCO prompts.

Figure 7: Qualitative comparison of CFG [ 17 ] alone, CFG with PLADIS, and CFG with GAG.

Figure 8: Qualitative comparison of APG [ 46 ] alone, APG with PLADIS, and APG with GAG.

Figure 9: Qualitative comparison combining PAG [ 1 ] with PLADIS and with GAG.

Figure 10: Qualitative comparison on Hyper-SDXL [ 42 ] .

Figure 11: Qualitative comparison on DMD2 [ 63 ] .

Figure 12: Qualitative comparison on FLUX.1-Schnell.

Figure 13: Qualitative comparison on FLUX.1-Dev.

Figure 14: Qualitative comparison of NAG [ 8 ] , PLADIS, and GAG on Hyper-SDXL.

Figure 15: Qualitative comparison on FLUX.2 (Klein-4B) [ 28 ] : baseline vs. GAG. GAG improves text–image alignment and detail without modifying the underlying weights or adding forward passes.

Figure 16: Qualitative comparison on FLUX.2 (Klein-9B) [ 28 ] : baseline vs. GAG. The directional decomposition transfers to the larger MMDiT backbone with consistent gains in compositional fidelity and visual detail.

Figure 17: Qualitative comparison on Qwen-Image [ 3 ] : baseline vs. GAG. GAG generalizes beyond the FLUX family to a different DiT-based backbone, showing consistent improvements in prompt adherence and texture quality.

Diffusion Model Guidance

Diffusion model guidance steers denoising toward desired conditions, rewards, constraints, or away from unwanted concepts.

See all 362 papers

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control
Oct 5, 2026Max Collins, Dasith de Silva Edirimuni, Jordan Vice +2Diffusion Model Guidance Classifier-Free Guidance
School of Physics, Maths and Computing The University of Western Australia Perth, WA 6009
Delta Score Matters! Spatial Adaptive Multi Guidance in Diffusion Models
Apr 29, 2026Haosen Li, Wenshuo Chen, Lei Wang +4Diffusion Model Guidance Classifier-Free Guidance
1The Hong Kong University of Science and Technology (Guangzhou) · 2Griffith University & Data61/CSIRO
Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models
Aug 17, 2026Artem Sergievskii, Artyom Turevich, Sergey KastryulinDiffusion Model Guidance Classifier-Free Guidance
HSE University · Yandex, HSE University

Oct 5, 2026J/K move · Enter open · S save

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control

Max Collins, Dasith de Silva Edirimuni, Jordan Vice, Tim French+1

School of Physics, Maths and Computing The University of Western Australia Perth, WA 6009

Diffusion Model Guidance Classifier-Free Guidance Stochastic Optimal Control Diffusion Sampling

Open paper AskPDF arXiv

Diffusion models generate samples by learning to reverse a fixed corruption process, and classifier-free guidance (CFG) is the standard mechanism for conditioning this process on a desired class or prompt. CFG can be applied at varying guidance strengths, and while higher strengths improve image quality and conditional alignment, too high a guidance strength can degrade image quality and diversity. Furthermore, CFG violates principled diffusion sampling dynamics, and existing explanations for why it works despite the violation disagree on the underlying theory or do not extend to deterministic samplers used in practice. We address both these issues. We first frame CFG sampling as a continuous-time optimal control problem, treating the sampling trajectory as a sequence of controls chosen to maximise the probability of the desired condition. Solving the resulting Hamilton--Jacobi--Bellman equation shows that CFG is recovered under specific path costs when using an unconstrained control set. We argue this lack of constraint is responsible for CFG's failure at high guidance strengths, since it permits the sampling path to move arbitrarily far from the current image estimate. To fix this, we propose curvature-aware CFG (CACFG), which constrains the control set to a hypersphere informed by the Gaussian regularisation used when training variational autoencoders. We show that the control inputs produced by CFG sampling routinely violate this bound, and that across diffusion models, datasets, and guidance schedules, CACFG achieves superior generative quality at mid-to-high guidance strengths with a less severe quality-diversity tradeoff than regular CFG.

Abstract

Diffusion Model Guidance

Explore similar work

CACFG: Curvature-Aware Classifier-Free Guidance and Optimal Control

Delta Score Matters! Spatial Adaptive Multi Guidance in Diffusion Models

Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models