Attention Control in Diffusion Models

Overview

The field studies how self-, cross-, and joint-attention route information between text, image or video regions, references, and denoising steps. Common approaches modify attention maps, logits, or values at inference; use masks and spatial routing to bind tokens to regions; share reference keys and values; or adapt attention and guidance across denoising steps, sometimes with training-time supervision. These controls target tasks such as multi-object composition, localized editing, style transfer, subject preservation, and video consistency, with evaluations using benchmarks such as T2I-CompBench, PIE-Bench, CountBench, and VBench. A central tension is that attention interventions can improve alignment or local control but may have entangled, distributed effects, making preservation and quality gains inconsistent across models and settings.

Latest papers

CardsList

75 papers6 papers in the last four weeks, down 45% on the four weeks before. 0.0% of all new papers.

  1. Memorization Lives in a Few Attention Heads: Mitigating Memorization in Diffusion Models via Projection-Based Suppression

    Nov 14, 2026Puwei Lian, Yujun Cai, Songze Li +2Memorization in Generative ModelsDiffusion Models

  2. Disentangled Stylized Image Generation via IP-Cross-Attention Rearrangement

    Nov 7, 2026Beibei Dong, Yueming Lyu, Yue Jiang +1Image GenerationT2I Diffusion Models

  3. Disen-Forcing: Disentangling Semantic Anchoring from Motion for Autoregressive Video Diffusion

    Nov 2, 2026Zhengyang Yu, Akio Hayakawa, Masato Ishii +5Video Diffusion ModelsAutoregressive Video Generation

  4. Copying As Defense: Protecting Images from Editing By Eliciting Model Copying

    Oct 26, 2026Salma Abdel Magid, Sanghyuk Chun, Eric Slyman +2Adversarial PerturbationsDiffusion-Based Image Editing

  5. Push-Pull Attentional Anchoring for Diffusion Concept Erasure

    Oct 20, 2026Nattanat Chatthee, Tagon Sompong, Ekapol Chuangsuwanich +1Concept ErasureDiffusion Model Unlearning

  6. QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing

    Oct 13, 2026Tiancheng Shen, Zilong Huang, Xiangtai Li +6Diffusion-Based Image EditingAttention Control in Diffusion Models

  7. TRACE: Time-Adaptive Residual Attention Control with Content-Style Decomposition for Training-Free Diffusion Style Transfer

    Oct 4, 2026Duc Khoan Le, Kim Ngoc Tran, Minh Nhat Le +5Image Style TransferDiffusion Model Guidance

  8. Where to Look Is Not How to Fix: Pre-Denoising Diagnostics and Modality-Dependent Control in Diffusion Composition

    Oct 2, 2026Fangzheng Wu, Brian SummaCompositional T2I GenerationT2I Diffusion Models

  9. Rethinking Memorization Mitigation in Diffusion Models: Reinforcing Text Conditioning

    Oct 1, 2026Hyungjun Joo, Sehwan Kim, Hyeonggeun Han +2Diffusion Model AlignmentAttention Control in Diffusion Models

  10. Motion Concept Unlearning in Video Diffusion Models

    Sep 29, 2026Ping Liu, Chi ZhangVideo Diffusion ModelsDiffusion Transformer

  11. Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing

    Sep 28, 2026Yanan Wang, Shengcai Liao, Guangyi Liu +1Video Diffusion ModelsAttention Control in Diffusion Models

  12. Attention-Scoped Guidance: Training-Free Spatial Control for Image Editing

    Sep 28, 2026Zeyan Li, Wei Zhou, Hadi Amirpour +3Classifier-Free GuidanceText-Guided Image Editing

  13. TSGate: Timestep-Aware Gated Attention for Diffusion Transformers

    Sep 28, 2026Boyu Zhang, Yifan Liu, Shuxia Lin +3Diffusion TransformerGated Attention

  14. DiffSynth-Music: Audio-Conditioned KV-Cache Adapters for Controllable Music Generation

    Sep 11, 2026Zhongjie Duan, Shengchuan Gao, Hong Zhang +1Controllable Music GenerationAudio Diffusion Models

  15. The Attention Triangle in Audio-Video Models

    Sep 3, 2026Sagi Polaczek, Noa Kraicer, Gal Metzer +4Audio-Video GenerationAttention Control in Diffusion Models

  16. Denoising Diffusion Generative Models Secretly Calculate Attentions

    Sep 1, 2026Farzan Haddadi, Leila Monfared, Ebrahim Rezaii +3Attention Control in Diffusion ModelsDiffusion-Based Image Generation

  17. Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models

    Aug 17, 2026Artem Sergievskii, Artyom Turevich, Sergey KastryulinDiffusion Model GuidanceClassifier-Free Guidance

  18. CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets

    Aug 14, 2026Jihun Park, Kyoungmin Lee, Jongmin Gim +6Subject-Driven Image GenerationIdentity-Preserving Image Generation

  19. SafeCA: Safe Cross-Attention Localization and Regulation for Text-to-Video Jailbreak Defense

    Aug 11, 2026Siyuan Liang, Yupeng Qiu, Junfeng Fang +3Video Diffusion ModelsAttention Control in Diffusion Models

  20. SC-Diff: Semantically Calibrated Diffusion for Visible-to-Infrared Image Translation

    Aug 9, 2026Junyin Zhang, Siyu Huang, Jianxiong Ye +4Image-to-Image TranslationLatent Diffusion Models

  21. ControlRef: Efficient Layout-Guided Multi-Instance Generation via Anchored 4D-RoPE

    Aug 7, 2026Yunkai Yang, Yudong Zhang, Xinying Chen +7Rotary Positional EmbeddingsDiffusion Transformer

  22. Rectify Then Diffuse: Disentangling Concepts Before Denoising Trajectory Unfolds

    Aug 4, 2026Ning Zhu, An Chen, Mengfei Zhao +4T2I GenerationAttention Control in Diffusion Models

  23. FreeShadow: Training-Free Shadow Removal via Illumination Transfer and Selective Content Preservation in Diffusion Models

    Jul 29, 2026Yinan Wang, Yan Huang, Yong Xu +1Diffusion Models for Image RestorationShadow Removal

  24. PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention

    Jul 28, 2026Zhengtao Yao, Runhao Li, Xupeng Chen +12Masked Diffusion ModelsAttention Control in Diffusion Models