Attention Control in Diffusion Models

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 33

All topics

RefRoute: Decoupling Conditioning Cost from References via Compact Residual Conditioning and Spatial Routing
Oct 6, 2026Wanning He, Yuyao Zhang, Yu-Wing TaiDiffusion Transformer Image Generation
Dartmouth College
Rethinking Memorization Mitigation in Diffusion Models: Reinforcing Text Conditioning
Oct 1, 2026Hyungjun Joo, Sehwan Kim, Hyeonggeun Han +2Diffusion Model Alignment Attention Control in Diffusion Models
Seoul National University · Konkuk University
Motion Concept Unlearning in Video Diffusion Models
Sep 29, 2026Ping Liu, Chi ZhangVideo Diffusion Models Diffusion Transformer
Computer Science and Engineering University of Nevada, Reno Reno, NV, USA · Department of Mathematics National University of Singapore Singapore
Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing
Sep 28, 2026Yanan Wang, Shengcai Liao, Guangyi Liu +1Video Diffusion Models Attention Control in Diffusion Models
Mohamed bin Zayed University of Artificial Intelligence · Institute of Foundation Models · United Arab Emirates University
Attention-Scoped Guidance: Training-Free Spatial Control for Image Editing
Sep 28, 2026Zeyan Li, Wei Zhou, Hadi Amirpour +3Classifier-Free Guidance Text-Guided Image Editing
Shanghai Jiao Tong University · Cardiff University · University of Klagenfurt +1
TSGate: Timestep-Aware Gated Attention for Diffusion Transformers
Sep 28, 2026Boyu Zhang, Yifan Liu, Shuxia Lin +3Diffusion Transformer Gated Attention
Alibaba Token Hub, Alibaba Group · Southeast University
DiffSynth-Music: Audio-Conditioned KV-Cache Adapters for Controllable Music Generation
Sep 14, 2026Zhongjie Duan, Shengchuan Gao, Hong Zhang +1Controllable Music Generation Audio Diffusion Models
Alibaba Group · Shanghai Jiao Tong University
Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation
Sep 11, 2026Niange Yu, Ye Tian, Biaolong Chen +5Diffusion Transformer Identity-Preserving Video Generation
Alibaba Group · Peking University
The Attention Triangle in Audio-Video Models
Sep 3, 2026Sagi Polaczek, Noa Kraicer, Gal Metzer +4Audio-Video Generation Attention Control in Diffusion Models
1Tel Aviv University · 2Simon Fraser University
Denoising Diffusion Generative Models Secretly Calculate Attentions
Sep 1, 2026Farzan Haddadi, Leila Monfared, Ebrahim Rezaii +3Attention Control in Diffusion Models Diffusion-Based Image Generation
School of Electrical Engineering, Iran University of Science & Technology, Tehran, IRAN · Independent researcher
SafeCA: Safe Cross-Attention Localization and Regulation for Text-to-Video Jailbreak Defense
Aug 11, 2026Siyuan Liang, Yupeng Qiu, Junfeng Fang +3Video Diffusion Models Attention Control in Diffusion Models
Nanyang Technological University · National University of Singapore
SC-Diff: Semantically Calibrated Diffusion for Visible-to-Infrared Image Translation
Aug 9, 2026Junyin Zhang, Siyu Huang, Jianxiong Ye +4Image-to-Image Translation Latent Diffusion Models
School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen, Guangdong 518107, P.R. China · Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong 518055, P.R. China · School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, P.R. China
ControlRef: Efficient Layout-Guided Multi-Instance Generation via Anchored 4D-RoPE
Aug 7, 2026Yunkai Yang, Yudong Zhang, Xinying Chen +7Rotary Positional Embeddings Diffusion Transformer
1Sun Yat-Sen University · 2Tsinghua University · 3Beijing Institute of Technology +1
Rectify Then Diffuse: Disentangling Concepts Before Denoising Trajectory Unfolds
Aug 4, 2026Ning Zhu, An Chen, Mengfei Zhao +4T2I Generation Attention Control in Diffusion Models
Glasgow College, University of Electronic Science and Technology of China · School of Mathematical Sciences, University of Electronic Science and Technology of China
FreeShadow: Training-Free Shadow Removal via Illumination Transfer and Selective Content Preservation in Diffusion Models
Jul 29, 2026Yinan Wang, Yan Huang, Yong Xu +1Diffusion Models for Image Restoration Shadow Removal
School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China · Pazhou Lab, Guangzhou 510005, China · Polytech Nantes, Universit´e de Nantes, Nantes 44306, France
PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention
Jul 28, 2026Zhengtao Yao, Runhao Li, Xupeng Chen +12Masked Diffusion Models Attention Control in Diffusion Models
University of Southern California · New York University · Columbia University +4
To Erase, or Not to Erase: Robust Training-Free Concept Erasure with Preservation aware Adaptive Ranked Subspace Expansion
Jul 26, 2026Shaswati Saha, Rajasekhar Anguluri, Manas GaurAttention Control in Diffusion Models Diffusion Model Unlearning
University of Maryland Baltimore County, USA
DORS: Dynamic Attention Routing for Diffusion-based Object Removal in Dense Scenes
Jul 18, 2026Haitong Tang, Haipeng Liu, Yang WangDiffusion Models for Image Restoration Attention Control in Diffusion Models
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China
Introspective Attention Modulation for Safe Text-to-Image Generation
Jul 16, 2026Basim Azam, Hossein Rahmani, Naveed AkhtarDiffusion Transformer T2I Generation
The University of Melbourne, Australia · Lancaster University, United Kingdom
Erasing Without Collateral Damage: Precise Concept Removal in Diffusion Models
Jul 6, 2026Parth Upman, Nishita Jain, Shreyank N GowdaAttention Control in Diffusion Models Diffusion-Based Image Editing
School of Computer Science, University of Nottingham, Nottingham, NG8 1BB, UK · Department of Computing, Imperial College London, London, SW7 2AZ, UK
Editing Everything Everywhere All at Once
Jun 30, 2026Fabio Quattrini, Carmine Zaccagnino, Enis Simsar +4Text-Guided Image Editing Attention Control in Diffusion Models
University of Modena and Reggio Emilia · ETH Zurich · Google
BindEdit: Taming Attention Leakage for Precise Multi-Object Image Editing
Jun 17, 2026Chaewon Park, Soyoon Lee, Naeun Lee +3Text-Guided Image Editing Attention Control in Diffusion Models
1Sookmyung Women’s University · 2Yonsei University · 3Samsung Research
Training-free sparse attention based on cumulative energy filtering
Jun 15, 2026Chunlu Li, Yixuan Pan, Bai Du +5Video Diffusion Models Diffusion Transformer
Huawei Technologies
Conditioning Matters: Stabilizing Inversion and Attention in Diffusion Image Editing
Jun 12, 2026Zheyuan Zhan, Hongchen Li, Can Wang +6Diffusion Model Inversion Text-Guided Image Editing
State Key Laboratory of Blockchain and Data Security, Zhejiang University · HangZhou High-Tech Zong (Binjiang) Institute of Blockchain and Data Security · College of Computer Science, Zhejiang University +1
Toward 360-Degree Indoor Panorama Editing via Tuning-Free Diffusion Model with Refocusing Cross-Attention
Jun 12, 2026Dinh-Khoi Vo, Nhut-Thanh Le-Hinh, Viet-Tham Huynh +3Diffusion Model Guidance Text-Guided Image Editing
University of Science, Ho Chi Minh, Vietnam · Vietnam National University, Ho Chi Minh, Vietnam · University of Dayton, Ohio, United States
Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models
May 25, 2026Saemi Moon, Suhyeon Jun, Seoyeon Lee +1Attention Control in Diffusion Models Diffusion Model Unlearning
CSE, POSTECH · GSAI, POSTECH
AI-T2I: Aggregating-and-Isolating Cross-Attention to Diffusion Models for Text-to-Image Synthesis
May 25, 2026Shipeng Cao, Biao Qian, Haipeng Liu +2Diffusion Model Alignment Cross-Attention
Key Laboratory of Knowledge Engineering With Big Data, Ministry of Education, Hefei University of Technology, China · Department of Automation, Tsinghua University, China
ClickRemoval: An Interactive Open-Source Tool for Object Removal in Diffusion Models
May 14, 2026Ledun Zhang, Yatu Ji, Xufei Zhuang +1Attention Control in Diffusion Models Image Inpainting
Inner Mongolia University of Technology Hohhot, Inner Mongolia, China
Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generation
Apr 23, 2026Boxun Xu, Yuming Du, Zichang Liu +7Autoregressive Diffusion Self-Attention
Meta Superintelligence Labs · University of California, Santa Barbara · Work done at Meta
AttentionBender: Manipulating Cross-Attention in Video Diffusion Transformers as a Creative Probe
Apr 22, 2026Adam Cole, Mick GriersonTransformer Interpretability Video Diffusion Models
University of the Arts London London, UK

Oct 6, 2026, cs.CVJ/K move · Enter open · S save

RefRoute: Decoupling Conditioning Cost from References via Compact Residual Conditioning and Spatial Routing

Wanning He, Yuyao Zhang, Yu-Wing Tai

Dartmouth College

Diffusion Transformer Image Generation Compositional Image Generation Attention Control in Diffusion Models Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Multi-reference image generation requires preserving the appearance of multiple subjects while composing them into a coherent scene. However, existing diffusion transformers commonly encode references as dense visual token grids and jointly process them with global attention, making conditioning increasingly expensive as the number and resolution of references grow. We present RefRoute, a framework that addresses both reference representation cost and attention overhead through two complementary mechanisms. Compact residual conditioning combines low-resolution latent tokens with lightweight residual features extracted from full-resolution pixels, reducing reference token counts while retaining fine-grained appearance cues. Condition routing and attention routing align reference tokens with their assigned target regions and restrict cross-reference interactions, while allowing selective reference access beyond region boundaries for scene integration. We further introduce RefRoute-Data for training many-reference generation models and ManyRef100, a benchmark spanning human, object, and mixed compositions with 10-17 references. After many-reference fine-tuning, RefRoute achieves an overall Weighted-Ref-VIEScore of 36.06 on ManyRef100, compared with 8.88 for FLUX.2-Klein-9B. Separate inference-cost evaluations show substantially slower latency growth as the reference count increases: at 16 references, our 50-step and 4-step configurations achieve

18.3\times

and

14.2\times

speedups over their corresponding FLUX baselines, respectively. These results establish compact reference representations and spatially routed attention as an effective approach to scalable many-reference image generation.

Momentum

Related topics

Latest papers 33