Diffusion Model Inference Acceleration

Follow this topic Ask about this topic

Momentum

40 papers in the last four weeks, up 100% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 335

All topics

Attend Locally, Remember Linearly: Linear Attention as Cross-Frame Memory for Autoregressive Video Diffusion
May 15, 2026Kunyang Li, Mubarak Shah, Yuzhang ShangMemory-Augmented Neural Networks Video Diffusion Models
Institute of Artificial Intelligence, University of Central Florida
Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schrödinger Samplers
May 15, 2026Bruno Trentini, Dejan Stancevic, Michael M. Bronstein +2Flow Matching Schrödinger Bridges
NVIDIA Corporation, Santa Clara, CA, USA · University of Oxford, Dept. of Computer Science, Oxford, UK · Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, NL +1
ElasticDiT: Efficient Diffusion Transformers via Elastic Architecture and Sparse Attention for High-Resolution Image Generation on Mobile Devices
May 15, 2026Kunpeng Du, Haizhen Xie, Sen Lu +11Diffusion Transformer Diffusion-Based Image Generation
Huawei Technologies
Efficient Image Synthesis with Sphere Latent Encoder
May 15, 2026Tung Do, Thuan Hoang Nguyen, Hao LiLatent Diffusion Models Few-Step Diffusion Sampling
Mohamed Bin Zayed University of Artificial Intelligence, UAE
Unlocking Complex Visual Generation via Closed-Loop Verified Reasoning
May 14, 2026Hanbo Cheng, Limin Lin, Ruo Zhang +2Image Generation Compositional T2I Generation
University of Science and Technology of China (USTC) · 2Independent Researcher
The Velocity Deficit: Initial Energy Injection for Flow Matching
May 14, 2026Linze Li, Zong-Wei Hong, Shen Zhang +4Flow Matching Image Generation
Jiiov Technology, Beijing, China.
HASTE: Training-Free Video Diffusion Acceleration via Head-Wise Adaptive Sparse Attention
May 14, 2026Xuzhe Zheng, Yuexiao Ma, Jing Xu +3Video Diffusion Models Diffusion Transformer
1Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China
Image Restoration via Diffusion Models with Dynamic Resolution
May 14, 2026Yang Zheng, Wen Li, Zhaoqiang LiuDiffusion Models for Image Restoration Diffusion-Based Inverse Problems
School of Computer Science and Engineering, University of Electronic Science and Technology of China.
CoReDiT: Spatial Coherence-Guided Token Pruning and Reconstruction for Efficient Diffusion Transformers
May 13, 2026Zhuojin Li, Hsin-Pai Cheng, Hong Cai +2Diffusion Transformer Visual Token Pruning
Qualcomm AI Research†
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
May 13, 2026Zhonghao Li, Chaoyu Liu, Qian ZhangNeural PDE Solvers Latent Diffusion Models
School of Science, Harbin Institute of Technology, Shenzhen, China · Department of Theoretical Physics and Applied Mathematics, University of Cambridge, Cambridge, United Kingdom
OP4KSR: One-Step Patch-Free 4K Super-Resolution with Periodic Artifact Suppression
May 13, 2026Chengyan Deng, Pengbin Yu, Zhentao Chen +6Diffusion Transformer Diffusion Model Inference Acceleration
School of Automation Engineering, University of Electronic Science and Technology of China · OPPO AI Center, OPPO Inc. · School of Intelligence Science and Technology, Nanjing University
BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuning
May 13, 2026Ruiheng Wang, Shuanghao Bai, Haoran Zhang +2Robotic Manipulation Vision-Language-Action Models
Xi'an Jiaotong University
Test-time Sparsity for Extreme Fast Action Diffusion
May 13, 2026Kangye Ji, Yuan Meng, Jianbo Zhou +3Diffusion Transformer Inference-Time Optimization
Tsinghua University · The Chinese University of Hong Kong
DiffST: Spatiotemporal-Aware Diffusion for Real-World Space-Time Video Super-Resolution
May 13, 2026Zheng Chen, Ruofan Yang, Jin Han +5Video Diffusion Models Video Super-Resolution
Shanghai Jiao Tong University · Huawei Noah’s Ark Lab · Duke University +1
FIS-DiT: Breaking the Few-Step Video Inference Barrier via Training-Free Frame Interleaved Sparsity
May 12, 2026Jian Tang, Jiawei Fan, Qingbin Liu +1Video Diffusion Models Diffusion Transformer
Platform and Content Group, Tencent
Principled Design of Diffusion-based Optimizers for Inverse Problems
May 12, 2026Julio Oscanoa, Irmak Sivgin, Cagan Alkan +4Inference-Time Optimization Diffusion-Based Inverse Problems
Department of Bioengineering · Deparment of Electrical Engineering · Department of Radiology, Stanford University, USA.
ChunkFlow: Communication-Aware Chunked Prefetching for Layerwise Offloading in Distributed Diffusion Transformer Inference
May 11, 2026Han Meng, Danny Willow Liu, Dong LiDiffusion Transformer GPU Acceleration
University of California, Merced · University of Chicago · University of California, Merced and Yotta Labs +1
Muninn: Your Trajectory Diffusion Model But Faster
May 11, 2026Gokul Puthumanaillam, Hao Jiang, Ruben Hernandez +4Trajectory Generation Diffusion-Based Planning
University of Illinois Urbana-Champaign. · Florida International University. · Providence College.
Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Models
May 10, 2026Yicheng Ji, Zhizhou Zhong, Jun Zhang +7Diffusion Model Caching KV-Cache Compression
ZJU · Video Rebirth · HKUST +1
Edit-Based Refinement for Parallel Masked Diffusion Language Models
May 10, 2026Houxing Ren, Mingjie Zhan, Zimu Lu +5Masked Diffusion Models Parallel Decoding
CUHK MMLab · SenseTime Research · Shenzhen Loop Area Institute +1
When Few Steps Are Enough: Training-Free Acceleration of Identity-Preserved Generation
May 10, 2026Dongqi ZhengIdentity-Preserving Image Generation Few-Step Diffusion Sampling
SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation
May 10, 2026Shanwen Tan, Hao Li, Jingtao Zhang +4Long Video Generation Memory-Augmented Video Generation
University of Science and Technology of China · Fudan University · Georgia Institute of Technology +1
Latency Analysis and Optimization of Alpamayo 1 via Efficient Trajectory Generation
May 9, 2026Yunseong Jeon, Namcheol Lee, Yoonsu Lee +4Efficient Inference End-to-End Autonomous Driving
Graduate School of Automobile and Mobility, Kookmin University, Seoul, Republic of Korea · Department of Electrical and Computer Engineering, Seoul National University, Seoul, Republic of Korea
SynerDiff: Synergetic Continuous Batching for Fast and Parallel Diffusion Model Inference
May 9, 2026Ziqi Zhou, Peng Yang, Yuxin Liang +2Diffusion Model Serving Diffusion Model Inference Acceleration
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China · The State Grid Hubei Electric Power Research Institute, Wuhan, China · Jincheng Yunxiang Big Data Technology Operation Co., Ltd., Shanxi, China
ElasticFlow: One-Step Physics-Consistent Policy with Elastic Time Horizons for Language-Guided Manipulation
May 9, 2026Kewei Chen, Yayu Long, Shuai Li +1Flow Matching Long-Horizon Robotic Manipulation
Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences · Chongqing School, University of Chinese Academy of Sciences · Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection
May 9, 2026Haohui Zhang, Zhiye Wang, Xiaoying Gan +2Parallel Decoding Diffusion Language Model Decoding
Shanghai Jiao Tong University
ECTraj: Enhanced Consistency Training for Multi-Agent Trajectory Prediction
May 9, 2026Alen Mrdovic, Qingze, Liu +6Trajectory Generation Multi-Agent Trajectory Prediction
Tony · Rutgers University, New Brunswick · The College of New Jersey
PET-Adapter: Test-Time Domain Adaptation for Full and Limited-Angle PET Image Reconstruction
May 8, 2026Rüveyda Yilmaz, Yuli Wu, Johannes Stegmaier +1Test-Time Adaptation Image Reconstruction
Institute of Imaging and Computer Vision, RWTH Aachen University, Germany · Machine Learning for Medical Data, Heinrich Heine University Düsseldorf, Germany
Stochastic Transition-Map Distillation for Fast Probabilistic Inference
May 8, 2026George Rapakoulias, Peter Garud, Lingjiong Zhu +1Diffusion Model Distillation MeanFlow
Department of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA · Department of Computer Science, Georgia Institute of Technology, Atlanta, GA · Department of Mathematics, Florida State University, Tallahassee, FL
LENS: Low-Frequency Eigen Noise Shaping for Efficient Diffusion Sampling
May 8, 2026Haewon Jeon, Si-Hyeon LeeDiffusion Model Distillation Few-Step Diffusion Sampling
School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon, South Korea

May 15, 2026, cs.CVJ/K move · Enter open · S save

Attend Locally, Remember Linearly: Linear Attention as Cross-Frame Memory for Autoregressive Video Diffusion

Kunyang Li, Mubarak Shah, Yuzhang Shang

Institute of Artificial Intelligence, University of Central Florida

Memory-Augmented Neural Networks Video Diffusion Models Self-Attention Linear Attention Autoregressive Video Generation Diffusion Model Inference Acceleration+1

Open paper AskPDF arXiv

Autoregressive (AR) video diffusion is a powerful paradigm for streaming and interactive video generation. However, its reliance on softmax self-attention leads to quadratic compute complexity in sequence length and memory usage due to key-value caching, which limits its scalability to long video horizons. Existing remedies (e.g., sparse attention and KV-cache compression) reduce per-step cost but still rely on a linearly growing cache or irreversibly discard past context, and thus fail to address linear memory growth and streaming context management. To address this scalability bottleneck, we propose ARL2 (Attend Locally, Remember Linearly), a hybrid attention module that replaces quadratic cross-frame attention with a fixed-size recurrent state. We decompose self-attention into two branches: an intra-frame softmax branch for spatial detail and local dependencies, and an inter-frame gated recurrent linear branch that maintains a fixed-size state for streaming context. Our key insight is that softmax attention captures fine-grained local interactions, while a recurrent state provides controllable long-range memory. This design achieves linear-time scaling with constant memory while improving temporal consistency over the full-softmax model. To prevent noisy intermediate states from corrupting memory, we update the recurrent state only after the denoised pass. To avoid within-frame information asymmetry, all tokens share the same pre-update state rather than sequential updates. To the best of our knowledge, this is the first work to convert a pretrained AR video diffusion model into a hybrid linear attention architecture, through an efficient two-stage training scheme for AR video. With 75% of layers replaced by hybrid linear attention, the model achieves up to 2.26 wall-clock speedup and 54% memory reduction, while maintaining comparable quality with improving temporal consistency.

Momentum

Related topics

Latest papers 335