Diffusion Model Serving

Follow this topic Ask about this topic

Diffusion model serving studies how to run image, video, audio, and language-generation workloads efficiently under real-time latency and throughput constraints. Systems use continuous and block-grained batching, adaptive GPU scheduling, caching and KV-cache reuse, selective computation, and model quantization to reduce wasted work and manage shared hardware. Evaluations include image and video generation workloads as well as diffusion language model tasks such as GSM8K and HumanEval, often measuring throughput, tail latency, service-level objectives, and output quality. A central challenge is that denoising costs vary across requests and steps, so predicting work and improving utilization must not undermine generation quality or deadline guarantees.

Latest papers

18 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

SAKURA: Serving Attention Key-Value State Under Rate Adaptation for Private Image Editing on Phones
Oct 21, 2026Shuncheng Jie, Desen Sun, Ri Hong +3Edge-Cloud Collaborative Inference Diffusion Model Serving
University of Waterloo Canada
DIFFLET: A Fast Inference Engine for Diffusion Transformer Models on AWS Trainium
Oct 16, 2026Bin Ma, Min Yu, Vijay Kodamalla +3Diffusion Model Inference Acceleration Diffusion Model Serving
Yotta Labs, USA · University of California, Merced, Merced, California, USA.
Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving
Sep 30, 2026Haoyu Zheng, Fangcheng Fu, Binhang Yuan +6Cost-Aware Inference Diffusion Model Serving
Wuhan University · Shanghai Jiao Tong University · The Hong Kong University of Science and Technology +2
Co-occurrence Patterns of LoRA Adapters in Production Diffusion Model Inference Services
Sep 20, 2026Tao Zhang, Bin Liao, Tao Zhou +1Diffusion Model Serving Low-Rank Adaptation
School of Information Engineering, Guizhou University of Traditional Chinese Medicine, Guiyang, 550025, China · School of Big Data and Statistics, Guizhou University of Finance and Economics, Guiyang, 550025, China · School of Mathematics and Statistics, Guizhou University of Finance and Economics, Guiyang, 550025, China +1
Serving Masked Diffusion LLMs: Characterization and Design Principles from Real Hardware
Aug 24, 2026Farhana Amin, Sabiha Afroz, Mona Moghadampanah +1Diffusion Model Serving Diffusion Language Model Inference
Dept. of Computer Science Virginia Tech Blacksburg, VA, USA
CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inference
Jul 31, 2026Yifan Wu, Yufeng Zhang, Kenli LiDiffusion Model Serving Diffusion Language Model Inference
College of Computer Science, Hunan University Changsha 410082, China
FlashDiff: Efficient Regional Execution and Scheduling for Diffusion Model Serving
Jul 13, 2026Yaqi Qiao, Ping He, Songrun Xie +4Diffusion Model Serving Adaptive Computation for Diffusion Models
University of Illinois Urbana-Champaign · Vanderbilt University · HKUST +2
BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
Jul 9, 2026Yuanjie Zhu, Liangwei Yang, Ke Xu +4LLM Inference Scheduling Diffusion Model Serving
University of Illinois Chicago, USA
Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
Jul 5, 2026Nitin Kedia, Saurabh Agarwal, Myungjin Lee +1LLM Inference Scheduling LLM Inference Serving
The University of Texas at Austin Austin, United States · Cisco Research Bellevue, United States
DiLaServe: High SLO Attainment Serving for Diffusion Language Models
Jun 27, 2026Tzu-Tao Chang, Benjamin Yuanyang Hong, Kiet Pham +1LLM Serving Diffusion Model Serving
University of Wisconsin-Madison USA
HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval
Jun 19, 2026Omin Kwon, Doyeon Kim, Jongseok Park +3KV-Cache Offloading Diffusion Model Serving
Seoul National University Seoul, Republic of Korea · UC Berkeley Berkeley, CA, USA
GF-DiT: Scheduling Parallelism for Diffusion Transformer Serving
Jun 11, 2026Xinwei Qiang, Yifan Hu, Shixuan Sun +6Diffusion Transformer Diffusion Model Serving
Shanghai Jiao Tong University Shanghai, China · Guizhou University Guiyang, China
FLARE: Diffusion for Hybrid Language Model
Jun 1, 2026Yuchen Zhu, Jing Shi, Chongjian Ge +9Diffusion Model Serving Diffusion Language Model Inference
1Adobe Research · 2Georgia Institute of Technology
SynerDiff: Synergetic Continuous Batching for Fast and Parallel Diffusion Model Inference
May 9, 2026Ziqi Zhou, Peng Yang, Yuxin Liang +2Diffusion Model Serving Diffusion Model Inference Acceleration
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China · The State Grid Hubei Electric Power Research Institute, Wuhan, China · Jincheng Yunxiang Big Data Technology Operation Co., Ltd., Shanxi, China
LegoDiffusion: Micro-Serving Text-to-Image Diffusion Workflows
Apr 9, 2026Lingyun Yang, Suyi Li, Tianyu Feng +10Diffusion Model Serving T2I Diffusion Models
Hong Kong University of Science and Technology · Alibaba Group
Beyond Few-Step Inference: Accelerating Video Diffusion Transformer Model Serving with Inter-Request Caching Reuse
Apr 6, 2026Hao Liu, Ye Huang, Chenghuan Huang +5Diffusion Model Caching Diffusion Model Inference Acceleration
Sun Yat-sen University, China · Independent Researcher
PersonalQ: Select, Quantize, and Serve Personalized Diffusion Models for Efficient Inference
Mar 24, 2026Qirui Wang, Qi Guo, Yiding Sun +4Diffusion Model Serving T2I Diffusion Models
School of Software Engineering, Xi’an Jiaotong University, China · Nankai University, China
Sphinx: Efficiently Serving Novel View Synthesis using Regression-Guided Selective Refinement
Nov 24, 2025Yuchen Xia, Souvik Kundu, Mosharaf Chowdhury +1Novel View Synthesis Diffusion Model Serving
University of Illinois Urbana-Champaign, Illinois, USA · Intel AI Group, USA · University of Michigan, Michigan, USA

Oct 21, 2026J/K move · Enter open · S save

SAKURA: Serving Attention Key-Value State Under Rate Adaptation for Private Image Editing on Phones

Shuncheng Jie, Desen Sun, Ri Hong, Yuyang Tian+2

University of Waterloo Canada

Edge-Cloud Collaborative Inference Diffusion Model Serving On-Device Inference KV-Cache Compression Diffusion-Based Image Editing Diffusion Model Compression

Open paper AskPDF arXiv

Diffusion transformers are widely used in image-editing services. To preserve both efficiency and privacy, existing systems commonly rely on cloud-device collaboration. Keeping request-specific inference on the device avoids sending personal images and prompts to the cloud. These workloads often use common templates, such as in face swapping and virtual try-on, enabling KV state to be precomputed and reused across requests. However, this reusable state is large and scales with the number of templates, making it impractical to store all of them on mobile devices. Storing them in the cloud addresses this capacity issue, but cache transfer is placed onto the inference path and can even be slower than on-device computation, especially under fluctuating mobile networks. We introduce SAKURA, a cloud-edge architecture that delivers reusable attention state to mobile devices under a bandwidth budget. It first uses a fine-grained token-importance-aware compression method to preserve image quality under limited transfer budgets. At runtime, SAKURA adaptively selects the cache representation for each upcoming shard based on network bandwidth. To enable efficient execution, we develop a pipelined runtime that overlaps cache transfer and reconstruction with on-device execution. We evaluate SAKURA using three editing tasks and two diffusion models, using a Snapdragon 8 Elite Gen 5 phone. SAKURA reduces latency by 1.82x and 3.27x over CacheGen adapted to DiT and on-device generation, respectively. It also reduces cache storage by 3.06x over uncompressed cache, while preserving image quality.