Diffusion Transformer

Also known as DiT

Diffusion Transformers (DiTs) use transformer networks to turn noise into data, typically by repeatedly predicting denoising updates or following a learned flow. Work explores latent- and pixel-space models, conditioning through text, images, geometry, or other modalities, and representation-learning methods that improve generation and downstream perception. To lower the cost of training and inference, researchers use sparse attention, token reduction, caching, quantization, distillation, and hardware-aware designs. A central challenge is balancing speed, memory, and model size against fidelity, temporal or spatial consistency, and reliable performance across tasks and benchmarks.

Latest papers

473 papers78 papers in the last four weeks, up 100% on the four weeks before. 0.5% of all new papers.

Test-time Sparsity for Extreme Fast Action Diffusion
May 13, 2026Kangye Ji, Yuan Meng, Jianbo Zhou +3Diffusion Transformer Inference-Time Optimization
Tsinghua University · The Chinese University of Hong Kong
Sparse Mixture-of-Experts Routing in Visual Diffusion Transformers:Diagnosis, Boundary Calibration and Evolutionary Roadmap from Routing Collapse to Selective Deadlock
May 12, 2026Haiying ShaVideo Diffusion Models Diffusion Transformer
Independent Researcher
FIS-DiT: Breaking the Few-Step Video Inference Barrier via Training-Free Frame Interleaved Sparsity
May 12, 2026Jian Tang, Jiawei Fan, Qingbin Liu +1Video Diffusion Models Diffusion Transformer
Platform and Content Group, Tencent
ChunkFlow: Communication-Aware Chunked Prefetching for Layerwise Offloading in Distributed Diffusion Transformer Inference
May 11, 2026Han Meng, Danny Willow Liu, Dong LiDiffusion Transformer GPU Acceleration
University of California, Merced · University of Chicago · University of California, Merced and Yotta Labs +1
HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer
May 11, 2026Qi Cai, Jingwen Chen, Chengmin Gao +22Diffusion Transformer Text-Guided Image Editing
What Concepts Lie Within? Detecting and Suppressing Risky Content in Diffusion Transformers
May 11, 2026Chenyu ZhangDiffusion Transformer T2I Generation
Tianjin University · Tianjin, China
Attention Sinks in Diffusion Transformers: A Causal Analysis
May 10, 2026Fangzheng Wu, Brian SummaTransformer Interpretability Diffusion Transformer
Department of Computer Science, Tulane University, New Orleans, LA, USA.
Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems
May 10, 2026Leduo Chen, Junchuan Zhao, Shengchen LiDiffusion Transformer Controllable Music Generation
XJTLU
BRIDGE: Background Routing and Isolated Discrete Gating for Coarse-Mask Local Editing
May 8, 2026Peilin Xiong, Honghui Yuan, Junwen Chen +1Diffusion Transformer Positional Encoding
Department of Informatics, The University of Electro-Communications, Tokyo, Japan
TextLDM: Language Modeling with Continuous Latent Diffusion
May 8, 2026Jiaxiu Jiang, Jingjing Ren, Wenbo Li +10Diffusion Transformer Latent Diffusion Models
Joy Future Academy · HIT · HKUST(GZ)
EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing
May 8, 2026Lan Chen, Qi Mao, Yiren Song +2Diffusion Transformer Diffusion Model Fine-Tuning
School of Information and Communication Engineering and the State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China · ShowLab, National University of Singapore, 119077, Singapore · State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing 100871, China
Not All Tokens Need 40 Steps: Heterogeneous Step Allocation in Diffusion Transformers for Efficient Video Generation
May 7, 2026Ernie Chu, Vishal M. PatelVideo Diffusion Models Diffusion Transformer
Johns Hopkins University Baltimore, MD 21218
Continuous Latent Diffusion Language Model
May 7, 2026Hongcan Guo, Qinyu Zhao, Yian Zhao +8Non-Autoregressive Text Generation Diffusion Transformer
ByteDance Seed · The Australian National University · Peking University +2
Why Do DiT Editors Drift? Plug-and-Play Low Frequency Alignment in VAE Latent Space
May 7, 2026Xiaoce Wang, Sifan Zhou, Kaifei Wang +4Diffusion Transformer Multi-Turn Image Editing
♠Tsinghua University · ♡Carnegie Mellon University · ♢Peking University +3
Mean Mode Screaming: Mean--Variance Split Residuals for 1000-Layer Diffusion Transformers
May 7, 2026Pengqi LuDeep Learning Optimization Representation Collapse
Beijing, China
Taming Outlier Tokens in Diffusion Transformers
May 6, 2026Xiaoyu Wu, Yifei Wang, Tsu-Jui Fu +3Diffusion Transformer Activation Outliers
Rice University · Apple
Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models
May 6, 2026Yifan F. Zhang, Fangjun Hu, Guangkuo Liu +2Diffusion Transformer Symmetry Breaking
Department of Electrical and Computer Engineering Princeton University Princeton, NJ 08544, USA · QuEra Computing Inc. 1284 Soldiers Field Road, Boston, MA 02135, USA · JILA and Department of Physics University of Colorado Boulder Boulder, CO 80309, USA +1
AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers
May 5, 2026Ruibin Min, Yexin Liu, Aimin Pan +5Diffusion Model Alignment Diffusion Transformer
Sun Yat-sen University · The Hong Kong University of Science and Technology · VNET Group +2
Ortho-Hydra: Orthogonalized Experts for DiT LoRA
May 5, 2026Seunghyun JiFine-Tuning Diffusion Transformer
Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE
May 4, 2026Yangming Shi, Shixiang Zhu, Tao Shen +14Diffusion Model Inference Acceleration Unified Multimodal Models
Mamoda Team, ByteDance
AttnRouter: Per-Category Attention Routing for Training-Free Image Editing on MMDiT
May 2, 2026Guandong Li, Mengxia YeDiffusion Transformer Text-Guided Image Editing
iFLYTEK · Aegon THTF
Soft Graph Diffusion Transformer for MIMO Detection
May 1, 2026Nan Jiang, Jiadong Hong, Lei Liu +3Diffusion Transformer Wireless Communications
College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou, China · Theory Lab, Central Research Institute, 2012 Labs, Huawei Technologies Co., Ltd., Hong Kong, China
YOSE: You Only Select Essential Tokens for Efficient DiT-based Video Object Removal
Apr 30, 2026Chenyang Wu, Lina Lei, Fan Li +6Diffusion Transformer Video Editing
1VCIP, CS, Nankai University · 3NKIARI, Shenzhen Futian · 2Huawei Noah’s Ark Lab
Beyond Fixed Formulas: Data-Driven Linear Predictor for Efficient Diffusion Models
Apr 29, 2026Zhirong Shen, Rui Huang, Jiacheng Liu +6Diffusion Transformer Diffusion Model Caching
Shanghai Jiao Tong University · University of Electronic Science and Technology of China · Shandong University +2
The Thinking Pixel: Recursive Sparse Reasoning in Multimodal Diffusion Latents
Apr 28, 2026Yuwei Sun, Yuxuan Yao, Hui Li +1Diffusion Transformer Conditional Image Generation
Shanghai Academy of AI for Science · Fudan University
Edit Where You Mean: Region-Aware Adapter Injection for Mask-Free Local Image Editing
Apr 26, 2026Honghao Cai, Xiangyuan Wang, Yunhao Bai +8Diffusion Transformer Text-Guided Image Editing
The Chinese University of Hong Kong, Shenzhen · Xiaohongshu Inc. · Peking University +2
Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling
Apr 26, 2026Zhen Ye, Xu Tan, Aoxiong Yin +8Talking Head Generation Audio-Video Generation
Hong Kong University of Science and Technology · Independent Researcher · Zhejiang University +3
OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Space
Apr 24, 2026Zhuding Liang, Tianyi Yan, Dubing Chen +6Diffusion Transformer Multi-Agent Systems
SKL-IOTSC, CIS, University of Macau, Macau, China · Li Auto Inc, Beijing, China
UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions
Apr 24, 2026Chunyu Qiang, Xiaopeng Wang, Kang Yin +11Text-to-Music Generation Text-to-Audio Generation
1Tianjin University, Tianjin, China · 2Kling Team, Kuaishou Technology, Beijing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China
Sculpt4D: Generating 4D Shapes via Sparse-Attention Diffusion Transformers
Apr 23, 2026Minghao Yin, Wenbo Hu, Jiale Xu +2Diffusion Transformer 3D Shape Generation
1The University of Hong Kong · 2ARC Lab, Tencent PCG

May 13, 2026J/K move · Enter open · S save

Test-time Sparsity for Extreme Fast Action Diffusion

Kangye Ji, Yuan Meng, Jianbo Zhou, Ye Li+2

Tsinghua University · The Chinese University of Hong Kong

Diffusion Transformer Inference-Time Optimization Diffusion Policy Diffusion Model Caching Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Action diffusion excels at high-fidelity action generation but incurs heavy computational costs owing to its iterative denoising nature. Despite current technologies showing promise in accelerating diffusion transformers by reusing the cached features, they struggle to adapt to policy dynamics arising from diverse perceptions and multi-round rollout iterations in open environments. We propose test-time sparsity to tackle this challenge, which aims to accelerate action diffusion by dynamically predicting prunable residual computations for each model forward at test time. However, two bottlenecks remain in this paradigm: 1) repetitive conditional encoding and pruning offset most potential speed gains, and 2) the features cached from previous denoising timesteps cannot constrain large pruning errors under aggressive sparsity. To address the first bottleneck, we design a highly parallelized inference pipeline that minimizes the non-decoder delay to milliseconds. Specifically, we first design a lightweight pruner that shares the encoder with the diffusion transformer. Then, we decouple the encoding and pruning from the autoregressive denoising loop by processing all denoising timesteps in parallel, and overlap the pruner with the decoder forward inference through asynchronism. To overcome the second bottleneck, we introduce an omnidirectional reusing strategy, which achieves 95% sparsity by selectively reusing features cached from the current forward, previous denoising timesteps, and earlier rollout iterations. To learn the rollout-level reusing strategies, we sample a few action trajectories to supervise the sparsified diffusion step by step. Extensive experiments demonstrate that our method reduces FLOPs by 92% and accelerates action generation by 5x, achieving lossless performance with an inference frequency of 47.5 Hz. Our code is available at https://github.com/ky-ji/Test-time-Sparsity.