Sparsity

Follow this topic Ask about this topic

Recent momentum

-13%

20 papers in the last 28 days · 0.3% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

6 new papers

A weekly snapshot of new work published in Sparsity.

Period ending 2026-09-14

9 new papers

A weekly snapshot of new work published in Sparsity.

Period ending 2026-09-07

4 new papers

A weekly snapshot of new work published in Sparsity.

193 papers

Latest in Sparsity

Browse all topics →

Post-Training Pruning for Diffusion Transformers
Jul 1, 2026Chengzhi Hu, Xuewen Liu, Jing Zhang +3Multimodal Diffusion Transformers Heterogeneous Pruning Scores
Entropy-Regularized Probabilistic Gates for Sparse Model Discovery in Scarce-Data Federated Learning
Jun 30, 2026Krishna Harsha Kovelakuntla Huthasana, Alireza Olama, Andreas LundellFederated Learning Entropy-Regularized Equilibrium Selection
Sparsity-Inducing Divergence Losses for Biometric Verification
Jun 30, 2026Dimitrios Koutsianos, Ladislav Mošner, Yannis Panagakis +1Bregman Divergences Sparsity
KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search
Jun 29, 2026Tao Feng, Xinke Jiang, Chao WuAgentic Reinforcement-Learning Framework Agentic Reasoning
On the Policy Gradient Foundations of Group Relative Policy Optimization: Credit Assignment, Gradient Sparsity, and Rank Collapse
Jun 28, 2026Amritansh Mishra, Supriyo Chakraborty, Berkcan KapusuzogluGroup Relative Policy Optimization Policy Gradient
Adaptive Iterative Hard Thresholding for Online High-dimensional Quantile Regression
Jun 26, 2026Zitian Zhou, Nan LinQuantile Regression Learning-Augmented Algorithms
Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring
Jun 26, 2026Muhammad HamzaSpiking Neural Networks Neuromorphic Hardware
End-to-End Dynamic Sparsity for Resource-Adaptive LLM Inference
Jun 26, 2026Yuhang Chen, Jinhao Duan, Ruichen Zhang +11LLM Inference Efficiency Sparsity
Beyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse Autoencoders
Jun 25, 2026Nathanaël Jacquier, Maria Vakalopoulou, Mahdi S. HosseiniImproving Sparse Autoencoders Sparsity
EPTS: Elastic Post-Training Sparsity for Efficient Large Language Model Compression
Jun 24, 2026Ke Xu, Jiaqi Wan, Wenhao Hu +2Large Language Model Compression Sparsity
Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Jun 23, 2026Thomas Fel, Matthew Kowal, Mozes Jacobs +22Implicit Neural Representations Sparsity
Chorus II: Cross-Request Sparsity Reuse for Efficient Image-to-Video Generation
Jun 23, 2026Hao Liu, Chenghuan Huang, Hao Liu +6Large-Scale Image Generation Sparsity
GES-TSP: Graph Edge Sparsification for TSP
Jun 23, 2026Tianfeng Chen, Xianyue LiSalesman Problem Combinatorial Optimization
PeLAP-A: Adaptive Latent Pruning for Lightweight Latent Diffusion Models
Jun 22, 2026Kissa Zahra, Zaib Un NisaLatent Diffusion Model Masked Autoencoders
On the Sparsity-Storage-Accuracy Tradeoff in Parsimoniously Activated Dictionary Learning
Jun 21, 2026Zihui Zhao, Yuanbo Tang, Yang LiUnsupervised Dictionary Learning Sparsity
On the Curse of Dimensionality in Private Sparse Covariance Estimation and PCA
Jun 20, 2026Syamantak Kumar, Shourya Pandey, Purnamrita Sarkar +1Differential Privacy Principal Component Analysis
Mitigating Early Training Collapse in CTR Models
Jun 20, 2026Ergun Biçici, Erkan ÇetinyamaçClick-Through Rate Prediction Collapse
ACE-GS: Acing the Trade-off with Accurate, Compact and Efficient 3D Gaussian Splatting
Jun 19, 2026Jijian Zhao3D Gaussian Splats 3D Object Understanding
Sparsity, Superposition, and Forgetting: A Mechanistic Study of Representation Retention in Continual Learning
Jun 18, 2026Jan Wasilewski, Jędrzej Kozal, Michał Woźniak +1Catastrophic Forgetting Forgetting
Compositionality Emerges in a Narrow Depth-Connectivity Regime: Architecture Constraints and Solution Manifolds
Jun 18, 2026Dat H. Do, Rushi Shah, Duc V. Le +1Compositionality Deep Network
Robust and Interpretable Adaptation of Equivariant Materials Foundation Models via Sparsity-promoting Fine-tuning
Jun 17, 2026Youngwoo Cho, Seunghoon Yi, Wooil Yang +6Interatomic Potentials Sparsity
Spatially Stratified Distillation for Heterogeneous Radar Place Recognition
Jun 17, 2026Sagun Singh Shrestha, Samuel Harding, Abdelwahed Khamis +2Sparse4D-Radar Radar
Effects of sparsity and superposition on loss in simple autoencoders
Jun 16, 2026Mriganka Basu Roy Chowdhury, Eric McLaughlin WeinerImproving Sparse Autoencoders Activation Functions
Sparsity Curse: Understanding RLVR Model Parameter Space from Model Merging
Jun 16, 2026Chenrui Wu, Zexi Li, Jiajun Bu +2Reinforcement Learning With Verifiable Reward Model Merging
ConSA: Controllable Sparsity in Hybrid Attention via Learnable Allocation
Jun 16, 2026Yao Chen, Yinqi Yang, Junyuan Shang +6Sliding-Window Attention Sparsity
Trustworthy MRI Reconstruction via Bayesian Uncertainty Quantification with Sparsity Prior Models
Jun 15, 2026Ahmed Karam Eldaly, Matteo Figini, Daniel C. AlexanderMagnetic Resonance Imaging Reconstruction Image Reconstruction
Beyond Layer Importance in Layer-wise Sparsity: An Inter-Layer Perturbation-Absorption Perspective
Jun 13, 2026Tao Jing, Ningxin Wu, Chen Kang +3Sparsity Large Language Model Compression
Decompose Sparsely Where You Should, Absorb Densely Where You Should No
Jun 12, 2026Ruixuan Deng, Zehao Jin, Zekun Wang +1Improving Sparse Autoencoders Sparsity
Zero-Inflated Gaussian Distributions Enable Parameter-Space Sparsity in Estimation-of-Distribution Algorithms
Jun 11, 2026Andreas Faust, Sven Nitzsche, Juergen BeckerSparsity Black-Box Optimization
Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation
Jun 11, 2026Guo Yu, Wenlin Liu, Yulan Hu +3On-Policy Distillation Rp-Opsd
Towards Data-free and Training-free Compression for Speech Foundation Models Using Parameter Clustering
Jun 10, 2026Haoning Xu, Zhaoqing Li, Huimeng Wang +4Large Language Model Compression Speech Large Language Models
Secure Aggregation with Top-K Sparsification in Decentralized Federated Learning
Jun 9, 2026Hengxuan Tang, Jinbao Zhu, Xiaohu TangFederated Learning Sparsity
SpenseGPT: Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference
Jun 9, 2026Jaeseong Lee, Seung-won Hwang, Samyam RajbhandariSparsity LLM Inference Efficiency
An Empirical Analysis of Optimization Dynamics and Sparsity Boundaries in Large-Scale Pedestrian Attribute Recognition
Jun 9, 2026Houssam El MirPedestrian Detection Focal Loss
ATN3D: Density-Aware LiDAR-Radar Early 3D Object Detection Under Extreme Sparsity
Jun 8, 2026Debojyoti Biswas, Xianbiao Hu3D Object Detection Lidar
Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Models
Jun 7, 2026Yang Zhou, Ranajoy Sadhukhan, Zhaofeng Sun +7Autoregressive Rollout Sparse Rewards
Sparsely gated tiny linear experts
Jun 5, 2026Simon SchugFeed-Forward Network Transformer Layers
Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads
Jun 4, 2026Ruoxi Sun, Quantong Qiu, Juntao Li +3Multi-Modal Large Language Models Cross-Modal Attention
ColBERTSaR: Sparsified ColBERT Index via Product Quantization
Jun 4, 2026Eugene Yang, Andrew Yates, Dawn Lawrie +3Retrieval Layer Sparsity
A Robust Optimization Approach to Sparse Principal Component Analysis
Jun 2, 2026David Vävinggren, Francis Bach, André M. H. Teixeira +2Principal Component Analysis Sparsity
CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search
Jun 1, 2026Cheonjun ParkHeterogeneous Pruning Scores Large Language Model Post-Training
MURMUR: An Efficient Inference System for Long-Form ASR
May 31, 2026Wei-Tzu Lee, Keisuke Kamahori, Baris KasikciSparsity
DAGGER: Gradient-Free Construction of Transiently Amplifying Networks under Hard Connectivity Constraints
May 31, 2026James C. FergusonSparsity Gradient-Based Optimization
HASTE: Hardware-Aware Dynamic Sparse Training for Large Output Spaces
May 31, 2026Nasib Ullah, Jinbin Zhang, Jean Lucien Randrianantenaina +2Sparsity Multi-Label Classification
Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling
May 30, 2026Qiao Xiao, Boqian Wu, Patrik Okanovic +6Sparsity Adam
Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits
May 29, 2026Salem Ameen, Sunil VaderaModern Deep Networks Heterogeneous Pruning Scores
VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning
May 29, 2026Hengbo Xu, Shengjie Jin, Yanbiao Ma +1Multimodal Reasoning Visual Token Pruning
Exploiting Chordal Sparsity for Globally Optimal Estimation with Factor Graphs
May 28, 2026Avinash Subramanian, Connor Holmes, Timothy D. Barfoot +2Factor Graphs State Estimation
Joint Model and Data Sparsification via the Marginal Likelihood
May 28, 2026Alexander Timans, Thomas Möllenhoff, Christian A. Naesseth +2Sparse Recovery Sparsity
Augmenting Attention with Exponentially Decaying Memory Improves Query-Aware KV Sparsity
May 27, 2026Xiuying Wei, Caglar GulcehreMemory-Augmented Large Language Models Efficient Long-Context Inference
Learning High-Dimensional Parity Functions with Product Networks using Gradient Descent
May 27, 2026Guillaume Larue, Louis-Adrien Dufrène, Quentin Lampin +2Two-Layer Neural Networks Neural Network
PrunePath: Towards Highly Structured Sparse Language Models
May 27, 2026Zhexuan Gu, Zixun Fu, Yancheng YuanPruning Sparsity
UNIQUE: Universal Top-k Sparse Attention for Training-free Inference and Sparsity-aware Training
May 26, 2026Keqi Deng, Shaoshi Ling, Ruchao Fan +1Dynamic Sparse Attention LLM Inference Efficiency
SparseOpt: Addressing Normalization-induced Gradient Skew in Sparse Training
May 26, 2026Mohammed Adnan, Rohan Jain, Tom Jacobs +4Batch Normalization Sparsity
Learning Sparse Compositional Functions with Norm-Constrained Neural Networks
May 25, 2026Shuo Huang, Lorenzo Fiorito, Lorenzo Rosasco +1Neural Network Sparsity
Relative Repairability: A Calibration-Based Diagnostic for High-Sparsity Post-Pruning Allocation
May 25, 2026Qishi Zhan, Liang He, Minxuan Hu +1Sparsity Resnet-18
Theoretical Analysis of Sparse Optimization with Reparameterization, Weight Decay, and Adaptive Learning Rate
May 24, 2026Huangyu Xu, Jingqin Yang, Qianqian Xu +1Sparsity Regularization Techniques
Inference Time Context Sparsity: Illusion or Opportunity?
May 22, 2026Sahil Joshi, Prithvi Dixit, Agniva Chowdhury +5LLM Inference Efficiency Dynamic Sparse Attention
Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers
May 22, 2026Shuhong Zheng, Michael Oechsle, Erik Sandström +3Self-Supervised Vision Transformers 3D Scene Understanding
Is Dimensionality a Barrier for Retrieval Models?
May 22, 2026Kiril Bangachev, Guy Bresler, Jonathan Kogan +1Retrieval Layer Embedding Dimension

Jul 1, 2026 · cs.CVJ/K move · Enter open · S save

Post-Training Pruning for Diffusion Transformers

Chengzhi Hu, Xuewen Liu, Jing Zhang, Mengjuan Chen+2

Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences

Diffusion Transformers (DiTs) have demonstrated impressive performance in image generation but suffer from substantial computational overhead and resource consumption. Post-training pruning offers a promising solution; however, due to DiTs' unique architectural design and parameter distribution, traditional pruning methods are inapplicable, leading to significant performance degradation. Specifically, prior methods developed for LLMs, which derive metrics through a series of approximations, amplify the relative contribution of weights in the saliency metric. In addition, weights in DiTs exhibit significantly larger magnitudes than those in LLMs. Moreover, existing pruning granularity overlooks variations in model structures. In this paper, we propose DiT-Pruning, which improves pruning performance by introducing customized saliency criteria and pruning granularity. We design a novel metric that balances the contributions of weights and activations from an energy-based perspective, enabling more effective identification of important elements. Furthermore, we observe distinct clustering patterns in the two-dimensional weight space. Accordingly, we adopt a clustering-aware pruning granularity, enabling effective sparse allocation. Extensive evaluations on various DiTs show that our method consistently preserves image quality, especially under high sparsity. For FLUX.1-dev at 512x512 resolution on MJHQ, DiT-Pruning achieves only a 0.001 loss in CLIP score at 50% sparsity, dramatically outperforming recent pruning methods.

Multimodal Diffusion Transformers Heterogeneous Pruning Scores Pruning Weight-Only Quantization Transformer Architectures Sparsity Image Generation Saliency

Open paper Ask about it PDF