Neural Network Compression

Follow this topic Ask about this topic

Neural network compression studies how to represent, train, or run models with fewer parameters and less memory or computation, making them practical on devices with limited resources. Common approaches include pruning weights or neurons, quantization, low-rank and tensor factorization, parameter sharing, and knowledge distillation; some methods also compress activations exchanged during distributed training. Work spans small edge models and microcontrollers as well as large language, vision, and diffusion models, evaluated on tasks such as image classification, language modeling, and video coding. A central challenge is that smaller parameter counts or fewer theoretical operations do not always mean faster inference, and aggressive compression can harm accuracy, robustness, or task-specific behavior.

Latest papers

218 papers28 papers in the last four weeks, up 8% on the four weeks before. 0.2% of all new papers.

Activation Compression in LLMs: Theoretical Analysis and Efficient Algorithm
May 2, 2026Wen-Da Wei, Han-Bin Fang, Yang-Di Liu +3LLM Compression Gradient Compression
Nanjing University, Nanjing, China · Tsinghua University, Beijing, China · Huazhong University of Science and Technology, Wuhan, China +1
Auto-FlexSwitch: Efficient Dynamic Model Merging via Learnable Task Vector Compression
Apr 30, 2026Junqi Gao, Dazhi Zhang, Zhichang Guo +3Task Vectors Task Vector Merging
School of Mathematics, Harbin Institute of Technology, Harbin, P. R. China · Shanghai Artificial Intelligence Laboratory, Shanghai, P. R. China
Parameter-Efficient Architectural Modifications for Translation-Invariant CNNs
Apr 30, 2026Nuria Alabau-Bosque, Jorge Vila-Tomas, Paula Dauden-Oliver +2Neural Network Robustness Convolutional Neural Networks
Image Processing Lab, Universitat de València, Carrer del Catedr´atic Jos´e Beltr´an Martinez, Paterna, 46980, Spain.
Random Cloud: Finding Minimal Neural Architectures Without Training
Apr 29, 2026Javier Gil BlázquezNeural Architecture Search Feedforward Neural Networks
QYOLO: Lightweight Object Detection via Quantum Inspired Shared Channel Mixing
Apr 29, 2026Garvit Kumar Mittal, Sahil Tomar, Sandeep KumarEfficient Neural Network Inference Quantum-Inspired ML
Central Research Laboratory Bharat Electronics Limited Ghaziabad, India
Hierarchical Spatio-Channel Clustering for Efficient Model Compression in Medical Image Analysis
Apr 25, 2026Sisipho Hamlomo, Marcellin Atemkeng, Habte Tadesse Likassa +5Low-Rank Compression Low-Rank Approximation
Department of Mathematics, Rhodes University, Makhanda 6140, South Africa · Department of Statistics, Rhodes University, Makhanda 6140, South Africa · National Institute for Theoretical and Computational Sciences, Stellenbosch 7600, South Africa +8
Efficient VQ-QAT and Mixed Vector/Linear quantized Neural Networks
Apr 25, 2026Terry Gou, Puneet GuptaQuantization-Aware Training Vector Quantization
LTBs-KAN: Linear-Time B-splines Kolmogorov-Arnold Networks
Apr 23, 2026Eduardo Said Merin-Martinez, Andres Mendez-Vazquez, Eduardo Rodriguez-TelloEfficient Neural Network Inference Kolmogorov-Arnold Networks
Cinvestav, Unidad Guadalajara · Av. del Bosque 1145, El Bajio · Zapopan, Jalisco, 45017, Mexico +3
Leveraging Kernel Symmetry for Joint Compression and Error Mitigation in Edge Model Transfer
Apr 19, 2026Anis Hamadouche, Mathini SellathuraiNeural Network Compression
School of Engineering & Physical Sciences Heriot-Watt University Edinburgh EH14 4AS, UK
Towards Joint Quantization and Token Pruning of Vision-Language Models
Apr 19, 2026Xinqing Li, Xin He, Xindong Zhang +3Vision-Language Models VLM Quantization
VCIP, College of Computer Science, Nankai University · School of Computer Science and Engineering, Tianjin University of Technology · OPPO Research Institute +3
LASER: Low-Rank Activation SVD for Efficient Recursion
Apr 19, 2026Ege Çakar, Ketan Ali Raghu, Lia ZhengLow-Rank Matrix Decomposition Recurrent Neural Networks
Harvard University Cambridge, MA, USA
A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exits
Apr 16, 2026Nekane Fernandez, Ivan Valdes, Steven Van Vaerenbergh +2Efficient Inference Edge Inference
Nekane Fernandez, Ivan Valdes, Idoia de la Iglesia and Julen Arratibel are with the Dept. of Connected and Distributed Intelligence, Ikerlan Technology Research Centre, Arrasate, Spain · Nekane Fernandez and Steven Van Vaerenbergh are with the Dept. of Mathematics, Statistics and Computation, Universidad de Cantabria, Santander, Spain
Exploiting Correlations in Federated Learning: Opportunities and Practical Limitations
Apr 16, 2026Adrian Edin, Michel Kieffer, Mikael Johansson +1Communication-Efficient Distributed Training Gradient Compression
Department of Electrical Engineering (ISY), Linköping University, SE-581 83 Linköping, Sweden · CentraleSupélec, CNRS, L2S, Université Paris-Saclay, Gif-sur-Yvette, FR-91192, France · Division of Decision and Control Systems, School of EECS, KTH Royal Institute of Technology, SE-100 44 Stockholm, Sweden
OFA-Diffusion Compression: Compressing Diffusion Model in One-Shot Manner
Apr 14, 2026Haoyang Jiang, Zekun Wang, Mingyang Yi +6Diffusion Model Compression Model Compression
Renmin University of China Beijing, China · Alibaba Inc. Hangzhou, China · Independent researcher +1
ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism
Apr 13, 2026Alan Aboudib, Rodrigo Lopez Portillo A., Kalei Brady +1Activation Compression Communication-Efficient Distributed Training
Macrocosmos AI
Saccade Attention Networks: Using Transfer Learning of Attention to Reduce Network Sizes
Apr 13, 2026Marc EstafanousEfficient ViTs Neural Network Compression
Johns Hopkins University · Neurobaby Corporation
Engineering Resource-constrained Software Systems with DNN Components: a Concept-based Pruning Approach
Apr 11, 2026Federico Formica, Andrea Rota, Aurora Francesca Zanenga +4Neural Network Pruning Neural Network Compression
University of Bergamo, Bergamo, Italy
TinyNeRV: Compact Neural Video Representations via Capacity Scaling, Distillation, and Low-Precision Inference
Apr 10, 2026Muhammad Hannan Akhtar, Ihab Amer, Tamer ShanablehNeural Video Compression Neural Network Compression
Department of Computer Science & Engineering, American University of Sharjah, Sharjah, United Arab Emirates
SOLAR: Communication-Efficient Model Adaptation via Subspace-Oriented Latent Adapter Reparametrization
Apr 9, 2026Seyed Mahmoud Sajjadi Mohammadabadi, Xiaolong Ma, Lei Yang +2Model Compression Neural Network Compression
University of Nevada, Reno, Reno, NV, USA · Argonne National Laboratory, Lemont, IL, USA · University of Houston, Houston, TX, USA +1
Learning is Forgetting: LLM Training As Lossy Compression
Apr 8, 2026Henry C. Conklin, Tom Hosking, Tan Yi-Chern +5Language Model Pretraining LLM Compression
Princeton University · Cohere
SL-FAC: A Communication-Efficient Split Learning Framework with Frequency-Aware Compression
Apr 8, 2026Zehang Lin, Miao Yang, Haihan Zhu +9Activation Compression Communication-Efficient Distributed Training
School of Computer and Information Engineering, Xiamen University of Technology, Xiamen, 361000, China · School of Artificial Intelligence, Hebei University of Technology · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong +5
The Lifecycle of the Spectral Edge: From Gradient Learning to Weight-Decay Compression
Apr 8, 2026Yongzhong XuNeural Network Training Dynamics Grokking
Training Transformers in Cosine Coefficient Space
Apr 6, 2026Mohamed Amine BergachTransformer Neural Network Compression
Illumina, San Diego, CA, USA
Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression
Apr 5, 2026Longsheng Zhou, Yu ShenNeural Network Compression Efficient Neural Network Inference
Institute of Advanced Technology University of Science and Technology of China Hefei, China · Supercomputing Center University of Science and Technology of China Hefei, China
AutoCompress: Critical Layer Isolation for Efficient Transformer Compression
Apr 4, 2026Archit ThoratNeural Network Compression LLM Compression
Independent Researcher
Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression
Apr 2, 2026Ruoling Qi, Yirui Liu, Xuaner Wu +6LLM Compression Low-Rank Compression
Institute of Artificial Intelligence (TeleAI), China Telecom · Shanghai Jiao Tong University · University of Maryland +2
A Hierarchical Importance-Guided Multi-objective Evolutionary Framework for Deep Neural Network Pruning
Apr 1, 2026Zak Khan, Azam Asilian BidgoliNeural Network Pruning Neural Network Compression
Department of Computer Science and Physics, Wilfrid Laurier University, Waterloo, N2L 3C5, Ontario, Canada
Big2Small: A Unifying Neural Network Framework for Model Compression
Mar 31, 2026Jing-Xiao Liao, Haoran Wang, Tao Li +4Model Compression Neural Network Compression
Frontier of Artificial Networks (FAN) Lab, Department of Data Science, City University of Hong Kong, Kowloon, Hong Kong SAR, China · Hong Kong Applied Science and Technology Research Institute (ASTRI) · Sichuan Engineering Research Center for Big Data Visual Analytics, Sichuan University of Science and Engineering, Zigong, China +3
OneComp: One-Line Revolution for Generative AI Model Compression
Mar 30, 2026Yuma Ichikawa, Keiji Kimura, Akihiro Yoshida +11Hardware-Aware Quantization Model Compression
Fujitsu Limited · RIKEN Center for AIP · Institute of Science Tokyo +2
Decompose, Mix, Adapt: A Unified Framework for Parameter-Efficient Neural Network Recombination and Compression
Mar 28, 2026Nazia Tasnim, Shrimai Prabhumoye, Bryan A. PlummerNeural Network Compression Parameter-Efficient Fine-Tuning
Boston University · NVIDIA, Boston University

May 2, 2026J/K move · Enter open · S save

Activation Compression in LLMs: Theoretical Analysis and Efficient Algorithm

Wen-Da Wei, Han-Bin Fang, Yang-Di Liu, Jiang-Xin Shi+2

Nanjing University, Nanjing, China · Tsinghua University, Beijing, China · Huazhong University of Science and Technology, Wuhan, China+1

LLM Compression Gradient Compression Low-Rank Compression Activation Compression LLM Training Neural Network Compression+1

Open paper AskPDF arXiv

Training large language models (LLMs) is highly memory-intensive, as training must store not only weights and optimizer states but also intermediate activations for backpropagation. While existing memory-efficient methods largely focus on gradients and optimizer states, activation compression is less well established due to the lack of LLM-tailored theory and guarantees. In this work, we develop a theoretical framework showing that activation compression is safe for linear operators when activation compression is unbiased, but problematic for nonlinear ones. We further derive gradient variance bound and establish convergence guarantees for applying activation compression to all linear operators under the standard

L

-smoothness assumption, showing that it does not change the convergence rate. Guided by the theory, we propose an activation-gradient co-compression method that reuses low-rank activation factors to compress linear-layer gradients without extra computation or additional gradient error. We conduct extensive experiments on Qwen and LLaMA models using a pretraining benchmark and multiple fine-tuning benchmarks to validate our theory and demonstrate competitive performance of our method in both accuracy and compression efficiency. We provide our code in the supplementary material for reproducibility.