Neural Network Pruning

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 83% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 161

All topics

XMerge: Cross-Axis Selection and Reconstructive Layer Merging for LLM Depth Compression
Sep 2, 2026Jundong Hu, Shekar RamachandranLLM Compression Model Compression
PayPal AI
A Closed-Loop Evaluation of Capability Loss and Recovery in Compressed Driving Policies
Sep 1, 2026Ahmad Alfan Alfian Irfan, Nur Ahmad Khatim, Mansur AriefAutonomous Driving Safety Evaluation Knowledge Distillation
Information Technology, Universitas Muhammadiyah Yogyakarta, Indonesia · Division of Information Science, Nara Institute of Science and Technology (NAIST), Japan · Industrial and Systems Engineering (ISE) Department and IRC for Smart Mobility and Logistics (IRC-SML), King Fahd University of Petroleum and Minerals (KFUPM), Saudi Arabia
Residual Sparsification via Output Importance for Compressing Mixture-of-Experts LLMs
Sep 1, 2026Seungwoo Jung, Dohyeok Kwon, Seungmin Cha +4Mixture-of-Experts Language Models LLM Compression
Department of Computer Science and Engineering, Korea University · Department of Computer Science and Artificial Intelligence, Dongguk University
Functional Degeneracy in Neural Networks: Measurement and Pruning
Aug 31, 2026Maria Matveev, Pascal Esser, Ayush Bharadwaj +2Structured Pruning Neural Network Compression
Department of Mathematics, LMU Munich · Munich Center for Machine Learning (MCML) · Independent +3
Breaking the Compression Barrier: Cross-Architecture Compression Boundary Learning via Reverse Regrowth
Aug 17, 2026Zhaocen Liu, Satvik Praveen, Yi ShengNeural Network Compression Neural Network Pruning
University of British Columbia Vancouver, British Columbia, CA · University of South Florida Tampa, Florida, USA
Unifying Depth and Width Pruning for LLMs via Binary Knapsack Optimization
Aug 13, 2026Palaash Goel, Ayan Sengupta, Akshay Nambi +1LLM Pruning Structured Pruning
Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi · Department of Electrical Engineering, Indian Institute of Technology Delhi · Microsoft Research (MSR) India
TACTICL: Task-Aware Compression of Tabular ICL Models
Aug 11, 2026Mykhailo Koshil, Matthias Feurer, Katharina EggenspergerModel Compression Tabular Foundation Models
TU Dortmund University, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany
InterPruner: Interactive Structured Pruning via Taylor-Implicit Criterion and Language-Prior Modulator for Multimodal Object Detection
Aug 11, 2026Qi Ming, Zihan Yang, Shaoguang Huang +6Multispectral Object Detection Infrared Object Detection
Beijing University of Technology · Southwest University · China University of Geosciences Wuhan +5
Domain-Aware Pruning: Sparsity and Domain Generalization via Regularized Probabilistic Masking
Aug 9, 2026Parham Sazdar, Mostafa Tavassolipour, Reshad HosseiniDomain Generalization OOD Generalization
Department of Electrical and Computer Engineering, University of Tehran, Tehran, Iran
A continually expandable foundation model for brain MRI
Aug 8, 2026Michail Mamalakis, Carmen Jimenez-Mesa, Yonghao Li +8Medical Imaging Foundation Models Continual Learning
1*Cancer Research UK Cambridge Institute, University of Cambridge Li Ka Shing Centre, Robinson Way, Cambridge, CB2 0RE, Cambridgeshire, United Kingdom. · Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Ave, Cambridge, CB3 0FD, Cambridgeshire, United Kingdom. · Department of Psychiatry, University of Cambridge, Hills Road, Cambridge, CB2 2QQ, Cambridgeshire, United Kingdom. +7
Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Models
Aug 7, 2026Minseok Kang, Hyunwoo Kim, Chanyoung Kim +3Vision-Language Models Efficient VLM Inference
Department of Artificial Intelligence, Chung-Ang University, Seoul, Korea · School of Computer Science and Engineering, Soongsil University, Seoul, Korea · Department of Computer Science, Kookmin University, Seoul, Korea
The Sparsity Whisperer
Aug 6, 2026Linghao Kong, Inimai Subramanian, Micah Adler +3LLM Pruning LLM Inference Acceleration
MIT · IST Austria · IBM +1
APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning
Aug 6, 2026Sadegh Jafari, Mohiuddin Bilwal, Fan Zhou +2Model Compression Mixed-Precision Quantization
Department of Computer Science Iowa State University Ames, IA, USA · Department of Agricultural and Biosystems Engineering Iowa State University Ames, IA, USA
Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling
Aug 5, 2026Sanjith Senthil, Rishidev ChaudhuriRecurrent Neural Networks Biologically Plausible Learning
1The Harker School, San Jose, CA 95129, USA · Department of Neurobiology, Physiology and Behavior, University of California, Davis, Davis CA 95616 · Department of Mathematics, University of California, Davis, Davis CA 95616 +1
Rethinking Reservoir Pruning: A Dynamical Perspective for Echo State Networks
Aug 5, 2026Sudip Laudari, Puspa Raj AdhikariDynamic Neural Networks Time Series Forecasting
Independent Researcher. · Advanced Materials Science and Engineering, Sungkyunkwan University.
Understanding Fault Tolerance of Adversarially Robust Pruned Models
Aug 4, 2026Manali Dangarikar, Cory MerkelAdversarial Training Neural Network Robustness
Brain Lab Rochester Institute of Technology Rochester, NY, USA
SlimVLM: Sensitivity-aware Dynamic Structured Pruning with Adaptive Visual Token Selection for Efficient Vision-Language Models
Aug 4, 2026Yaozhi Wen, Jialong Guo, Zhenliang Ni +2Efficient VLM Inference Structured Pruning
Huawei Technologies
When Compression Scores Cannot Decide: Information Boundaries for Group-Robust LLM Pruning
Aug 3, 2026Andrew ZhangModel Selection Mixture-of-Experts Pruning
KTH Royal Institute of Technology
SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits
Aug 1, 2026Kazi Ahmed Asif Fuad, Lizhong ChenKolmogorov-Arnold Networks Neural Network Quantization
Department of EECS Oregon State University Corvallis, OR 97331
F-WANDA: Fisher-Reweighted Post-Training Pruning for Sustainable Deployment of Large Language Models
Aug 1, 2026Himanshu MishraLLM Pruning LLM Compression
University of British Columbia
Simplifying Neural Networks During Training
Jul 30, 2026Lorenzo Sciandra, Samuele Fonio, Roberto EspositoNeural Network Training Dynamics Neural Network Compression
Department of Computer Science, University of Turin, Turin, Italy
Prox: Training-Free FFN Activation Sparsity via Approximate Intermediate-Channel Salience in LLMs
Jul 30, 2026Jinyi Liu, Wei Chen, Pengyu Chen +4Activation Sparsity Structured Sparsity
Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · School of Computer Science, Nanjing University of Posts and Telecommunications
Lottery Tickets Are Not Deployment Tickets
Jul 29, 2026Bum Jun KimNeural Network Pruning
The University of Tokyo
Unified Static-Dynamic Pruning for Efficient LLM Inference
Jul 24, 2026Jinhyeok Kim, Yejoon Lee, Jaeyoung DoEfficient Neural Network Inference LLM Pruning
Seoul National University Seoul, South Korea
Hilbert Operator for Progressive Encoding (HOPE): A Mathematical Framework for Deconstructing Learned Representations in Deep Networks
Jul 23, 2026Hossein Mobahi, Peter L. BartlettNeural Network Compression Neural Network Pruning
Google DeepMind · University of California, Berkeley
Neural Feature Governance: Extending Atom Prevalence
Jul 23, 2026Idris Karel Seunda Ekwe, Patrick Tenga Shako, Ernest Parfait FokouéBayesian Neural Networks Uncertainty Quantification
African Institute for Mathematical Sciences (AIMS), Cameroon · Rochester Institute of Technology (RIT), US
Importance-Aware OBS Pruning for Diffusion Models
Jul 22, 2026Ba-Thinh Lam, Srijan Das, Hieu LeNeural Network Compression Neural Network Pruning
Department of Computer Science UNC Charlotte NC, USA
PRISM: Sensitivity-Aware PolynoMial PRuning for EffIcient Neural Network Encryption
Jul 20, 2026Sahaj Majavdia, Mahdi TaheriEfficient Neural Network Inference Homomorphic Encryption
Brandenburgische Technische Universität Cottbus-Senftenberg, Germany · Humboldt University of Berlin, Germany · Tallinn University of Technology, Tallinn, Estonia
CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning
Jul 20, 2026Zhiren Gong, Zihao Zeng, Zijie Wang +3LLM Pruning LLM Compression
College of Computing and Data Science, Nanyang Technological University, Singapore · Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore
Staged Depth-Pruning Distillation of a Flow-Matching Text-to-Speech Teacher: A Compact Hindi Speech Synthesizer
Jul 19, 2026Sivateja TrikutamTTS Synthesis Low-Resource TTS Synthesis

Sep 2, 2026, cs.LGJ/K move · Enter open · S save

XMerge: Cross-Axis Selection and Reconstructive Layer Merging for LLM Depth Compression

Jundong Hu, Shekar Ramachandran

PayPal AI

LLM Compression Model Compression Neural Network Pruning

Open paper AskPDF arXiv

Removing complete transformer layers preserves a standard serving architecture, but existing depth-compression methods can lose substantial quality, and the loss varies unpredictably across models. We introduce XMerge, a post-training method with two components. Cross-axis selection identifies a block with low relative-magnitude and angular hidden-state change, and local boundary reconstruction re-fits the adjacent surviving block to match the original two-block output. XMerge uses no task labels or end-to-end fine-tuning, and it introduces neither architectural changes nor additional inference-time parameters. Across seven Llama and Qwen backbones (0.5B-8B), five published baselines, and three layer-reduction levels, its advantage over baselines is largest at the most aggressive removal: at k=4 it ranks first on six of seven backbones on CORE (a 22-task aggregate) and, separately, on six of seven on MMLU (five of seven on both at once), while avoiding the large perplexity increases of several competing operators. In a task-level bootstrap, the 95% confidence intervals for the three largest CORE margins exclude zero; the remaining margins are consistent with ties. Across the 14 (model, regime) cells it is also the only evaluated operator that never collapses, ranking top-2 in both zero-shot and in-context regimes; on a first calibration probe (one backbone) it is the best-calibrated operator. Ablations show that local reconstruction provides most of the gain, while cross-axis fusion helps when the two selection axes disagree. The additional construction cost is recovered through per-token decode savings after roughly tens of thousands of requests.

Momentum

Related topics

Latest papers 161