Transformer Pruning

Latest papers

34 papers12 papers in the last four weeks, up 140% on the four weeks before. 0.1% of all new papers.

TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
Oct 19, 2026Liana Patel, Rishabh Ranjan, Ion Stoica +2Token Pruning LLM Inference Acceleration
Stanford University · UC Berkeley
BMuon: Budgeting Muon's Spectral Geometry for Structured Compressibility
Oct 16, 2026Rida Assalouh, Tom Jacobs, Rebekka Burkholz +2Muon Optimizer Structured Sparsity
Inria, Paris, France · ´Ecole polytechnique, Institut Polytechnique de Paris, Palaiseau, France · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany +3
Your Transformer Is Secretly Elastic
Oct 14, 2026Walter Simoncini, Dheeraj Varghese, Gertjan J. Burghouts +1Elastic Neural Networks Transformer Pruning
University of Amsterdam · TNO, Intelligent Imaging
Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too
Oct 13, 2026Mehdi Makni, Natalia Ponomareva, Rahul MazumderLLM Pruning Neural Network Pruning
Operations Research Center, Massachusetts Institute of Technology · Google Research
Retraining-Free Depth Pruning of Vision Transformers: A Cross Domain Study of Layer Redundancy
Oct 10, 2026Alessandro Viespoli, Loris NanniTransformer Pruning Efficient ViTs
University of Padua
Tabular Foundation Models Are Effectively Shallow
Oct 9, 2026Irene Cannistraci, Julia E. VogtTransformer Pruning Tabular Foundation Models
Department of Computer Science ETH Zurich Switzerland
EntroPrefill: Renyi-Guided Context Pruning with Conditional Stability Guarantees for Retrieval-Augmented Generation
Oct 7, 2026Inbasekaran SToken Pruning Transformer Pruning
SRM Institute of Science and Technology, India
A Self-Pruning Transformer: Extreme KV-Cache Compression with Universal Attention
Oct 6, 2026Davis Wertheimer, Haochen Shen, Ahan Gupta +6KV-Cache Compression Transformer Attention
IBM Research, USA · SSAIL Lab, University of Illinois Urbana-Champaign
QK-Wanda: Coupling Queries and Keys for Unstructured Pruning
Oct 1, 2026Ivan Ilin, Peter RichtárikLLM Pruning Neural Network Pruning
KAUST
DIET: Deletion-response Expert Trimming for Video Diffusion Transformers
Sep 29, 2026Jiachang Zhang, Teng Hu, Bohao Feng +4Mixture-of-Experts Pruning Video Diffusion Models
Xi’an Jiao Tong University, China · Shanghai Jiao Tong University, China · Alibaba Token Hub, Alibaba Group, China +1
Does the VGGT Family Need All Its Layers?
Sep 29, 2026Fengyi Zhang, Holger Caesar, Xiangyu Sun +3Multi-View 3D Reconstruction Transformer Pruning
The University of Queensland · Delft University of Technology · Harbin Institute of Technology
Output-aware Residual Stream Pruning for Large Language Models
Sep 28, 2026Chayne Thrash, Kevin Chen, Soheil KolouriLLM Pruning LLM Compression
Department of Computer Science Vanderbilt University Nashville, TN 37235, USA
Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery
Sep 23, 2026Rasmus Aagaard, Nicki Skafte DetlefsenSpeech Foundation Models Automatic Speech Recognition
Technical University of Denmark · Laerdal Medical
Forward-Free LLM Depth Pruning via Weight Redundancy
Sep 9, 2026Vincent-Daniel Yun, Woosang LimLLM Pruning LLM Inference Acceleration
University of Southern California, United States · 2Seoul National University, Republic of Korea
TAP-Path: Task-Adaptive Structural and Token Pruning for Efficient and Trustworthy Pathology Foundation Models
Sep 3, 2026Mehedi Hasan, Ashfak Yeafi, Md Khairul IslamComputational Pathology Pathology Foundation Models
Department of Computer Science and Engineering, Brac University, Dhaka, Bangladesh · EEE, Khulna University of Engineering & Technology, Khulna, Bangladesh · Mathematics and Computer Science, Hobart and William Smith Colleges, Geneva, NY, USA
XMerge: Cross-Axis Selection and Reconstructive Layer Merging for LLM Depth Compression
Sep 2, 2026Jundong Hu, Shekar RamachandranTransformer Pruning LLM Compression
PayPal AI
SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs
Aug 25, 2026Ali Bahri, Hang Li, Hongliang Li +1LLM Pruning LLM Compression
Huawei Noah’s Ark Lab, Canada · Huawei Noah’s Ark Lab, Hong Kong SAR, China
Resource-Efficient Pruning for Transformer via Low-Rank Importance Estimation
Aug 25, 2026Peng Liu, Huibing Zeng, Yiqun Zhang +2LLM Pruning Transformer Pruning
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China · College of Information and Artificial Intelligence, Yangzhou University, Yangzhou 225000, China
Magnitude Profile Pruning: Calibration-Free Structured Attention Head Removal for Transformer Compression
Aug 11, 2026Kasun Dewage, Marianna Pensky, Heranga K. Rathnasekara +1LLM Compression Transformer Pruning
University of Central Florida Orlando, Florida, USA · Old Dominion University Norfolk, Virginia, USA
CausalGate: Causal Importance Distillation for Transformer Module Pruning
Jul 21, 2026Kiran Nair, Smriti Regmi, Rodrigue RizkEfficient Transformer Inference LLM Pruning
USD Artificial Intelligence Research Lab, Department of Computer Science University of South Dakota, Vermillion, SD 57069, USA
CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning
Jul 20, 2026Zhiren Gong, Zihao Zeng, Zijie Wang +3LLM Pruning LLM Compression
College of Computing and Data Science, Nanyang Technological University, Singapore · Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore
PALS: Percentile-Aware Layerwise Sparsity for LLM Pruning
Jul 8, 2026Yazdan Jamshidi, Alexey ShvetsLLM Pruning LLM Compression
Palo Alto Networks
CascadeFormer: Depth-Tapered Transformers Motivated by Gradient Fan-in Asymmetry
Jun 25, 2026Huzama Ahmad, Cao Viet Hai Nam, Se-Young YunEfficient Transformer Inference Transformer
KAIST
PrunePath: Towards Highly Structured Sparse Language Models
May 27, 2026Zhexuan Gu, Zixun Fu, Yancheng YuanMixture-of-Experts Pruning LLM Pruning
Department of Applied Mathematics, The Hong Kong Polytechnic University
Prune, Update and Trim: Robust Structured Pruning for Large Language Models
May 18, 2026Diego Coello de Portugal Mecke, Tom Hanika, Lars Schmidt-ThiemeLLM Pruning LLM Inference Acceleration
ISMLL & DARC VWFS · University of Hildesheim · Hildesheim, Germany +1
No Free Swap: Protocol-Dependent Layer Redundancy in Transformers
May 15, 2026Gabriel GarciaTransformer Transformer Pruning
Independent Researcher
Ghosted Layers: Unconstrained Activation Alignment for Recovering Layer-Pruned LLMs
May 15, 2026Vincent-Daniel Yun, Junhyuk Jo, Sai Praneeth Karimireddy +1LLM Pruning LLM Compression
University of Southern California, USA · Inha University, Republic of Korea
MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
Apr 14, 2026Gabriel Afriat, Xiang Meng, Shibal Ibrahim +2LLM Pruning Neural Network Pruning
Operations Research Center Massachusetts Institute of Technology · Google · OpenAI +1
Structural Sensitivity in Compressed Transformers: Relative Error Propagation and Layer Removal
Mar 22, 2026Abhinaba Basu, Kumkum Basu, Koushik DebTransformer Pruning LLM Compression
Indian Institute of Information Technology Allahabad, India · National Institute of Electronics and Information Technology (NIELIT), India · Indian Institute of Technology Patna, India +2
ROSE: Reordered SparseGPT for More Accurate One-Shot Large Language Models Pruning
Mar 6, 2026Mingluo Su, Huan WangLLM Pruning Neural Network Pruning
Westlake University

Oct 19, 2026J/K move · Enter open · S save

TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning

Liana Patel, Rishabh Ranjan, Ion Stoica, Matei Zaharia+1

Stanford University · UC Berkeley

Token Pruning LLM Inference Acceleration Long-Context Language Model Inference Transformer Pruning LLM Context Compression

Open paper AskPDF arXiv

Expert human readers process large inputs (e.g., code repositories and long documents) by skimming to closely read only what matters. On the other hand, language models process all input tokens through each model layer, resulting in significant processing costs for long-context inputs. We introduce TokenGate, a small, drop-in component that extends existing trained models to enable end-to-end learnable token pruning over long-context inputs for fast and memory-efficient prefill. To enable end-to-end learning despite the non-differentiable top-K operator for token pruning, we introduce token-gated attention, a mechanism inspired by Mixture-of-Experts that re-weights attention logits using token importance scores. Across diverse tasks, TokenGate achieves

1.4

11.3\times

faster time-to-first token (TTFT) and

1.5

3.4\times

lower peak memory at no accuracy degradation for prefill in transformers. Compared to the best-performing prior methods for token-pruning, TokenGate gives up to

6.4\times

faster prefill and

2\times

lower peak memory. TokenGate natively extends both standard transformers (e.g., Qwen3) and hybrid-attention models (e.g., Qwen3.5) with simple lightweight adaptation comparable to standard parameter-efficient fine-tuning. We make our code available at: https://github.com/tokengate-models/TokenGate.