LLM Compression

LLM: Large Language Model

LLM compression studies how to make language models smaller or cheaper to run while retaining useful abilities. Common approaches include low-bit quantization, pruning weights, layers, or experts, low-rank factorization, distillation into smaller models, and compressing prompts or KV caches. Methods increasingly use calibration data, sensitivity estimates, or adaptive budget allocation, often paired with specialized inference kernels to turn compact representations into practical speedups. A central challenge is that quality, reasoning, reliability, and actual runtime can change in different ways: fewer parameters or tokens do not always mean faster execution or preserved behavior.

Latest papers

283 papers44 papers in the last four weeks, up 52% on the four weeks before. 0.3% of all new papers.

SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skipping
Dec 15, 2025Yu-Chen Lu, Sheng-Feng Yu, Hui-Hsien Weng +5LLM Compression Low-Rank Compression
National Yang Ming Chiao Tung University · Macronix International Co., Ltd. · Skymizer Taiwan Inc. +1
Data-Free Pruning of Self-Attention Layers in LLMs
Dec 3, 2025Dhananjay Saikumar, Blesson VargheseLLM Inference Acceleration LLM Compression
School of Computer Science University of St Andrews United Kingdom
BudgetMem: Training-Free Selective Memory for Cost-Efficient Long-Context Processing in Language Models
Nov 7, 2025Chandra Vamsi Krishna Alla, Harish Naidu Gaddam, Manohar Kommi +1Memory-Efficient Optimization LLM Compression
University of Texas at Arlington
PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
Nov 6, 2025Yushu Zhao, Zheng Wang, Minjia ZhangLLM Compression Mixture-of-Experts Models
1Tsinghua University · †Work done while intern at UIUC. · University of Illinois Urbana-Champaign.
Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
Oct 7, 2025Ryan Solgi, Parsa Madinei, Jiayi Tian +4Model Compression LLM Compression
University of California-Santa Barbara, USA · Amazon, USA
PATCH: Learnable Tile-level Hybrid Sparsity for LLMs
Sep 27, 2025Younes Hourri, Mohammad Mozaffari, Maryam Mehri DehnaviLLM Pruning LLM Compression
Department of Computer Science University of Toronto
CompLLM: Compression for Long Context Q&A
Sep 23, 2025Gabriele Berton, Jayakrishnan Unnikrishnan, Son Tran +1LLM Compression Long-Context Language Model Inference
Amazon · Center For Research in Computer Vision, University of Central Florida
GeLaCo: An Evolutionary Approach to Layer Compression
Jul 14, 2025David Ponce, Thierry Etchegoyhen, Javier Del SerModel Compression LLM Compression
Fundación Vicomtech, Basque Research and Technology Alliance (BRTA) · University of the Basque Country UPV/EHU · TECNALIA. Basque Research and Technology Alliance (BRTA)
Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models
May 23, 2025Viktoriia Chekalina, Daniil Moskovskiy, Tatiana Matveeva +2Model Compression LLM Compression
1Fusion Brain · 2MSU · 3AXXX +2
A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
May 19, 2025Jitai Hao, Qiang Huang, Hao Liu +3Language Model Pretraining LLM Pruning
School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen · Baidu Inc. · Leiden University +1
Optimizing Singular Spectrum for Large Language Model Compression
Feb 20, 2025Dengjie Li, Tiancheng Shen, Yao Zhou +3Model Compression LLM Compression
A Survey of Transformer-based Language Models with Focus on Efficiency
May 15, 2024Wazib Ansar, Saptarsi Goswami, Amlan ChakrabartiLLM Compression Energy-Efficient ML
A. K. Choudhury School of IT University of Calcutta, Kolkata, India · Department of Computer Science Bangabasi Morning College Kolkata, India
Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models
Nov 8, 2023Rocktim Jyoti Das, Mukul Ranjan, Mingjie Sun +2LVLM Pruning LLM Pruning
Mohamed bin Zayed University of AI · Carnegie Mellon University

Dec 15, 2025J/K move · Enter open · S save

SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skipping

Yu-Chen Lu, Sheng-Feng Yu, Hui-Hsien Weng, Pei-Shuo Wang+4

National Yang Ming Chiao Tung University · Macronix International Co., Ltd. · Skymizer Taiwan Inc.+1

LLM Compression Low-Rank Compression

Open paper AskPDF arXiv

Large language models (LLM) have achieved remarkable performance across a wide range of tasks. However, their substantial parameter sizes pose significant challenges for deployment on edge devices with limited computational and memory resources. Low-rank compression is a promising approach to address this issue, as it reduces both computational and memory costs, making LLM more suitable for resource-constrained environments. Nonetheless, naïve low-rank compression methods require a significant reduction in the retained rank to achieve meaningful memory and computation savings. For a low-rank model, the ranks need to be reduced by more than half to yield efficiency gains. Such aggressive truncation, however, typically results in substantial performance degradation. To address this trade-off, we propose SkipCat, a novel low-rank compression framework that enables the use of higher ranks while achieving the same compression rates. First, we introduce an intra-layer shared low-rank projection method, where multiple matrices that share the same input use a common projection. This reduces redundancy and improves compression efficiency. Second, we propose a block skipping technique that omits computations and memory transfers for selected sub-blocks within the low-rank decomposition. These two techniques jointly enable our compressed model to retain more effective ranks under the same compression budget. Experimental results show that, without any additional fine-tuning, our method outperforms previous low-rank compression approaches by 7% accuracy improvement on zero-shot tasks under the same compression rate. These results highlight the effectiveness of our rank-maximized compression strategy in preserving model performance under tight resource constraints.