LLM Compression

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 92% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 178

All topics

Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression
Jun 5, 2026Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi +1Mixed-Precision Quantization LLM Quantization
UiT The Arctic University of Norway · University of Oslo, Norway
SigmaScale: LLM Compression with SVD-based Low-Rank Decomposition and Learned Scaling Matrices
Jun 5, 2026Ernests Lavrinovics, Marco Letizia, Roy Janco +3Model Compression LLM Compression
Department of Computer Science, Aalborg University Copenhagen, Denmark · MaLGa-DIBRIS, University of Genoa, Genoa, Italy · INFN, Sezione di Genova, Genoa, Italy +2
YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition
Jun 4, 2026PSBC LLM Team, Huawei LLM Team, Ruihan Long +56LLM Compression LLM Inference Acceleration
LLMCodec: Adapting Video Codecs for Efficient Weight Compression of Large Language Models
Jun 4, 2026Rui Wang, Yan Zhao, Li Song +1Model Compression LLM Quantization
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University. Shanghai, China
Learned Subspace Compression for Communication-Efficient Pipeline Parallelism
Jun 3, 2026Paul Janson, Edouard Oyallon, Eugene BelilovskyLLM Compression Communication-Efficient Distributed Training
Concordia University · Mila Quebec AI Institute · CNRS, Sorbonne University
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts
Jun 3, 2026Jiangyang He, Shaolin Zhu, Deyi XiongMixture-of-Experts Pruning LLM Pruning
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China
Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines
Jun 2, 2026Justice Owusu Agyemang, Jerry John Kponyo, Kwame Opuni-Boachie Obour Agyekum +3LLM Compression Token Pruning
1VIA Cybersecurity Lab, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana · 2Quantum and Assistive Technologies Lab, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana
LLM Compression with Jointly Optimizing Architectural and Quantization choices
Jun 2, 2026Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi +1Mixed-Precision Quantization LLM Quantization
UiT The Arctic University of Norway · University of Oslo, Norway
Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression
Jun 2, 2026Artur Zagitov, Alexander Miasnikov, Maxim Krutikov +5LLM Compression Tensor Decomposition
BRAIn Lab, Moscow, Russia
LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection
Jun 2, 2026Liulu He, XuanAng Liu, Juntao Liu +8LLM Quantization LLM Compression
Nanjing University, China · China Mobile Research Institute, China · Xiaomi Corporation, China +2
From Layers to Submodules: Rethinking Granularity in Replacement-Based LLM Compression
Jun 1, 2026Elia Cunegatti, Marcus Vukojevic, Erik Nielsen +1LLM Pruning LLM Compression
University of Trento
HMPO: Hybrid Median-length Policy Optimization for Chain-of-Thought Compression
Jun 1, 2026Minghui Zheng, Hongxu Chen, Huimin Ren +8LLM Compression RL for Language Model Reasoning
Li Auto Inc.
Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction
Jun 1, 2026Yujia Tong, Yuxi Wang, Yunyang Wan +3LLM Quantization LLM Evaluation
Wuhan University of Technology · Nanyang Technological University
CRePE: Convolution-aware Relative Importance in Post-training Pruning with Efficient Search
Jun 1, 2026Cheonjun ParkLLM Pruning LLM Compression
Hankuk University of Foreign Studies
LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning
May 31, 2026Mengmeng Ji, Ravi Shanker Raju, Jonathan Lingjie Li +1LLM Compression LLM Fine-Tuning
SambaNova Systems, Inc. San Jose, CA, USA
Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limits
May 29, 2026Snigdha Chandan KhilarModel Compression LLM Compression
Independent Researcher
Kronecker Embeddings: Byte-Level Structured Token Representations for Parameter-Efficient Language Models
May 28, 2026Rohan ShravanLanguage Model Pretraining LLM Compression
The School of AI Bengaluru, India
ConMoE: Expert-Pool Consolidation via Prototype Reassignment for MoE Compression
May 28, 2026Yilun Yao, Jiaming Pan, Elsie Dai +3Mixture-of-Experts Language Models LLM Compression
Peking University
Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization
May 28, 2026Junlin He, Yihong Tang, Tong Nie +5LLM Pruning LLM Compression
The Hong Kong Polytechnic University, Hong Kong SAR, China · McGill University, Montreal, QC, Canada
Apertus LLM Family Expansion via Distillation and Quantization
May 27, 2026Andrei Panferov, Davit Melikidze, Martin Jaggi +1LLM Quantization LLM Compression
ISTA · EPFL · ETH Zürich +1
Knowledge Offloading: Decomposing LLMs into Sparse Backbones and Memory Modules
May 27, 2026Karim Galliamov, Rochelle Choenni, Ivan TitovLLM Pruning LLM Compression
University of Amsterdam · University of Edinburgh
Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor
May 27, 2026Guoxin Ma, Yibing Liu, Chengzhengxu Li +7LLM Compression Long-Context Inference
Baidu Inc., Beijing, China · Xi’an Jiaotong University · City University of Hong Kong +1
Pruning and Distilling Mixture-of-Experts into Dense Language Models
May 27, 2026Junhyuck Kim, Jihun Yun, Haechan Kim +3Mixture-of-Experts Pruning LLM Pruning
1KRAFTON · 2KAIST
ZipRL: Adaptive Multi-Turn Context Compression with Hindsight Response Replay
May 27, 2026Zhexin Hu, Li Wang, Xiaohan Wang +4LLM Compression Reinforcement Learning with Verifiable Rewards
1Meituan · Institute of Software, Chinese Academy of Sciences
Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts
May 27, 2026Liu O. Martin, Lucas Bandarkar, Nanyun PengMixture-of-Experts Pruning LLM Pruning
University of California, Los Angeles
Locality-Aware Redundancy Pruning for LLM Depth Compression
May 27, 2026Vincent-Daniel Yun, Youngrae Kim, Woosang Lim +3LLM Pruning LLM Compression
University of Southern California, United States · Neural Superintelligence Lab, MODULABS, Republic of Korea · Seoul National University, Republic of Korea +1
Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs
May 26, 2026Wenhui Tan, Minghao Li, Xiaoqian Ma +5Multi-Token Prediction LLM Compression
Gaoling School of Artificial Intelligence, Renmin University of China · AI Platform, Xiaohongshu Inc. · University of Electronic Science and Technology of China +1
AGORA: Adapter-Grounded Observation-Action Retention for Inference-Free Prompt Compression in LLM Agents
May 26, 2026Haoran Zhang, Zhaohua SunLLM Compression LLM Agents
AI Agent Technologies (Hong Kong) Limited · Department of Mechanical Engineering, The University of Hong Kong
Dense2MoE: Pushing the Pareto Frontier of On-Device LLMs via Unified Pruning and Upcycling
May 26, 2026Fengfa Li, Hongjin Ji, Yifeng Ding +2Mixture-of-Experts Pruning LLM Pruning
Li Auto · The Chinese University of Hong Kong, Shenzhen
The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works
May 25, 2026Guanghui Wang, Kaiwen Lv Kacuila, Zhiyong Yang +5LLM Compression Language Model Distillation
School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China · Alibaba Group, Hangzhou, China · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China +2

Jun 5, 2026, cs.AIJ/K move · Enter open · S save

Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression

Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi, Phuong Hoai Ha

UiT The Arctic University of Norway · University of Oslo, Norway

Mixed-Precision Quantization LLM Quantization LLM Pruning LLM Compression Post-Training Quantization Neural Network Compression

Open paper AskPDF arXiv

Recently, the efficiency of Large Language Models (LLMs) deployment has become a critical concern in practical applications. While post-training quantization (PTQ) and structural pruning are established techniques for reducing memory footprint and inference latency, most existing PTQ approaches optimize quantization errors on a per-layer basis, overlooking how errors accumulate and propagate through the network, often resulting in suboptimal solutions. Traditional pipelines also tend to apply pruning and quantization in isolation or sequentially, further compounding sub-optimality. We introduce a novel end-to-end framework that addresses these limitations in two key ways. First, we propose a novel mixed-precision PTQ strategy that directly minimizes global error propagation across the entire model, rather than isolating layer-wise errors. Building on this, we develop a novel joint optimization approach that simultaneously learns structural pruning decisions and mixed-precision quantization policies within a unified search space. Extensive experiments show that, at ultra-low precisions (1-3 bits), our quantization method reduces WikiText perplexity by up to 21% compared to state-of-the-art (SoTA) weight-activation quantization baselines. Against leading weight-only quantization methods, it achieves up to 59% and 85% lower perplexity on WikiText and C4, respectively. Compared to the SoTA joint pruning-and-quantization techniques, our proposed method delivers superior perplexity and reasoning performance at ultra-low bits.

Momentum

Related topics

Latest papers 178