LLM Compression

LLM: Large Language Model

LLM compression studies how to make language models smaller or cheaper to run while retaining useful abilities. Common approaches include low-bit quantization, pruning weights, layers, or experts, low-rank factorization, distillation into smaller models, and compressing prompts or KV caches. Methods increasingly use calibration data, sensitivity estimates, or adaptive budget allocation, often paired with specialized inference kernels to turn compact representations into practical speedups. A central challenge is that quality, reasoning, reliability, and actual runtime can change in different ways: fewer parameters or tokens do not always mean faster execution or preserved behavior.

Latest papers

283 papers44 papers in the last four weeks, up 52% on the four weeks before. 0.3% of all new papers.

DCKV: Learned Transform Coding of the Prompt KV Cache in Hybrid Language Models
Oct 27, 2026Egor Maximov, Aleksei Merkelov, Vasiliy Alekseev +5KV-Cache Compression KV-Cache Quantization
MIL Team · DAIMLD
Balanced State Subspaces for Mamba-2 State Compression
Oct 27, 2026Shang Tingyu, Cao WenfeiLLM Compression State Space Language Models
School of Mathematics and Statistics, Shaanxi Normal University, Xi’an 710119, China
Advancing SVD-based LLM Compression via Layer-Wise Error Model Search
Oct 23, 2026Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler +8LLM Compression Low-Rank Compression
Technical University of Munich, Germany · BMW Group, Germany · University of Glasgow, United Kingdom +1
Learning Structured Sparsity for Large Models: Bridging Model Compressibility and Efficient Inference
Oct 22, 2026Tianxiang Xia, Kai Lion, Niao He +3LLM Compression LLM Inference Acceleration
ETH Zürich, Zürich · Max Planck Institute for Intelligent Systems, Tübingen · Vienna University of Economics and Business, Vienna +1
Low-Bit N-Gram Embeddings for Language Models
Oct 22, 2026Stephen Chung, Wenyu DuLLM Compression Low-Bit Quantization
DualverseAI
Extreme Compression of Per-Layer Embeddings via Cross-Layer Latent Vector Quantization
Oct 22, 2026Jiacheng Liu, Yaxin Luo, Xiaohan Zhao +5Embedding Compression LLM Compression
Mohamed bin Zayed University of Artificial Intelligence · Westlake University · Zhejiang University +2
Compressing Onboard Language Models for Space Robots under a Behavioral-Fidelity Requirement
Oct 21, 2026Jia Deng, Zizhe Wang, Xin Lu +2LLM Compression LLM Quantization
Astroup Lab, Beijing, China
ACE-Compression: An Ability-Centric Evaluation of Compressed Large Language Models
Oct 20, 2026Jiahui Ning, Liang Chen, Yuchen Liang +2LLM Compression LLM Evaluation
City University of Hong Kong · Tongji University · Rensselaer Polytechnic Institute +1
Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models
Oct 19, 2026Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin +7Mixture-of-Experts Pruning Mixture-of-Experts Language Models
LinkedIn · MIT · Google
LOOP: Recovery-Aligned Iterative Layer Pruning via On-Policy Distillation
Oct 19, 2026Liang Xu, Shufan Shen, Zhongni Hou +4LLM Pruning Neural Network Pruning
State Key Lab. of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Meituan
N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models
Oct 15, 2026Donghoon Han, Minseop Jung, Dong Woo Kwak +2Memory Compression LLM Compression
Dnotitia Inc. · KAIST
Embeddings Are Overpriced? A Frozen 16 MB Vocabulary for Small and Ternary Language Models
Oct 15, 2026Sai Kiran BathulaEmbedding Compression Small Language Models
QLNI, New South Wales, Australia
From Long Context to Selective Memory:A Survey of Memory-Centric Compression for Large Language Models
Oct 15, 2026Chaogui GouLLM Context Management Memory-Efficient Inference
USTB
Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming
Oct 13, 2026Woosang Lim, Haneul Yoo, Vincent-Daniel Yun +1LLM Pruning LLM Compression
Seoul National University, Republic of Korea · New York University, United States · University of Southern California, United States
SLEM: Spectrum-Aware Latent Expert Merging
Oct 12, 2026Dohyeon Kim, Bedionita Soro, Bruno Andreis +1Language Model Merging Mixture-of-Experts Language Models
KAIST · University of Oxford · Slater Labs
The Fragility and Dynamics of LLM Pruning-Recovery Steps
Oct 11, 2026Luis Frentzen Salim, Komang Rinartha, Chia-Chun Wu +3LLM Pruning LLM Compression
Institute of Information Science, Academia Sinica · National Taiwan University of Science and Technology · AI Research Center, Hon Hai Research Institute
SARATHI: Decoupled Data-Free Probing and Adaptive Reconstruction for Structured LLM Pruning
Oct 11, 2026Sankar Behera, Dhruv Singh, Satyadev Ahlawat +1LLM Pruning Structured Pruning
Department of Computer Science and Engineering, IIT Jammu · Department of Mathematics, IIT Jammu · Department of Electrical Engineering, IIT Jammu
GA-SVD: Generation-aware Singular Value Decomposition for LLM Compression
Oct 11, 2026Shengkun Tang, Cong Zeng, Zongfang Liu +2LLM Compression Low-Rank Compression
MBZUAI · Technology Innovation Institute UAE · Zhejiang Univeristy +2
WUSPruner: Second-Moment-Optimal Transforms for Structured Pruning
Oct 11, 2026Shengkun Tang, Dan Alistarh, Zhiqiang ShenLLM Pruning Structured Pruning
MBZUAI · ISTA
Beyond Adjacent Layers: Graph-Guided Layer Fusion for Compressing Large Language Models
Oct 10, 2026Muhammad Usama, Yeoreum JungLLM Compression Language Model Merging
Braindeck Inc Seoul, Republic of Korea
zip2zip++: Towards Inference-Time Token Compression with Minimal Quality Loss
Oct 9, 2026Saibo Geng, Andrea Gentilini, Xinxian Ma +3Token Compression LLM Compression
EPFL · Technical University of Munich · Zuse School ELIZA +1
Spectral Weight Decay: Inducing Low-Rank Structure in Neural Network Weights
Oct 8, 2026Dmitrii Andriianov, Andrey Veprikov, Aleksandr BeznosikovWeight Decay Nuclear Norm Minimization
Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · SB AI Lab · Innopolis University
LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs
Oct 8, 2026Xiaobing Yu, Peijie Qiu, Jin Yang +5Language Model Editing Memory-Efficient Fine-Tuning
Yale University · Washington University in St. Louis · Icahn School of Medicine at Mount Sinai +1
Dynamics as Code: On Model Compression via Dynamic System
Oct 8, 2026Fan Gao, Wei Su, Juntong Fan +4Model Compression Neural Network Compression
School of Information Science and Engineering, Lanzhou University, Lanzhou, Gansu Province, China · Department of Data Science, City University of Hong Kong, Hong Kong, China SAR · Department of Mathematics, City University of Hong Kong, Hong Kong, China SAR +1
Language Modeling is Monotone Compression
Oct 8, 2026Noam Mazor, Andrew Morgan, Rafael PassLLM Compression Next-Token Prediction
New York University. · Cornell Tech. · Cornell Tech, Technion and Tel-Aviv University.
OrBIT: Structure-Guided Embedding Compression
Oct 7, 2026Yunied Puig, Amit Kumar JaiswalEmbedding Compression LLM Compression
Jay Chaudhry Software Innovation Centre Indian Institute of Technology (BHU) Varanasi, India
SemanticFold: Latent Sequence Compression SeparatesLanguage Modeling, Decodability, and Reasoning
Oct 7, 2026Mingyan Liu, Min HuangLLM Context Compression LLM Compression
Activation-Aware Weight Tensorization: A Calibration-Time Preconditioner for Tensor-Network LLM Compression
Oct 7, 2026Alessandro Beatini, Marco Maronese, Emanuele RodolàLLM Compression Tensor Networks
Sapienza University of Rome · Independent Researcher
Shared Low-rank Basis Factorization for Data-free Mixture-of-Experts Compression
Oct 7, 2026Tianxiao Cao, Jiahe Shao, Yuning Qiu +3LLM Compression Mixture-of-Experts Language Models
Kyoto University · The University of Tokyo · RIKEN AIP
A Self-Pruning Transformer: Extreme KV-Cache Compression with Universal Attention
Oct 6, 2026Davis Wertheimer, Haochen Shen, Ahan Gupta +6KV-Cache Compression Transformer Attention
IBM Research, USA · SSAIL Lab, University of Illinois Urbana-Champaign

Oct 27, 2026J/K move · Enter open · S save

DCKV: Learned Transform Coding of the Prompt KV Cache in Hybrid Language Models

Egor Maximov, Aleksei Merkelov, Vasiliy Alekseev, Ivan Krivorotov+4

MIL Team · DAIMLD

KV-Cache Compression KV-Cache Quantization LLM Compression

Open paper AskPDF arXiv

Hybrid language models keep a key-value (KV) cache only in their few full-attention layers, but this cache is still the part of a prompt's state that grows with the prompt and that must be stored or moved to reuse it. We present DCKV, a differentiable codec for the KV cache and the first KV-cache codec that is trained end to end. Its transform, its quantizer and its entropy models, which are conditioned on the prompt tokens and on already decoded neighbouring positions, are optimized together against the frozen language model, both for the code length that the entropy models assign and for the change of the model's output. On Qwen3.5-2B, DCKV stores a compressed token in more than 55 times fewer bytes than bf16 at a loss of at most 1.8 points on three long-context benchmarks, and in more than 100 times fewer bytes at a loss of at most 3.6 points.