LLM Compression

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 92% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 178

All topics

FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression
Apr 22, 2026Ye Qiao, Yian Wang, Zhiheng Chen +2LLM Compression GPU Acceleration
University of California, Irvine Irvine, California, USA
Variance Is Not Importance: Structural Analysis of Transformer Compressibility Across Model Scales
Apr 22, 2026Samuel SalfatiLLM Compression Post-Training Quantization
fraQtl AI Research
GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Models
Apr 21, 2026Ziyang Wang, Jiangfeng Xiao, Chuan Xiao +3LLM Pruning LLM Compression
Beijing Institute of Technology, Zhuhai · Shenzhen University · Osaka University
Modular Representation Compression: Adapting LLMs for Efficient and Effective Recommendations
Apr 20, 2026Yunjia Xi, Menghui Zhu, Jianghao Lin +4LLM Compression Embedding Compression
Shanghai Jiao Tong University Shanghai, China · Huawei Noah’s Ark Lab Shanghai, China · Antai College of Economics and Management, Shanghai Jiao Tong University Shanghai, China
Predicting LLM Compression Degradation from Spectral Statistics
Apr 20, 2026Mingxue XuLLM Compression Low-Rank Matrix Decomposition
Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom
Neural Garbage Collection: Learning to Forget while Learning to Reason
Apr 20, 2026Michael Y. Li, Jubayer Ibn Hamid, Emily B. Fox +1LLM Compression Memory-Efficient Optimization
Stanford University
CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning
Apr 19, 2026Yangsong Lan, Hongliang Dai, Piji LiLLM Pruning LLM Compression
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China.
D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
Apr 18, 2026Junlin Li, Shuangyong Song, Guodong Du +6LLM Quantization LLM Compression
Harbin Institute of Technology, Shenzhen, China · TeleAI of China Telecom · The Hong Kong Polytechnic University +1
Optimizing Korean-Centric LLMs via Token Pruning
Apr 17, 2026Hoyeol Kim, Hyeonwoo KimMultilingual Language Model Evaluation LLM Pruning
College of Computing · Independent Researcher
Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting
Apr 17, 2026Chi Liu, Xin Chen, Xu Zhou +2LLM Compression Language Model Distillation
PayPal AI
Compressing Sequences in the Latent Embedding Space: $K$ -Token Merging for Large Language Models
Apr 16, 2026Zihao Xu, John Harvill, Ziwei Fan +3Token Merging LLM Compression
♢Rutgers University · ♡Mistral AI · ♠AWS AI Labs +3
When Less Latent Leads to Better Relay: Information-Preserving Compression for Latent Multi-Agent LLM Collaboration
Apr 14, 2026Yiping Li, Zhiyu An, Wan DuLLM Compression KV Caching
Department of Computer Science and Engineering University of California, Merced Merced, CA 95343
Effective Distillation to Hybrid xLSTM Architectures
Mar 16, 2026Lukas Hauzenberger, Niklas Schmidinger, Thomas Schmied +7LLM Compression Language Model Distillation
1NXAI · 2Johannes Kepler University Linz
Leech Lattice Vector Quantization for Efficient LLM Compression
Mar 11, 2026Tycho F. A. van der Ouderaa, Mart van Baalen, Paul Whatmough +1LLM Quantization LLM Compression
Qualcomm AI Research
Curvature-Weighted Capacity Allocation: A Minimum Description Length Framework for Layer-Adaptive Large Language Model Optimization
Mar 1, 2026Theophilus Amaefuna, Hitesh Vaidya, Anshuman Chhabra +1LLM Pruning LLM Compression
1Bellini College of Artificial Intelligence, Cybersecurity and Computing University of South Florida, Tampa, Florida, USA
QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs
Feb 11, 2026Kanghyun Noh, Jinheon Choi, Yulhwa KimLLM Quantization LLM Compression
Department of Semiconductor Convergence Engineering, Sungkyunkwan University · Department of Electrical and Computer Engineering, Sungkyunkwan University · Department of Semiconductor Systems Engineering, Sungkyunkwan University
ButterflyMoE: Compression-Scalable Ternary Experts via Structured Butterfly Orbits
Jan 20, 2026Aryan KarmoreTernary Quantization Mixture-of-Experts Language Models
Indian Institute of Information Technology, Nagpur
Data-Free Pruning of Self-Attention Layers in LLMs
Dec 3, 2025Dhananjay Saikumar, Blesson VargheseLLM Pruning LLM Compression
School of Computer Science University of St Andrews United Kingdom
BudgetMem: Training-Free Selective Memory for Cost-Efficient Long-Context Processing in Language Models
Nov 7, 2025Chandra Vamsi Krishna Alla, Harish Naidu Gaddam, Manohar Kommi +1LLM Compression Long-Context Language Model Inference
University of Texas at Arlington
PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
Nov 6, 2025Yushu Zhao, Zheng Wang, Minjia ZhangLLM Compression Mixture-of-Experts Models
1Tsinghua University · †Work done while intern at UIUC. · University of Illinois Urbana-Champaign.
Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
Oct 7, 2025Ryan Solgi, Parsa Madinei, Jiayi Tian +4Model Compression LLM Compression
University of California-Santa Barbara, USA · Amazon, USA
PATCH: Learnable Tile-level Hybrid Sparsity for LLMs
Sep 27, 2025Younes Hourri, Mohammad Mozaffari, Maryam Mehri DehnaviLLM Pruning LLM Compression
Department of Computer Science University of Toronto
CompLLM: Compression for Long Context Q&A
Sep 23, 2025Gabriele Berton, Jayakrishnan Unnikrishnan, Son Tran +1LLM Compression Long-Context Language Model Inference
Amazon · Center For Research in Computer Vision, University of Central Florida
GeLaCo: An Evolutionary Approach to Layer Compression
Jul 14, 2025David Ponce, Thierry Etchegoyhen, Javier Del SerModel Compression LLM Compression
Fundación Vicomtech, Basque Research and Technology Alliance (BRTA) · University of the Basque Country UPV/EHU · TECNALIA. Basque Research and Technology Alliance (BRTA)
Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models
May 23, 2025Viktoriia Chekalina, Daniil Moskovskiy, Tatiana Matveeva +2Model Compression LLM Compression
1Fusion Brain · 2MSU · 3AXXX +2
A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
May 19, 2025Jitai Hao, Qiang Huang, Hao Liu +3Language Model Pretraining LLM Pruning
School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen · Baidu Inc. · Leiden University +1
A Survey of Transformer-based Language Models with Focus on Efficiency
May 15, 2024Wazib Ansar, Saptarsi Goswami, Amlan ChakrabartiLLM Compression Energy-Efficient ML
A. K. Choudhury School of IT University of Calcutta, Kolkata, India · Department of Computer Science Bangabasi Morning College Kolkata, India
LLM Compression by Block Removal with Constrained Binary Optimization
Date pendingDavid Jansen, Roman Rausch, Ali Hashemi +2LLM Pruning LLM Compression
Multiverse Computing · Donostia International Physics Center Ikerbasque Foundation for Science

Apr 22, 2026, cs.LGJ/K move · Enter open · S save

FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression

Ye Qiao, Yian Wang, Zhiheng Chen, Hyoukjun Kwon+1

University of California, Irvine Irvine, California, USA

LLM Compression GPU Acceleration Vector Quantization LLM Inference Acceleration Product Quantization Post-Training Quantization+1

Open paper AskPDF arXiv

Compressing large language models (LLMs) for deployment on commodity GPUs remains challenging: conventional scalar quantization is limited to fixed bit-widths (e.g., 8/4/3-bit), offers only a few discrete compression points, and typically requires calibration data. We present FASQ (Flexible Accelerated Subspace Quantization), a calibration-free framework that applies product quantization to LLM weight matrices. By tuning two parameters, sub-vector size and codebook cardinality, FASQ exposes a continuous design space spanning 27-49% of the original FP16 model size, filling compression gaps that fixed-bit schemes cannot reach. On Meta-Llama-3-8B, FASQ surpasses 4-bit GPTQ and AWQ in accuracy (67.1-67.7 avg.) at 37-42% model size, with consistent results on Qwen3-8B and Qwen3.5-9B-Base. To make product quantization practical at inference time, we design custom CUDA kernels: a LUT-free direct-compute GEMV for decode and an output-stationary double-buffered LUT GEMM for prefill, both with split-K parallelism. On an RTX~3090, FASQ achieves 45.2 tok/s decode at effective 4-bit (2.56x memory reduction) and 51.8 tok/s at effective 3-bit (2.80x), both surpassing FP16 tensor-core performance (43.9 tok/s) and delivering 1.6 to 1.8x the throughput of AWQ, 2.5 to 2.5x of GPTQ, and 4.3 to 5x of RTN. FASQ is the only compressed method that accelerates decode beyond FP16, offering calibration-free compression, continuous size-quality trade-offs, and real-time inference on a single consumer GPU.

Momentum

Related topics

Latest papers 178