LLM Quantization

LLM: Large Language Model

Overview

LLM quantization studies how to represent model weights, activations, and attention key-value caches with fewer bits while preserving useful behavior and making deployment more efficient. Post-training methods such as GPTQ and AWQ use calibration data to round weights, while quantization-aware training, mixed-precision allocation, rotations, and outlier handling aim to reduce errors that accumulate across layers. Work also co-designs number formats and GPU, CPU, and edge-device kernels, evaluating trade-offs with language-model perplexity, zero-shot and reasoning benchmarks, memory, latency, and energy. Four-bit inference is often practical, but performance at very low bit widths remains model- and task-dependent, and an open challenge is ensuring that gains in aggregate accuracy do not hide losses in reasoning, safety, fairness, or uncertainty.

Latest papers

CardsList

443 papers68 papers in the last four weeks, down 13% on the four weeks before. 0.5% of all new papers.

  1. SPQ: An Ensemble Technique for Large Language Model Compression

    Feb 20, 2026Jiamin Yao, Eren GultepeLLM CompressionLLM Quantization

  2. 1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization

    Feb 17, 2026Sohir Maskey, Constantin Eichenberg, Johannes Messner +1QuantizationLLM Quantization

  3. QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs

    Feb 11, 2026Kanghyun Noh, Jinheon Choi, Yulhwa KimLLM QuantizationLLM Compression

  4. Investigating Social Bias Changes in Quantized Language Models

    Feb 5, 2026Stanley Z. Hua, Sanae Lotfi, Irene Y. ChenLLM QuantizationSocial Bias in Language Models

  5. MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models

    Feb 3, 2026Maximilian Kleinegger, Elvir Crnčević, Dan AlistarhLLM QuantizationLLM Inference Acceleration

  6. Benford's Law as a Distributional Prior for Post-Training Quantization of Large Language Models

    Jan 29, 2026Arthur Negrão, Pedro Silva, Vander L. S. Freitas +2LLM Quantization4-Bit Quantization

  7. Preserving Fairness and Safety in Quantized LLMs Through Critical Weight Protection

    Jan 17, 2026Muhammad Alif Al Hakim, Alfan Farizki Wicaksono, Fajri KotoLLM QuantizationMultilingual Language Model Evaluation

  8. ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs

    Jan 12, 2026Haoqian Meng, Yilun Luo, Yafei Zhao +3LLM Quantization4-Bit Quantization

  9. Beyond local reconstruction: Geometry-preserving output alignment for binary LLM quantization

    Dec 25, 2025Dung Anh Hoang, Cuong Pham, Cuong Nguyen +3LLM QuantizationBinary Quantization

  10. Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models

    Nov 21, 2025Cuong Pham, Hoang Anh Dung, Cuong C. Nguyen +4LLM QuantizationPost-Training Quantization

  11. You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

    Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Neural Network QuantizationMixed-Precision Quantization

  12. LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

    Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision QuantizationLLM Quantization

  13. On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs

    Sep 22, 2025Rongguang Ye, Ming Tang, Edith C. H. NgaiMixed-Precision QuantizationLLM Quantization

  14. Fair-GPTQ: Bias-Aware Quantization for Large Language Models

    Sep 18, 2025Irina Proskurina, Guillaume Metzler, Julien VelcinLLM QuantizationSocial Bias in Language Models

  15. End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost

    Aug 21, 2025Qitao Tan, Xiaoying Song, Ningxi Cheng +11On-Device Language Model Fine-TuningLLM Quantization

  16. Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method

    Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision QuantizationLLM Quantization

  17. A Comprehensive Evaluation on Quantization Techniques for Large Language Models

    Jul 23, 2025Yutong Liu, Cairong Zhao, Guosheng HuLLM QuantizationPost-Training Quantization

  18. FPTQuant: Function-Preserving Transforms for LLM Quantization

    Jun 5, 2025Boris van Breugel, Yelysei Bondarenko, Paul Whatmough +1LLM QuantizationLLM Inference Acceleration

  19. Highly Efficient and Effective LLMs with Multi-Boolean Architectures

    May 28, 2025Ba-Hien Tran, Van Minh NguyenLLM QuantizationBinary Neural Networks

  20. FlexQuant: Elastic Quantization Framework for Locally Hosted LLM on Edge Devices

    Jan 13, 2025Yuji Chai, Mujin Kwun, David Brooks +1LLM QuantizationOn-Device Language Model Inference

  21. TernaryLLM: Ternarized Large Language Model

    Jun 11, 2024Tianqi Chen, Zhe Li, Weixiang Xu +6LLM QuantizationTernary Quantization