Overview

Low-bit quantization studies how to represent model weights, activations, attention caches, and training states with fewer bits, reducing memory use and enabling faster or more energy-efficient inference. Post-training methods use calibration data, error reconstruction, Hessian information, rotations, and outlier handling, while quantization-aware training adapts models to the errors they will encounter at deployment. Mixed precision and scalar or vector codebooks allocate limited precision to sensitive components, with specialized kernels and hardware formats needed to turn compression into real speed and energy gains. The central tension is that aggressive compression can damage generation, reasoning, or long-context behavior even when standard accuracy metrics look stable, and improvements in quantization quality do not always translate into faster end-to-end systems.

Latest papers

CardsList

254 papers23 papers in the last four weeks, down 49% on the four weeks before. 0.2% of all new papers.

  1. Scaling Laws for Precision in High-Dimensional Linear Regression

    Feb 22, 2026Dechen Zhang, Xuan Tang, Yingyu Liang +1High-Dimensional Linear RegressionScaling Laws

  2. Uniform error bounds for quantized dynamical models

    Feb 17, 2026Abdelkader Metakalard, Fabien Lauer, Kevin Colin +1System IdentificationGeneralization Bounds

  3. 1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization

    Feb 17, 2026Sohir Maskey, Constantin Eichenberg, Johannes Messner +1QuantizationLLM Quantization

  4. CoreQ: Mismatch-Corrected Successive Rounding for LLM Quantization

    Feb 5, 2026Seohyeon Cha, Huancheng Chen, Dongjun Kim +4LLM QuantizationPost-Training Quantization

  5. MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models

    Feb 3, 2026Maximilian Kleinegger, Elvir Crnčević, Dan AlistarhLLM QuantizationLLM Inference Acceleration

  6. QuantKAN: A Unified Quantization Framework for Kolmogorov Arnold Networks

    Nov 24, 2025Kazi Ahmed Asif Fuad, Lizhong ChenQuantization-Aware TrainingKolmogorov-Arnold Networks

  7. Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models

    Nov 21, 2025Cuong Pham, Hoang Anh Dung, Cuong C. Nguyen +4LLM QuantizationPost-Training Quantization

  8. Activation Quantization of Vision Encoders Needs Prefixing Registers

    Oct 6, 2025Seunghyeon Kim, Taesun Yeom, Jinho Kim +3Activation OutliersActivation Quantization

  9. LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

    Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision QuantizationLLM Quantization

  10. End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost

    Aug 21, 2025Qitao Tan, Xiaoying Song, Ningxi Cheng +11On-Device Language Model Fine-TuningLLM Quantization

  11. Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method

    Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision QuantizationLLM Quantization

  12. A Comprehensive Evaluation on Quantization Techniques for Large Language Models

    Jul 23, 2025Yutong Liu, Cairong Zhao, Guosheng HuLLM QuantizationPost-Training Quantization

  13. FPTQuant: Function-Preserving Transforms for LLM Quantization

    Jun 5, 2025Boris van Breugel, Yelysei Bondarenko, Paul Whatmough +1LLM QuantizationLLM Inference Acceleration