LLM Quantization

LLM: Large Language Model

Overview

LLM quantization studies how to represent model weights, activations, and attention key-value caches with fewer bits while preserving useful behavior and making deployment more efficient. Post-training methods such as GPTQ and AWQ use calibration data to round weights, while quantization-aware training, mixed-precision allocation, rotations, and outlier handling aim to reduce errors that accumulate across layers. Work also co-designs number formats and GPU, CPU, and edge-device kernels, evaluating trade-offs with language-model perplexity, zero-shot and reasoning benchmarks, memory, latency, and energy. Four-bit inference is often practical, but performance at very low bit widths remains model- and task-dependent, and an open challenge is ensuring that gains in aggregate accuracy do not hide losses in reasoning, safety, fairness, or uncertainty.

Latest papers

CardsList

443 papers68 papers in the last four weeks, down 13% on the four weeks before. 0.5% of all new papers.

  1. RUQuant: Towards Refining Uniform Quantization for Large Language Models

    Apr 5, 2026Han Liu, Haotian Gao, Changya Li +4LLM QuantizationActivation Quantization

  2. Improving Quantized Model Performance in Qualitative Analysis with Multi-Pass Prompt Verification

    Apr 4, 2026Aisvarya Adeseye, Jouni Isoaho, Adeyemi AdeseyeLLM QuantizationLow-Bit Quantization

  3. Fast NF4 Dequantization Kernels for Large Language Model Inference

    Apr 2, 2026Xiangbo Qi, Chaoyi Jiang, Murali AnnavaramLLM Inference AccelerationGPU Kernel Optimization

  4. AdaHOP: Fast and Accurate Low-Precision Training via Outlier-Pattern-Aware Rotation

    Apr 2, 2026Seonggon Kim, Alireza Khodamoradi, Pranathi Vasireddy +2Walsh-Hadamard TransformLow-Precision Training

  5. Adaptive Block-Scaled Data Types

    Mar 30, 2026Jack Cook, Hyemin S. Lee, Kathryn Le +4Neural Network QuantizationLLM Quantization

  6. GlowQ: Group-Shared LOw-Rank Approximation for Quantized LLMs

    Mar 26, 2026Selim An, Il hong Suh, Yeseong KimLLM QuantizationLLM Inference Acceleration

  7. SliderQuant: Accurate Post-Training Quantization for LLMs

    Mar 26, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +3LLM QuantizationPost-Training Quantization

  8. APreQEL: Adaptive Mixed Precision Quantization For Edge LLMs

    Mar 24, 2026Meriem Bouzouad, Yuan-Hao Chang, Jalil BoukhobzaHardware-Aware QuantizationLLM Quantization

  9. FAAR: Format-Aware Adaptive Rounding for NVFP4

    Mar 23, 2026Hanglin Li, Shuchang Tian, Chen Lin +2LLM Quantization4-Bit Quantization

  10. DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

    Mar 20, 2026Xiaoming Yu, Shize Tang, Guanghua Yu +4QuantizationLLM Quantization

  11. Weight Clustering Reveals the Importance of Relative Rank in Large Language Models

    Mar 18, 2026Borja Aizpurua, Sukhbinder Singh, Román OrúsLLM CompressionLLM Quantization

  12. BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization

    Mar 17, 2026Ji-Fu Li, Manyi Zhang, Xiaobo Xia +4LLM QuantizationNeural Network Quantization

  13. Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization

    Mar 17, 2026Francesco Pio Monaco, Elia Cunegatti, Flavio Vella +1LLM CompressionLLM Pruning

  14. Leech Lattice Vector Quantization for Efficient LLM Compression

    Mar 11, 2026Tycho F. A. van der Ouderaa, Mart van Baalen, Paul Whatmough +1LLM QuantizationLLM Compression

  15. The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training

    Mar 11, 2026Hengjie Cao, Zhendong Huang, Mengyi Chen +15LLM Quantization4-Bit Quantization

  16. TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Fly

    Mar 11, 2026Toshiaki Koike-Akino, Jing Liu, Ye WangLLM QuantizationNeural Network Quantization

  17. SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization

    Mar 9, 2026Yeonsik Park, Hyeonseong Kim, Seungkyu ChoiLLM QuantizationPost-Training Quantization

  18. Diagnosing FP4 inference: a layer-wise and block-wise sensitivity analysis of NVFP4 and MXFP4

    Mar 5, 2026Musa Cim, Burak Topcu, Mahmut Taylan KandemirQuantization Sensitivity AnalysisLLM Quantization

  19. Sparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsity

    Mar 5, 2026Di Zhang, Xun Wu, Shaohan Huang +9LLM Inference EfficiencyLLM Quantization

  20. WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization

    Mar 5, 2026Egor Lifar, Semyon Savkin, Or Ordentlich +1Neural Network QuantizationLLM Quantization

  21. Dissecting Quantization Error: A Concentration-Alignment Perspective

    Mar 4, 2026Marco Federici, Boris van Breugel, Paul Whatmough +1LLM Quantization4-Bit Quantization

  22. FreeAct: Freeing Activations for LLM Quantization

    Mar 2, 2026Xiaohao Liu, Xiaobo Xia, Manyi Zhang +6LLM QuantizationActivation Quantization

  23. pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training

    Feb 26, 2026Wenzheng Zhang, Bingzheng Liu, Yang Hu +3LLM QuantizationLow-Bit Quantization

  24. MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM

    Feb 21, 2026Dongwei Wang, Jinhee Kim, Seokho Han +8LLM QuantizationNeural Network Quantization