Post-Training Quantization

Also known as PTQ

Ask about this topic

Overview

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

CardsList

360 papers70 papers in the last four weeks, up 46% on the four weeks before. 0.4% of all new papers.

  1. Leech Lattice Vector Quantization for Efficient LLM Compression

    Mar 11, 2026Tycho F. A. van der Ouderaa, Mart van Baalen, Paul Whatmough +1LLM QuantizationLLM Compression

    Qualcomm AI Research

  2. Quantization Robustness of Monotone Operator Equilibrium Networks

    Mar 11, 2026James Li, Philip H. W. Leong, Thomas ChaffeyQuantization RobustnessNeural Network Quantization

    School of Electrical and Computer Engineering, The University of Sydney, NSW, Australia

  3. SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization

    Mar 9, 2026Yeonsik Park, Hyeonseong Kim, Seungkyu ChoiLLM QuantizationPost-Training Quantization

    Kyung Hee University · Yonsei University

  4. Evolution Strategy-Based Calibration for Low-Bit Quantization of Speech Models

    Mar 9, 2026Lucas RakotoarivonyLow-Bit QuantizationActivation Quantization

    Thales, cortAIx Labs, 1 Av. Augustin Fresnel, 91120 Palaiseau, France

  5. Post Training Quantization for Efficient Dataset Condensation

    Mar 7, 2026Linh-Tam Tran, Sung-Ho BaeDataset CondensationPost-Training Quantization

    Kyung Hee University, Republic of Korea

  6. WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization

    Mar 5, 2026Egor Lifar, Semyon Savkin, Or Ordentlich +1Neural Network QuantizationLLM Quantization

    MIT · Independent Researcher · Hebrew University of Jerusalem

  7. MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models

    Mar 5, 2026Lulu Hu, Wenhu Xiao, Xin Chen +4Post-Training QuantizationNeural Network Quantization

    Alibaba Cloud Computing, Alibaba Group

  8. Dissecting Quantization Error: A Concentration-Alignment Perspective

    Mar 4, 2026Marco Federici, Boris van Breugel, Paul Whatmough +1LLM Quantization4-Bit Quantization

    Qualcomm AI Research

  9. Activation Outliers in Transformer Quantization: Reproduction, Statistical Analysis, and Deployment Tradeoffs

    Mar 4, 2026Pranav Kumar KaliaperumalActivation OutliersPost-Training Quantization

    M.S. Computer Science, University of Colorado Denver, Aurora, Colorado, USA

  10. Bielik-Q2-Sharp: A Comparative Study of Extreme 2-bit Quantization Methods for a Polish 11B Language Model

    Mar 4, 2026Jakub PrejznerQuantizationLLM Quantization

    BitSharp, Independent Researcher, Rzeszów, Poland

  11. Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization

    Feb 27, 2026Chenwei Jia, Baoting Li, Xuchong Zhang +3VLM QuantizationNeural Network Quantization

    State Key Laboratory of Human-Machine Hybrid Augmented Intelligence Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University

  12. QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Models

    Feb 23, 2026Jingxuan Zhang, Yunta Hsieh, Zhongwei Wan +5Efficient VLA Model InferenceVLA Model Post-Training

    The Ohio State University · University of Michigan · City University of Hong Kong

  13. MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM

    Feb 21, 2026Dongwei Wang, Jinhee Kim, Seokho Han +8LLM QuantizationNeural Network Quantization

    University of Arizona · Panasonic AI Lab · Duke University +2

  14. Joint Post-Training Quantization of Vision Transformers with Learned Prompt-Guided Data Generation

    Feb 21, 2026Shile Li, Markus Karmann, Onay UrfaliogluPost-Training QuantizationNeural Network Quantization

    vivo Tech Research GmbH · Torr Vision Group, Universtiy of Oxford

  15. SPQ: An Ensemble Technique for Large Language Model Compression

    Feb 20, 2026Jiamin Yao, Eren GultepeLLM CompressionLLM Quantization

    Dept. of Computer Science, Southern Illinois University Edwardsville, Edwardsville, USA

  16. S2D: Selective Spectral Decay for Quantization-Friendly Conditioning of Neural Activations

    Feb 16, 2026Arnav Chavan, Nahush Lele, Udbhav Bamba +3Activation OutliersActivation Quantization

    Amazon · Carnegie Mellon University

  17. Investigating Social Bias Changes in Quantized Language Models

    Feb 5, 2026Stanley Z. Hua, Sanae Lotfi, Irene Y. ChenLLM QuantizationSocial Bias in Language Models

    UC Berkeley & UCSF · Meta Superintelligence Labs

  18. MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models

    Feb 3, 2026Maximilian Kleinegger, Elvir Crnčević, Dan AlistarhLLM QuantizationLLM Inference Acceleration

    1Vienna University of Technology, Vienna, Austria · Institute of Science & Technology Austria (ISTA), Vienna, Austria · 3Red Hat AI, Boston, USA

  19. Benford's Law as a Distributional Prior for Post-Training Quantization of Large Language Models

    Jan 29, 2026Arthur Negrão, Pedro Silva, Vander L. S. Freitas +2LLM Quantization4-Bit Quantization

    Postgraduate Program in Computer Science Federal University of Ouro Preto · Computing Department Federal University of Ouro Preto

  20. Activation Sensitivity as a Unifying Principle for Post-Training Quantization

    Jan 15, 2026Bruce Changlong XuLLM QuantizationPost-Training Quantization

  21. ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs

    Jan 12, 2026Haoqian Meng, Yilun Luo, Yafei Zhao +3LLM Quantization4-Bit Quantization

    School of Computer Science and Technology, Tianjin University

  22. QuantKAN: A Unified Quantization Framework for Kolmogorov Arnold Networks

    Nov 24, 2025Kazi Ahmed Asif Fuad, Lizhong ChenQuantization-Aware TrainingKolmogorov-Arnold Networks

    Department of EECS · Oregon State University · Corvallis, OR 97331

  23. Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference

    Nov 19, 2025Kexin Chu, Dawei Xiang, Zixu Shen +3Mixed-Precision QuantizationGPU Acceleration

    University of Connecticut · University of California, Santa Cruz · Independent Researcher

  24. You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

    Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Weight-Only QuantizationMixed-Precision Quantization

    1Technion—Israel Institute of Technology · 2Intuit · 3Ben-Gurion University of the Negev +1

  25. Quantization Effects of Artificial Neural Networks for Embedded Edge-Computing Applications

    Nov 7, 2025Alperen Aksoy, Ilja Bekman, Vesselin Dimitrov +6Neural Network QuantizationBinary Neural Networks

    Peter Grünberg Institute (PGI), Integrated Computing Architectures (ICA | PGI-4), Forschungszentrum Jülich GmbH, Germany · Center for Particle Physics Siegen, Department für Physik, Universität Siegen, Germany · Faculty of Engineering, JARA-FIT Institute for Quantum Information, Forschungszentrum Jülich GmbH and RWTH Aachen University, Germany +1

  26. LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

    Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision QuantizationLLM Quantization

    University of Illinois Urbana-Champaign · Work done as an intern at HP Inc. · University of California, Los Angeles +1

  27. SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models

    Aug 5, 2025Yufei Xue, Yushi Huang, Lunjie Zhu +2Vision-Language ModelsVLM Quantization

    Institute of Artificial Intelligence (TeleAI), China Telecom · The Hong Kong University of Science and Technology

  28. Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method

    Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision QuantizationLLM Quantization

    Beihang University · Communication University of China · Beijing Jiaotong University

  29. A Comprehensive Evaluation on Quantization Techniques for Large Language Models

    Jul 23, 2025Yutong Liu, Cairong Zhao, Guosheng HuLLM QuantizationPost-Training Quantization

    School of Computer Science and Technology, Tongji University, Shanghai 201804, China · School of Engineering Math and Technology, University of Bristol, BS8 1QU, UK

  30. On Orthogonal Recurrent Neural Networks Quantization

    Feb 5, 2024Armand Foucault, Franck Mamalet, François MalgouyresRecurrent Neural NetworksNeural Network Quantization

    IMT · Institut de Mathématiques de Toulouse, UMR5219. Université de Toulouse, CNRS. UPS IMT, F-31062 Toulouse Cedex 9, France · Comue de Toulouse +1