Post-Training Quantization

Also known as PTQ

Ask about this topic

Overview

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

CardsList

366 papers72 papers in the last four weeks, up 50% on the four weeks before. 0.4% of all new papers.

  1. MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference

    Jul 20, 2026Simla Burcu Harma, Danila Mishin, Zhengyuan Su +7Mixed-Precision QuantizationLLM Quantization

    EcoCloud, EPFL · Tsinghua University · MangoBoost Inc. +2

  2. Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models

    Jul 16, 2026Ryona NodaLLM QuantizationBinary Quantization

    Ryo Noda

  3. Quantize with Confidence? An Empirical Study of Quantization for Code Generation

    Jul 15, 2026Saima Afrin, Md. Zahidul Haque, Antonio MastropaoloLLM QuantizationCode Generation Evaluation

    AURA Lab · Department of Computer Science · William & Mary +1

  4. ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level

    Jul 15, 2026Chethan Reddy G. PLLM QuantizationTernary Quantization

  5. Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models

    Jul 13, 2026Peng Xia, Junbiao Pang, Muhammad Ayub SabirActivation QuantizationPost-Training Quantization

    School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China

  6. RDQ: Residual Distribution Quantization for Large Language Models

    Jul 11, 2026Prateek SinghLLM QuantizationPost-Training Quantization

  7. Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts

    Jul 10, 2026Renuka Oladri, Mohan Vamsi Varadaraju Priya, Jerry WuLLM QuantizationLLM Evaluation

    Science Academy University of Maryland College Park, MD, USA · Electrical and Computer Engineering University of Maryland College Park, MD, USA

  8. The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs

    Jul 9, 2026Baha Rababah, Cuneyt Gurcan Akcora, Carson K. LeungLLM EvaluationQuantization Sensitivity Analysis

    Department of Computer Science, University of Manitoba, Canada · AI Initiative, University of Central Florida, USA

  9. It Takes Few to TANGO: A Quantized Distributed Model for Binaural Speech Enhancement

    Jul 9, 2026Zahra Benslimane, Pierre Chouteau, Martyna Poreba +4Quantization-Aware TrainingSpeech Enhancement

    Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France · Université de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France

  10. A Quantized Native Runtime for On-Device Semantic Audio Generation

    Jul 9, 2026Matteo Spanio, Antonio RodàText-to-Audio GenerationControllable Music Generation

    Centro di Sonologia Computazionale (CSC) · Dept. of Information Engineering, University of Padova

  11. Closing the Null Space: Guidance-Aware Quantization for Classifier-Free Diffusion

    Jul 9, 2026Abdullah Al Shafi, Sumaiya Rahim SumaMixed-Precision QuantizationClassifier-Free Guidance

    Department of Computer Science and Engineering Khulna University of Engineering & Technology Khulna-9203, Bangladesh

  12. KronQ: LLM Quantization via Kronecker-Factored Hessian

    Jul 8, 2026Donghyun Lee, Yuhang Li, Ruokai Yin +1Mixed-Precision QuantizationLLM Quantization

    University of Southern California · Yale University

  13. Lyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic

    Jul 5, 2026Anis Hamadouche, Amir HussainLyapunov-Based ControlQuantization-Aware Training

    The School of Engineering & Physical Sciences Heriot-Watt University Edinburgh EH14 4AS, UK · SDAIA-KFUPM Joint Research Centre for Artificial Intelligence King Fahd University of Petroleum and Minerals Dhahran, Saudi Arabia

  14. HiFA4: Training-Free 4-bit FlashAttention on Ascend HIF4 NPUs for LLM Inference

    Jul 5, 2026Hui Dong, Yanzhao Li, Jie Gao +5Neural Network QuantizationLLM Quantization

    Huawei Technologies

  15. OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers

    Jul 2, 2026Donghyun Lee, Jitesh Chavan, Duy Nguyen +5Video Diffusion ModelsDiffusion Transformer

    Cantina Labs · University of Southern California · *Work done during an internship at Cantina Labs. +1

  16. SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models

    Jul 2, 2026Qi Lyu, Jiahua Dong, Baichen Liu +7Model CompressionVision-Language Models

    State Key Laboratory of Robotics and Intelligent Systems · Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences +2

  17. Boundary-Aware Quantization: Finite-Scale Decision Geometry of Neural Classifiers

    Jul 1, 2026O. M. KiselevWeight-Only QuantizationPost-Training Quantization

    Innopolis University, Innopolis, Republic of Tatarstan, Russian Federation

  18. Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization

    Jul 1, 2026Fei Wang, Chao Xue, Taoran Liu +3Mixed-Precision QuantizationLLM Quantization

    South China University of Technology · Ocean University of China · Shenzhen Campus of Sun Yat-sen University +1

  19. W4A4 Quantization for Inference on Wan2.2-I2V-A14B

    Jun 28, 2026Yidong Chen, Chengyu Shi, Jiahao LiuDiffusion Model QuantizationPost-Training Quantization

    Tsinghua University

  20. CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs

    Jun 25, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +2LLM QuantizationTernary Quantization

    1Intel Labs China

  21. Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models

    Jun 24, 2026Xinyu Lian, Walid Krichene, Beichen Huang +4LLM Inference EfficiencyLLM Quantization

    University of Illinois Urbana-Champaign · Microsoft · Anyscale +1

  22. GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generation

    Jun 22, 2026Jette Oberländer, Jan Finkbeiner, Catherine M. Schöfmann +1Mixed-Precision QuantizationLLM Quantization

    Fakultät für Informatik, RWTH Aachen, Aachen, 52074, Germany · Peter Grünberg Institut, Forschungszentrum Jülich GmbH, Jülich, 52425, Germany · Fakultät für Elektrotechnik und Informationstechnik, RWTH Aachen, Aachen, 52062, Germany

  23. HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models

    Jun 22, 2026Yuval Domb, Hadar Sackstein, Tomer SolbergLLM QuantizationDiffusion Model Quantization

  24. Efficient Network Inference via Hardware-Aware Architecture Search, Model Pruning & Quantization

    Jun 22, 2026Lucas Heublein, Mark Deutel, Axel Plinge +1Efficient Neural Network InferenceHardware-Aware NAS

    Fraunhofer Institute for Integrated Circuits IIS, 90411 Nürnberg, Germany

  25. ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers

    Jun 20, 2026Changjun Li, Runqing Jiang, Lian Xu +3Vision TransformerActivation Quantization

    School of Electronics and Communication Engineering, Shenzhen Campus, Sun Yat-sen University, Shenzhen, China · Computer Science and Software Engineering, The University of Western Australia, Perth, Australia · University of Oxford, Oxford, United Kingdom

  26. An Empirical Study of OpenPangu Quantization on Ascend NPUs

    Jun 19, 2026Tong Shi, Jiacheng Wang, Hui Xie +4LLM QuantizationAI Accelerator Inference

    State Key Laboratory of Complex & Critical Software Environment, Beihang University · School of Artificial Intelligence, Beihang University · School of Computer Science and Engineering, Beihang University

  27. Mix-QVLA: Task-Evidence-Aware Mixed-Precision Quantization of Vision-Language-Action Models

    Jun 17, 2026Navin Ranjan, Andreas SavakisMixed-Precision QuantizationVLM Quantization

    Rochester Institute of Technology Rochester, New York 14623, USA

  28. MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs

    Jun 15, 2026Yuanteng Chen, Peisong Wang, Zhilei Liu +9Mixed-Precision QuantizationLLM Quantization

    Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Zhongguancun Academy

  29. Shift-and-Sum Quantization for Visual Autoregressive Models

    Jun 15, 2026Jaehyeon Moon, Bumsub HamVisual Autoregressive ModelsAutoregressive Image Generation

    1Yonsei University · 2Articron

  30. Signed Symmetric Quantization for Few-Bit Integers

    Jun 12, 2026Ian Colbert, Eashan Dash, Pablo Monteagudo-Lago +5LLM QuantizationWeight-Only Quantization

    AMD