Post-Training Quantization

Also known as PTQ

Ask about this topic

Overview

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

CardsList

359 papers84 papers in the last four weeks, up 133% on the four weeks before. 0.5% of all new papers.

  1. BiLoCo: Binary Low-Rank Corrections for Post-Training Quantization

    Oct 23, 2026David Jin, Beshr IslamBouli, Tarushii Goel +2LLM QuantizationPost-Training Quantization

    Massachusetts Institute of Technology

  2. MPQ: A Message-Passing View of Post-Training Quantization

    Oct 23, 2026Yanshuo Chen, Chenxi Liu, Reza Shirkavand +1QuantizationLLM Quantization

    Department of Computer Science, University of Maryland, College Park, MD 20742

  3. Evaluating the Applicability of LLM Inference Optimizations on Apple Silicon

    Oct 22, 2026Kyrylo Yemets, Andrii Yaroshevych, Victor Muryn +5LLM InferenceOn-Device Language Model Inference

    MacPaw AI&Research, Kyiv, Ukraine · Department of Artificial Intelligence, Lviv Polytechnic National University, Lviv, Ukraine · Institute of Mathematics of NAS of Ukraine, Kyiv, Ukraine

  4. ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models

    Oct 22, 2026Jiaqi Chen, Xianglong Yan, Chengzhu Bao +2Mixture-of-Experts QuantizationPost-Training Quantization

    Shanghai Jiao Tong University

  5. PACE: Periodic and Channel-Aligned Encoding of Four-Bit Recurrent States

    Oct 21, 2026Patrick Haller, Filipe Laitenberger, Alan AkbikRecurrent-State QuantizationLLM Quantization

    Humboldt-Universität zu Berlin

  6. Compressing Onboard Language Models for Space Robots under a Behavioral-Fidelity Requirement

    Oct 21, 2026Jia Deng, Zizhe Wang, Xin Lu +2LLM CompressionLLM Quantization

    Astroup Lab, Beijing, China

  7. Extreme Low-State Quantized Transformer for Electricity-Load Forecasting

    Oct 20, 2026Muhammad Sabbir Alam, Harindra Sandun Mavikumbure, Victor Cobilean +2Low-Bit QuantizationQuantization-Aware Training

    Department of Mechanical and Nuclear Engineering, Virginia Commonwealth University, Richmond, VA, 23284, USA · Department of Computer Science, Virginia Commonwealth University, Richmond, VA, 23284, USA · Department of Electrical and Computer Engineering, Virginia Commonwealth University, Richmond, VA, 23284, USA

  8. Beyond 4 bits: Data-Centric Post-training Quantization and Recovery

    Oct 20, 2026Rhea Sanjay Sukthanker, Riccardo Grazzi, James Hensman +1LLM QuantizationPost-Training Quantization

    Applied Sciences Group (ASG), Microsoft · Microsoft Research

  9. When PTQ Calibration Helps or Hurts: Dose-Driven Collapse and Validation-Gated Abstention in Quantized Text Embedders

    Oct 19, 2026Hyojung Han, Ryangkyung Kang, Youngeun Kim +1Post-Training QuantizationNeural Network Quantization

    ThakiCloud · Korea University

  10. LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models

    Oct 19, 2026Desheng Xu, Hanglin Li, Tianchen Song +2LLM QuantizationPost-Training Quantization

    Li Auto Inc.

  11. Reliability-Preserving Compression for Deployable Medical Image Segmentation

    Oct 18, 2026Sagar Kumar, Shailja Sharma, Tirthajit Baruah +1Model CompressionNeural Network Compression

    Indian Institute of Science, India · Indian Institute of Science (IISc), India

  12. SpikeQ: Spike-Guided Post-Training Quantization for Spiking Neural Networks

    Oct 16, 2026Kai Sun, Peibo Duan, Zhanglu Yan +3Spiking Neural NetworksPost-Training Quantization

    Monash University · National University of Singapore · Northeastern University

  13. RoleQ: Role-Aligned Scales for Converting Language Models to Ternary

    Oct 15, 2026Johannes Knittel, Alec Helbling, Hanspeter PfisterLLM QuantizationNeural Network Quantization

    Harvard University · Georgia Tech

  14. Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference

    Oct 15, 2026Sunghyun Wee, Geunjae Choi, Hyeonjin Kim +2LLM QuantizationNeural Network Quantization

    Seoul National University · LG Electronics

  15. Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors

    Oct 14, 2026Yifan LiuQuantization Sensitivity AnalysisMixture-of-Experts Quantization

    UIUC MCS Student

  16. RoLaQ: Joint Rotation and Range Calibration for 2-Bit KV Caches in Multi-Head Latent Attention

    Oct 14, 2026Shiyu Ji, Peng Ling, Chen Lin +4KV-Cache QuantizationLow-Bit Quantization

    Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, China · Tsinghua University, China · Baidu Inc., China

  17. SpectrumQuant: Spectrum Alignment for LLM Quantization

    Oct 14, 2026Dwaipayan Haldar, Indranil Patra, Azhar Yousuf +1LLM QuantizationRotation-Based Quantization

    Indian Institute of Science Bengaluru-560012, Karnataka, India

  18. Not All Frames Are Equal:Difference-Aware Quantization for Ultra-Low-Bit ASR

    Oct 14, 2026Woori Jeon, Jungmin SoLow-Bit QuantizationAutomatic Speech Recognition

    Sogang University, Seoul, Korea

  19. MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantization

    Oct 13, 2026Yangchengyu Zhou, Zhiping Wu, Runchang Jiang +5KV-Cache QuantizationMemory-Efficient Inference

    Nanjing University · The Hong Kong University of Science and Technology · Geely +1

  20. Detecting Unreliable Classifications from Quantized Models

    Oct 13, 2026Ananya Kotla, Jorge T. GomezPost-Training QuantizationClassification

  21. Towards Green AI: Evaluating Energy Efficiency for Quantized LLM Inference

    Oct 13, 2026Petar BastaLLM Inference EfficiencyLLM Quantization

    University of Waterloo Computer Science Waterloo, Ontario, Canada

  22. LoopQuant: Accurate and Efficient Post-Training Quantization for Looped Transformers

    Oct 12, 2026Zhixiong Zhao, Zukang Xu, Xing Hu +3Post-Training QuantizationLLM Quantization

    School of Integrated Circuits, Peking University · Houmo AI

  23. Hardware-Friendly Token-Group Activation Quantization for Low-Bit Mamba Super-Resolution

    Oct 12, 2026Jinwoo Chung, Jangho KimActivation QuantizationPost-Training Quantization

    Kookmin University Seoul, Republic of Korea

  24. When Does Compensating Inherited Quantization Error Help? Mean-Square Guarantees for Layer-Wise Quantization and Their Limits

    Oct 12, 2026Tianhao GuoQuantization Error CompensationNeural Network Quantization

    College of Physics and Electronic Engineering, Shanxi University, Taiyuan 030006, China

  25. HGA-Quant: Heterogeneity-Aware Quantization for AutoRegressive Visual Generation Models

    Oct 11, 2026Xinmeng Liu, Xianglong Yan, Kaicheng Yang +2Post-Training QuantizationNeural Network Quantization

    Shanghai Jiao Tong University · Xi’an University of Electronic Science and Technology

  26. RECAP: Calibrating Low-Bit Reasoning Models on the History They Read

    Oct 11, 2026Chi Ruan, Yujie Chen, Zixiang Zhao +2LLM QuantizationPost-Training Quantization

    SpinEntropy · ETH Zurich · PolyU

  27. VLCQ: Post-training quantization for deep neural networks using variable length coding

    Oct 10, 2026Reem Abdel-Salam, Ahmed H. Abdel-Gawad, Amr G. WassalPost-Training QuantizationNeural Network Quantization

    Computer Engineering Department, Faculty of Engineering, Cairo University, Giza, Egypt

  28. FidelityQ: Preserving Predictive Fidelity in Ultra-Low-Bit Language Model Quantization

    Oct 10, 2026Yujie Chen, Weiyan Zheng, Yiwu Yao +7Low-Bit QuantizationNeural Network Quantization

    The Hong Kong Polytechnic University · Huawei Technologies Ltd. · The University of Hong Kong +2

  29. TETRIS-4DGS: Efficient Post-Training Compression of 4D Gaussian Splatting

    Oct 10, 2026Chengzhen Wu, Kelei Liu, Sicheng Li +2Gaussian Splatting CompressionAdaptive Quantization

    Zhejiang University

  30. DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs

    Oct 9, 2026Mahsa Salmani, Sayeh Sharify, Hesham MostafaLLM QuantizationPost-Training Quantization

    d-Matrix