Post-Training Quantization

Also known as PTQ

Ask about this topic

Overview

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

CardsList

360 papers70 papers in the last four weeks, up 46% on the four weeks before. 0.4% of all new papers.

  1. When W4A4 Breaks Camouflaged Object Detection: Token-Group Dual-Constraint Activation Quantization

    Apr 18, 2026Tianqi Li, Wenyu Fang, Xin He +34-Bit QuantizationActivation Quantization

    VCIP, College of Computer Science, Nankai University · School of Computer Science and Engineering, Tianjin University of Technology · Institute for Infocomm Research, A*STAR +2

  2. When Flat Minima Fail: Characterizing INT4 Quantization Collapse After FP32 Convergence

    Apr 16, 2026Marcus ArmstrongLLM Quantization4-Bit Quantization

    Department of Computer Science University of Houston Houston, TX 77004

  3. Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference

    Apr 16, 2026Nenad Banfic, David Fan, Kunal Vaishnavi +5Efficient Neural Network InferenceStreaming ASR

    CoreAI, Microsoft

  4. Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate

    Apr 15, 2026Jaemin Kim, Sungkyun Kim, Junyeol Lee +1LLM QuantizationLanguage Model Calibration

    Seoul National University · Hanyang University

  5. DA-PTQ: Drift-Aware Post-Training Quantization for Efficient Vision-Language-Action Models

    Apr 13, 2026Siyuan Xu, Tianshi Wang, Fengling Li +2Post-Training QuantizationEfficient VLA Models

    Tongji University Shanghai, China · University of Technology Sydney Sydney, Australia

  6. ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximation

    Apr 13, 2026Suyoung Kim, Sunghyun Wee, Hyeonjin Kim +3LLM QuantizationPost-Training Quantization

    Seoul National University, Seoul, Republic of Korea · LG Electronics, Seoul, Republic of Korea

  7. CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts

    Apr 12, 2026Xiangyang Yin, Xingyu Liu, Tianhua Xia +5Mixture-of-Experts QuantizationNeural Network Quantization

    Courant Institute of Mathematical Sciences, New York University · Tandon School of Engineering, New York University · Cerebras Systems Inc.

  8. SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Models

    Apr 11, 2026Han Liu, Haotian Gao, Xiaotong Zhang +5LLM QuantizationPost-Training Quantization

    Dalian University of Technology, Dalian, China · Peking University, Beijing, China · Shenzhen MSU-BIT University, Shenzhen, China +1

  9. Initialisation Determines the Basin: Efficient Codebook Optimisation for Extreme LLM Quantization

    Apr 9, 2026Ian W. Kennedy, Nafise Sadat MoosaviLLM QuantizationNeural Network Quantization

    Department of Computer Science, University of Sheffield, Sheffield, UK

  10. Rethinking Residual Errors in Compensation-based LLM Quantization

    Apr 9, 2026Shuaiting Li, Juncan Deng, Kedong Xu +5Quantization Error CompensationLLM Quantization

    Zhejiang University · vivo Mobile Communication Co., Ltd

  11. Weight Group-wise Post-Training Quantization for Medical Foundation Model

    Apr 9, 2026Yineng Chen, Peng Huang, Aozhong Zhang +9Post-Training QuantizationLow-Bit Quantization

    University at Albany, SUNY · Southwest Jiaotong University · Purdue University +1

  12. MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization

    Apr 8, 2026Zhixiong Zhao, Zukang Xu, Zhixuan Chen +1Mixture-of-Experts InferenceMixture-of-Experts Quantization

    Houmo AI

  13. Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees

    Apr 7, 2026Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai +4Mixture-of-Experts QuantizationNeural Network Quantization

    Rensselaer Polytechnic Institute · IBM Research

  14. 3DTurboQuant: Training-Free Near-Optimal Quantization for 3D Reconstruction Models

    Apr 7, 2026Jae Joong Lee3DGS CompressionPost-Training Quantization

    Department of Computer Science, Purdue University

  15. RUQuant: Towards Refining Uniform Quantization for Large Language Models

    Apr 5, 2026Han Liu, Haotian Gao, Changya Li +4LLM QuantizationActivation Quantization

    Dalian University of Technology Dalian, China · Peking University Beijing, China · Macao Polytechnic University Macao, China

  16. Zero-Shot Quantization via Weight-Space Arithmetic

    Apr 3, 2026Daniele Solombrino, Antonio Andrea Gargiulo, Alessandro Zirilli +3Quantization RobustnessQuantization

    Sapienza University of Rome · Sapienza University of Rome / Paradigma

  17. QAPruner: Quantization-Aware Vision Token Pruning for Multimodal Large Language Models

    Apr 3, 2026Xinhao Wang, Zhonyu Xia, Zhiwei Lin +2Visual Token PruningLVLM Pruning

    Wangxuan Institute of Computer Technology, Peking University

  18. PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression

    Mar 30, 2026Caio VicentinoLLM QuantizationNeural Network Quantization

  19. Adaptive Block-Scaled Data Types

    Mar 30, 2026Jack Cook, Hyemin S. Lee, Kathryn Le +4Neural Network QuantizationLLM Quantization

    Massachusetts Institute of Technology · NVIDIA

  20. OneComp: One-Line Revolution for Generative AI Model Compression

    Mar 30, 2026Yuma Ichikawa, Keiji Kimura, Akihiro Yoshida +11Hardware-Aware QuantizationModel Compression

    Fujitsu Limited · RIKEN Center for AIP · Institute of Science Tokyo +2

  21. SliderQuant: Accurate Post-Training Quantization for LLMs

    Mar 26, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +3LLM QuantizationPost-Training Quantization

    Intel Labs China · BUPT

  22. FAAR: Format-Aware Adaptive Rounding for NVFP4

    Mar 23, 2026Hanglin Li, Shuchang Tian, Chen Lin +2LLM Quantization4-Bit Quantization

    Li Auto Inc.

  23. DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

    Mar 20, 2026Xiaoming Yu, Shize Tang, Guanghua Yu +4QuantizationLLM Quantization

    Yuanbao & Hunyuan AI Infra Team

  24. 6Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models

    Mar 19, 2026Rundong Su, Jintao Zhang, Zhihang Yuan +3Diffusion Model QuantizationMixed-Precision Quantization

    Fudan University · Tsinghua University · Shanghai Jiao Tong University

  25. RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference

    Mar 18, 2026Arpit Singh Gautam, Saurabh JhaLLM QuantizationNeural Network Quantization

  26. Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients

    Mar 18, 2026Ziwei Xiang, Fanhu Zeng, Hongjian Fang +6VLM QuantizationPost-Training Quantization

    State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA · School of Artificial Intelligence, UCAS · Beijing National Research Center for Information Science and Technology +3

  27. Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Sampling

    Mar 18, 2026Sooyoung Ryu, Mathieu Salzmann, Saqib JavedDiffusion Model QuantizationDiffusion Model Sampling

    Department of Computer Science and Engineering, Seoul National University, Seoul, South Korea · School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland · Meta Reality Labs, Redmond, USA

  28. BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization

    Mar 17, 2026Ji-Fu Li, Manyi Zhang, Xiaobo Xia +4LLM QuantizationNeural Network Quantization

    Huawei Technologies · University of Science and Technology of China

  29. Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization

    Mar 17, 2026Francesco Pio Monaco, Elia Cunegatti, Flavio Vella +1LLM CompressionLLM Pruning

    University of Trento

  30. RESQ: A Unified Framework for REliability- and Security Enhancement of Quantized Deep Neural Networks

    Mar 16, 2026Ali Soltan Mohammadi, Samira Nazari, Ali Azarpeyvand +5Adversarial RobustnessQuantization Robustness

    University of Zanjan, Zanjan, Iran · Brandenburg Technical University, Cottbus, Germany · Tallinn University of Technology, Tallinn, Estonia +1