Post-Training Quantization

Also known as PTQ

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

360 papers71 papers in the last four weeks, up 48% on the four weeks before. 0.4% of all new papers.

Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models
Jul 13, 2026Peng Xia, Junbiao Pang, Muhammad Ayub SabirActivation Quantization Post-Training Quantization
School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China
RDQ: Residual Distribution Quantization for Large Language Models
Jul 11, 2026Prateek SinghLLM Quantization Post-Training Quantization
Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
Jul 10, 2026Renuka Oladri, Mohan Vamsi Varadaraju Priya, Jerry WuLLM Quantization LLM Evaluation
Science Academy University of Maryland College Park, MD, USA · Electrical and Computer Engineering University of Maryland College Park, MD, USA
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
Jul 9, 2026Baha Rababah, Cuneyt Gurcan Akcora, Carson K. LeungLLM Evaluation Quantization Sensitivity Analysis
Department of Computer Science, University of Manitoba, Canada · AI Initiative, University of Central Florida, USA
It Takes Few to TANGO: A Quantized Distributed Model for Binaural Speech Enhancement
Jul 9, 2026Zahra Benslimane, Pierre Chouteau, Martyna Poreba +4Quantization-Aware Training Speech Enhancement
Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France · Université de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France
A Quantized Native Runtime for On-Device Semantic Audio Generation
Jul 9, 2026Matteo Spanio, Antonio RodàText-to-Audio Generation Controllable Music Generation
Centro di Sonologia Computazionale (CSC) · Dept. of Information Engineering, University of Padova
Closing the Null Space: Guidance-Aware Quantization for Classifier-Free Diffusion
Jul 9, 2026Abdullah Al Shafi, Sumaiya Rahim SumaMixed-Precision Quantization Classifier-Free Guidance
Department of Computer Science and Engineering Khulna University of Engineering & Technology Khulna-9203, Bangladesh
KronQ: LLM Quantization via Kronecker-Factored Hessian
Jul 8, 2026Donghyun Lee, Yuhang Li, Ruokai Yin +1Mixed-Precision Quantization LLM Quantization
University of Southern California · Yale University
Lyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic
Jul 5, 2026Anis Hamadouche, Amir HussainLyapunov-Based Control Quantization-Aware Training
The School of Engineering & Physical Sciences Heriot-Watt University Edinburgh EH14 4AS, UK · SDAIA-KFUPM Joint Research Centre for Artificial Intelligence King Fahd University of Petroleum and Minerals Dhahran, Saudi Arabia
HiFA4: Training-Free 4-bit FlashAttention on Ascend HIF4 NPUs for LLM Inference
Jul 5, 2026Hui Dong, Yanzhao Li, Jie Gao +5Neural Network Quantization LLM Quantization
Huawei Technologies
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers
Jul 2, 2026Donghyun Lee, Jitesh Chavan, Duy Nguyen +5Video Diffusion Models Diffusion Transformer
Cantina Labs · University of Southern California · *Work done during an internship at Cantina Labs. +1
SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models
Jul 2, 2026Qi Lyu, Jiahua Dong, Baichen Liu +7Model Compression Vision-Language Models
State Key Laboratory of Robotics and Intelligent Systems · Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences +2
Boundary-Aware Quantization: Finite-Scale Decision Geometry of Neural Classifiers
Jul 1, 2026O. M. KiselevWeight-Only Quantization Post-Training Quantization
Innopolis University, Innopolis, Republic of Tatarstan, Russian Federation
Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization
Jul 1, 2026Fei Wang, Chao Xue, Taoran Liu +3Mixed-Precision Quantization LLM Quantization
South China University of Technology · Ocean University of China · Shenzhen Campus of Sun Yat-sen University +1
W4A4 Quantization for Inference on Wan2.2-I2V-A14B
Jun 28, 2026Yidong Chen, Chengyu Shi, Jiahao LiuDiffusion Model Quantization Post-Training Quantization
Tsinghua University
CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs
Jun 25, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +2LLM Quantization Ternary Quantization
1Intel Labs China
Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
Jun 24, 2026Xinyu Lian, Walid Krichene, Beichen Huang +4LLM Inference Efficiency LLM Quantization
University of Illinois Urbana-Champaign · Microsoft · Anyscale +1
GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generation
Jun 22, 2026Jette Oberländer, Jan Finkbeiner, Catherine M. Schöfmann +1Mixed-Precision Quantization LLM Quantization
Fakultät für Informatik, RWTH Aachen, Aachen, 52074, Germany · Peter Grünberg Institut, Forschungszentrum Jülich GmbH, Jülich, 52425, Germany · Fakultät für Elektrotechnik und Informationstechnik, RWTH Aachen, Aachen, 52062, Germany
HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models
Jun 22, 2026Yuval Domb, Hadar Sackstein, Tomer SolbergLLM Quantization Diffusion Model Quantization
Efficient Network Inference via Hardware-Aware Architecture Search, Model Pruning & Quantization
Jun 22, 2026Lucas Heublein, Mark Deutel, Axel Plinge +1Efficient Neural Network Inference Hardware-Aware NAS
Fraunhofer Institute for Integrated Circuits IIS, 90411 Nürnberg, Germany
ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers
Jun 20, 2026Changjun Li, Runqing Jiang, Lian Xu +3Vision Transformer Activation Quantization
School of Electronics and Communication Engineering, Shenzhen Campus, Sun Yat-sen University, Shenzhen, China · Computer Science and Software Engineering, The University of Western Australia, Perth, Australia · University of Oxford, Oxford, United Kingdom
An Empirical Study of OpenPangu Quantization on Ascend NPUs
Jun 19, 2026Tong Shi, Jiacheng Wang, Hui Xie +4LLM Quantization AI Accelerator Inference
State Key Laboratory of Complex & Critical Software Environment, Beihang University · School of Artificial Intelligence, Beihang University · School of Computer Science and Engineering, Beihang University
Mix-QVLA: Task-Evidence-Aware Mixed-Precision Quantization of Vision-Language-Action Models
Jun 17, 2026Navin Ranjan, Andreas SavakisMixed-Precision Quantization VLM Quantization
Rochester Institute of Technology Rochester, New York 14623, USA
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
Jun 15, 2026Yuanteng Chen, Peisong Wang, Zhilei Liu +9Mixed-Precision Quantization LLM Quantization
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Zhongguancun Academy
Shift-and-Sum Quantization for Visual Autoregressive Models
Jun 15, 2026Jaehyeon Moon, Bumsub HamVisual Autoregressive Models Autoregressive Image Generation
1Yonsei University · 2Articron
Signed Symmetric Quantization for Few-Bit Integers
Jun 12, 2026Ian Colbert, Eashan Dash, Pablo Monteagudo-Lago +5LLM Quantization Weight-Only Quantization
AMD
Realizing Native INT8 Compute for Diffusion Transformers on Consumer GPUs: A Fused INT8 GEMM Kernel for Ideogram 4.0
Jun 12, 2026Ali Asaria, Tony Salomone, Deep GandhiGPU Kernel Optimization Diffusion Transformer
Transformer Lab
Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Score
Jun 11, 2026Mariya Pavlova, Harrison Bo Hua Zhu, Lidia Vitanova +2Mixed-Precision Quantization Time Series Modeling
Department of Computing, Imperial College London, London, United Kingdom · Department of Public Health, University of Copenhagen, Copenhagen, Denmark · GATE Institute, Sofia University St. Kliment Ohridski, Sofia, Bulgaria +1
TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
Jun 11, 2026Zhixiong Zhao, Zukang Xu, Zhixuan Chen +3Mixed-Precision Quantization LLM Quantization
Houmo AI, China · 2Southeast University, Nanjing, China
Multi-Bitwidth Quantization for LLMs Using Additive Codebooks
Jun 11, 2026Liza Babaoglu, Shuangyi Chen, Ashish KhistiLLM Quantization Weight-Only Quantization
University of Toronto

Jul 13, 2026J/K move · Enter open · S save

Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models

Peng Xia, Junbiao Pang, Muhammad Ayub Sabir

School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China

Activation Quantization Post-Training Quantization Low-Bit Quantization

Open paper AskPDF arXiv

Post-training quantization (PTQ) compresses deep neural networks for deployment under limited memory and computational budgets. However, low-bit (i.e., 2-bit or 4-bit) PTQ often suffers from substantial performance degradation. Most existing PTQ methods operate on an unconstrained full-precision (FP) model and primarily address quantization errors through post-hoc reconstruction. We argue that low-bit PTQ accuracy is limited not only by post-quantization error minimization, but also by the quantization-error tolerance of a FP model itself. In this paper, we propose Efficient Tuning Before Quantization (ETBQ), a pre-conditioning tuning stage for Stochastic Gradient Descent (SGD)-optimized models before PTQ. During tuning, the FP model is optimized under perturbations sampled from the error distributions of weight and activation quantization, guiding the model toward a loss-landscape region that is less sensitive to the subsequent PTQ. Unlike QAT, ETBQ does not train a fake-quantized deployment model, which is computationally and memory intensive. Instead, ETBQ outputs a FP model that can be used by any PTQ backend. Experiments on CIFAR-100, Tiny-ImageNet, ImageNet, and Cityscapes provide consistent evidence that ETBQ improves low-bit PTQ across diverse tasks. Under W2A4 settings, e.g., ETBQ improves over naive PTQ by 2.14% top-1 accuracy on Tiny-ImageNet and by 5.80% mIoU on Cityscapes. Code is available at https://github.com/xpxpxp2001xpxpxp/ETBQ.