Post-Training Quantization

Also known as PTQ

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

360 papers70 papers in the last four weeks, up 46% on the four weeks before. 0.4% of all new papers.

Scale Sensitivity in Low-Bit Post-Training Quantization: Curvature of the Quantization Error Landscape
Sep 29, 2026Jonas von Berg, Massimiliano Datres, Carlo Kneißl +1Post-Training Quantization Neural Network Quantization
Ludwig-Maximilians-Universität München Munich Center for Machine Learning (MCML) Konrad Zuse School of Excellence in Reliable AI (DAAD) · University of Tromso, DLR-German Aerospace Center
Calibrate the Decisions That Change the Future: On-Policy Post-Training Quantization for Multimodal Large Language Models
Sep 29, 2026Wenxiao Fan, Jingling Fu, Lichen Ma +6LLM Quantization VLM Quantization
School of Computer Science and Technology, Beijing Institute of Technology · Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching
Sep 29, 2026Zunhai Su, Yuxuan Sun, Jianchao Tan +5LLM Inference Acceleration Attention Quantization
The University of Hong Kong · Meituan LongCat Team · South China University of Technology +1
Replay the Curvature: Accurate and Scalable NVFP4 Quantization for Large Language Model Inference
Sep 29, 2026Ruiyi Ding, Jie Li, Kang He +4LLM Quantization Post-Training Quantization
KlingAI Research · University of Science and Technology of China · Fudan University +2
IMC-CLINIC: Coupled Loss-Informed Newton Iterations for Clipping in Analog In-Memory Computing
Sep 28, 2026Yung-Chin Chen, Chia-Yu Chen, Naveen VermaLLM Quantization Compute-in-Memory
Princeton University, NJ, USA · EnCharge AI, CA, USA
SubRot: Signed Gradient Subspace Calibration for VLM Rotation Quantization
Sep 28, 2026Zhenhao Shang, Haizhao Jing, Haokui Zhang +4VLM Quantization Rotation-Based Quantization
Northwest Polytechnical University · Nanjing University of Science and Technology · Intellifusion +1
P4Q: Co-designing Token Pruning and Quantization for Vision-Language Model Acceleration
Sep 28, 2026Haizhao Jing, Zhenhao Shang, Haokui Zhang +2VLM Quantization Efficient VLM Inference
Northwest Polytechnical University · Intellifusion
From Attention Sensitivity to Layer Role: Revisiting Mixed-Precision Quantization of Transformers
Sep 28, 2026Nafiseh HosseinpourFardi, Negar Alihadi, Mahmoudreza Babaei +2Mixed-Precision Quantization LLM Quantization
Sharif University of Technology, Tehran, Iran · GISMA University of Applied Sciences, Potsdam, Germany · BRAINS, Brandenburg Research Center for Applied Intelligent Systems, Potsdam, Germany +3
Beyond Reconstruction Loss in Post-Training Quantization: Balanced Fitting for Large Vision-Language Models
Sep 28, 2026Minchan Kang, Kyeonghye Park, Seungyeon Sa +3VLM Quantization Large Vision-Language Models
Korea Advanced Institute of Science and Technology (KAIST) · Hanbat National University
QuantForge: Discovering Residual Decompositions for MXFP4 Post-Training Quantization
Sep 28, 2026Qiulin Shang, Zhoutong Wu, Jie Hu +1LLM Quantization Large Language Model-Guided Optimization
Peking University
GLF-Q: Global-Local Feature-based Quantization for Vision Transformers
Sep 28, 2026Peilin Sun, Guang Liang, Jin Tong +1Post-Training Quantization Low-Bit Quantization
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China · Zhongguancun Academy, Beijing 100094, China
Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantization
Sep 27, 2026I Kennedy, T KennedyMixed-Precision Quantization LLM Quantization
baa.ai Auckland, New Zealand
Chameleon: Dynamic Format Adapter for Efficient Diffusion
Sep 27, 2026Arnab Sanyal, Sandeep ChinchaliDiffusion Model Quantization Post-Training Quantization
UT SWARM Lab, Department of Electrical & Computer Engineering The University of Texas, Austin TX 78712
PulseQuant: Propagation-Guided Subspace Correction for 4-Bit Video Diffusion Transformers
Sep 27, 2026Yutong Wang, Xingtong Ge, Enhuai Liu +4Video Diffusion Models Diffusion Transformer
The University of Sydney · The Hong Kong University of Science and Technology · The Chinese University of Hong Kong +1
Q-WAM: 4-Bit Quantization of World Action Models with Action-Subspace Protection
Sep 27, 2026Arash Akbari, Arman Akbari, Jingwu Luo +7Mixed-Precision Quantization 4-Bit Quantization
Northeastern University · EmbodyX · University of Minnesota - Twin Cities +1
PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers
Sep 26, 2026Yanlong Chen, Yining Chen, Song Zhang +2LLM Quantization Rotation-Based Quantization
Nanyang Technological University, Singapore · Nanjing University of Posts and Telecommunications, Nanjing, China · Nanjing University, Nanjing, China +1
Depth Laws for the Precision Floor of Trained Neural Networks: Amplification, Residual Scaling, and a Quantization-Aware Training Paradox
Sep 25, 2026Ahmad S. TarawnehNeural Network Quantization Post-Training Quantization
Faculty of Information Technology Mutah University Al-Karak, Jordan
Generalization behavior of OPTQ and the role of regularization
Sep 25, 2026Erin George, Rayan SaabNeural Network Quantization Post-Training Quantization
Softmax Reparameterization for Output-Head Quantization
Sep 25, 2026Asim Kadav, Christian Flores, Chirag Arora +5LLM Quantization Post-Training Quantization
Adobe Search, Discovery & ContentAI
Teacher-Anchored Selection of Post-Training Quantized Models under Domain Shift
Sep 25, 2026Alejandro Rodriguez Dominguez, Muhammad Shahzad, Xia HongModel Selection Post-Training Quantization
Department of Computer Science, University of Reading, Reading, United Kingdom · Quantitative Analysis and Artificial Intelligence Department, Miralta Finance Bank S.A., Madrid, Spain · Lecturer, Albert School, Madrid, Spain
G $^2$ PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation
Sep 25, 2026Ruikang Liu, Haoli Bai, Yuxuan Sun +8LLM Quantization Post-Training Quantization
ZTE Corporation · The Chinese University of Hong Kong · Northwestern Polytechnical University +2
Low-Bit Recurrent States in Hybrid Language Models
Sep 25, 2026Hongren Chen, Jiayang HeHybrid Language Models Recurrent-State Quantization
Quantizing Looped Transformers: Feedback Exposure and Calibration Blindness
Sep 25, 2026Nux LiLLM Quantization Post-Training Quantization
Meta
AERIAL: Adversarial Evaluation of Robustness in Accuracy-Preserving Low-Precision EEG Decoders
Sep 24, 2026Saim Rehman, Muhammad ShafiqueQuantization-Aware Training EEG Decoding
eBRAIN Lab, Division of Engineering, New York University Abu Dhabi (NYUAD), Abu Dhabi, UAE
GHOST-Q: Towards Studying Grounding Hallucinations Overlooked Under Same-score TradeOffs in Quantized VLMS
Sep 24, 2026Saim Rehman, Muhammad ShafiqueVLM Evaluation VLM Quantization
eBRAIN Lab, Division of Engineering, New York University Abu Dhabi (NYUAD), Abu Dhabi, UAE
Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures
Sep 24, 2026Se Un Park, Yutae Kim, Junyoung ParkTTS Evaluation Weight-Only Quantization
UX Factory, Inc.
Temporal Taxation Compounds Under Post-Training Compression of Whisper Models
Sep 23, 2026Srishti Ginjala, Eric Fosler-Lussier, Christopher W. Myers +1ASR Evaluation Speech Foundation Models
The Ohio State University, Columbus, OH, USA · Air Force Research Laboratory, USA
Predicting Quantization Price for Selecting PTQ Configurations Before Deployment
Sep 23, 2026Junbin Qiu, Jian Mu, Weitong Zhang +1Post-Training Quantization Neural Network Quantization
Disaggregated Quantization: Specializing LLM Prefill and Decode
Sep 22, 2026Andrei Panferov, Maximilian Kleinegger, Sweta Priyadarshi +2LLM Quantization LLM Inference
NVIDIA · ISTA
RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models
Sep 21, 2026Zhiping Wu, Dongdong Ren, Yangchengyu Zhou +4VLM Quantization Large Vision-Language Models
State Key Laboratory for Novel Software Technology and the School of Intelligence Science and Technology, Nanjing University, Suzhou Campus, Suzhou 215163, China

Sep 29, 2026J/K move · Enter open · S save

Scale Sensitivity in Low-Bit Post-Training Quantization: Curvature of the Quantization Error Landscape

Jonas von Berg, Massimiliano Datres, Carlo Kneißl, Gitta Kutyniok

Ludwig-Maximilians-Universität München Munich Center for Machine Learning (MCML) Konrad Zuse School of Excellence in Reliable AI (DAAD) · University of Tromso, DLR-German Aerospace Center

Post-Training Quantization Neural Network Quantization LLM Quantization Quantization Sensitivity Analysis Low-Bit Quantization

Open paper AskPDF arXiv

Post-training quantization (PTQ) methods in the GPTQ family minimize a layer-wise reconstruction error on a uniform grid whose scale must be chosen; the common max-based choice degrades sharply at low bit-widths. We study how sensitive this objective is to the scale. For a layer with i.i.d. Gaussian weights and calibration activations of sufficiently large effective rank, we prove that, as the width grows, the normalized round-to-nearest loss converges with high probability, uniformly over all scales, to the mean-squared error of a uniform quantizer applied to a standard Gaussian; we verify the effective-rank condition for wide, randomly initialized MLPs with odd Lipschitz activations and isotropic Gaussian calibration data. The limiting objective has a unique nondegenerate minimizer, whose scale decreases strictly with the number of levels and whose curvature with respect to relative scale errors decays approximately exponentially with the bit-width. GPTQ experiments on five LLMs show the same trend: the scale rule changes perplexity substantially at 2--3 bits and negligibly from 6 bits on, and a local measure of GPTQ scale sensitivity decreases with bit-width in line with the Gaussian curvature. The Gaussian-optimal scale fails on raw weights; after Hadamard incoherence processing it matches the best searched rule at 3 bits and above without any search, but remains clearly worse at 2 bits.