Post-Training Quantization

Also known as PTQ

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

366 papers72 papers in the last four weeks, up 50% on the four weeks before. 0.4% of all new papers.

SCULPT: Training Edge Vision Models for Post-Training Quantization Readiness
Sep 1, 2026Bharadwaj Kavuri, Sourav Babu-PK, Varadhraj Ellapan +2Edge Computing Activation Quantization
Valeo Vision Systems, Tuam, Ireland · Valeo India Pvt. Ltd, Chennai, India
The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally
Sep 1, 2026Jundong Hu, Shekar RamachandranMixed-Precision Quantization LLM Quantization
PayPal AI
QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization
Aug 31, 2026Yipin Guo, Arun M George, Jie Fu +3LLM Quantization Ternary Quantization
University of Notre Dame
TopGQ: Fast GNN Post-Training Quantization Leveraging Topology Information
Aug 31, 2026Dain Kwon, Kanghyun Choi, Hyeyoon Lee +4Graph Neural Networks Activation Quantization
Seoul National University Seoul, South Korea
OCGQuant: Outlier-Companion Grouping for NVFP4 Quantization
Aug 30, 2026Yishan Yao, Binjun Li, Hanling Yi +5Neural Network Quantization LLM Quantization
South China University of Technology · Intellifusion Inc.
REAL-Q: E2E LLM Quantization via Dynamic Gradient Descent
Aug 30, 2026Qian Zhang, Yaoming Li, Zhewen Tan +9LLM Quantization Post-Training Quantization
Peking University · Northeastern University · ZTE Corporation
H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference
Aug 28, 2026Hao Yu, Zheng Li, Dayiheng Liu +1LLM Quantization Neural Network Quantization
Qwen Team, Alibaba Inc.
A Method for Layer Bit-Width Allocation in LLM Quantization via Performance Maximization Under a Quality-Degradation Constraint
Aug 28, 2026Artem SafronovLLM Quantization LLM Inference Acceleration
A Layer Importance Metric for Quantization Accounting for the Speed-Quality Trade-off in Autoregressive Models
Aug 27, 2026Artem SafronovQuantization Sensitivity Analysis Neural Network Quantization
DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization
Aug 27, 2026Tao Zhang, Jianchao Tan, Pingwei Sun +6Mixed-Precision Quantization LLM Quantization
South China University of Technology · Meituan · East China Normal University
Quantization-Triggered Backdoors in Language Models: Cross-Quantizer Transferability and the Validation--Deployment Gap
Aug 27, 2026Jacopo Dardini, Claudio Stanzione, Giordano Colò +1LLM Backdoor Attacks Backdoor Attacks
Department of Engineering, University of Bologna, Bologna, Italy · Luiss Guido Carli University, Rome, Italy · Live Tech, Rome, Italy Department of Artificial Intelligence, Rome, Italy +1
Transforms for LLM Quantization: The Great Inversion and Format Co-Design
Aug 25, 2026Ehsan JokarLLM Quantization Neural Network Quantization
Quantization Effects on Bangla Language Understanding in Large Language Models: A Systematic Evaluation
Aug 25, 2026Ismail Hossain, Nafi Ullah Shafin, Mohammad Abdullah Al MuminLLM Quantization Post-Training Quantization
Institute of Information and Communication Technology (IICT) Shahjalal University of Science and Technology (SUST), Sylhet, Bangladesh
SandwichQuant: Which Parameters Matter Before and After Quantization?
Aug 25, 2026Peng Xia, Junbiao PangNeural Network Quantization Quantization Error Compensation
School of Information Science and Technology, Beijing University of Technology Beijing, China
Compression Trinity: Exploring Sparsity, Quantization, and Low-Rank Approximations for LLM Compression
Aug 25, 2026Mohammad MozaffariLLM Compression Neural Network Compression
Department of Computer Science University of Toronto
Activation-Weighted Seeded Residual Coding for Low-Bit LLM Weight Repair
Aug 24, 2026Zehao Liu, Chuangchuang Fang, Yang RenLLM Quantization Weight-Only Quantization
Huawei Technologies Research & Development (UK) Limited, Edinburgh, United Kingdom
SSDi8: Accurate and Efficient 8-bit Quantization for State Space Duality
Aug 22, 2026Hyunwoo Kim, Byoungchan Ko, Minseok Kang +5Post-Training Quantization State Space Models
Department of Artificial Intelligence, Chung-Ang University · School of Computer Science and Engineering, Soongsil University · Department of Electrical and Computer Engineering, Seoul National University +2
Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs
Aug 21, 2026Luka Ribar, Jeevan Bhoot, Douglas OrrVLM Quantization Efficient VLM Inference
Graphcore Research · Arm
Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Models
Aug 21, 2026Zhen Yang, Sizai Hou, Kaiwen Zheng +4LLM Quantization Uncertainty Calibration
Yale University · The Hong Kong University of Science and Technology · The Hong Kong University of Science and Technology (Guangzhou) +4
Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs
Aug 21, 2026Bakbergen Ryskulov, Iker García-Ferrero, David Montero +5Quantization-Aware Distillation LLM Quantization
Multiverse Computing
Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs
Aug 19, 2026Shayan Shahrabi-Farahani, Dara RahmatiLLM Quantization Post-Training Quantization
Shahid Beheshti University · Shahid Beheshti University, Tehran, Iran
Efficient INT8 Inference of Small NLP Models on Server CPUs with PyTorch Native Stack
Aug 18, 2026Weiwen Xia, Yuxin Cui, E CaoEfficient Transformer Inference Inference-Time Optimization
Intel Corporation
PRQuant: Permutation Residual Quantization for Low-Overhead Inference
Aug 16, 2026Peiran Wang, Anqi Wang, Jiaying Zhao +10LLM Quantization Low-Bit Quantization
Huawei Technologies Co., Ltd. · Tsinghua University. · Nanjing University of Science and Technology.
FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy
Aug 16, 2026Qingyao Yang, Runming Yang, He Xiao +7Binary Quantization LLM Inference Acceleration
The University of Hong Kong · Harbin Institute of Technology
SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization
Aug 16, 2026Gunjun Lee, Sehwan Son, Younjoo Lee +2LLM Quantization Post-Training Quantization
Seoul National University Seoul, South Korea
QuaSAR: Quantization Compensation via Stable Activation-Aware Rank Truncation
Aug 14, 2026Lin-Fa Lee, Yi-Yu Chang, Kuo-Hei YehQuantization Error Compensation Neural Network Quantization
Department of Institute of Artificial Intelligence Innovation National Yang Ming Chiao Tung University Hsinchu, Taiwan
Post-training Quantization for Hybrid Iterative Generative Models
Aug 14, 2026Jing Gao, Junyi Wu, Wei Wang +2Post-Training Quantization Neural Network Quantization
Institute of Information Science, Beijing Jiaotong University, Beijing, China · University of Illinois Chicago, Chicago, IL, USA
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation
Aug 13, 2026Shuhan Wang, Yilin Luo, Nan Xu +1LLM Quantization 4-Bit Quantization
NAS-Driven Hardware Accelerator Exploration for Edge AI and Quantization Effects on the Pareto Space
Aug 13, 2026Eleftherios Mylonas, Angelos Kouprizas, Michael Birbas +14-Bit Quantization Hardware-Aware NAS
Dept. of ECE University of Patras Patras, Greece
Damage Predicts Recovery: When Calibration Data Matters in Compressing Financial LLMs
Aug 12, 2026Junyi Ye, Mengjia Yu, Debapriya Hazra +1Language Model Calibration Post-Training Quantization
School of Computing, Montclair State University Montclair, New Jersey, USA · Department of Computer Science, New Jersey Institute of Technology Newark, New Jersey, USA

Sep 1, 2026J/K move · Enter open · S save

SCULPT: Training Edge Vision Models for Post-Training Quantization Readiness

Bharadwaj Kavuri, Sourav Babu-PK, Varadhraj Ellapan, Pullarao Maddu+1

Valeo Vision Systems, Tuam, Ireland · Valeo India Pvt. Ltd, Chennai, India

Edge Computing Activation Quantization Post-Training Quantization Low-Bit Quantization

Open paper AskPDF arXiv

Edge vision models are difficult to deploy on resource-constrained hardware, making low-bit post-training quantization (PTQ) attractive. In practice, standard FP32 training often produces heavy-tailed activation distributions whose outliers destabilize activation quantization: preserving the full range wastes quantization bins on rare extremes, while aggressive clipping causes information loss. Existing solutions typically rely on quantization-aware training (QAT), which adds training complexity and bit-width coupling, or advanced PTQ procedures that repair the model after training. We present SCULPT (Statistical Clipping and Uniform Loss for Post-Training), a training-time method that improves PTQ readiness during ordinary FP32 fine-tuning. SCULPT combines a topology-aware activation regularizer that suppresses quantization-hostile skewness and kurtosis with a stable percentile-based clipping mechanism that learns deployment-ready activation bounds. Unlike QAT, SCULPT does not simulate quantization during optimization; unlike post hoc outlier-repair PTQ methods, it does not require runtime activation transformations. The learned clipping bounds can be exported directly into a standard PTQ workflow for low-bit deployment, including INT8 and lower-bit settings such as W4A8.