Post Training Quantization

Post-training quantization (PTQ) aims to reduce the computational cost and memory footprint of large neural networks, particularly large language models (LLMs) and vision transformers (ViTs), without retraining. Current research focuses on improving PTQ accuracy at extremely low bit-widths (e.g., 2-4 bits) through techniques like vector quantization, adaptive quantization schemes (e.g., per-channel, mixed-precision), and optimization strategies that minimize quantization error by addressing issues such as outliers and activation distribution. This work is significant because efficient quantization is crucial for deploying large models on resource-constrained devices, enabling broader accessibility and reducing the environmental impact of AI.

Papers

May 18, 2023

PTQD: Accurate Post-Training Quantization for Diffusion Models
Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, Bohan Zhuang
Diffusion Model Denoising Process Post Training Quantization Multiplier Free Quantization Iterative Denoising Quantization Noise

April 19, 2023

Improving Post-Training Quantization on Object Detection with Task Loss-Guided Lp Metric
Lin Niu, Jiawei Liu, Zhihang Yuan, Dawei Yang, Xinggang Wang, Wenyu Liu
Object Detection Post Training Quantization Metric Learning Loss Detection Loss Quantization Performance

April 18, 2023

Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, Xianglong Liu
Large Language Model Post Training Quantization Transformer Language Model Accurate Quantization Outlier Removal Fine Grained Quantization

April 3, 2023

RPTQ: Reorder-based Post-training Quantization for Large Language Models
Zhihang Yuan, Lin Niu, Jiawei Liu, Wenyu Liu, Xinggang Wang, Yuzhang Shang, Guangyu Sun, Qiang Wu, Jiaxiang Wu, Bingzhe Wu
Post Training Quantization Large Scale Language Model Multiplier Free Quantization Product Quantization Memory Reduction

March 25, 2023

Towards Accurate Post-Training Quantization for Vision Transformer
Yifu Ding, Haotong Qin, Qinghua Yan, Zhenhua Chai, Junjie Liu, Xiaolin Wei, Xianglong Liu
Vision Transformer Post Training Quantization Quantization Challenge

March 23, 2023

Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance
Zhihang Yuan, Jiawei Liu, Jiaxiang Wu, Dawei Yang, Qiang Wu, Guangyu Sun, Wenyu Liu, Xinggang Wang, Bingzhe Wu
Deep Neural Network Post Training Quantization Special Emphasis Worst Case Performance

March 21, 2023

Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective
Yuexiao Ma, Huixia Li, Xiawu Zheng, Xuefeng Xiao, Rui Wang, Shilei Wen, Xin Pan, Fei Chao, Rongrong Ji
Quantization Operator Post Training Quantization Multiplier Free Quantization Compression Technique Model Quantization

March 15, 2023

ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation
Zhewei Yao, Xiaoxia Wu, Cheng Li, Stephen Youn, Yuxiong He
Large Language Model Low Rank Post Training Quantization Comprehensive Study Weight Quantization Activation Quantization

February 2, 2023

Mixed Precision Post Training Quantization of Neural Networks with Sensitivity Guided Search
Clemens JS Schaefer, Elfie Guo, Caitlin Stanton, Xiaofan Zhang, Tom Jablin, Navid Lambert-Shirzad, Jian Li, Chiachen Chou, Siddharth Joshi, Yu Emma Wang
Neural Network Post Training Quantization Quantization Technique Mixed Precision Quantization Large Scale Machine Learning Learning Search

December 16, 2022

RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers
Zhikai Li, Junrui Xiao, Lianwei Yang, Qingyi Gu
Vision Transformer Quantization Operator Post Training Quantization Accurate Quantization GPUSQ ViT

December 14, 2022

PD-Quant: Post-Training Quantization based on Prediction Difference Metric
Jiawei Liu, Lin Niu, Zhihang Yuan, Dawei Yang, Xinggang Wang, Wenyu Liu
Fine Tuning Quantization Operator Post Training Quantization Quantization Parameter Metric Prediction

December 9, 2022

Genie: Show Me the Data for Quantization
Yongkweon Jeon, Chungman Lee, Ho-young Kim
Raw Data Quantization Operator Post Training Quantization Zero Shot Quantization

December 5, 2022

QFT: Post-training quantization via fast joint finetuning of all degrees of freedom
Alex Finkelstein, Ella Fuchs, Idan Tal, Mark Grobman, Niv Vosco, Eldad Meller
Fine Tuning Post Training Quantization Multiplier Free Quantization Different Degree Freedom Matter Quantization Step Degree of Freedom Parameterized Network Bit Weight Quantization

December 3, 2022

Make RepVGG Greater Again: A Quantization-aware Approach
Xiangxiang Chu, Liang Li, Bo Zhang
Supervised ImageNet Post Training Quantization Quantization Performance Heterogeneous Quantization

November 29, 2022

NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers
Yijiang Liu, Huanrui Yang, Zhen Dong, Kurt Keutzer, Li Du, Shanghang Zhang
Vision Transformer Post Training Quantization Activation Quantization

November 28, 2022

Post-training Quantization on Diffusion Models
Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, Yan Yan
Diffusion Model Denoising Process Post Training Quantization Denoising Diffusion Noise Estimation

November 18, 2022

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, Song Han
Large Language Model Post Training Quantization Quantization Challenge

November 17, 2022

CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformers
Natalia Frumkin, Dibakar Gope, Diana Marculescu
Vision Transformer Loss Function Contrastive Loss Quantization Operator Post Training Quantization Contrastive Method Weight Quantization Quantization Scale

November 13, 2022

Long-Range Zero-Shot Generative Deep Network Quantization
Yan Luo, Yangcheng Gao, Zhao Zhang, Haijun Zhang, Mingliang Xu, Meng Wang
Deep Learning Model Post Training Quantization Zero Shot Quantization

October 31, 2022

GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, Dan Alistarh
Transformer Megatron Decepticons Generative Question Post Training Quantization Pre Trained Transformer Real Time Inference Zero Shot Quantization Ternary Quantization

Post Training Quantization

Papers

PTQD: Accurate Post-Training Quantization for Diffusion Models

Improving Post-Training Quantization on Object Detection with Task Loss-Guided Lp Metric

Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling

RPTQ: Reorder-based Post-training Quantization for Large Language Models

Towards Accurate Post-Training Quantization for Vision Transformer

Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance

Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective

ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation

Mixed Precision Post Training Quantization of Neural Networks with Sensitivity Guided Search

RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers

PD-Quant: Post-Training Quantization based on Prediction Difference Metric

Genie: Show Me the Data for Quantization

QFT: Post-training quantization via fast joint finetuning of all degrees of freedom

Make RepVGG Greater Again: A Quantization-aware Approach

NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers

Post-training Quantization on Diffusion Models

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformers

Long-Range Zero-Shot Generative Deep Network Quantization

GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers