Post Training Quantization

Post-training quantization (PTQ) aims to reduce the computational cost and memory footprint of large neural networks, particularly large language models (LLMs) and vision transformers (ViTs), without retraining. Current research focuses on improving PTQ accuracy at extremely low bit-widths (e.g., 2-4 bits) through techniques like vector quantization, adaptive quantization schemes (e.g., per-channel, mixed-precision), and optimization strategies that minimize quantization error by addressing issues such as outliers and activation distribution. This work is significant because efficient quantization is crucial for deploying large models on resource-constrained devices, enabling broader accessibility and reducing the environmental impact of AI.

Papers

November 10, 2023

Post-training Quantization with Progressive Calibration and Activation Relaxing for Text-to-Image Diffusion Models
Siao Tang, Xin Wang, Hong Chen, Chaoyu Guan, Zewen Wu, Yansong Tang, Wenwu Zhu
Text to Image Diffusion Model Stable Diffusion Post Training Quantization Targeted Activation Penalty Diffusion Based Text to Image Continual Calibration

November 9, 2023

Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization
Jangwhan Lee, Minsoo Kim, Seungcheol Baek, Seok Joong Hwang, Wonyong Sung, Jungwook Choi
Post Training Quantization Multiplier Free Quantization Computational Efficiency Activation Quantization Weight Update Weight Only Quantization W4A8 Quantization

October 30, 2023

Exploring Post-Training Quantization of Protein Language Models
Shuang Peng, Fei Yang, Ning Sun, Sheng Chen, Yanfeng Jiang, Aimin Pan
Post Training Quantization Protein Language Model Protein Prediction Task

October 26, 2023

ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers
Zhewei Yao, Reza Yazdani Aminabadi, Stephen Youn, Xiaoxia Wu, Elton Zheng, Yuxiong He
Neural Network Post Training Quantization Quantization Technique Dynamic Quantization Transformer XL W4A8 Quantization Intensive Operator

October 25, 2023

LLM-FP4: 4-Bit Floating-Point Quantized Transformers
Shih-yang Liu, Zechun Liu, Xijie Huang, Pingcheng Dong, Kwang-Ting Cheng
Post Training Quantization Activation Quantization Integer Quantization Floating Point Quantization Quantized Transformer

October 12, 2023

QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
Jing Liu, Ruihao Gong, Xiuying Wei, Zhiwei Dong, Jianfei Cai, Bohan Zhuang
Post Training Quantization Quantization Aware Training Bitwidth Quantization

October 5, 2023

EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models
Yefei He, Jing Liu, Weijia Wu, Hong Zhou, Bohan Zhuang
Fine Tuning Post Training Quantization Quantization Technique

September 29, 2023

September 26, 2023

Efficient Post-training Quantization with FP8 Formats
Haihao Shen, Naveen Mellempudi, Xin He, Qun Gao, Chang Wang, Mengni Wang
Post Training Quantization Neural Compression FP8 Format

September 20, 2023

EPTQ: Enhanced Post-Training Quantization via Hessian-guided Network-wise Optimization
Ofir Gordon, Elad Cohen, Hai Victor Habi, Arnon Netzer
Quantization Operator Post Training Quantization Hessian Free Image Quantization

September 11, 2023

Understanding the Impact of Post-Training Quantization on Large Language Models
Somnath Roy
Large Language Model Neural Network Global Impact Post Training Quantization Inference Speed

September 5, 2023

QuantEase: Optimization-based Quantization for Language Models
Kayhan Behdin, Ayan Acharya, Aman Gupta, Qingquan Song, Siyu Zhu, Sathiya Keerthi, Rahul Mazumder
Language Model Quantization Operator Post Training Quantization Layer Wise Quantization Non Uniform Quantization

September 4, 2023

Softmax Bias Correction for Quantized Generative Models
Nilesh Prasad Pandey, Marios Fournarakis, Chirag Patel, Markus Nagel
Post Training Quantization Quantization Step Quantization Noise Quantization Parameter Vector Quantized Generative Adversarial Network Softmax Output

August 22, 2023

Towards Clip-Free Quantized Super-Resolution Networks: How to Tame Representative Images
Alperen Kalay, Bahri Batuhan Bilecen, Mustafa Ayazoglu
Super Resolution Post Training Quantization Super Resolution Network Quantization Level Representative Dataset Maximally Tense Image

August 15, 2023

Gradient-Based Post-Training Quantization: Challenging the Status Quo
Edouard Yvinec, Arnaud Dapogny, Kevin Bailly
Quantization Operator Post Training Quantization Quantization Aware Training Chaotic Regime Quantization Step

June 8, 2023

Augmenting Hessians with Inter-Layer Dependencies for Mixed-Precision Post-Training Quantization
Clemens JS Schaefer, Navid Lambert-Shirzad, Xiaofan Zhang, Chiachen Chou, Tom Jablin, Jian Li, Elfie Guo, Caitlin Stanton, Siddharth Joshi, Yu Emma Wang
Hessian Matrix Post Training Quantization Quantization Technique Latency Accuracy Independent Layer

June 4, 2023

Temporal Dynamic Quantization for Diffusion Models
Junhyuk So, Jungwon Lee, Daehyun Ahn, Hyungjun Kim, Eunhyeok Park
Diffusion Model Post Training Quantization Quantization Aware Training Quantization Technique Dynamic Quantization

June 1, 2023

FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization
Jung Hyun Lee, Jeonghoon Kim, Se Jung Kwon, Dongsoo Lee
Post Training Quantization Quantization Aware Training Data Divide Different Quantization Stochastic Rounding

May 30, 2023

Towards Accurate Post-training Quantization for Diffusion Models
Changyuan Wang, Ziwei Wang, Xiuwei Xu, Yansong Tang, Jie Zhou, Jiwen Lu
Diffusion Model Post Training Quantization Data Free Quantization Quantization Learning Quantization Function

Post Training Quantization

Papers

Post-training Quantization with Progressive Calibration and Activation Relaxing for Text-to-Image Diffusion Models

Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization

Exploring Post-Training Quantization of Protein Language Models

ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers

LLM-FP4: 4-Bit Floating-Point Quantized Transformers

QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models

EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models

Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings

PB-LLM: Partially Binarized Large Language Models

Efficient Post-training Quantization with FP8 Formats

EPTQ: Enhanced Post-Training Quantization via Hessian-guided Network-wise Optimization

Understanding the Impact of Post-Training Quantization on Large Language Models

QuantEase: Optimization-based Quantization for Language Models

Softmax Bias Correction for Quantized Generative Models

Towards Clip-Free Quantized Super-Resolution Networks: How to Tame Representative Images

Gradient-Based Post-Training Quantization: Challenging the Status Quo

Augmenting Hessians with Inter-Layer Dependencies for Mixed-Precision Post-Training Quantization

Temporal Dynamic Quantization for Diffusion Models

FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization

Towards Accurate Post-training Quantization for Diffusion Models