Post Training Quantization

Post-training quantization (PTQ) aims to reduce the computational cost and memory footprint of large neural networks, particularly large language models (LLMs) and vision transformers (ViTs), without retraining. Current research focuses on improving PTQ accuracy at extremely low bit-widths (e.g., 2-4 bits) through techniques like vector quantization, adaptive quantization schemes (e.g., per-channel, mixed-precision), and optimization strategies that minimize quantization error by addressing issues such as outliers and activation distribution. This work is significant because efficient quantization is crucial for deploying large models on resource-constrained devices, enabling broader accessibility and reducing the environmental impact of AI.

Papers

October 14, 2022

September 19, 2022

SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision
Rong Tian, Zijing Zhao, Weijie Liu, Haoyan Liu, Weiquan Mao, Zhe Zhao, Kan Zhou
Post Training Quantization Inference Framework Mixed Precision Inference Engine Text Processing INT4 Quantization

September 12, 2022

FP8 Formats for Deep Learning
Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, Naveen Mellempudi, Stuart Oberman, Mohammad Shoeybi, Michael Siu, Hao Wu
Deep Learning Post Training Quantization Binary Code FP8 Format INT4 Quantization Modern Neural Network Architecture

August 25, 2022

Efficient Adaptive Activation Rounding for Post-Training Quantization
Zhengyi Li, Cong Guo, Zhanda Zhu, Yangjie Zhou, Yuxian Qiu, Xiaotian Gao, Jingwen Leng, Minyi Guo
Post Training Quantization Quantization Error Weight Quantization Activation Quantization Adaptive Activation

August 24, 2022

Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning
Elias Frantar, Sidak Pal Singh, Dan Alistarh
Deep Neural Network New Framework Model Compression Edge Pruning Post Training Quantization Compression Technique

August 23, 2022

Adaptation of MobileNetV2 for Face Detection on Ultra-Low Power Platform
Simon Narduzzi, Engin Türetken, Jean-Philippe Thiran, L. Andrea Dunbar
Adaptation Concern Post Training Quantization Face Detection L MobileNet

August 19, 2022

FP8 Quantization: The Power of the Exponent
Andrey Kuzmin, Mart Van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, Tijmen Blankevoort
Real Power Post Training Quantization Efficient Inference Exponential Mechanism P Bit Neural Network Inference Low Bit Floating Point Quantization

July 31, 2022

July 13, 2022

Learning Representations for CSI Adaptive Quantization and Feedback
Valentina Rizzello, Matteo Nerini, Michael Joham, Bruno Clerckx, Wolfgang Utschick
Representation Learning Human Feedback Post Training Quantization Channel State Information Quantization Technique Uniform Quantization Adaptive Quantization Downlink Adaptive Quantization

July 7, 2022

Attention Round for Post-Training Quantization
Huabin Diao, Gongyan Li, Shaoyun Xu, Yuexing Hao
Quantization Operator Post Training Quantization Attention Head Mixed Precision Quantization Quantization Step Quantization Performance

June 4, 2022

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, Yuxiong He
High Efficiency Transformer Based Model Open Source Large Language Model Post Training Quantization Large Scale Transformer

April 30, 2022

Towards Feature Distribution Alignment and Diversity Enhancement for Data-Free Quantization
Yangcheng Gao, Zhao Zhang, Richang Hong, Haijun Zhang, Jicong Fan, Shuicheng Yan
Post Training Quantization Distribution Alignment Model Quantization Diversity Enhancement Data Free Quantization

February 15, 2022

Post-Training Quantization for Cross-Platform Learned Image Compression
Dailan He, Ziming Yang, Yuan Chen, Qi Zhang, Hongwei Qin, Yan Wang
Image Compression Post Training Quantization Efficient Inference Multiplier Free Quantization Learned Image Compression Rate Distortion Performance

February 10, 2022

Quantune: Post-training Quantization of Convolutional Neural Networks using Extreme Gradient Boosting for Fast Deployment
Jemin Lee, Misun Yu, Yongin Kwon, Taeho Kim
Convolutional Neural Network Quantization Operator Post Training Quantization Multiplier Free Quantization Convolutional Neural Network Model Real Time Deployment Extreme Gradient Boosting

January 26, 2022

Post-training Quantization for Neural Networks with Provable Guarantees
Jinjie Zhang, Yixuan Zhou, Rayan Saab
Neural Network Supervised ImageNet Post Training Quantization Mixed Precision Quantization Convolutional Architecture Provable Guarantee Integer Quantization

January 17, 2022

UWC: Unit-wise Calibration Towards Rapid Network Compression
Chen Lin, Zheyang Li, Bo Peng, Haoji Hu, Wenming Tan, Ye Ren, Shiliang Pu
Post Training Quantization Multiplier Free Quantization Quantization Error Network Compression Post Training Feature Reconstruction Optimal Calibration

November 24, 2021

PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization
Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, Guangyu Sun
Vision Transformer Post Training Quantization Multiplier Free Quantization Quantization Technique Uniform Quantization

November 23, 2021

HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performance
Huanrui Yang, Xiaoxuan Yang, Neil Zhenqiang Gong, Yiran Chen
Neural Network Stochastic Gradient Descent Robust Optimization Post Training Quantization Anti Unification Fixed Point Enhancing Generalization Quantization Performance

Post Training Quantization

Papers

Post-Training Quantization for Energy Efficient Realization of Deep Neural Networks

Accelerating RNN-based Speech Enhancement on a Multi-Core MCU with Mixed FP16-INT8 Post-Training Quantization

SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision

FP8 Formats for Deep Learning

Efficient Adaptive Activation Rounding for Post-Training Quantization

Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning

Adaptation of MobileNetV2 for Face Detection on Ultra-Low Power Platform

FP8 Quantization: The Power of the Exponent

Symmetry Regularization and Saturating Nonlinearity for Robust Quantization

CoNLoCNN: Exploiting Correlation and Non-Uniform Quantization for Energy-Efficient Low-precision Deep Convolutional Neural Networks

Learning Representations for CSI Adaptive Quantization and Feedback

Attention Round for Post-Training Quantization

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

Towards Feature Distribution Alignment and Diversity Enhancement for Data-Free Quantization

Post-Training Quantization for Cross-Platform Learned Image Compression

Quantune: Post-training Quantization of Convolutional Neural Networks using Extreme Gradient Boosting for Fast Deployment

Post-training Quantization for Neural Networks with Provable Guarantees

UWC: Unit-wise Calibration Towards Rapid Network Compression

PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization

HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performance