Quantization Function

Quantization functions aim to reduce the computational cost and memory footprint of neural networks by representing weights and activations with fewer bits, a crucial step for deploying models on resource-constrained devices. Current research focuses on improving the accuracy of quantized models through techniques like quantization-aware training, which incorporates quantization into the training process, and the development of adaptive quantization functions tailored to different layers or activation distributions within a network (e.g., using ResNet, transformers, and spiking neural networks). These advancements are significant because they enable the efficient deployment of deep learning models in various applications, including mobile and embedded systems, while mitigating the accuracy loss typically associated with quantization.

Papers

May 8, 2024

Custom Gradient Estimators are Straight-Through Estimators in Disguise
Matt Schoenbauer, Daniele Moro, Lukasz Lew, Andrew Howard
Quantization Aware Training Gradient Estimator Digital STEALTH Metric Small Sized Convolutional Neural Network Straight Through Estimator Quantization Function

April 17, 2024

QGen: On the Ability to Generalize in Quantization Aware Training
MohammadHossein AskariHemmat, Ahmadreza Jeddi, Reyhane Askari Hemmat, Ivan Lazarevich, Alexander Hoffman, Sudhakar Sah, Ehsan Saboori, Yvon Savaria, Jean-Pierre David
Quantization Operator Critique Ability ImageNet Dataset Quantization Aware Training Generalization Property Quantization Noise Quantization Function

September 22, 2023

Poster: Self-Supervised Quantization-Aware Knowledge Distillation
Kaiqi Zhao, Ming Zhao
Self Supervised Quantization Operator Quantization Aware Training Visually Effective Poster Quantization Aware Knowledge Distillation Quantization Function

May 30, 2023

August 17, 2022

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets
Zhijun Tu, Xinghao Chen, Pengju Ren, Yunhe Wang
Binary Neural Network Discrete Distribution Binary Network Optimal Set Quantization Function

Quantization Function

Papers

Custom Gradient Estimators are Straight-Through Estimators in Disguise

QGen: On the Ability to Generalize in Quantization Aware Training

Poster: Self-Supervised Quantization-Aware Knowledge Distillation

Low Precision Quantization-aware Training in Spiking Neural Networks with Differentiable Quantization Function

Towards Accurate Post-training Quantization for Diffusion Models

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets