Quantized Neural Network

Quantized neural networks (QNNs) aim to reduce the computational cost and memory footprint of deep learning models by representing weights and activations using lower-precision integer arithmetic, rather than 32-bit floating-point numbers. Current research focuses on improving the accuracy of QNNs through techniques like quantization-aware training, exploring different quantization schemes (e.g., mixed-precision, stochastic quantization), and developing efficient algorithms for training and verification. This field is significant because QNNs enable the deployment of deep learning on resource-constrained devices, impacting applications ranging from mobile and edge computing to embedded systems and Internet of Things (IoT) devices.

Papers

November 7, 2022

AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks
Louis Leconte, Sholom Schechtman, Eric Moulines
Neural Network Deep Neural Network Constrained Optimization Quantized Neural Network Annealing Machine Stochastic First Order Method Fast AltGDA Type Algorithm

August 30, 2022

ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization
Cong Guo, Chen Zhang, Jingwen Leng, Zihan Liu, Fan Yang, Yunxin Liu, Minyi Guo, Yuhao Zhu
DNN Accelerator Low Bit Quantization Quantized Neural Network Dynamic Quantization Data Adaptation Ant Colony

July 19, 2022

Green, Quantized Federated Learning over Wireless Networks: An Energy-Efficient Design
Minsu Kim, Walid Saad, Mohammad Mozaffari, Merouane Debbah
Wireless Network Multiplier Free Quantization Quantized Neural Network Baseline Algorithm

May 25, 2022

A Low Memory Footprint Quantized Neural Network for Depth Completion of Very Sparse Time-of-Flight Depth Maps
Xiaowen Jiang, Valerio Cambareri, Gianluca Agresti, Cynthia Ifeyinwa Ugwu, Adriano Simonetto, Fabien Cardinaux, Pietro Zanuttigh
Depth Map Depth Completion Dense Depth Map Quantized Neural Network Indoor Body Perception

April 1, 2022

QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration
Zirui Xu, Fuxun Yu, Jinjun Xiong, Xiang Chen
Quantized Neural Network Quadratic Neural Network Architecture Optimization Design Exploration

February 15, 2022

Navigating Local Minima in Quantized Spiking Neural Networks
Jason K. Eshraghian, Corey Lammie, Mostafa Rahimi Azghadi, Wei D. Lu
Spiking Neural Network Quantized Neural Network Local Minimum

November 29, 2021

Mixed Precision of Quantization of Transformer Language Models for Speech Recognition
Junhao Xu, Shoukang Hu, Jianwei Yu, Xunying Liu, Helen Meng
Speech Recognition Quantization Operator Transformer Language Model Quantized Neural Network Precision Transformer Quantized Transformer

November 15, 2021

On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks
Minsu Kim, Walid Saad, Mohammad Mozaffari, Merouane Debbah
Deep Neural Network Multidimensional Local Precision Rate Energy Policy Research Resource Constrained Device Quantized Neural Network Federated Learning Convergence

November 10, 2021

An Underexplored Dilemma between Confidence and Calibration in Quantized Neural Networks
Guoxuan Xia, Sangwon Ha, Tiago Azevedo, Partha Maji
Calibration Performance Post Training Quantization High Confidence Better Robustness Quantized Neural Network Convolution Based

Quantized Neural Network

Papers

AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks

ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization

Green, Quantized Federated Learning over Wireless Networks: An Energy-Efficient Design

A Low Memory Footprint Quantized Neural Network for Depth Completion of Very Sparse Time-of-Flight Depth Maps

QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration

Navigating Local Minima in Quantized Spiking Neural Networks

Mixed Precision of Quantization of Transformer Language Models for Speech Recognition

On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks

An Underexplored Dilemma between Confidence and Calibration in Quantized Neural Networks