Neural Network Quantization

Neural network quantization aims to reduce the memory footprint and computational cost of deep learning models by representing their weights and activations using lower precision numbers (e.g., 1-bit, 2-bit, 4-bit), thereby enabling deployment on resource-constrained devices. Current research focuses on developing efficient quantization algorithms, including mixed-precision techniques that assign different bitwidths to different layers, and exploring novel quantization schemes beyond uniform quantization to minimize accuracy loss. This area is crucial for advancing the practical applicability of large language models and other computationally intensive neural networks, impacting fields ranging from mobile device applications to energy-efficient edge computing.

Papers

June 10, 2024

Efficient Neural Compression with Inference-time Decoding
C. Metz, O. Bichler, A. Dupret
Inference Time Mixed Precision Quantization Neural Compression Fixed Point Neural Network Quantization Decoder Architecture

June 8, 2024

Towards Lightweight Speaker Verification via Adaptive Neural Network Quantization
Bei Liu, Haoyu Wang, Yanmin Qian
Mixed Precision Quantization Neural Network Quantization Adaptive Quantization Precision Quantization

March 15, 2024

Quantization Effects on Neural Networks Perception: How would quantization change the perceptual field of vision models?
Mohamed Amine Kerkouri, Marouane Tliba, Aladine Chetouani, Alessandro Bruno
Vision Model Quantization Operator Neural Network Quantization Perceptual Space Quantization Challenge

February 23, 2024

GPTVQ: The Blessing of Dimensionality for LLM Quantization
Mart van Baalen, Andrey Kuzmin, Markus Nagel, Peter Couperus, Cedric Bastoul, Eric Mahurin, Tijmen Blankevoort, Paul Whatmough
Vector Quantization Data Dimensionality LLM Quantization Neural Network Quantization Blessing Generation Integer Quantization Multi Codebook Quantization Scale

December 12, 2023

IDKM: Memory Efficient Neural Network Quantization via Implicit, Differentiable k-Means
Sean Jaffe, Ambuj K. Singh, Francesco Bullo
K$ Mean Implicit Language Weight Quantization Neural Network Quantization Differentiable Clustering

September 20, 2023

SPFQ: A Stochastic Algorithm and Its Error Analysis for Neural Network Quantization
Jinjie Zhang, Rayan Saab
Neural Network Quantization Operator Quantization Technique Error Analysis Neural Network Quantization Stochastic Quantization Stochastic Search

August 10, 2023

NUPES : Non-Uniform Post-Training Quantization via Power Exponent Search
Edouard Yvinec, Arnaud Dapogny, Kevin Bailly
Quantization Operator Multiplier Free Quantization Neural Network Weight Exponential Mechanism Non Uniform Neural Network Quantization Uniform Quantization Non Uniform Quantization

July 20, 2023

Quantized Feature Distillation for Network Quantization
Ke Zhu, Yin-Yin He, Jianxin Wu
Vision Transformer Multiplier Free Quantization Feature Distillation Neural Network Quantization Network Quantization Quantization Aware Knowledge Distillation

July 6, 2023

Pruning vs Quantization: Which is Better?
Andrey Kuzmin, Markus Nagel, Mart van Baalen, Arash Behboodi, Tijmen Blankevoort
Neural Network Deep Neural Network Quantization Operator Edge Pruning Quantization Technique Neural Network Quantization Layer Wise Pruning

July 2, 2023

Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuning
Jun Chen, Shipeng Bai, Tianxin Huang, Mengmeng Wang, Guanzhong Tian, Yong Liu
Mixed Precision Neural Network Quantization Data Free Quantization Low Precision Quantization

June 23, 2023

QNNRepair: Quantized Neural Network Repair
Xidan Song, Youcheng Sun, Mustafa A. Mustafa, Lucas C. Cordeiro
Quantized Neural Network DNN Repair Neural Network Quantization

May 16, 2023

MINT: Multiplier-less INTeger Quantization for Energy Efficient Spiking Neural Networks
Ruokai Yin, Yuhang Li, Abhishek Moitra, Priyadarshini Panda
Neural Network Quantization Uniform Quantization Integer Quantization Membrane Potential

April 7, 2023

AutoQNN: An End-to-End Framework for Automatically Quantizing Neural Networks
Cheng Gong, Ye Lu, Surong Dai, Deng Qian, Chenkun Du, Tao Li
End to End Neural Network Quantization State of the Art Quantization

February 10, 2023

A Practical Mixed Precision Algorithm for Post-Training Quantization
Nilesh Prasad Pandey, Markus Nagel, Mart van Baalen, Yin Huang, Chirag Patel, Tijmen Blankevoort
Quantization Operator Mixed Precision Quantization Mixed Precision Neural Network Quantization Quantization Noise

December 21, 2022

Automatic Network Adaptation for Ultra-Low Uniform-Precision Quantization
Seongmin Park, Beomseok Kwon, Jieun Lim, Kyuyoung Sim, Tae-Ho Kim, Jungwook Choi
Quantization Error Adaptive Network Neural Network Quantization Precision Quantization

December 10, 2022

Vertical Layering of Quantized Neural Networks for Heterogeneous Inference
Hai Wu, Ruifei He, Haoru Tan, Xiaojuan Qi, Kaibin Huang
Multi Layer Quantization Aware Training Quantized Neural Network Neural Network Quantization Layer Representation

September 18, 2022

PIM-QAT: Neural Network Quantization for Processing-In-Memory (PIM) Systems
Qing Jin, Zhiyu Chen, Jian Ren, Yanyu Li, Yanzhi Wang, Kaiyuan Yang
System Description Neuromorphic Hardware Processing in Memory Neural Network Quantization

July 20, 2022

Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approach
Jiseok Youn, Jaehun Song, Hyung-Sin Kim, Saewoong Bahk
Neural Network Quantization Bitwidth Quantization

July 4, 2022

BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformation
Geon Park, Jaehong Yoon, Haiyang Zhang, Xing Zhang, Sung Ju Hwang, Yonina C. Eldar
Neural Network Quantization Aware Training Neural Network Quantization Extreme Quantization Task Adaptive Feature

February 10, 2022

F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization
Qing Jin, Jian Ren, Richard Zhuang, Sumant Hanumante, Zhengang Li, Zhiyu Chen, Yanzhi Wang, Kaiyuan Yang, Sergey Tulyakov
Fine Tuning Quantization Technique Neural Network Quantization Network Quantization Multiplication Task