Product Quantization

Product quantization is a model compression technique that reduces the memory footprint and computational cost of deep learning models by representing data using lower-bit representations. Current research focuses on improving the accuracy of quantized models, particularly for large language models and computer vision tasks, through techniques like quantization-aware training, optimized quantization schemes (e.g., mixed-precision quantization, product quantization with Gumbel), and novel loss functions that address the inharmony between different model tasks. These advancements are significant for deploying large models on resource-constrained devices and accelerating inference speed in various applications, including object detection, natural language processing, and approximate nearest neighbor search.

Papers

November 12, 2024

Towards Low-bit Communication for Tensor Parallel LLM Inference
Harry Dong, Tyler Johnson, Minsik Cho, Emad Soroush
Language Model LLM Inference Multiplier Free Quantization Inference Efficiency Low Bit Product Quantization Tensor Parallelism

August 5, 2024

HQOD: Harmonious Quantization for Object Detection
Long Huang, Zhiwei Dong, Song-Lu Chen, Ruiyao Zhang, Shutong Ti, Feng Chen, Xu-Cheng Yin
Object Detector Product Quantization Task Specific Loss Image Quantization

July 22, 2024

Differentiable Product Quantization for Memory Efficient Camera Relocalization
Zakaria Laskar, Iaroslav Melekhov, Assia Benbihi, Shuzhe Wang, Juho Kannala
Weight Quantization Camera Relocalization Product Quantization Descriptor Extraction Efficient Dequantization

July 3, 2024

GPTQT: Quantize Large Language Models Twice to Push the Efficiency
Yipin Guo, Yilin Lang, Qinyuan Ren
High Efficiency GPT Neo Post Training Quantization Generative Large Language Model Product Quantization Linear Quantization

May 23, 2024

Distilling Vision-Language Pretraining for Efficient Cross-Modal Retrieval
Young Kyun Jang, Donghyun Kim, Ser-nam Lim
Vision Language Cross Modal Cross Modal Retrieval Product Quantization Cross Modal Hashing

April 9, 2024

AiSAQ: All-in-Storage ANNS with Product Quantization for DRAM-free Information Retrieval
Kento Tatsuno, Daisuke Miyashita, Taiga Ikeda, Kiyoshi Ishiyama, Kazunari Sumiyoshi, Jun Deguchi
Information Retrieval Approximate Nearest Neighbor Search Quantized Neural Network Product Quantization Graph Based Approximate Nearest Neighbor

March 22, 2024

Magic for the Age of Quantized DNNs
Yoshihide Sawada, Ryuji Saiin, Kazuma Suetake
Speech Based Age Quantization Aware Training LangId Magic Spell Batch Normalization Layer Quantized Neural Network Product Quantization New Normalization Technique Quantised Neural Network

December 12, 2023

Expand-and-Quantize: Unsupervised Semantic Segmentation Using High-Dimensional Space and Product Quantization
Jiyoung Kim, Kyuhong Shim, Insu Lee, Byonghyo Shim
Semantic Segmentation Unsupervised Setting Unsupervised Semantic Segmentation Prompt Expansion High Dimensional Space Product Quantization Clustering Performance

December 10, 2023

QMGeo: Differentially Private Federated Learning via Stochastic Quantization with Mixed Truncated Geometric Distribution
Zixi Wang, M. Cenk Gursoy
Differential Privacy Privacy Guarantee Private Federated Learning Product Quantization Stochastic Quantization

November 27, 2023

PIPE : Parallelized Inference Through Post-Training Quantization Ensembling of Residual Expansions
Edouard Yvinec, Arnaud Dapogny, Kevin Bailly
Deep Neural Network Scientific Inference Multiplier Free Quantization High Performing 1D ConvNet Residual Model Product Quantization Software Evolution Ternary Quantization

November 25, 2023

CUCL: Codebook for Unsupervised Continual Learning
Chen Cheng, Jingkuan Song, Xiaosu Zhu, Junchen Zhu, Lianli Gao, Hengtao Shen
Contrastive Loss Unsupervised Method Codebook Learning Product Quantization Unsupervised Continual Learning Supervised Continual Learning

September 29, 2023

MixQuant: Mixed Precision Quantization with a Bit-width Optimization Search
Eliska Kloberdanz, Wei Le
Mixed Precision Quantization Efficient Deep Product Quantization State of the Art Quantization Optimal Quantization

August 15, 2023

EQ-Net: Elastic Quantization Neural Networks
Ke Xu, Lei Han, Ye Tian, Shangshang Yang, Xingyi Zhang
Product Quantization Zero Shot Quantization Weight Sharing Supernet

May 25, 2023

PQA: Exploring the Potential of Product Quantization in DNN Hardware Acceleration
Ahmed F. AbouElhamayed, Angela Cui, Javier Fernandez-Marques, Nicholas D. Lane, Mohamed S. Abdelfattah
Full Potential DNN Accelerator Quantized Neural Network Product Quantization

May 10, 2023

Post-training Model Quantization Using GANs for Synthetic Data Generation
Athanasios Masouris, Mansi Sharma, Adrian Boguszewski, Alexander Kozlov, Zhuo Wu, Raymond Lo
Generative Adversarial Network Synthetic Data GAN Model Synthetic Data Generation Post Training Product Quantization Model Re Quantization

April 3, 2023

RPTQ: Reorder-based Post-training Quantization for Large Language Models
Zhihang Yuan, Lin Niu, Jiawei Liu, Wenyu Liu, Xinggang Wang, Yuzhang Shang, Guangyu Sun, Qiang Wu, Jiaxiang Wu, Bingzhe Wu
Post Training Quantization Large Scale Language Model Multiplier Free Quantization Product Quantization Memory Reduction

August 13, 2022

PECAN: A Product-Quantized Content Addressable Memory Network
Jie Ran, Rui Lin, Jason Chun Lok Li, Jiajun Zhou, Ngai Wong
Neural Network Deep Neural Network Content Addressable Memory Product Quantization

July 20, 2022

Mixed-Precision Inference Quantization: Radically Towards Faster inference speed, Lower Storage requirement, and Lower Loss
Daning Cheng, Wenguang Chen
Multiplier Free Quantization Mixed Precision Quantization Model Quantization Product Quantization Balanced Loss Mixed Quantization Quantization Model

March 28, 2022

REx: Data-Free Residual Quantization Error Expansion
Edouard Yvinec, Arnaud Dapgony, Matthieu Cord, Kevin Bailly
Deep Neural Network Multiplier Free Quantization High Performing 1D ConvNet Product Quantization Residual Quantization Ternary Quantization

March 9, 2022

Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexes
Yunjiang Jiang, Han Zhang, Yiming Qiu, Yun Xiao, Bo Long, Wen-Yun Yang
Singular Value Decomposition Coordinate Descent Product Quantization Index Decomposition Inner Product Rotation Invariant Learning