Quantization Technique

Quantization techniques aim to reduce the memory footprint and computational cost of deep learning models by representing their weights and activations using fewer bits, thereby accelerating inference and enabling deployment on resource-constrained devices. Current research focuses on developing novel quantization algorithms for various architectures, including large language models (LLMs), diffusion models, and vision transformers, often employing strategies like post-training quantization (PTQ) and quantization-aware training (QAT) to minimize accuracy loss. This area is crucial for advancing the practical applicability of increasingly complex deep learning models across diverse fields, from natural language processing and image generation to speech recognition and computer vision.

Papers

November 9, 2024

Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques
Jahid Hasan
General Analysis Quantization Operator Quantization Aware Training Mixed Precision Quantization Quantization Technique

October 30, 2024

A Comprehensive Study on Quantization Techniques for Large Language Models
Jiedong Lang, Zhehao Guo, Shuyu Huang
Full Model Large Model Large Language Comprehensive Study Quantization Technique Quantization Step Different Quantization

October 17, 2024

AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations
Qian Tao, Wenyuan Yu, Jingren Zhou
Large Language Model Quantization Operator Layer Wise Quantization Technique KV Cache

September 3, 2024

Robust Clustering on High-Dimensional Data with Stochastic Quantization
Anton Kozyriev, Vladimir Norkin
High Dimensional Data Quantization Technique Mini Batch Gradient Descent Robust Clustering Stochastic Quantization

August 16, 2024

ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
Chao Zeng, Songwei Liu, Yusheng Xie, Hong Liu, Xiaojian Wang, Miao Wei, Shu Yang, Fangmin Chen, Xing Mei
Multiplier Free Quantization Quantization Technique Matrix Multiplication Low Bit Quantization Bayesian Quadrature

July 28, 2024

Temporal Feature Matters: A Framework for Diffusion Model Quantization
Yushi Huang, Ruihao Gong, Xianglong Liu, Jing Liu, Yuhang Li, Jiwen Lu, Dacheng Tao
Diffusion Model New Framework Post Training Quantization Temporal Feature Quantization Technique Dynamic Reconstruction

July 23, 2024

Pixel Embedding: Fully Quantized Convolutional Neural Network with Differentiable Lookup Table
Hiroyuki Tokunaga, Joel Nicholls, Daria Vazhenina, Atsunori Kanemura
Convolutional Neural Network Deep Neural Network Supervised ImageNet Tetromino Pixel Quantization Technique Resolution Input

July 12, 2024

Optimization of DNN-based speaker verification model through efficient quantization technique
Yeona Hong, Woo-Jin Chung, Hong-Goo Kang
Optimization Purpose Speaker Verification Quantization Technique

June 15, 2024

Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolbox
Yijun Liu, Yuan Meng, Fang Wu, Shenhao Peng, Hang Yao, Chaoyu Guan, Chen Tang, Xinzhu Ma, Zhi Wang, Wenwu Zhu
Large Language Model New Benchmark General Analysis Multiplier Free Quantization Quantization Technique Flexible Toolkit LLM Quantization

June 12, 2024

Examining Post-Training Quantization for Mixture-of-Experts: A Benchmark
Pingzhi Li, Xiaolong Jin, Yu Cheng, Tianlong Chen
Large Language Model Natural Language Processing New Benchmark Mixture of Expert Post Training Quantization Quantization Technique Activation Quantization

June 5, 2024

QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead
Amir Zandieh, Majid Daliri, Insu Han
Quantization Technique Key Value Cache Cache Quantization Johnson Lindenstrauss Transform

June 3, 2024

DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs
Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang, Linzhan Mou, Linqi Song, Zhenan Sun, Ying Wei
Large Language Model Quantization Technique Rotation Angle Machine Permutation Activation Quantization Low Bit Quantization

May 24, 2024

Athena: Efficient Block-Wise Post-Training Quantization for Large Language Models Using Second-Order Matrix Derivative Information
Yanshu Wang, Wenyang He, Tong Yang
Text Generation Natural Language Processing Task Post Training Quantization Quantization Technique

May 14, 2024

Neural Speech Coding for Real-time Communications using Constant Bitrate Scalar Quantization
Andreas Brendel, Nicola Pia, Kishan Gupta, Lyonel Behringer, Guillaume Fuchs, Markus Multrus
Neural Audio Quantization Technique Neural Speech

May 9, 2024

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit
Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Xianglong Liu, Dacheng Tao
Large Language Model New Benchmark Post Training Quantization Best Practice Quantization Technique LLM dCache

March 29, 2024

Accurate Block Quantization in LLMs with Outliers
Nikita Trukhanov, Ilya Soloveychik
Medical LLM Quantization Technique KV Cache Accurate Quantization Chip Memory Floating Point Format Block Quantization

February 28, 2024

Evaluating Quantized Large Language Models
Shiyao Li, Xuefei Ning, Luning Wang, Tengxuan Liu, Xiangsheng Shi, Shengen Yan, Guohao Dai, Huazhong Yang, Yu Wang
Large Language Model Post Training Quantization Quantization Technique

February 26, 2024

A Comprehensive Evaluation of Quantization Strategies for Large Language Models
Renren Jin, Jiangcun Du, Wuwei Huang, Wei Liu, Jian Luan, Bin Wang, Deyi Xiong
Large Language Model Comprehensive Evaluation Quantization Technique Instruction Tuned Large Language Model

February 7, 2024

ApiQ: Finetuning of 2-Bit Quantized Large Language Model
Baohao Liao, Christian Herold, Shahram Khadivi, Christof Monz
Large Language Model Fine Tuning Quantization Technique Memory Efficient Fine Tuning Bit Width Quantization

December 9, 2023

Efficient Quantization Strategies for Latent Diffusion Models
Yuewei Yang, Xiaoliang Dai, Jialiang Wang, Peizhao Zhang, Hongbo Zhang
Latent Diffusion Model Quantization Technique Quantization Loss

Quantization Technique

Papers

Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques

A Comprehensive Study on Quantization Techniques for Large Language Models

AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations

Robust Clustering on High-Dimensional Data with Stochastic Quantization

ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models

Temporal Feature Matters: A Framework for Diffusion Model Quantization

Pixel Embedding: Fully Quantized Convolutional Neural Network with Differentiable Lookup Table

Optimization of DNN-based speaker verification model through efficient quantization technique

Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolbox

Examining Post-Training Quantization for Mixture-of-Experts: A Benchmark

QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead

DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs

Athena: Efficient Block-Wise Post-Training Quantization for Large Language Models Using Second-Order Matrix Derivative Information

Neural Speech Coding for Real-time Communications using Constant Bitrate Scalar Quantization

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit

Accurate Block Quantization in LLMs with Outliers

Evaluating Quantized Large Language Models

A Comprehensive Evaluation of Quantization Strategies for Large Language Models

ApiQ: Finetuning of 2-Bit Quantized Large Language Model

Efficient Quantization Strategies for Latent Diffusion Models