Quantization Performance

Quantization performance focuses on efficiently representing neural network weights and activations using lower precision (e.g., INT4, INT8) to reduce memory footprint and accelerate inference, particularly crucial for deploying models on resource-constrained devices. Current research emphasizes improving post-training quantization (PTQ) techniques through methods like vector quantization, trellis coded quantization, and outlier mitigation strategies, often applied to transformer-based models (LLMs) and convolutional networks (e.g., RepVGG). These advancements are significant for enabling the deployment of large and complex models on edge devices and improving the efficiency of various applications, including image classification, object detection, and natural language processing.

Papers

November 21, 2024

Quantization without Tears
Minghao Fu, Hao Yu, Jie Shao, Junjie Zhou, Ke Zhu, Jianxin Wu
Quantization Operator Multiplier Free Quantization Quantization Technique Network Quantization Quantization Performance Image Quantization Tear Film

November 8, 2024

MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization
Akshat Ramachandran, Souvik Kundu, Tushar Krishna
Foundational Model Inlier Classification Quantization Performance

October 29, 2024

Data Generation for Hardware-Friendly Post-Training Quantization
Lior Dikstein, Ariel Lapid, Arnon Netzer, Hai Victor Habi
Quantization Operator Post Training Quantization Data Generation Zero Shot Quantization Quantization Performance Heterogeneous Quantization

October 22, 2024

Pyramid Vector Quantization for LLMs
Tycho F. A. van der Ouderaa, Maximilian L. Croci, Agrin Hilmkil, James Hensman
Large Language Model Medical LLM Vector Quantization Quantization Performance

July 17, 2024

Toward INT4 Fixed-Point Training via Exploring Quantization Error for Gradients
Dohyung Kim, Junghyup Lee, Jeimin Jeon, Jaehyeon Moon, Bumsub Ham
Quantization Error Quantization Level Network Quantization Quantization Performance

June 17, 2024

QTIP: Quantization with Trellises and Incoherence Processing
Albert Tseng, Qingyao Sun, David Hou, Christopher De Sa
Quantization Operator Post Training Quantization Vector Quantization Novel Encoding Extreme Quantization Tree Structure Quantization Performance Incoherence Processing

June 16, 2024

Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models
Dominik Wagner, Ilja Baumann, Korbinian Riedhammer, Tobias Bocklet
Automatic Speech Recognition Transformer Based Model Post Training Quantization Speech Model Outlier Removal Quantization Performance Gated Attention

May 22, 2024

Two Heads are Better Than One: Neural Networks Quantization with 2D Hilbert Curve-based Output Representation
Mykhailo Uss, Ruslan Yermolenko, Olena Kolodiazhna, Oleksii Shashko, Ivan Safonov, Volodymyr Savin, Yoonjae Yeo, Seowon Ji, Jaeyun Jeong
Structured Output Post Training Quantization Human Head Quantization Aware Training Quantization Error Quantization Performance 2 Dimensional Hilbert

January 29, 2024

LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detection
Sifan Zhou, Liang Li, Xinyu Zhang, Bo Zhang, Shipeng Bai, Miao Sun, Ziyu Zhao, Xiaobo Lu, Xiangxiang Chu
Point Cloud Quantization Operator Quantization Performance Lidar Based 3D Detection Cross LiDAR 3D Detection LiDAR Object Detection Network

September 25, 2023

On Calibration of Modern Quantized Efficient Neural Networks
Joey Kuang, Alexander Wong
Calibration Performance Calibration Error Quantized Neural Network Quantization Performance

June 13, 2023

SqueezeLLM: Dense-and-Sparse Quantization
Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W. Mahoney, Kurt Keutzer
Generative Modeling Generative Large Language Model GPU Inference Squeeze Flow Quantization Performance

April 19, 2023

Improving Post-Training Quantization on Object Detection with Task Loss-Guided Lp Metric
Lin Niu, Jiawei Liu, Zhihang Yuan, Dawei Yang, Xinggang Wang, Wenyu Liu
Object Detection Post Training Quantization Metric Learning Loss Detection Loss Quantization Performance

December 3, 2022

Make RepVGG Greater Again: A Quantization-aware Approach
Xiangxiang Chu, Liang Li, Bo Zhang
Supervised ImageNet Post Training Quantization Quantization Performance Heterogeneous Quantization

November 30, 2022

Quadapter: Adapter for GPT-2 Quantization
Minseop Park, Jaeseong You, Markus Nagel, Simyung Chang
Language Model Transformer Language Model Quantization Aware Training Quantization Error Long Short Range Adapter Quantization Performance

July 7, 2022

Attention Round for Post-Training Quantization
Huabin Diao, Gongyan Li, Shaoyun Xu, Yuexing Hao
Quantization Operator Post Training Quantization Attention Head Mixed Precision Quantization Quantization Step Quantization Performance

November 23, 2021

HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performance
Huanrui Yang, Xiaoxuan Yang, Neil Zhenqiang Gong, Yiran Chen
Neural Network Stochastic Gradient Descent Robust Optimization Post Training Quantization Anti Unification Fixed Point Enhancing Generalization Quantization Performance