Quantization Scale

Quantization scale optimization aims to reduce the precision of neural network weights and activations, thereby decreasing model size and improving inference speed without significant accuracy loss. Current research focuses on developing efficient algorithms, such as evolutionary search and contrastive learning, to find optimal quantization scales, particularly for large language models (LLMs) and vision transformers (ViTs), often employing techniques like vector quantization and lookup tables. These advancements are crucial for deploying large models on resource-constrained devices and accelerating inference, impacting both the efficiency of AI systems and their accessibility.

Papers

July 15, 2024

Fast Matrix Multiplications for Lookup Table-Quantized LLMs
Han Guo, William Brandon, Radostin Cholakov, Jonathan Ragan-Kelley, Eric P. Xing, Yoon Kim
Large Language Model Matrix Multiplication Weight Only Quantization Kernel Performance Quantization Scale

February 23, 2024

GPTVQ: The Blessing of Dimensionality for LLM Quantization
Mart van Baalen, Andrey Kuzmin, Markus Nagel, Peter Couperus, Cedric Bastoul, Eric Mahurin, Tijmen Blankevoort, Paul Whatmough
Vector Quantization Data Dimensionality LLM Quantization Neural Network Quantization Blessing Generation Integer Quantization Multi Codebook Quantization Scale

August 21, 2023

Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers
Natalia Frumkin, Dibakar Gope, Diana Marculescu
Vision Transformer Quantization Operator Loss Landscape Weight Quantization Local Minimum Extreme Quantization Quantization Scale

May 24, 2023

RAND: Robustness Aware Norm Decay For Quantized Seq2seq Models
David Qiu, David Rim, Shaojin Ding, Oleg Rybakov, Yanzhang He
Automatic Speech Recognition Model Compression Long Form Quantization Noise Quantization Scale

March 23, 2023

Scaled Quantization for the Vision Transformer
Yangyang Chang, Gerald E. Sobelman
Vision Transformer Quantization Operator Quantization Technique Vision Transformer Network Integer Quantization Quantization Scale

November 17, 2022

CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformers
Natalia Frumkin, Dibakar Gope, Diana Marculescu
Vision Transformer Loss Function Contrastive Loss Quantization Operator Post Training Quantization Contrastive Method Weight Quantization Quantization Scale