Quantization Operator

Quantization is a model compression technique that reduces the precision of numerical representations in neural networks, aiming to decrease computational costs and memory footprint while preserving model accuracy. Current research focuses on applying quantization to various deep learning architectures, including Vision Transformers (ViTs), large language models (LLMs), and diffusion models, often employing post-training quantization (PTQ) methods to avoid retraining the entire model. This work is significant because it enables the deployment of large, computationally expensive models on resource-constrained devices, impacting fields like healthcare, edge computing, and natural language processing by making advanced AI more accessible and efficient.

Papers

November 17, 2024

EfQAT: An Efficient Framework for Quantization-Aware Training
Saleh Ashkboos, Bram Verhoef, Torsten Hoefler, Evangelos Eleftheriou, Martino Dazzi
Pre Trained Model Quantization Operator Multidimensional Local Precision Rate Quantization Aware Training Efficient Framework

November 11, 2024

HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignment
Yannis Belkhiter, Giulio Zizzo, Sergio Maffeis
Language Model Fine Grained Global Impact Transformer Architecture Jailbreak Attack Quantization Operator LLM Safety Model Alignment

November 9, 2024

Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques
Jahid Hasan
General Analysis Quantization Operator Quantization Aware Training Quantization Technique Mixed Precision Quantization

November 8, 2024

When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantization
Jacob Nielsen, Lukas Galke, Peter Schneider-Kamp
Language Model Multi Layer Quantization Operator Transformer Based Language Model P Bit Decoder Only Language Model Binary Forward Exploration

November 4, 2024

"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization
Eldar Kurtic, Alexandre Marques, Shubhra Pandit, Mark Kurtz, Dan Alistarh
Large Language Model Quantization Operator Activation Quantization Weight Only Quantization Inference Performance Different Quantization Accuracy Efficiency Trade

October 29, 2024

Data Generation for Hardware-Friendly Post-Training Quantization
Lior Dikstein, Ariel Lapid, Arnon Netzer, Hai Victor Habi
Quantization Operator Post Training Quantization Data Generation Zero Shot Quantization Quantization Performance Heterogeneous Quantization

October 26, 2024

You Never Know: Quantization Induces Inconsistent Biases in Vision-Language Foundation Models
Eric Slyman, Anirudh Kanneganti, Sanghyun Hong, Stefan Lee
Vision Language Model Quantization Operator Multiplier Free Quantization Unimodal Model Quantization Parameter

October 24, 2024

TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction
Yuhang Li, Priyadarshini Panda
Large Language Model Natural Language Processing Full State Reconstruction Quantization Operator Post Training Quantization Multi Modal Outer Arithmetic Block Weight Only Quantization

October 17, 2024

AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations
Qian Tao, Wenyuan Yu, Jingren Zhou
Large Language Model Quantization Operator Layer Wise Quantization Technique KV Cache

October 11, 2024

QEFT: Quantization for Efficient Fine-Tuning of LLMs
Changhun Lee, Jun-gyu Jin, Younghyun Cho, Eunhyeok Park
Large Language Model Fine Tuning Medical LLM Parameter Efficient Fine Tuning Quantization Operator Inference Speed

October 10, 2024

October 8, 2024

October 6, 2024

Continuous Approximations for Improving Quantization Aware Training of LLMs
He Li, Jianhang Hong, Yuanzhuo Wu, Snehal Adbol, Zonglin Li
Large Language Model Model Compression Quantization Operator Quantization Aware Training Approximation Theory

September 26, 2024

P4Q: Learning to Prompt for Quantization in Visual-language Models
Huixin Sun, Runqi Wang, Yanjing Li, Xianbin Cao, Xiaolong Jiang, Yao Hu, Baochang Zhang
Quantization Operator Pre Trained Vision Language Model Style PROMPT Visual Language Model Quantization Aware Training Multimodal Task Low Bit Quantization

September 25, 2024

PTQ4RIS: Post-Training Quantization for Referring Image Segmentation
Xiaoyan Jiang, Hang Yang, Kaiying Zhu, Xihe Qiu, Shibo Zhao, Sifan Zhou
Quantization Operator Referring Image Segmentation

September 20, 2024

PTQ4ADM: Post-Training Quantization for Efficient Text Conditional Audio Diffusion Models
Jayneel Vora, Aditya Krishnan, Nader Bouacida, Prabhu RV Shankar, Prasant Mohapatra
Quantization Operator Denoising Diffusion Model Text to Speech Synthesis Quantization Noise

September 14, 2024

Privacy-Preserving SAM Quantization for Efficient Edge Intelligence in Healthcare
Zhikai Li, Jing Zhang, Qingyi Gu
Segment Anything Model Quantization Operator Healthcare System Low Bit Quantization Sensitive Patient Data Free Quantization

September 10, 2024

Rate-Constrained Quantization for Communication-Efficient Federated Learning
Shayan Mohajer Hamidi, Ali Bereyhi
Quantization Operator Compression Technique Quantization Error Weight Only Quantization Gradient Coding

Quantization Operator

Papers

EfQAT: An Efficient Framework for Quantization-Aware Training

HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignment

Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques

When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantization

"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization

Data Generation for Hardware-Friendly Post-Training Quantization

You Never Know: Quantization Induces Inconsistent Biases in Vision-Language Foundation Models

TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction

AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations

QEFT: Quantization for Efficient Fine-Tuning of LLMs

Q-VLM: Post-training Quantization for Large Vision-Language Models

M$^2$-ViT: Accelerating Hybrid Vision Transformers with Two-Level Mixed Quantization

QT-DoG: Quantization-aware Training for Domain Generalization

Integrated Encoding and Quantization to Enhance Quanvolutional Neural Networks

Continuous Approximations for Improving Quantization Aware Training of LLMs

P4Q: Learning to Prompt for Quantization in Visual-language Models

PTQ4RIS: Post-Training Quantization for Referring Image Segmentation

PTQ4ADM: Post-Training Quantization for Efficient Text Conditional Audio Diffusion Models

Privacy-Preserving SAM Quantization for Efficient Edge Intelligence in Healthcare

Rate-Constrained Quantization for Communication-Efficient Federated Learning