Quantization Operator

Quantization is a model compression technique that reduces the precision of numerical representations in neural networks, aiming to decrease computational costs and memory footprint while preserving model accuracy. Current research focuses on applying quantization to various deep learning architectures, including Vision Transformers (ViTs), large language models (LLMs), and diffusion models, often employing post-training quantization (PTQ) methods to avoid retraining the entire model. This work is significant because it enables the deployment of large, computationally expensive models on resource-constrained devices, impacting fields like healthcare, edge computing, and natural language processing by making advanced AI more accessible and efficient.

Papers

June 10, 2024

The Impact of Quantization on Retrieval-Augmented Generation: An Analysis of Small LLMs
Mert Yazan, Suzan Verberne, Frederik Situmeang
Large Language Model General Analysis Global Impact Retrieval Augmented Generation Long Context Quantization Operator Retrieval Model

June 5, 2024

Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning
Inwoo Hwang, Yunhyeok Kwak, Suhyung Choi, Byoung-Tak Zhang, Sanghack Lee
Reinforcement Learning Native Robustness Causal Structure Quantization Operator Causal Dynamic Learning Fine Grained Causal

June 1, 2024

Privacy-Aware Randomized Quantization via Linear Programming
Zhongteng Cai, Xueru Zhang, Mohammad Mahdi Khalili
Differential Privacy Quantization Operator Linear Programming Differential Privacy Mechanism Privacy Accuracy Different Quantization Privacy Aware

May 31, 2024

May 30, 2024

P$^2$-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer
Huihong Shi, Xin Cheng, Wendong Mao, Zhongfeng Wang
Vision Transformer Quantization Operator Low Rate ACCELERATION Mixed Precision Quantization ViT Model Model Re Quantization

May 28, 2024

May 27, 2024

CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs
Haoyu Wang, Bei Liu, Hang Shao, Bo Xiao, Ke Zeng, Guanglu Wan, Yanmin Qian
Large Language Model Continuum Limit Quantization Operator Low Bit LLM Quantization Weight Only Quantization Parameter Quantization Class Centroid

May 16, 2024

The Effect of Quantization in Federated Learning: A R\'enyi Differential Privacy Perspective
Tianqu Kang, Lumin Liu, Hengtao He, Jun Zhang, S. H. Song, Khaled B. Letaief
Differential Privacy Mixed Effect Membership Inference Attack Quantization Operator PRIvacy Leakage Multiplier Free Quantization

May 12, 2024

Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization
Xinyuan Zhang, Jiang Liu, Zehui Xiong, Yudong Huang, Gaochang Xie, Ran Zhang
Generative Artificial Intelligence Quantization Operator Large Language Model Inference Edge Intelligence Batch Data Edge Resource

May 7, 2024

KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
Tianyi Zhang, Jonah Yi, Zhaozhuo Xu, Anshumali Shrivastava
Quantization Operator Large Language Model Inference P Bit Key Value Cache Channel Quality Key Value Cache Compression

May 6, 2024

May 1, 2024

When Quantization Affects Confidence of Large Language Models?
Irina Proskurina, Luc Brun, Guillaume Metzler, Julien Velcin
Language Model Quantization Operator High Confidence Quantization Loss Bit Weight

April 17, 2024

QGen: On the Ability to Generalize in Quantization Aware Training
MohammadHossein AskariHemmat, Ahmadreza Jeddi, Reyhane Askari Hemmat, Ivan Lazarevich, Alexander Hoffman, Sudhakar Sah, Ehsan Saboori, Yvon Savaria, Jean-Pierre David
Quantization Operator Critique Ability ImageNet Dataset Quantization Aware Training Generalization Property Quantization Noise Quantization Function

April 15, 2024

Quantization of Large Language Models with an Overdetermined Basis
Daniil Merkulov, Daria Cherniuk, Alexander Rudikov, Ivan Oseledets, Ekaterina Muravleva, Aleksandr Mikhalev, Boris Kashin
Quantization Operator Data Compression Compression Algorithm Interpretable Basis

April 8, 2024

Quantization Operator

Papers

The Impact of Quantization on Retrieval-Augmented Generation: An Analysis of Small LLMs

Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning

Privacy-Aware Randomized Quantization via Linear Programming

LCQ: Low-Rank Codebook based Quantization for Large Language Models

Effective Interplay between Sparsity and Quantization: From Theory to Practice

Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs

P$^2$-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer

Exploiting LLM Quantization

MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization

CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs

The Effect of Quantization in Federated Learning: A R\'enyi Differential Privacy Perspective

Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization

KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization

Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer

PTQ4SAM: Post-Training Quantization for Segment Anything

When Quantization Affects Confidence of Large Language Models?

QGen: On the Ability to Generalize in Quantization Aware Training

Quantization of Large Language Models with an Overdetermined Basis

David and Goliath: An Empirical Evaluation of Attacks and Defenses for QNNs at the Deep Edge

Investigating the Impact of Quantization on Adversarial Robustness