Accurate Quantization

Accurate quantization aims to reduce the precision of numerical representations in deep learning models (e.g., weights and activations) without significantly sacrificing performance. Current research focuses on developing novel quantization techniques for various architectures, including transformers and diffusion models, often employing mixed-precision strategies and addressing challenges like outlier values and imbalanced distributions through methods such as Fisher information analysis, activation regularization, and scale reparameterization. These advancements are crucial for deploying large-scale models on resource-constrained devices, improving inference speed and reducing memory footprint across diverse applications like natural language processing, image generation, and object detection.

Papers

October 17, 2024

Quamba: A Post-Training Quantization Recipe for Selective State Space Models
Hung-Yueh Chiang, Chi-Chih Chang, Natalia Frumkin, Kai-Chiang Wu, Diana Marculescu
Large Language Model Full Model Long Sequence Selective State Space Accurate Quantization Per Tensor

October 16, 2024

Channel-Wise Mixed-Precision Quantization for Large Language Models
Zihan Chen, Bike Xie, Jundong Li, Cong Shen
Mixed Precision Quantization Channel Wise Weight Only Quantization Accurate Quantization Non Uniform Quantization

October 9, 2024

QuAILoRA: Quantization-Aware Initialization for LoRA
Neal Lawton, Aishwarya Padmakumar, Judith Gaspers, Jack FitzGerald, Anoop Kumar, Greg Ver Steeg, Aram Galstyan
Large Language Model Quantization Error Safe LoRA Accurate Quantization Driven Initialization

October 3, 2024

SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
Jintao Zhang, Jia wei, Haofeng Huang, Pengle Zhang, Jun Zhu, Jianfei Chen
Transformer Based Transformer Architecture Multiplier Free Quantization Pixel Attention Accurate Quantization

September 25, 2024

AlignedKV: Reducing Memory Access of KV-Cache with Precision-Aligned Quantization
Yifan Tan, Haoze Wang, Chao Yan, Yangdong Deng
LD Align KV Cache Accurate Quantization Different Quantization Memory Access Cache Quantization

July 3, 2024

Fisher-aware Quantization for DETR Detectors with Critical-category Objectives
Huanrui Yang, Yafeng Huang, Zhen Dong, Denis A Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Yuan Du, Kurt Keutzer, Shanghang Zhang
Loss Landscape High Performance Task Performance DETR Based Detector Accurate Quantization Mixed Quantization

June 4, 2024

ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
Tianchen Zhao, Tongcheng Fang, Enshu Liu, Rui Wan, Widyadewi Soedarmadji, Shiyao Li, Zinan Lin, Guohao Dai, Shengen Yan, Huazhong Yang, Xuefei Ning, Yu Wang
Diffusion Model Video Generation Diffusion Transformer Accurate Quantization

May 22, 2024

QGait: Toward Accurate Quantization for Gait Recognition with Binarized Input
Senmao Tian, Haoyu Gao, Gangyi Hong, Shuyun Wang, JingJie Wang, Xin Yu, Shunli Zhang
Gait Recognition Binarization Method Human Silhouette Accurate Quantization

April 4, 2024

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
Aniruddha Nrusimha, Mayank Mishra, Naigang Wang, Dan Alistarh, Rameswar Panda, Yoon Kim
Large Language Model Global Impact Post Training Quantization Quantization Aware Training Activation Quantization Accurate Quantization Task Specific Channel Input Quantization

March 29, 2024

Accurate Block Quantization in LLMs with Outliers
Nikita Trukhanov, Ilya Soloveychik
Medical LLM Quantization Technique KV Cache Accurate Quantization Chip Memory Floating Point Format Block Quantization

February 27, 2024

Adaptive quantization with mixed-precision based on low-cost proxy
Junzhe Chen, Qiao Yang, Senmao Tian, Shunli Zhang
Mixed Precision Quantization Model Quantization Zero Cost Proxy Adaptive Quantization Accurate Quantization Quantization Parameter

February 8, 2024

RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization
Zhikai Li, Xuewen Liu, Jing Zhang, Qingyi Gu
Post Training Quantization Large Transformer Model Accurate Quantization Large Scale Transformer

February 6, 2024

QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning
Haoxuan Wang, Yuzhang Shang, Zhihang Yuan, Junyi Wu, Junchi Yan, Yan Yan
Diffusion Model Multiplier Free Quantization Bit Quantization Accurate Quantization QueST

November 9, 2023

Reducing the Side-Effects of Oscillations in Training of Quantized YOLO Networks
Kartik Gupta, Akshay Asthana
Training Data Quantization Aware Training YOLO Model Side Effect Non Converging Artificial Oscillation Accurate Quantization Network Quantization

June 21, 2023

Beyond Learned Metadata-based Raw Image Reconstruction
Yufei Wang, Yi Yu, Wenhan Yang, Lanqing Guo, Lap-Pui Chau, Alex C. Kot, Bihan Wen
Lossy Image Compression Accurate Quantization RAW Image sRGB Image

April 18, 2023

Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, Xianglong Liu
Large Language Model Post Training Quantization Transformer Language Model Accurate Quantization Outlier Removal Fine Grained Quantization

December 16, 2022

RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers
Zhikai Li, Junrui Xiao, Lianwei Yang, Qingyi Gu
Vision Transformer Quantization Operator Post Training Quantization Accurate Quantization GPUSQ ViT

December 6, 2022

CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification
Lirui Xiao, Huanrui Yang, Zhen Dong, Kurt Keutzer, Li Du, Shanghang Zhang
Mixed Precision Quantization Accurate Quantization

July 13, 2022

DiverGet: A Search-Based Software Testing Approach for Deep Neural Network Quantization Assessment
Ahmed Haj Yahmed, Houssem Ben Braiek, Foutse Khomh, Sonia Bouzidi, Rania Zaatour
Deep Neural Network State of the Art Quantization Operator Quantization Technique Search Based Accurate Quantization

November 29, 2021

Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
Zechun Liu, Kwang-Ting Cheng, Dong Huang, Eric Xing, Zhiqiang Shen
Accurate Quantization Non Uniform Quantization Straight Through Estimator

Accurate Quantization

Papers

Quamba: A Post-Training Quantization Recipe for Selective State Space Models

Channel-Wise Mixed-Precision Quantization for Large Language Models

QuAILoRA: Quantization-Aware Initialization for LoRA

SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration

AlignedKV: Reducing Memory Access of KV-Cache with Precision-Aligned Quantization

Fisher-aware Quantization for DETR Detectors with Critical-category Objectives

ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation

QGait: Toward Accurate Quantization for Gait Recognition with Binarized Input

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization

Accurate Block Quantization in LLMs with Outliers

Adaptive quantization with mixed-precision based on low-cost proxy

RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization

QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning

Reducing the Side-Effects of Oscillations in Training of Quantized YOLO Networks

Beyond Learned Metadata-based Raw Image Reconstruction

Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling

RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers

CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification

DiverGet: A Search-Based Software Testing Approach for Deep Neural Network Quantization Assessment

Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation