Quantization Granularity

Quantization granularity, the precision level used to represent numerical values in neural networks, is crucial for balancing model size, inference speed, and accuracy. Current research focuses on optimizing quantization techniques for various architectures, including Vision Transformers (ViTs) and Large Language Models (LLMs), often employing mixed-precision approaches that tailor the granularity to different parts of the network. These advancements aim to significantly reduce computational costs and memory requirements without sacrificing performance, impacting both the efficiency of deep learning research and the deployment of resource-constrained applications.

Papers

October 9, 2024

Scaling Laws for Mixed quantization in Large Language Models
Zeyu Cao, Cheng Zhang, Pedro Gimenes, Jianqiao Lu, Jianyi Cheng, Yiren Zhao
Scaling Law Mixed Precision Quantization Mixed Quantization Low Precision Quantization Quantization Granularity

June 13, 2024

MGRQ: Post-Training Quantization For Vision Transformer With Mixed Granularity Reconstruction
Lianwei Yang, Zhikai Li, Junrui Xiao, Haisong Gong, Qingyi Gu
Vision Transformer Quantization Operator Post Training Quantization Multi Granularity Quantization Granularity

May 1, 2024

Gradient-based Automatic Mixed Precision Quantization for Neural Networks On-Chip
Chang Sun, Thea K. Årrestad, Vladimir Loncar, Jennifer Ngadiuba, Maria Spiropulu
Multiplier Free Quantization Quantization Aware Training Mixed Precision Quantization Uniform Quantization Chip Learning Quantization Granularity

March 15, 2024

Quantization Avoids Saddle Points in Distributed Optimization
Yanan Bo, Yongqiang Wang
Optimization Purpose Nonconvex Optimization Saddle Point Distributed Optimization Stochastic Quantization Quantization Granularity

November 16, 2023

I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization
Yunshan Zhong, Jiawei Hu, Mengzhao Chen, Rongrong Ji
Vision Transformer Continuum Limit Vision Task Layer Wise Quantization Deeper ViT S 54 Vision Transformer Quantization Quantization Granularity

October 8, 2023

Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?
Cheng Zhang, Jianyi Cheng, Ilia Shumailov, George A. Constantinides, Yiren Zhao
Large Language Model LLM Quantization 8 Bit Quantization Quantization Granularity

Quantization Granularity

Papers

Scaling Laws for Mixed quantization in Large Language Models

MGRQ: Post-Training Quantization For Vision Transformer With Mixed Granularity Reconstruction

Gradient-based Automatic Mixed Precision Quantization for Neural Networks On-Chip

Quantization Avoids Saddle Points in Distributed Optimization

I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization

Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?