Quantization Robust Parameter

Quantization robust parameters aim to create neural network models that maintain accuracy even when their parameters are reduced to lower precision (e.g., 8-bit or even 2-bit integers), crucial for deploying models on resource-constrained devices. Current research focuses on developing methods to predict these robust parameters, often employing graph hypernetworks or incorporating quantization-aware training strategies to improve model resilience. This work is significant because it addresses the trade-off between model size/speed and accuracy, enabling efficient deployment of deep learning in various applications while mitigating the vulnerabilities introduced by quantization.

Papers

September 24, 2023

GHN-QAT: Training Graph Hypernetworks to Predict Quantization-Robust Parameters of Unseen Limited Precision Neural Networks
Stone Yun, Alexander Wong
Quantization Aware Training Low Precision Graph Hypernetworks Quantization Robust Parameter

August 4, 2023

RobustMQ: Benchmarking Robustness of Quantized Models
Yisong Xiao, Aishan Liu, Tianyuan Zhang, Haotong Qin, Jinyang Guo, Xianglong Liu
Native Robustness Adversarial Robustness Quantization Model Intrinsic Robustness Quantization Robust Parameter

April 8, 2023

Benchmarking the Robustness of Quantized Models
Yisong Xiao, Tianyuan Zhang, Shunchang Liu, Haotong Qin
Adversarial Attack Native Robustness Quantization Operator Quantization Model Quantization Robust Parameter

October 17, 2022

ODG-Q: Robust Quantization via Online Domain Generalization
Chaofan Tao, Ngai Wong
Adversarial Attack Domain Generalization Quantization Error Quantization Robust Parameter

August 26, 2022

GHN-Q: Parameter Prediction for Unseen Quantized Convolutional Architectures via Graph Hypernetworks
Stone Yun, Alexander Wong
Convolutional Neural Network Architecture Model Prediction Graph Hypernetworks Quantization Robust Parameter

July 31, 2022

Symmetry Regularization and Saturating Nonlinearity for Robust Quantization
Sein Park, Yeongsang Jang, Eunhyeok Park
Native Robustness Post Training Quantization Quantization Aware Training Quantization Error Saturation Effect Learned Symmetry Quantization Robust Parameter

March 11, 2022

Wireless Quantized Federated Learning: A Joint Computation and Communication Design
Pavlos S. Bouzinis, Panagiotis D. Diamantoulakis, George K. Karagiannidis
Faster Convergence Quantization Error Wireless System Convergence Time Stochastic Quantization Joint Distribution Adaptation Quantization Robust Parameter

January 19, 2022

Q-ViT: Fully Differentiable Quantization for Vision Transformer
Zhexin Li, Tong Yang, Peisong Wang, Jian Cheng
Vision Transformer Uniform Quantization Vision Transformer Quantization Quantization Robust Parameter