Neural Network Quantization

Ask about this topic

Overview

Neural network quantization studies how to use lower-precision numbers for model weights, activations, caches, and training states while preserving useful behavior. Post-training quantization adjusts a trained model using calibration data or reconstruction objectives, while quantization-aware training exposes the model to low precision during optimization; rotations, outlier handling, and mixed-precision allocation are common techniques. Work spans language models and KV caches, vision and diffusion models, and edge devices, with quality measured using benchmarks such as ImageNet, language-model perplexity, reasoning tasks, and generative quality. A central tension is that lower bit widths do not guarantee faster or safer deployment: accuracy and behavior can shift, and real gains depend on hardware, kernels, and the task being evaluated.

Latest papers

CardsList

405 papers64 papers in the last four weeks, up 42% on the four weeks before. 0.4% of all new papers.

  1. Benford's Law as a Distributional Prior for Post-Training Quantization of Large Language Models

    Jan 29, 2026Arthur Negrão, Pedro Silva, Vander L. S. Freitas +2LLM Quantization4-Bit Quantization

    Postgraduate Program in Computer Science Federal University of Ouro Preto · Computing Department Federal University of Ouro Preto

  2. Preserving Fairness and Safety in Quantized LLMs Through Critical Weight Protection

    Jan 17, 2026Muhammad Alif Al Hakim, Alfan Farizki Wicaksono, Fajri KotoLLM QuantizationMultilingual Language Model Evaluation

    Faculty of Computer Science, Universitas Indonesia · Mohamed bin Zayed University of Artificial Intelligence

  3. Activation Sensitivity as a Unifying Principle for Post-Training Quantization

    Jan 15, 2026Bruce Changlong XuLLM QuantizationPost-Training Quantization

  4. ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs

    Jan 12, 2026Haoqian Meng, Yilun Luo, Yafei Zhao +3LLM Quantization4-Bit Quantization

    School of Computer Science and Technology, Tianjin University

  5. QuantKAN: A Unified Quantization Framework for Kolmogorov Arnold Networks

    Nov 24, 2025Kazi Ahmed Asif Fuad, Lizhong ChenQuantization-Aware TrainingKolmogorov-Arnold Networks

    Department of EECS · Oregon State University · Corvallis, OR 97331

  6. Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference

    Nov 19, 2025Kexin Chu, Dawei Xiang, Zixu Shen +3Mixed-Precision QuantizationGPU Acceleration

    University of Connecticut · University of California, Santa Cruz · Independent Researcher

  7. You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

    Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Weight-Only QuantizationMixed-Precision Quantization

    1Technion—Israel Institute of Technology · 2Intuit · 3Ben-Gurion University of the Negev +1

  8. Quantization Effects of Artificial Neural Networks for Embedded Edge-Computing Applications

    Nov 7, 2025Alperen Aksoy, Ilja Bekman, Vesselin Dimitrov +6Neural Network QuantizationBinary Neural Networks

    Peter Grünberg Institute (PGI), Integrated Computing Architectures (ICA | PGI-4), Forschungszentrum Jülich GmbH, Germany · Center for Particle Physics Siegen, Department für Physik, Universität Siegen, Germany · Faculty of Engineering, JARA-FIT Institute for Quantum Information, Forschungszentrum Jülich GmbH and RWTH Aachen University, Germany +1

  9. Activation Quantization of Vision Encoders Needs Prefixing Registers

    Oct 6, 2025Seunghyeon Kim, Taesun Yeom, Jinho Kim +3Activation OutliersActivation Quantization

    POSTECH · NYU Langone Health2 · Google3

  10. QuantV2X: A Fully Quantized Multi-Agent System for Cooperative Perception

    Sep 3, 2025Seth Z. Zhao, Huizhi Zhang, Zhaowei Li +11V2X CommunicationCooperative Perception

    UCLA · UW-Madison · NCSU +3

  11. SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models

    Aug 5, 2025Yufei Xue, Yushi Huang, Lunjie Zhu +2Vision-Language ModelsVLM Quantization

    Institute of Artificial Intelligence (TeleAI), China Telecom · The Hong Kong University of Science and Technology

  12. Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method

    Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision QuantizationLLM Quantization

    Beihang University · Communication University of China · Beijing Jiaotong University

  13. GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers

    Jun 13, 2025Guang Liang, Xinyao Liu, Jianxin WuQuantization-Aware Training4-Bit Quantization

    State Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · University of Science and Technology of China, Hefei, China

  14. NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache

    May 23, 2025Donghyun Son, Euntae Choi, Sungjoo YooKV CachingVector Quantization

    Seoul National University

  15. On Orthogonal Recurrent Neural Networks Quantization

    Feb 5, 2024Armand Foucault, Franck Mamalet, François MalgouyresRecurrent Neural NetworksNeural Network Quantization

    IMT · Comue de Toulouse