Neural Network Quantization

Ask about this topic

Overview

Neural network quantization studies how to use lower-precision numbers for model weights, activations, caches, and training states while preserving useful behavior. Post-training quantization adjusts a trained model using calibration data or reconstruction objectives, while quantization-aware training exposes the model to low precision during optimization; rotations, outlier handling, and mixed-precision allocation are common techniques. Work spans language models and KV caches, vision and diffusion models, and edge devices, with quality measured using benchmarks such as ImageNet, language-model perplexity, reasoning tasks, and generative quality. A central tension is that lower bit widths do not guarantee faster or safer deployment: accuracy and behavior can shift, and real gains depend on hardware, kernels, and the task being evaluated.

Latest papers

CardsList

405 papers64 papers in the last four weeks, up 42% on the four weeks before. 0.4% of all new papers.

  1. Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees

    Apr 7, 2026Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai +4Mixture-of-Experts QuantizationNeural Network Quantization

    Rensselaer Polytechnic Institute · IBM Research

  2. MUXQ: Mixed-to-Uniform Precision MatriX Quantization via Low-Rank Outlier Decomposition

    Apr 6, 2026Seoungsub Lee, In Seo Kim, Seon Wook KimLLM QuantizationNeural Network Quantization

    School of Electrical Engineering, Korea University, Seoul, Korea

  3. Integer-Only Operations on Extreme Learning Machine Test Time Classification

    Apr 6, 2026Emerson Lopes Machadoa, Cristiano Jacques Miosso, Ricardo Pezzuol JacobiEfficient Neural Network InferenceNeural Network Quantization

    Dept. of Computer Science, University of Brasilia, Brazil · University of Brasilia at Gama, Brazil

  4. Zero-Shot Quantization via Weight-Space Arithmetic

    Apr 3, 2026Daniele Solombrino, Antonio Andrea Gargiulo, Alessandro Zirilli +3Quantization RobustnessQuantization

    Sapienza University of Rome · Sapienza University of Rome / Paradigma

  5. Fast NF4 Dequantization Kernels for Large Language Model Inference

    Apr 2, 2026Xiangbo Qi, Chaoyi Jiang, Murali AnnavaramLLM Inference AccelerationGPU Kernel Optimization

    University of Southern California Los Angeles, USA

  6. Integer-State Dynamics of Quantized Spiking Neural Networks for Efficient Hardware Acceleration

    Apr 1, 2026Lei ZhangSpiking Neural NetworksNeuromorphic Computing

    Faculty of Engineering and Applied Science, University of Regina

  7. MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference

    Apr 1, 2026Zifei Xu, Sayeh Sharify, Hesham MostafaQuantization-Aware TrainingNeural Network Quantization

    d-Matrix, Santa Clara, CA 95054

  8. PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression

    Mar 30, 2026Caio VicentinoLLM QuantizationNeural Network Quantization

  9. Adaptive Block-Scaled Data Types

    Mar 30, 2026Jack Cook, Hyemin S. Lee, Kathryn Le +4Neural Network QuantizationLLM Quantization

    Massachusetts Institute of Technology · NVIDIA

  10. OneComp: One-Line Revolution for Generative AI Model Compression

    Mar 30, 2026Yuma Ichikawa, Keiji Kimura, Akihiro Yoshida +11Hardware-Aware QuantizationModel Compression

    Fujitsu Limited · RIKEN Center for AIP · Institute of Science Tokyo +2

  11. TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization

    Mar 29, 2026Dipkumar PatelKV-Cache CompressionKV-Cache Quantization

    LLMs Research Inc.

  12. PQuantML: A Tool for End-to-End Hardware-aware Model Compression

    Mar 27, 2026Roope Niemi, Anastasiia Petrovych, Arghya Ranjan Das +9Model CompressionNeural Network Compression

    European Center for Nuclear Research (CERN), CH-1211 Geneva, Switzerland · Purdue University, West Lafayette, IN 47907, USA · California Institute of Technology, Pasadena, CA 91125, United States +4

  13. SliderQuant: Accurate Post-Training Quantization for LLMs

    Mar 26, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +3LLM QuantizationPost-Training Quantization

    Intel Labs China · BUPT

  14. APreQEL: Adaptive Mixed Precision Quantization For Edge LLMs

    Mar 24, 2026Meriem Bouzouad, Yuan-Hao Chang, Jalil BoukhobzaHardware-Aware QuantizationLLM Quantization

    Lab-STICC, CNRS UMR 6285 , ENSTA, Institut Polytechnique de Paris, Brest, France · National Taiwan University, Taipei, Taiwan

  15. From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips

    Mar 24, 2026Alan T. L. Bacellar, Sathvik Chemudupati, Shashank Nag +4Neural Network RobustnessNeural Network Quantization

    The University of Texas at Austin · Federal University of Rio de Janeiro · Instituto de Telecomunicações, Porto, Portugal

  16. Efficient Fine-Tuning Methods for Portuguese Question Answering: A Comparative Study of PEFT on BERTimbau and Exploratory Evaluation of Generative LLMs

    Mar 22, 2026Mariela M. Nina, Caio Veloso Costa, Lilian Berton +1Parameter-Efficient Fine-TuningQuestion Answering

    Institute of Science and Technology, Federal University of São Paulo (UNIFESP), São José dos Campos, SP, Brazil

  17. DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

    Mar 20, 2026Xiaoming Yu, Shize Tang, Guanghua Yu +4QuantizationLLM Quantization

    Yuanbao & Hunyuan AI Infra Team

  18. Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition

    Mar 20, 2026Calvin Ang, Sungyoon Kim, Mert PilanciNeural Network QuantizationActivation Quantization

    Stanford University

  19. Prune-then-Quantize or Quantize-then-Prune? Understanding the Impact of Compression Order in Joint Model Compression

    Mar 19, 2026Minjun Kim, Jaehyeon Choi, Hyunwoo Yang +3Model CompressionNeural Network Quantization

    Seoul National University, Seoul, South Korea

  20. SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning

    Mar 19, 2026Minjun Kim, Jongjin Kim, U KangNeural Network QuantizationQuantization-Aware Training

    Seoul National University, Seoul, South Korea

  21. RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference

    Mar 18, 2026Arpit Singh Gautam, Saurabh JhaLLM QuantizationNeural Network Quantization

  22. Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients

    Mar 18, 2026Ziwei Xiang, Fanhu Zeng, Hongjian Fang +6VLM QuantizationPost-Training Quantization

    State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA · School of Artificial Intelligence, UCAS · Beijing National Research Center for Information Science and Technology +3

  23. Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Sampling

    Mar 18, 2026Sooyoung Ryu, Mathieu Salzmann, Saqib JavedDiffusion Model QuantizationDiffusion Model Sampling

    Department of Computer Science and Engineering, Seoul National University, Seoul, South Korea · School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland · Meta Reality Labs, Redmond, USA

  24. QuantFL: Sustainable Federated Learning for Edge IoT via Pre-Trained Model Quantisation

    Mar 18, 2026Charuka Herath, Yogachandran Rahulamathavan, Varuna De Silva +1Federated LearningNeural Network Quantization

    Institute of Digital Technologies, Loughborough University, UK

  25. Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity

    Mar 18, 2026Hengyuan Zhang, Xinrong Chen, Zunhai Su +10Mixed-Precision QuantizationNeural Network Quantization

    The University of Hong Kong · Peking University · Tsinghua University +2

  26. KANtize: Exploring Low-bit Quantization of Kolmogorov-Arnold Networks for Efficient Inference

    Mar 18, 2026Sohaib Errabii, Olivier Sentieys, Marcello TraiolaLow-Bit QuantizationEfficient Neural Network Inference

    Univ Rennes, Inria, CNRS, IRISA, Rennes, France

  27. Mitigating Premature Discretization with Progressive Quantization for Robust Vector Tokenization

    Mar 17, 2026Wenhao Zhao, Qiran Zou, Zhouhan Lin +1Vector QuantizationVisual Tokenization

    National University of Singapore, Singapore · Shanghai Jiao Tong University, Shanghai, China

  28. Understanding Quantization of Optimizer States in LLM Pre-training: Dynamics of State Staleness and Effectiveness of State Resets

    Mar 17, 2026Kristi Topollai, Anna ChoromanskaLow-Precision TrainingLow-Bit Quantization

    New York University

  29. BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization

    Mar 17, 2026Ji-Fu Li, Manyi Zhang, Xiaobo Xia +4LLM QuantizationNeural Network Quantization

    Huawei Technologies · University of Science and Technology of China

  30. Federated Learning of Binary Neural Networks: Enabling Low-Cost Inference

    Mar 16, 2026Nitin Priyadarshini Shankar, Soham Lahiri, Sheetal Kalyani +1Binary Neural NetworksFederated Learning

    Department of Electrical Engineering, Indian Institute of Technology Madras · Department of Computer Science and Engineering, Jadavpur University