Mixed-Precision Quantization

Ask about this topic

Overview

Mixed-precision quantization studies how to use more bits for parts of a model that are sensitive to numerical error and fewer bits elsewhere, reducing storage and computation while preserving useful behavior. Common approaches include post-training quantization and quantization-aware training, with precision chosen using calibration data, sensitivity measures such as Hessian or KL-based scores, or budgeted search methods such as knapsack optimization and reinforcement learning. The work spans LLM weights and KV caches, mixture-of-experts, diffusion models, and edge devices, and is evaluated on measures such as perplexity, reasoning accuracy, image quality, latency, and energy use. A central challenge is that sensitivity estimates and average bit budgets do not always predict end-to-end quality or real hardware speed, so allocations must be validated against the target workload and device.

Latest papers

CardsList

168 papers26 papers in the last four weeks, up 24% on the four weeks before. 0.2% of all new papers.

  1. Activation Outliers in Transformer Quantization: Reproduction, Statistical Analysis, and Deployment Tradeoffs

    Mar 4, 2026Pranav Kumar KaliaperumalActivation OutliersPost-Training Quantization

    M.S. Computer Science, University of Colorado Denver, Aurora, Colorado, USA

  2. Practical FP4 Training for Large-Scale MoE Models on Hopper GPUs

    Mar 3, 2026Wuyue Zhang, Chongdong Huang, Chunbo You +3Low-Precision TrainingLLM Training

    Zhejiang Lab, Hangzhou, China

  3. SHIELD8-UAV: Sequential 8-bit Hardware Implementation of a Precision-Aware 1D-F-CNN for Low-Energy UAV Acoustic Detection and Temporal Tracking

    Mar 1, 2026Susmita Ghanta, Karan Nathwani, Rohit ChaurasiyaFPGA-Based Neural Network AccelerationEdge Inference

    Dept. of Electrical Engineering, Indian Institute of Technology Jammu, Jagti 181221, India

  4. FlashOptim: Optimizers for Memory-Efficient Training

    Feb 26, 2026Jose Javier Gonzalez Ortiz, Abhay Gupta, Christopher Rinard +1Memory-Efficient TrainingMixed-Precision Quantization

    Databricks AI Research

  5. Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators

    Feb 26, 2026Yuhao Liu, Salim Ullah, Akash KumarMixed-Precision QuantizationFPGA-Based Neural Network Acceleration

    Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany · Ruhr University Bochum, Germany

  6. FairQuant: Fairness-Aware Mixed-Precision Quantization for Medical Image Classification

    Feb 26, 2026Thomas Woergaard, Raghavendra SelvanMixed-Precision QuantizationAlgorithmic Fairness

    Department of Computer Science, University of Copenhagen, Denmark

  7. SigmaQuant: Hardware-Aware Heterogeneous Quantization Method for Edge DNN Inference

    Feb 25, 2026Qunyou Liu, Pengbo Yu, Marina Zapater +1Hardware-Aware QuantizationNeural Network Quantization

    Embedded Systems Laboratory (ESL), EPFL, Switzerland · University of Applied Sciences and Arts Western Switzerland (HES-SO), Switzerland

  8. AutoQRA: Joint Optimization of Mixed-Precision Quantization and Low-rank Adapters for Efficient LLM Fine-Tuning

    Feb 25, 2026Changhai Zhou, Shiyang Zhang, Yuhua Zhou +5LLM Fine-TuningQuantization-Aware LoRA

    Fudan University · Yale University · Zhejiang University +1

  9. CORVET: A CORDIC-Powered, Resource-Frugal Mixed-Precision Vector Processing Engine for High-Throughput AIoT applications

    Feb 22, 2026Sonu Kumar, Mohd Faisal Khan, Mukul Lokhande +1AI Accelerator InferenceEfficient Neural Network Inference

    Centre for Advanced Electronics, IIT Indore. · NSDCS Research Group, Dept. of Electrical Engineering at IIT Indore.

  10. MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM

    Feb 21, 2026Dongwei Wang, Jinhee Kim, Seokho Han +8LLM QuantizationNeural Network Quantization

    University of Arizona · Panasonic AI Lab · Duke University +2

  11. Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training

    Feb 21, 2026Seyed Morteza EmadiTransformer AttentionAttention Quantization

    UNC-Chapel Hill, Kenan-Flagler Business School, Chapel Hill, NC, USA

  12. UFO: Unlocking Ultra-Efficient Quantized Private Inference with Protocol and Algorithm Co-Optimization

    Feb 21, 2026Wenxuan Zeng, Chao Yang, Tianshi Xu +4Secure Multi-Party ComputationConvolutional Neural Networks

    Institute for Artificial Intelligence, Peking University, China · School of Software and Microelectronics, Peking University, China · School of Integrated Circuits, Peking University, China +1

  13. Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantization

    Dec 8, 2025Haidong Kang, Jun Du, Guo YuMixed-Precision QuantizationLarge Language Model-Guided Optimization

    School of Computer and Communication Engineering, Northeastern University · School of Software Engineering, Beijing Jiaotong University

  14. Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference

    Nov 19, 2025Kexin Chu, Dawei Xiang, Zixu Shen +3Mixed-Precision QuantizationGPU Acceleration

    University of Connecticut · University of California, Santa Cruz · Independent Researcher

  15. You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

    Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Weight-Only QuantizationMixed-Precision Quantization

    1Technion—Israel Institute of Technology · 2Intuit · 3Ben-Gurion University of the Negev +1

  16. LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

    Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision QuantizationLLM Quantization

    University of Illinois Urbana-Champaign · Work done as an intern at HP Inc. · University of California, Los Angeles +1

  17. On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs

    Sep 22, 2025Rongguang Ye, Ming Tang, Edith C. H. NgaiMixed-Precision QuantizationLLM Quantization

    Southern University of Science and Technology · The University of Hong Kong

  18. Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method

    Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision QuantizationLLM Quantization

    Beihang University · Communication University of China · Beijing Jiaotong University