Mixed-Precision Quantization

Overview

Mixed-precision quantization studies how to use more bits for parts of a model that are sensitive to numerical error and fewer bits elsewhere, reducing storage and computation while preserving useful behavior. Common approaches include post-training quantization and quantization-aware training, with precision chosen using calibration data, sensitivity measures such as Hessian or KL-based scores, or budgeted search methods such as knapsack optimization and reinforcement learning. The work spans LLM weights and KV caches, mixture-of-experts, diffusion models, and edge devices, and is evaluated on measures such as perplexity, reasoning accuracy, image quality, latency, and energy use. A central challenge is that sensitivity estimates and average bit budgets do not always predict end-to-end quality or real hardware speed, so allocations must be validated against the target workload and device.

Latest papers

CardsList

189 papers25 papers in the last four weeks, down 17% on the four weeks before. 0.2% of all new papers.

  1. UFO: Unlocking Ultra-Efficient Quantized Private Inference with Protocol and Algorithm Co-Optimization

    Feb 21, 2026Wenxuan Zeng, Chao Yang, Tianshi Xu +4Secure Multi-Party ComputationConvolutional Neural Networks

  2. Dynamic Mixed-Precision Routing for Efficient Multi-step LLM Interaction

    Feb 2, 2026Yuanzhe Li, Jianing Deng, Jingtong Hu +3LLM Inference AccelerationMixed-Precision Quantization

  3. Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantization

    Dec 8, 2025Haidong Kang, Jun Du, Guo YuMixed-Precision QuantizationLarge Language Model-Guided Optimization

  4. Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference

    Nov 19, 2025Kexin Chu, Dawei Xiang, Zixu Shen +3Mixed-Precision QuantizationGPU Acceleration

  5. You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

    Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Neural Network QuantizationMixed-Precision Quantization

  6. QuantDemoire: Quantization with Outlier Aware for Image Demoiréing

    Oct 5, 2025Zheng Chen, Kewei Zhang, Xiaoyang Liu +4QuantizationPost-Training Quantization

  7. LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

    Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision QuantizationLLM Quantization

  8. On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs

    Sep 22, 2025Rongguang Ye, Ming Tang, Edith C. H. NgaiMixed-Precision QuantizationLLM Quantization

  9. Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method

    Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision QuantizationLLM Quantization