Mixed-Precision Quantization

Ask about this topic

Overview

Mixed-precision quantization studies how to use more bits for parts of a model that are sensitive to numerical error and fewer bits elsewhere, reducing storage and computation while preserving useful behavior. Common approaches include post-training quantization and quantization-aware training, with precision chosen using calibration data, sensitivity measures such as Hessian or KL-based scores, or budgeted search methods such as knapsack optimization and reinforcement learning. The work spans LLM weights and KV caches, mixture-of-experts, diffusion models, and edge devices, and is evaluated on measures such as perplexity, reasoning accuracy, image quality, latency, and energy use. A central challenge is that sensitivity estimates and average bit budgets do not always predict end-to-end quality or real hardware speed, so allocations must be validated against the target workload and device.

Latest papers

CardsList

168 papers26 papers in the last four weeks, up 24% on the four weeks before. 0.2% of all new papers.

  1. Mixed-Precision Information Bottlenecks for On-Device Trait-State Disentanglement in Bipolar Agitation Detection

    May 4, 2026Joydeep ChandraMixed-Precision QuantizationInformation Bottleneck

    Department of Computer Science & Technology, Tsinghua University, Beijing, China.

  2. EdgeLPR: On the Deep Neural Network trade-off between Precision and Performance in LiDAR Place Recognition

    May 4, 2026Pierpaolo Serio, Hetian Wang, Zixiang Wei +4Mixed-Precision QuantizationEfficient Neural Network Inference

    Department of Information Engineering, University of Pisa, Pisa, Italy. · School of Engineering and Material Science, Queen Mary University of London, London, U.K. · Research and Development, Toyota Material Handling Manufacturing Italy, Bologna, Italy.

  3. WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization

    May 4, 2026Wei Tao, Xiaoyang Qu, Peiqiang Wang +4Mixed-Precision QuantizationVLM Quantization

    Huazhong University of Science and Technology, Wuhan, Hubei, China · Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, Guangdong, China · Tsinghua University, Shenzhen International Graduate School, Shenzhen, Guangdong, China

  4. GETA-3DGS: Automatic Joint Structured Pruning and Quantization for 3D Gaussian Splatting

    May 3, 2026Baobing Zhang, Wanxin Sui3DGS CompressionMixed-Precision Quantization

    Robotics Research Group, School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield, AL10 9AB, United Kingdom · Department of Electronic and Electrical Engineering, Brunel University London, Kingston Lane, Uxbridge, Middlesex, UB8 3PH, United Kingdom

  5. CoQuant: Joint Weight-Activation Subspace Projection for Mixed-Precision LLMs

    Apr 29, 2026Zhe Ding, Su Pan, Duowei PanMixed-Precision QuantizationLLM Quantization

    School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing, China · Amazon AGI, Seattle, WA, USA

  6. QuantClaw: Precision Where It Matters for OpenClaw

    Apr 24, 2026Manyi Zhang, Ji-Fu Li, Zhongao Sun +5Mixed-Precision QuantizationLLM Quantization

    Huawei Technologies · National University of Singapore · University of Science and Technology of China

  7. RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory

    Apr 22, 2026Fei Zuo, Zikang Zhou, Hao Cong +2LLM Inference EfficiencyMixed-Precision Quantization

    1BA TechWorks (BMW Group) · 2National University of Singapore · 3Tsinghua University

  8. On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks

    Apr 22, 2026Aarav Gupta, Gururaj Deshpande, Chandreyi ChakrabortyMixed-Precision QuantizationDiffusion Model Quantization

    College of Computing · Georgia Institute of Technology

  9. LoRaQ: Optimized Low Rank Approximation for 4-bit Quantization

    Apr 20, 2026Yann Bouquet, Alireza Khodamoradi, Sophie Yáng Shen +2Mixed-Precision QuantizationDiffusion Transformer

    Research and Advancement Development Team, Advanced Micro Devices, Inc, USA · Computer Vision Laboratory, Ecole Polytechnique F´ed´erale de Lausanne, Lausanne, Switzerland.

  10. A KL Lens on Quantization: Fast, Forward-Only Sensitivity for Mixed-Precision SSM-Transformer Models

    Apr 15, 2026Jason Kong, Nilesh Prasad Pandey, Flavio Ponzina +1Quantization Sensitivity AnalysisLLM Quantization

  11. DA-PTQ: Drift-Aware Post-Training Quantization for Efficient Vision-Language-Action Models

    Apr 13, 2026Siyuan Xu, Tianshi Wang, Fengling Li +2Post-Training QuantizationEfficient VLA Models

    Tongji University Shanghai, China · University of Technology Sydney Sydney, Australia

  12. Device-Conditioned Neural Architecture Search for Efficient Robotic Manipulation

    Apr 11, 2026Yiming Wu, Huan Wang, Zhenghao Chen +2Hardware-Aware NASRobotic Manipulation

    The University of Hong Kong, China · Westlake University, China · The University of Newcastle, Australia

  13. EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation

    Apr 10, 2026Shu-Hao Zhang, Le-Tong Huang, Xiang-Sheng Deng +5Quantization-Aware DistillationLLM Quantization

    National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210063, China · School of Intelligent Science and Technology, Nanjing University, Suzhou 215163, China · Microsoft AI, Beijing 100080, China +1

  14. MP-ISMoE: Mixed-Precision Interactive Side Mixture-of-Experts for Efficient Transfer Learning

    Apr 10, 2026Yutong Zhang, Zimeng Wu, Shangcai Liao +2Transfer LearningMemory-Efficient Fine-Tuning

    State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China · School of Computer Science and Engineering, Beihang University, Beijing, China · College of Information Technology, United Arab Emirates University, Al Ain, Abu Dhabi, United Arab Emirates

  15. STQuant: Spatio-Temporal Adaptive Framework for Optimizer Quantization in Large Multimodal Model Training

    Apr 8, 2026Minglu Liu, Cunchen Hu, Liangliang Xu +3Neural Network QuantizationMemory-Efficient Training

    Xidian University Xian, Shaanxi, China · China Telecom Cloud Computing Research Institute Beijing, China

  16. Don't Waste Bits! Adaptive KV-Cache Quantization for Lightweight On-Device LLMs

    Apr 6, 2026Sayed Pedram Haeri Boroujeni, Niloufar Mehrabi, Patrick Woods +2KV-Cache QuantizationOn-Device Language Model Inference

    Clemson University, USA

  17. Diagonal-Tiled Mixed-Precision Attention for Efficient Low-Bit MXFP Inference

    Apr 5, 2026Yifu Ding, Xinhao Zhang, Jinyang GuoLLM Inference AccelerationGPU Kernel Optimization

    State Key Laboratory of Complex & Critical Software Environment (SKLCCSE), Beihang University · School of Computer Science and Engineering, Beihang University · Beijing Jiaotong University +1

  18. AdaHOP: Fast and Accurate Low-Precision Training via Outlier-Pattern-Aware Rotation

    Apr 2, 2026Seonggon Kim, Alireza Khodamoradi, Pranathi Vasireddy +2Walsh-Hadamard TransformLow-Precision Training

    Advanced Micro Devices, Inc. · POSTECH

  19. AdaLoRA-QAT: Adaptive Low-Rank and Quantization-Aware Segmentation

    Apr 1, 2026Prantik Deb, Srimanth Dhondy, N. Ramakrishna +3Medical Image SegmentationMedical Imaging Foundation Models

    International Institute of Information Technology (IIIT-H), Hyderabad, India · Nizam’s Institute of Medical Sciences (NIMS), Hyderabad, India · The Alan Turing Institute, London and University College London, United Kingdom

  20. MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference

    Apr 1, 2026Zifei Xu, Sayeh Sharify, Hesham MostafaQuantization-Aware TrainingNeural Network Quantization

    d-Matrix, Santa Clara, CA 95054

  21. OneComp: One-Line Revolution for Generative AI Model Compression

    Mar 30, 2026Yuma Ichikawa, Keiji Kimura, Akihiro Yoshida +11Hardware-Aware QuantizationModel Compression

    Fujitsu Limited · RIKEN Center for AIP · Institute of Science Tokyo +2

  22. TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization

    Mar 29, 2026Dipkumar PatelKV-Cache CompressionKV-Cache Quantization

    LLMs Research Inc.

  23. Sustainability Is Not Linear: Quantifying Performance, Energy, and Privacy Trade-offs in On-Device Intelligence

    Mar 27, 2026Eziyo Ehsani, Luca Giamattei, Ivano Malavolta +1On-Device Language Model InferenceEnergy-Efficient ML

    ∗University of Naples Federico II, Naples, Italy · †Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

  24. APreQEL: Adaptive Mixed Precision Quantization For Edge LLMs

    Mar 24, 2026Meriem Bouzouad, Yuan-Hao Chang, Jalil BoukhobzaHardware-Aware QuantizationLLM Quantization

    Lab-STICC, CNRS UMR 6285 , ENSTA, Institut Polytechnique de Paris, Brest, France · National Taiwan University, Taipei, Taiwan

  25. Rewriting TTS Inference Economics: Lightning V2 on Tenstorrent Achieves 4x Lower Cost Than NVIDIA L40S

    Mar 24, 2026Ranjith M. S., Akshat Mandloi, Sudarshan KamathTTS SynthesisAI Accelerator Inference

    Smallest AI

  26. DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edge

    Mar 19, 2026Yuegui Huang, Zhiyuan Fang, Weiqi Luo +3Mixture-of-Experts InferenceMixture-of-Experts Quantization

    Sun Yat-sen University, Guangzhou, China · Tencent, Shenzhen, China

  27. 6Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models

    Mar 19, 2026Rundong Su, Jintao Zhang, Zhihang Yuan +3Diffusion Model QuantizationMixed-Precision Quantization

    Fudan University · Tsinghua University · Shanghai Jiao Tong University

  28. RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference

    Mar 18, 2026Arpit Singh Gautam, Saurabh JhaLLM QuantizationNeural Network Quantization

  29. Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity

    Mar 18, 2026Hengyuan Zhang, Xinrong Chen, Zunhai Su +10Mixed-Precision QuantizationNeural Network Quantization

    The University of Hong Kong · Peking University · Tsinghua University +2

  30. TMPDiff: Temporal Mixed-Precision for Diffusion Models

    Mar 14, 2026Basile Lewandowski, Simon Kurz, Aditya Shankar +3Diffusion Model QuantizationDiffusion Model Inference Acceleration

    University of Neuchâtel, Switzerland · TU Dortmund University, Germany · LAMARR Institute, Germany +2