Mixed-Precision Quantization

Overview

Mixed-precision quantization studies how to use more bits for parts of a model that are sensitive to numerical error and fewer bits elsewhere, reducing storage and computation while preserving useful behavior. Common approaches include post-training quantization and quantization-aware training, with precision chosen using calibration data, sensitivity measures such as Hessian or KL-based scores, or budgeted search methods such as knapsack optimization and reinforcement learning. The work spans LLM weights and KV caches, mixture-of-experts, diffusion models, and edge devices, and is evaluated on measures such as perplexity, reasoning accuracy, image quality, latency, and energy use. A central challenge is that sensitivity estimates and average bit budgets do not always predict end-to-end quality or real hardware speed, so allocations must be validated against the target workload and device.

Latest papers

CardsList

189 papers25 papers in the last four weeks, down 17% on the four weeks before. 0.2% of all new papers.

  1. CertiSum: Bitwise-Identical MXFP4 Training Across Platforms and Parallelism Layouts

    Nov 16, 2026Steve Rhyner, Yubo Gao, Stefan de Lasa +2ML ReproducibilityMixed-Precision Training

  2. Gradient-Guided Hardware-Native MIXFP4

    Nov 12, 2026Jian-Jia Chen, Brian J. Chan, Wen-Hung Lee +8QuantizationLLM Quantization

  3. Joint Quantization and Dynamic Partitioning for Context-Aware Edge Vision Transformers

    Nov 10, 2026Zhu Jiaqi, Jain Prakriti, Katare Dewant +1Edge InferenceVehicular Networks

  4. AutoEMatic: Runtime-Adaptive Mixed-Precision LLM Inference via Automatic Exponent-Mantissa Refinement

    Nov 6, 2026Akshat Ramachandran, Reena Elangovan, Rangharajan Venkatesan +4LLM QuantizationMixed-Precision Quantization

  5. Coordinated Mixed-Precision KV Cache Quantization for Efficient LLM-based Multi-agent Systems

    Nov 6, 2026Weizhong Huang, Jinchao Zhang, Xiawu ZhengKV-Cache QuantizationMixed-Precision Quantization

  6. TriAttention: Unified Sparse Mixed-Precision Attention for Long-Context LLM Inference

    Nov 5, 2026Hanzhi Zhang, Qiao Zhang, Qinglei Cao +4LLM Inference AccelerationAttention Quantization

  7. TileQuant: Tile as the Precision-Allocation Unit for LLM Mixed-Precision Post-Training Quantization

    Nov 4, 2026Hanzhi Zhang, Qiao Zhang, Qinglei Cao +4LLM QuantizationPost-Training Quantization

  8. Quantizing Recurrent Language Models Is Easier Than You Think

    Nov 1, 2026Philipp Nazari, Neehal Tumma, Daniela Rus +1LLM QuantizationRecurrent-State Quantization

  9. SMART-AIMC: Sensitivity-Aware Mixed ADC Resolution with Fine-Grained Tunability for Analog In-Memory Computing

    Nov 1, 2026Yung-Chin Chen, Ourania Stouraiti, Naveen VermaCompute-in-MemoryEnergy-Efficient ML

  10. MatMoE: Matryoshka Mixture-of-Experts with Dynamic Mixed-Precision Quantization for Efficient Inference

    Nov 1, 2026Renjie Wei, Haochen Huang, Chuyu Qiu +3Mixture-of-Experts InferenceMixture-of-Experts Quantization

  11. Jet-QSparse: Accelerating Agentic LLM Inference with 4-Bit Sparse Attention

    Oct 31, 2026Haozhan Tang, Hao Kang, Meng Lu +7LLM Inference AccelerationKV-Cache Quantization

  12. Integer-Exact Microscaling: Running MXFP4 Language Models on INT8 Matrix Units

    Oct 31, 2026Shinyoung Kang, Junhyung Ahn, Yujun Kim +1LLM QuantizationNeural Network Quantization

  13. Quantize by Doing: Task-Guided Mixed-Precision Quantization of Vision-Language-Action Models

    Oct 28, 2026Jiuyi Xu, Xiao Hu, Meida Chen +3Mixed-Precision QuantizationVLM Quantization

  14. ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning

    Oct 26, 2026Rhea Sanjay Sukthanker, Pashmina CameronLLM QuantizationLow-Bit Quantization

  15. Different Tasks, Different Timesteps: Temporal Task-Aware Quantization for Diffusion Language Models

    Oct 22, 2026Ofek Avivi, Eliron Rahimi, Avi Mendelson +1Neural Network QuantizationMixed-Precision Quantization

  16. FIGFLEX: A Mixed-Precision FP-INT GEMM Accelerator Co-Designed with Low-Bit LLM Quantization

    Oct 21, 2026Faraz Tahmasebi, Haocheng Xu, Andrew Trauger +4LLM Inference AccelerationLLM Quantization

  17. Spike-Aware Gradient-Based Mixed-Precision Quantization of Spiking Neural Networks for Vision Applications

    Oct 21, 2026Rajshekhar Rakshit, Devgokul B., Rakshitha Kalkura +1Mixed-Precision QuantizationSpiking Neural Networks

  18. PACE: Periodic and Channel-Aligned Encoding of Four-Bit Recurrent States

    Oct 21, 2026Patrick Haller, Filipe Laitenberger, Alan AkbikRecurrent-State QuantizationLLM Quantization

  19. Reliability-Preserving Compression for Deployable Medical Image Segmentation

    Oct 18, 2026Sagar Kumar, Shailja Sharma, Tirthajit Baruah +1Model CompressionNeural Network Compression

  20. Measuring the Task Value of Precision Upgrades in Closed-Loop Agents

    Oct 17, 2026Sicheng Lyu, Xinyu Wang, Yu Gu +3LLM Agent ReliabilityLLM Agent Evaluation