Mixed-Precision Quantization

Follow this topic Ask about this topic

Mixed-precision quantization studies how to use more bits for parts of a model that are sensitive to numerical error and fewer bits elsewhere, reducing storage and computation while preserving useful behavior. Common approaches include post-training quantization and quantization-aware training, with precision chosen using calibration data, sensitivity measures such as Hessian or KL-based scores, or budgeted search methods such as knapsack optimization and reinforcement learning. The work spans LLM weights and KV caches, mixture-of-experts, diffusion models, and edge devices, and is evaluated on measures such as perplexity, reasoning accuracy, image quality, latency, and energy use. A central challenge is that sensitivity estimates and average bit budgets do not always predict end-to-end quality or real hardware speed, so allocations must be validated against the target workload and device.

Latest papers

168 papers26 papers in the last four weeks, up 24% on the four weeks before. 0.2% of all new papers.

Realizing Low-Precision Storage and Sparse Execution in a Folded FPGA Speech Enhancer
Oct 26, 2026Clément LarocheFPGA-Based Neural Network Acceleration Hardware-Aware Quantization
Audio Research, GN A/S, Ballerup, Denmark
End-to-End MXFP8 Reinforcement Learning on Blackwell: What Matching Buys, When It Pays Off, and How It Fails
Oct 26, 2026Wengee ZhangRL Fine-Tuning Mixed-Precision Quantization
ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning
Oct 26, 2026Rhea Sanjay Sukthanker, Pashmina CameronLLM Quantization Low-Bit Quantization
Microsoft Applied Sciences Group
Different Tasks, Different Timesteps: Temporal Task-Aware Quantization for Diffusion Language Models
Oct 22, 2026Ofek Avivi, Eliron Rahimi, Avi Mendelson +1Neural Network Quantization Mixed-Precision Quantization
Technion – Israel Institute of Technology · INSIGHT Lab, Ben-Gurion University of the Negev
ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models
Oct 22, 2026Jiaqi Chen, Xianglong Yan, Chengzhu Bao +2Mixture-of-Experts Quantization Post-Training Quantization
Shanghai Jiao Tong University
FIGFLEX: A Mixed-Precision FP-INT GEMM Accelerator Co-Designed with Low-Bit LLM Quantization
Oct 21, 2026Faraz Tahmasebi, Haocheng Xu, Andrew Trauger +4LLM Inference Acceleration LLM Quantization
University of California, Irvine, Irvine, CA, USA
Spike-Aware Gradient-Based Mixed-Precision Quantization of Spiking Neural Networks for Vision Applications
Oct 21, 2026Rajshekhar Rakshit, Devgokul B., Rakshitha Kalkura +1Mixed-Precision Quantization Spiking Neural Networks
Indian Institute Of Technology Madras, Chennai, India
PACE: Periodic and Channel-Aligned Encoding of Four-Bit Recurrent States
Oct 21, 2026Patrick Haller, Filipe Laitenberger, Alan AkbikRecurrent-State Quantization LLM Quantization
Humboldt-Universität zu Berlin
Reliability-Preserving Compression for Deployable Medical Image Segmentation
Oct 18, 2026Sagar Kumar, Shailja Sharma, Tirthajit Baruah +1Model Compression Neural Network Compression
Indian Institute of Science, India · Indian Institute of Science (IISc), India
Measuring the Task Value of Precision Upgrades in Closed-Loop Agents
Oct 17, 2026Sicheng Lyu, Xinyu Wang, Yu Gu +3LLM Agent Reliability LLM Agent Evaluation
McGill University · University of Manitoba · Ohio University
FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper
Oct 16, 2026Wei WangEfficient Transformer Inference GPU Kernel Optimization
Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors
Oct 14, 2026Yifan LiuQuantization Sensitivity Analysis Mixture-of-Experts Quantization
UIUC MCS Student
Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantization
Oct 14, 2026Renjie Xie, Juncheng Yang, Aoting Hu +5Mixed-Precision Quantization Mixture-of-Experts Quantization
Nanjing University of Posts and Telecommunications · Tylogi AI Lab / TAIL · Anhui University of Technology +4
PN-Mix: Pair-Norm Guided Mixed-Precision Key Cache Quantization for Efficient Long-Output LLM Inference
Oct 14, 2026Gil Jae Song, Sungjoo YooKV-Cache Quantization Mixed-Precision Quantization
Seoul National University
Mixed Precision Scaling Laws for Quantized Pretraining
Oct 13, 2026Jinuk Kim, Rajarshi Saha, Muhammed Fatih Balin +1Mixed-Precision Quantization LLM Quantization
Seoul National University · Amazon Annapurna Labs
Context-Aware Precision Scheduling for Small Language Models
Oct 11, 2026Mukund Athmaram, Manoj Rao, Liang MaEfficient Language Model Inference On-Device Language Model Inference
Advanced Micro Devices Inc. (AMD) · Cornell University
The Price of Precision: A Calibrated Cost Model for Mixed-Precision LLM Serving
Oct 11, 2026Mukund Athmaram, Manoj Rao, Liang MaLLM Serving LLM Inference Acceleration
Advanced Micro Devices Inc. (AMD) · Cornell University
HGA-Quant: Heterogeneity-Aware Quantization for AutoRegressive Visual Generation Models
Oct 11, 2026Xinmeng Liu, Xianglong Yan, Kaicheng Yang +2Post-Training Quantization Neural Network Quantization
Shanghai Jiao Tong University · Xi’an University of Electronic Science and Technology
KV4-RL: Unlocking Faster Asynchronous Rollouts for RLVR with INT4 KV Cache Quantization
Oct 10, 2026Sihwan Park, Sung-Yub Kim, Doohyuk Jang +1KV-Cache Quantization RL Post-Training
KAIST · NAVER Cloud · AITRICS
RAMP: Recurrent Adaptive Mixed Precision for Efficient Prefill in Recurrent-Depth Language Models
Oct 9, 2026Ke Wan, Chen ChenRecurrent Language Models LLM Inference Acceleration
Department of Computer Science, University of Virginia
Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization
Oct 7, 2026Samy Houache, Yann Traonmilin, Jean-François AujolPost-Training Quantization Neural Network Quantization
IMB, UB · UB, IMB
Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders
Oct 6, 2026Hyojung Han, Jongmin Kim, Seung-Hun JeonPost-Training Quantization Mixed-Precision Quantization
ThakiCloud
Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
Oct 6, 2026Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2LLM Quantization Quantization-Aware Training
ISTA · EPFL, ISTA
AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation
Oct 5, 2026Hanzhi Zhang, Qiao Zhang, Qinglei Cao +4Mixed-Precision Quantization LLM Quantization
LLaVi Lab, Computer Science & Engineering, University of North Texas
StagQ: Constraint-Driven Multi-Precision Weight Quantization for LLMs
Oct 5, 2026Zhe Wei, Mengqi Guo, Yuan Yuan +3Mixed-Precision Quantization LLM Quantization
Huawei Technologies Ltd.
Backward-State Policy Is Part of the Learning Algorithm
Sep 30, 2026Shuxiao Xie, Shuyang Xie, Dezhi Ran +2Mixed-Precision Quantization Backpropagation
Beijing Tongming Lake Information Technology Application Innovation Center (TLAIC), China
Behavioral Capacity Certificates for Quantized Language Models
Sep 29, 2026Arian Eamaz, Mojtaba SoltanalianLLM Quantization KV-Cache Quantization
Department of Electrical and Computer Engineering University of Illinois Chicago
Fiona: Accelerating FHE Inference with Packing-Aware Ternary Weights
Sep 28, 2026Yiteng Peng, Zhibo Liu, Dongwei Xiao +1Mixed-Precision Quantization Efficient Neural Network Inference
The Hong Kong University of Science and Technology · State Key Laboratory of Novel Software Technology, Nanjing University
From Attention Sensitivity to Layer Role: Revisiting Mixed-Precision Quantization of Transformers
Sep 28, 2026Nafiseh HosseinpourFardi, Negar Alihadi, Mahmoudreza Babaei +2Mixed-Precision Quantization LLM Quantization
Sharif University of Technology, Tehran, Iran · GISMA University of Applied Sciences, Potsdam, Germany · BRAINS, Brandenburg Research Center for Applied Intelligent Systems, Potsdam, Germany +3
Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantization
Sep 27, 2026I Kennedy, T KennedyMixed-Precision Quantization LLM Quantization
baa.ai Auckland, New Zealand

Oct 26, 2026J/K move · Enter open · S save

Realizing Low-Precision Storage and Sparse Execution in a Folded FPGA Speech Enhancer

Clément Laroche

Audio Research, GN A/S, Ballerup, Denmark

FPGA-Based Neural Network Acceleration Hardware-Aware Quantization Structured Sparsity Speech Enhancement Mixed-Precision Quantization

Open paper AskPDF arXiv

Hardware-aware training often optimizes FPGA resources, latency, or bit operations, but folding changes what compression saves: shared arithmetic decouples precision, weight storage, and scheduled work. We derive training objectives from the measured costs of a folded streaming speech-enhancement accelerator. Mixed-precision weights reduce on-chip storage; structured channel sparsity removes entire operations from the engine schedule. We deploy a bit-exact implementation on the Kria KV260. The best sparse model eliminates 47.3% of convolution MACs, cutting frame time by 45% (1.86 to 1.03 ms) and dynamic energy by 60% (83 to 33

μ

J/frame) against a storage-priced model measured in the same session, with higher speech quality. Relative to the most efficient dense build measured separately, frame time falls by 42% and energy by 48%. Reducing weight width from 8 to 6 bits lowers dynamic power by 27% but changes frame time by less than 5%. Plain channel group-lasso removes more work than cycle-cost-weighted lasso. Thus, folded-accelerator training should price storage and removable work separately, rather than rely on a single bit-operation metric.