Mixed-Precision Quantization

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 100

All topics

RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
Apr 22, 2026Fei Zuo, Zikang Zhou, Hao Cong +2LLM Inference Efficiency Mixed-Precision Quantization
1BA TechWorks (BMW Group) · 2National University of Singapore · 3Tsinghua University
On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
Apr 22, 2026Aarav Gupta, Gururaj Deshpande, Chandreyi ChakrabortyMixed-Precision Quantization Diffusion Model Quantization
College of Computing · Georgia Institute of Technology
LoRaQ: Optimized Low Rank Approximation for 4-bit Quantization
Apr 20, 2026Yann Bouquet, Alireza Khodamoradi, Sophie Yáng Shen +2Mixed-Precision Quantization Diffusion Transformer
Research and Advancement Development Team, Advanced Micro Devices, Inc, USA · Computer Vision Laboratory, Ecole Polytechnique F´ed´erale de Lausanne, Lausanne, Switzerland.
Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators
Feb 26, 2026Yuhao Liu, Salim Ullah, Akash KumarMixed-Precision Quantization FPGA-Based Neural Network Acceleration
Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany · Ruhr University Bochum, Germany
Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantization
Dec 8, 2025Haidong Kang, Jun Du, Guo YuMixed-Precision Quantization Large Language Model-Guided Optimization
School of Computer and Communication Engineering, Northeastern University · School of Software Engineering, Beijing Jiaotong University
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Mixed-Precision Quantization LLM Quantization
1Technion—Israel Institute of Technology · 2Intuit · 3Ben-Gurion University of the Negev +1
LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision Quantization LLM Quantization
University of Illinois Urbana-Champaign · Work done as an intern at HP Inc. · University of California, Los Angeles +1
On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
Sep 22, 2025Rongguang Ye, Ming Tang, Edith C. H. NgaiMixed-Precision Quantization LLM Quantization
Southern University of Science and Technology · The University of Hong Kong
Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method
Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision Quantization LLM Quantization
Beihang University · Communication University of China · Beijing Jiaotong University
Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
Date pendingKexin Chu, Dawei Xiang, Zixu Shen +3Mixed-Precision Quantization GPU Acceleration
University of Connecticut · University of California, Santa Cruz · Independent Researcher

Apr 22, 2026, cs.LGJ/K move · Enter open · S save

RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory

Fei Zuo, Zikang Zhou, Hao Cong, Xiaoyan Xi+1

1BA TechWorks (BMW Group) · 2National University of Singapore · 3Tsinghua University

LLM Inference Efficiency Mixed-Precision Quantization LLM Quantization KV Caching Memory-Efficient Optimization Rate-Distortion Optimization+1

Open paper AskPDF arXiv

Large language models cache all previously computed key-value (KV) pairs during generation, and this KV cache grows linearly with sequence length, making it a primary memory bottleneck for serving. Quantizing the KV cache to fewer bits reduces this cost, yet all current quantizers assign the same bit-width to every attention head, ignoring the large variation in head importance. A natural idea is to allocate more bits to important heads and fewer to the rest. We show, however, that such mixed-precision allocation has a hidden pitfall: each quantizer follows a different distortion curve D(b)=alpha*beta^{-b}, and the decay rate beta varies from 3.6 to 5.3 across quantizer designs. Applying one quantizer's distortion model to another inverts the allocation order and makes performance worse than uniform quantization. We call this failure mode distortion model mismatch and propose RateQuant to resolve it. RateQuant fits a per-quantizer distortion model from a small calibration set, then solves the resulting bit-allocation problem in closed form via reverse waterfilling from rate-distortion theory. On Qwen3-8B at 2.5 average bits, calibrated RateQuant reduces KIVI's perplexity from 49.3 to 14.9 (70% reduction) and improves QuaRot by 6.6 PPL. The entire calibration takes 1.6 s on a single GPU and adds zero overhead at inference time.

Momentum

Related topics

Latest papers 100