Quantization

Latest papers

48 papers15 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin
Jun 8, 2026Hanyang Li, Jianhao Ma, Ying CuiNeural Network Optimization Straight-Through Estimator
Department of IEOR, University of California, Berkeley · Department of Statistics and Data Science, University of Pennsylvania
Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference
Jun 1, 2026Chi-Wei Huang, Chia-Chi TsaiLLM Quantization Rotation-Based Quantization
National Cheng Kung University
Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression
May 26, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +3KV-Cache Compression LLM Quantization
Massachusetts Institute of Technology, Cambridge, MA, USA · University of Toronto, Toronto, Canada · IBM Research, Cambridge, MA, USA.
JacQuant: STE-Free Quantization-Aware Training via Learned Jacobian Surrogates
May 25, 2026Kai Yi, Vignesh Vivekraja, Harshit Khaitan +1LLM Quantization Quantization-Aware Training
Meta AI
QuoVLA: Quotient Space for Vision-Language-Action Models
May 24, 2026Xuan Wang, Yinan Wu, Haoran Duan +1Quantization Representation Learning
Department of Automation Tsinghua University
When Bits Break Recourse: Counterfactual-Faithful Quantization
May 16, 2026Chaymae Yahyati, Ismail Lamaakal, Khalid El Makkaoui +1Mixed-Precision Quantization Counterfactual Evaluation
Mohammed First University
Compander-Aligned Query Geometry for Quantized Zeroth-Order Optimization
May 11, 2026Yao Shu, Zilin ZhuLLM Quantization Zeroth-Order Optimization
HKUST-GZ
Quantizing With Randomized Hadamard Transforms: Efficient Heuristic Now Proven
May 7, 2026Ran Ben-Basat, William Kuszmaul, Michael Mitzenmacher +2Quantization Rotation-Based Quantization
UCL and Broadcom · Carnegie Mellon University · Harvard University +2
GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling
Apr 20, 2026Alireza Dadgarnia, Soroush Tabesh, Mahdi Nikdan +4LLM Quantization Post-Training Quantization
ISTA · ETH Zürich · ISTA & Red Hat AI +1
A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work
Apr 20, 2026Ran Ben-Basat, Yaniv Ben-Itzhak, Gal Mendelson +3Residual VQ Vector Quantization
UCL and Broadcom · VMware Research by Broadcom · North Carolina State University +2
Zero-Shot Quantization via Weight-Space Arithmetic
Apr 3, 2026Daniele Solombrino, Antonio Andrea Gargiulo, Alessandro Zirilli +3Quantization Robustness Quantization
Sapienza University of Rome · Sapienza University of Rome / Paradigma
DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression
Mar 20, 2026Xiaoming Yu, Shize Tang, Guanghua Yu +4Quantization LLM Quantization
Yuanbao & Hunyuan AI Infra Team
Dissecting Quantization Error: A Concentration-Alignment Perspective
Mar 4, 2026Marco Federici, Boris van Breugel, Paul Whatmough +1LLM Quantization 4-Bit Quantization
Qualcomm AI Research
Bielik-Q2-Sharp: A Comparative Study of Extreme 2-bit Quantization Methods for a Polish 11B Language Model
Mar 4, 2026Jakub PrejznerQuantization LLM Quantization
BitSharp, Independent Researcher, Rzeszów, Poland
Boosting Entropy with Bell Box Quantization
Mar 2, 2026Ningfeng Yang, Tor M. AamodtQuantization Neural Network Quantization
University of British Columbia
NextMem: Towards Latent Factual Memory for LLM-based Agents
Feb 26, 2026Zeyu Zhang, Rui Li, Xiaoyan Zhao +4Quantization LLM Agent Memory
Renmin University of China · The Chinese University of Hong Kong · National University of Singapore +1
1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization
Feb 17, 2026Sohir Maskey, Constantin Eichenberg, Johannes Messner +1Quantization LLM Quantization
Aleph Alpha Research · Graphcore Research
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Weight-Only Quantization Mixed-Precision Quantization
1Technion—Israel Institute of Technology · 2Intuit · 3Ben-Gurion University of the Negev +1

Jun 8, 2026J/K move · Enter open · S save

Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin

Hanyang Li, Jianhao Ma, Ying Cui

Department of IEOR, University of California, Berkeley · Department of Statistics and Data Science, University of Pennsylvania

Neural Network Optimization Straight-Through Estimator Quantization-Aware Training Neural Network Quantization Post-Training Quantization Low-Bit Quantization+1

Open paper AskPDF arXiv

Post-training quantization (PTQ) converts a trained full-precision model into low-bit weights without task-level retraining, while quantization-aware training (QAT) incorporates quantization into the training loop. Although PTQ is efficient and often accurate at moderate bitwidths, it can fail sharply at aggressive bitwidths; QAT is more expensive but can often recover the lost accuracy. We propose a unified geometric framework that explains both PTQ failure and QAT recovery. We model full-precision training as following a low-loss river inside a wider valley: a normal neighborhood of the river forms a nearly flat basin, while leaving this basin incurs a sharp loss increase. When the quantization grid is comparable to the basin width, local PTQ objectives, including rounding and Hessian-based second-order reconstruction, can select a high-loss deployed quantized point outside the basin even when nearby low-loss quantized points exist. In this regime, straight-through-estimator-based QAT has a useful bias: it evaluates gradients at the deployed quantized weights while updating latent full-precision weights, causing the gradient to sense the valley wall and acquire an inward component that steers subsequent quantized iterates back into the basin. We formalize this mechanism through a local landscape model, construct a geometric PTQ failure mode, and prove finite-time QAT recovery under local quantizer-compatibility assumptions. Experiments across vision and language models under multiple neural-network quantization schemes corroborate the predicted basin-crossing failure of PTQ and the corresponding recovery mechanism of QAT.