LLM Quantization

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

33 papers in the last four weeks, up 43% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 251

All topics

When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization
Oct 8, 2026Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu +6LLM Quantization Post-Training Quantization
University of Science and Technology of China · University College London · Shanghai University of Finance and Economics +5
OnlineQAT: On-Policy Distillation for Ultra-Low-Bit Large Language Models
Oct 7, 2026Wenjun Wang, Heng Li, Yanggan Gu +1LLM Quantization Quantization-Aware Training
The Hong Kong Polytechnic University · Sun Yat-sen University · PolyU-Daya Bay Technology and Innovation Research Institute
CurveTQ: Rotation-Free Trellis Quantization of LLM Weights via Curvature-Weighted Search
Oct 6, 2026Guanhua Ding, Zi Wang, Ruichao Li +1LLM Quantization Neural Network Quantization
Black Sesame Technologies
Few Bits, One Law: Toward W2A4KV2
Oct 6, 2026Kai Yi, Tarek Elgamal, Sruthikesh Surineni +3LLM Quantization Low-Bit Quantization
Meta AI
Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
Oct 6, 2026Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2LLM Quantization Quantization-Aware Training
ISTA · EPFL, ISTA
Align, Then Correct: Training-Free Two-Stage Low-Rank Compensation for Extremely Quantized Large Language Models
Oct 6, 2026Seobin Song, Geonho Lee, Janghwan Lee +1LLM Quantization Post-Training Quantization
Hanyang University, Seoul, Republic of Korea · Qualcomm AI Research, Qualcomm Korea YH, Seoul, Republic of Korea
ApexQuant: Data-Free Elastic Quantization by Residual Re-Isotropization
Oct 6, 2026Aksel Fristrup, Sumit Pandey, Ankit KariryaaLLM Quantization Rotation-Based Quantization
Henselian (henselian.com) · Department of Computer Science University of Copenhagen Denmark · Towards Deep Learning
Lost in the bf16 Cast: Exporting Ternary Language Models Can Revert Most Low-Learning-Rate Code Changes
Oct 6, 2026Avichal Sahai, Nishant Raj, Animesh SrivastavaLLM Quantization Ternary Quantization
Ofbusiness
TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models
Oct 6, 2026Xin Wang, Hao Yu, Zhengyang Zhuge +9LLM Quantization Quantization-Aware Training
Alibaba Token Hub, Alibaba Group · Ohio State University
Activation Denoising: A Robustness View on Parallel vs Sequential LLM Quantization
Oct 5, 2026Yan Scholten, Rachel Lawrence, James Hensman +3LLM Quantization Post-Training Quantization
Dept. of Computer Science & Munich Data Science Institute, Technical University of Munich · Microsoft Research Cambridge, UK
AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation
Oct 5, 2026Hanzhi Zhang, Qiao Zhang, Qinglei Cao +4Mixed-Precision Quantization LLM Quantization
LLaVi Lab, Computer Science & Engineering, University of North Texas
Differentiable Bit-Widths: Co-optimizing Pruning and Quantization via SVD for Ultra-Efficient LLM Compression
Oct 5, 2026Hankyul Kang, Jongbin RyuLLM Quantization LLM Pruning
Ajou University
StagQ: Constraint-Driven Multi-Precision Weight Quantization for LLMs
Oct 5, 2026Zhe Wei, Mengqi Guo, Yuan Yuan +3Mixed-Precision Quantization LLM Quantization
Huawei Technologies Ltd.
Understanding the Weight Averaging Mechanism in LLM Training for Post-Training Quantization
Oct 4, 2026Hanzhang Wang, Tianqi Shen, Zonglin Liu +3LLM Quantization Post-Training Quantization
City University of Hong Kong · The University of Hong Kong
Loopy: Low-Bit Quantization Framework for Looped Language Models
Oct 4, 2026Zeyu LI, Yipu ZHANG, Jintao Chen +2LLM Quantization Post-Training Quantization
The Hong Kong University of Science and Technology · Duke Kunshan University
The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantization
Oct 1, 2026Chao Li, Shigeng Wang, Anbang YaoLLM Quantization Post-Training Quantization
Intel Labs China
XOR-Trellis: Ultra-Low-Complexity Dequantization and Curvature-Aware Hadamard-Free LLM Quantization
Sep 30, 2026Xiaofan Que, Nir Elkayam, Spandan Pyakurel +2LLM Quantization LLM Inference Acceleration
Arm, Inc. · Rochester Institute of Technology
RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models
Sep 30, 2026Chengzhu Bao, Xianglong Yan, Tianao Zhang +3LLM Quantization Overthinking in Language Models
Shanghai Jiao Tong University
QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs
Sep 30, 2026Weili Xu, Jisen Li, Yuqing Jian +8LLM Quantization Quantization-Aware Training
Together AI · University of Illinois Urbana-Champaign · The University of Texas at Austin
Low-Discrepancy Dither for Quantized Recurrent State Caches
Sep 30, 2026Snigdha Chandan KhilarLLM Quantization Neural Network Quantization
Independent Researcher
JARQ: Joint Alternating Refinement for Quantization
Sep 29, 2026Xinyu Wang, Sicheng Lyu, Xiao-Wen ChangLLM Quantization Weight-Only Quantization
McGill University
ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights
Sep 29, 2026Jonathan Mei, Sang Hyub Kim, Oliver Knitter +2LLM Quantization Weight-Only Quantization
Applications R&D IonQ College Park, MD 20740, USA
Security-Enhanced Seed-Based Weight Quantization for Large Language Models
Sep 29, 2026Qiuyu Ren, Sudipta Paria, Aritra Dasgupta +1LLM Quantization LLM Compression
Department of Electrical and Computer Engineering (ECE) University of Florida Gainesville, FL 32611, USA
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
Sep 29, 2026Bingchen Yao, Haobo Xu, Haokun Lin +6LLM Quantization LLM Inference Acceleration
Zhejiang University · Tsinghua University · NLPR & MAIS, Institute of Automation, CAS +3
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
Sep 29, 2026Yi Pan, Haocheng Xi, Kan Zhu +10LLM Quantization LLM Inference Acceleration
UC Berkeley · University of Washington · MIT +2
Calibrate the Decisions That Change the Future: On-Policy Post-Training Quantization for Multimodal Large Language Models
Sep 29, 2026Wenxiao Fan, Jingling Fu, Lichen Ma +6LLM Quantization VLM Quantization
School of Computer Science and Technology, Beijing Institute of Technology · Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
Bits Under ZK-LLM: Evaluating Zero-Knowledge-Friendly Quantization for Verifiable Private LLM Inference
Sep 29, 2026Taeung Yoon, Yupeng Zhang, Xiaojing LiaoLLM Quantization LLM Inference
Siebel School of Computing and Data Science University of Illinois Urbana-Champaign
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
Sep 28, 2026Mehdi Makni, Ryan Lucas, Rahul MazumderLLM Quantization Rotation-Based Quantization
Operations Research Center Massachusetts Institute of Technology
IMC-CLINIC: Coupled Loss-Informed Newton Iterations for Clipping in Analog In-Memory Computing
Sep 28, 2026Yung-Chin Chen, Chia-Yu Chen, Naveen VermaLLM Quantization Compute-in-Memory
Princeton University, NJ, USA · EnCharge AI, CA, USA
Tetra: Serving Leech-Lattice Quantized LLMs at 2.7 Bits per Parameter
Sep 28, 2026Pier-Jean MalandrinoLLM Quantization LLM Inference Acceleration
Scub, Bordeaux, France

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization

Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu, Baihua He+5

University of Science and Technology of China · University College London · Shanghai University of Finance and Economics+5

LLM Quantization Post-Training Quantization Weight-Only Quantization

Open paper AskPDF arXiv

Weight-only post-training quantization (PTQ) relies heavily on reconstruction loss minimization to preserve model quality at low precision. We show that the weights favored by minimizing this loss need not yield better model performance on new tasks. In fact, we find that lower reconstruction loss can even degrade model performance on the same calibration data. Our analysis further shows that weights with lower reconstruction loss on calibration data can have higher loss than other weights when the distribution of input activations changes. Motivated by these observations and our analysis, we propose Distributionally Robust Quantization (DRQ), a post-hoc refinement process that minimizes worst-case reconstruction loss over a constrained set of input activation distributions. DRQ refines the integer codes representing quantized weights within the existing quantization grid, keeping quantization parameters and inference operators unchanged. Extensive experiments show that DRQ improves models quantized by six representative PTQ methods, including AWQ, GPTQ, and ParoQuant, and delivers gains across both dense and mixture-of-experts large language models. These results establish DRQ as a general post-hoc refinement framework for weight-only PTQ, achieving better downstream performance without adding inference overhead.

Momentum

Related topics

Latest papers 251