Weight-Only Quantization

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 55

All topics

KronQ: LLM Quantization via Kronecker-Factored Hessian
Jul 8, 2026Donghyun Lee, Yuhang Li, Ruokai Yin +1Mixed-Precision Quantization LLM Quantization
University of Southern California · Yale University
Boundary-Aware Quantization: Finite-Scale Decision Geometry of Neural Classifiers
Jul 1, 2026O. M. KiselevWeight-Only Quantization Post-Training Quantization
Innopolis University, Innopolis, Republic of Tatarstan, Russian Federation
Spectral Perturbation of the Empirical Fisher Information Matrix under Weight Quantization
Jun 26, 2026Rahid Zahid Alekberli, Hikmat KarimovInformation Geometry Weight-Only Quantization
Institute of Defense Technologies and Cybersecurity, Azerbaijan Technical University, Baku, Azerbaijan
GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generation
Jun 22, 2026Jette Oberländer, Jan Finkbeiner, Catherine M. Schöfmann +1Mixed-Precision Quantization LLM Quantization
Fakultät für Informatik, RWTH Aachen, Aachen, 52074, Germany · Peter Grünberg Institut, Forschungszentrum Jülich GmbH, Jülich, 52425, Germany · Fakultät für Elektrotechnik und Informationstechnik, RWTH Aachen, Aachen, 52062, Germany
On the Expressive Power of Weight Quantization in Large Language Models
Jun 20, 2026Shao-Qun ZhangLLM Quantization Weight-Only Quantization
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210063, China. · School of Intelligent Science and Technology, Nanjing University, Suzhou 215163, China.
MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization
Jun 14, 2026Yangjia Hu, Haodong Wang, Zicong Hong +8LLM Quantization 4-Bit Quantization
HKUST · EPFL · MetaX Integrated Circuits Co., Ltd
Signed Symmetric Quantization for Few-Bit Integers
Jun 12, 2026Ian Colbert, Eashan Dash, Pablo Monteagudo-Lago +5LLM Quantization Weight-Only Quantization
AMD
Multi-Bitwidth Quantization for LLMs Using Additive Codebooks
Jun 11, 2026Liza Babaoglu, Shuangyi Chen, Ashish KhistiLLM Quantization Weight-Only Quantization
University of Toronto
Optimal Post-Training Quantization Scales and Where to Find Them
Jun 9, 2026Juan Amboage, Pablo Monteagudo-Lago, Ian Colbert +2LLM Quantization Weight-Only Quantization
AMD
Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models
Jun 3, 2026Rayyan Abdalla, Amir Hussein, Min Wu +1LLM Quantization LLM Inference Acceleration
LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection
Jun 2, 2026Liulu He, XuanAng Liu, Juntao Liu +8LLM Quantization LLM Compression
Nanjing University, China · China Mobile Research Institute, China · Xiaomi Corporation, China +2
LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMs
May 28, 2026Jung Hyun Lee, June Yong Yang, Jungwook Choi +1LLM Quantization Weight-Only Quantization
Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea · LG AI Research, Seoul, South Korea · Hanyang University, Seoul, South Korea +1
BitTP: The Lightweight Trajectory Prediction Model with BitLLM for Edge-Devices
May 28, 2026Mincheol Kang, Hyunjin Lim, Bomin Kang +1Multi-Agent Trajectory Prediction Weight-Only Quantization
1KAIST, Republic of Korea · 2DGIST, Republic of Korea
QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
May 25, 2026Preetam Sharma, Kacper DobekLLM Quantization Rotation-Based Quantization
Independent Research · Institute of Computing Science Poznan University of Technology, Poznan, Poland
Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization
May 24, 2026Gorgi PavlovLLM Quantization Rotation-Based Quantization
Lehigh University
Motion-Compensated Weight Compression
May 23, 2026Ismail LamaakalPredictive Coding Weight-Only Quantization
Multidisciplinary Faculty of Nador · Mohammed Premier University · Oujda 60000, Morocco
EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecture
May 22, 2026Bowen Duan, Cong Guo, Chiyue Wei +8Vector Quantization LLM Inference Acceleration
Duke University, USA
A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models
May 14, 2026Earl KillianMixed-Precision Quantization LLM Quantization
High-Rate Quantized Matrix Multiplication II
May 13, 2026Or Ordentlich, Yury PolyanskiyLLM Quantization Rate-Distortion Theory
the Hebrew University of Jerusalem, Israel · MIT, USA
LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss
May 9, 2026Euntae Choi, Sumin Song, Sungjoo YooLLM Quantization Quantization-Aware Training
Seoul National University
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
May 9, 2026Beshr IslamBouli, David JinLLM Quantization 4-Bit Quantization
Massachusetts Institute of Technology, Cambridge, MA, USA.
OSAQ: Outlier Self-Absorption for Accurate Low-bit LLM Quantization
May 6, 2026Zhikai Li, Zhen Dong, Xuewen Liu +2LLM Quantization Weight-Only Quantization
Institute of Automation, Chinese Academy of Sciences · University of California, Santa Barbara.
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Mixed-Precision Quantization LLM Quantization
1Technion—Israel Institute of Technology · 2Intuit · 3Ben-Gurion University of the Negev +1
Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method
Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision Quantization LLM Quantization
Beihang University · Communication University of China · Beijing Jiaotong University
LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization
Date pendingHaoyu Wang, Xingyu Yu, Haiyan Zhao +2LLM Quantization Quantization-Aware Training
Department of Computer Science and Technology, Tsinghua University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · College of Computer Science and Technology, National University of Defense Technology, Hunan, China.

Jul 8, 2026, cs.LGJ/K move · Enter open · S save

KronQ: LLM Quantization via Kronecker-Factored Hessian

Donghyun Lee, Yuhang Li, Ruokai Yin, Priyadarshini Panda

University of Southern California · Yale University

Mixed-Precision Quantization LLM Quantization Rotation-Based Quantization Weight-Only Quantization Post-Training Quantization Neural Network Quantization

Open paper AskPDF arXiv

Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining. Existing second-order PTQ methods, including GPTQ, construct quantization objectives exclusively from input activation statistics, effectively assuming that all output channels contribute equally to the layer-wise reconstruction objective. We propose KronQ, a PTQ framework that challenges this assumption by introducing the gradient covariance into the quantization pipeline. Under the Kronecker-factored Hessian approximation, the quantization loss depends jointly on both the activation and gradient covariances, and KronQ exploits this at two complementary levels. (1) KronQ introduces bidirectional incoherence processing, extending the existing input-side random rotation to the output dimension using the gradient covariance, reducing weight magnitude variance across both input and output dimensions. (2) KronQ derives a new sensitivity metric for inter-layer mixed-precision allocation, driven by the gradient and activation Hessian traces. Notably, in the case of 2-bit weight-only quantization on LLaMA-3-70B, while GPTQ and GPTAQ diverge or produce degenerate quantizations (>2000 perplexity on WikiText-2), KronQ achieves 7.93 perplexity.

Momentum

Related topics

Latest papers 55