Weight-Only Quantization

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 55

All topics

When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization
Oct 8, 2026Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu +6LLM Quantization Post-Training Quantization
University of Science and Technology of China · University College London · Shanghai University of Finance and Economics +5
CurveTQ: Rotation-Free Trellis Quantization of LLM Weights via Curvature-Weighted Search
Oct 6, 2026Guanhua Ding, Zi Wang, Ruichao Li +1LLM Quantization Neural Network Quantization
Black Sesame Technologies
StagQ: Constraint-Driven Multi-Precision Weight Quantization for LLMs
Oct 5, 2026Zhe Wei, Mengqi Guo, Yuan Yuan +3Mixed-Precision Quantization LLM Quantization
Huawei Technologies Ltd.
JARQ: Joint Alternating Refinement for Quantization
Sep 29, 2026Xinyu Wang, Sicheng Lyu, Xiao-Wen ChangLLM Quantization Weight-Only Quantization
McGill University
ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights
Sep 29, 2026Jonathan Mei, Sang Hyub Kim, Oliver Knitter +2LLM Quantization Weight-Only Quantization
Applications R&D IonQ College Park, MD 20740, USA
Security-Enhanced Seed-Based Weight Quantization for Large Language Models
Sep 29, 2026Qiuyu Ren, Sudipta Paria, Aritra Dasgupta +1LLM Quantization LLM Compression
Department of Electrical and Computer Engineering (ECE) University of Florida Gainesville, FL 32611, USA
Tetra: Serving Leech-Lattice Quantized LLMs at 2.7 Bits per Parameter
Sep 28, 2026Pier-Jean MalandrinoLLM Quantization LLM Inference Acceleration
Scub, Bordeaux, France
Fiona: Accelerating FHE Inference with Packing-Aware Ternary Weights
Sep 28, 2026Yiteng Peng, Zhibo Liu, Dongwei Xiao +1Mixed-Precision Quantization Efficient Neural Network Inference
The Hong Kong University of Science and Technology · State Key Laboratory of Novel Software Technology, Nanjing University
EntroPack: Fast and Accurate Entropy-Coded Weight Compression at Arbitrary Bitrates
Sep 28, 2026Hong Zhang, Zhongjie Duan, Yingda ChenEfficient Neural Network Inference Weight-Only Quantization
Alibaba Inc.
PulseQuant: Propagation-Guided Subspace Correction for 4-Bit Video Diffusion Transformers
Sep 27, 2026Yutong Wang, Xingtong Ge, Enhuai Liu +4Video Diffusion Models Diffusion Transformer
The University of Sydney · The Hong Kong University of Science and Technology · The Chinese University of Hong Kong +1
Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures
Sep 24, 2026Se Un Park, Yutae Kim, Junyoung ParkTTS Evaluation Weight-Only Quantization
UX Factory, Inc.
Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement
Sep 22, 2026Akihiro Yoshida, Yuma IchikawaMixed-Precision Quantization LLM Quantization
Fujitsu Limited, Institute of Science Tokyo · Fujitsu Limited, RIKEN Center for AIP
SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models
Sep 21, 2026Kewei Zhang, Zheng Chen, Haotong Qin +1VLM Quantization Efficient VLM Inference
Shanghai Jiao Tong University · The Hong Kong Polytechnic University
Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization
Sep 20, 2026Kuanlin Chen, Chen-Wei Kuo, Cheng-En OuAttention Head Analysis 4-Bit Quantization
Independent Researcher Taoyuan, Taiwan · National Tsing Hua University Hsinchu, Taiwan
Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families
Sep 14, 2026Hyojung HanText Embedding Evaluation Weight-Only Quantization
ThakiCloud
Structured Transforms for Low-Overhead Quantization of Language Models
Sep 11, 2026Daria Cherniuk, Alexander Rudikov, Boris Kashin +1LLM Quantization Weight-Only Quantization
Institute of Numerical Mathematics · Steklov Mathematical Institute
Scaling Post-Training Ternarisation to Qwen3-8B Capability Retention, Reproduction, Lossless Packing, and Packed Execution
Sep 10, 2026Anirudh Malik, M Sparsh Mehra, Poojith DevanLLM Quantization Ternary Quantization
OneBit AI
Quantization Amplifies Determinism, Not Bias: Scale-Dependent Behavioral Effects of Serving-Time Weight Compression
Sep 7, 2026Dachi KurtskhaliaLLM Quantization Weight-Only Quantization
Impel
Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights
Sep 2, 2026Pier-Jean MalandrinoLLM Inference Vector Quantization
Scub
Post-Training Ternarization of Qwen3-4B Capability, Effective Bit Budget, Storage Compression, and Deployment
Sep 2, 2026Anirudh Malik, M Sparsh Mehra, Poojith DevanLLM Quantization Ternary Quantization
OneBit AI
FAMPWQ: Fisher Information-based Adaptive Mixed Precision Weight Quantization for Effective LLM Inference
Aug 24, 2026Gongwei Lee, Ji Liu, Juncheng Jia +1Mixed-Precision Quantization LLM Quantization
School of Computer Science and Technology, Soochow University, Suzhou, China · Hithink Research, Hangzhou, China · Electronic Engineering, Tsinghua University, Beijing, China
Activation-Weighted Seeded Residual Coding for Low-Bit LLM Weight Repair
Aug 24, 2026Zehao Liu, Chuangchuang Fang, Yang RenLLM Quantization Weight-Only Quantization
Huawei Technologies Research & Development (UK) Limited, Edinburgh, United Kingdom
ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization
Aug 11, 2026He-Yen Hsieh, H. T. KungLLM Quantization Weight-Only Quantization
Harvard University
Quantization Degradation in Large Language Models: A Signal-Noise Perspective
Aug 8, 2026Chenxi Zhou, Pengfei Cao, Jinyu Ye +5LLM Quantization Weight-Only Quantization
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China +1
CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights
Aug 7, 2026Xuetian GaoLLM Quantization LLM Inference Acceleration
TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models
Aug 4, 2026Seokho Han, Dongwei Wang, Jinhee Kim +4Mixed-Precision Quantization Diffusion Model Quantization
Department of Electrical and Computer Engineering, Sungkyunkwan University, Korea · Department of Electrical and Computer Engineering, University of Arizona, USA · Department of Electrical and Computer Engineering, Duke University, USA +1
ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads
Aug 3, 2026Şuayp Talha Kocabay, Talha Rüzgar Akkuş, Kamer Ali YukselLLM Quantization LLM Compression
Independent Researcher · aiXplain, Inc.
PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
Jul 17, 2026Yuchen Yang, Yifan Zhao, Anisha Dasgupta +1LLM Quantization Mixture-of-Experts Language Models
University of Illinois Urbana-Champaign
ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level
Jul 15, 2026Chethan Reddy G. PLLM Quantization Ternary Quantization
BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression
Jul 9, 2026Yuantian Shao, Peisong Wang, Zhilei Liu +6LLM Compression Weight-Only Quantization
Nanjing University of Science and Technology, Nanjing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · Huawei, China
KronQ: LLM Quantization via Kronecker-Factored Hessian
Jul 8, 2026Donghyun Lee, Yuhang Li, Ruokai Yin +1Mixed-Precision Quantization LLM Quantization
University of Southern California · Yale University
Boundary-Aware Quantization: Finite-Scale Decision Geometry of Neural Classifiers
Jul 1, 2026O. M. KiselevWeight-Only Quantization Post-Training Quantization
Innopolis University, Innopolis, Republic of Tatarstan, Russian Federation
Spectral Perturbation of the Empirical Fisher Information Matrix under Weight Quantization
Jun 26, 2026Rahid Zahid Alekberli, Hikmat KarimovInformation Geometry Weight-Only Quantization
Institute of Defense Technologies and Cybersecurity, Azerbaijan Technical University, Baku, Azerbaijan
GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generation
Jun 22, 2026Jette Oberländer, Jan Finkbeiner, Catherine M. Schöfmann +1Mixed-Precision Quantization LLM Quantization
Fakultät für Informatik, RWTH Aachen, Aachen, 52074, Germany · Peter Grünberg Institut, Forschungszentrum Jülich GmbH, Jülich, 52425, Germany · Fakultät für Elektrotechnik und Informationstechnik, RWTH Aachen, Aachen, 52062, Germany
On the Expressive Power of Weight Quantization in Large Language Models
Jun 20, 2026Shao-Qun ZhangLLM Quantization Weight-Only Quantization
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210063, China. · School of Intelligent Science and Technology, Nanjing University, Suzhou 215163, China.
MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization
Jun 14, 2026Yangjia Hu, Haodong Wang, Zicong Hong +8LLM Quantization 4-Bit Quantization
HKUST · EPFL · MetaX Integrated Circuits Co., Ltd
Signed Symmetric Quantization for Few-Bit Integers
Jun 12, 2026Ian Colbert, Eashan Dash, Pablo Monteagudo-Lago +5LLM Quantization Weight-Only Quantization
AMD
Multi-Bitwidth Quantization for LLMs Using Additive Codebooks
Jun 11, 2026Liza Babaoglu, Shuangyi Chen, Ashish KhistiLLM Quantization Weight-Only Quantization
University of Toronto
Optimal Post-Training Quantization Scales and Where to Find Them
Jun 9, 2026Juan Amboage, Pablo Monteagudo-Lago, Ian Colbert +2LLM Quantization Weight-Only Quantization
AMD
Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models
Jun 3, 2026Rayyan Abdalla, Amir Hussein, Min Wu +1LLM Quantization LLM Inference Acceleration
LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection
Jun 2, 2026Liulu He, XuanAng Liu, Juntao Liu +8LLM Quantization LLM Compression
Nanjing University, China · China Mobile Research Institute, China · Xiaomi Corporation, China +2
LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMs
May 28, 2026Jung Hyun Lee, June Yong Yang, Jungwook Choi +1LLM Quantization Weight-Only Quantization
Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea · LG AI Research, Seoul, South Korea · Hanyang University, Seoul, South Korea +1
BitTP: The Lightweight Trajectory Prediction Model with BitLLM for Edge-Devices
May 28, 2026Mincheol Kang, Hyunjin Lim, Bomin Kang +1Multi-Agent Trajectory Prediction Weight-Only Quantization
1KAIST, Republic of Korea · 2DGIST, Republic of Korea
QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
May 25, 2026Preetam Sharma, Kacper DobekLLM Quantization Rotation-Based Quantization
Independent Research · Institute of Computing Science Poznan University of Technology, Poznan, Poland
Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization
May 24, 2026Gorgi PavlovLLM Quantization Rotation-Based Quantization
Lehigh University
Motion-Compensated Weight Compression
May 23, 2026Ismail LamaakalPredictive Coding Weight-Only Quantization
Multidisciplinary Faculty of Nador · Mohammed Premier University · Oujda 60000, Morocco
EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecture
May 22, 2026Bowen Duan, Cong Guo, Chiyue Wei +8Vector Quantization LLM Inference Acceleration
Duke University, USA
A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models
May 14, 2026Earl KillianMixed-Precision Quantization LLM Quantization
High-Rate Quantized Matrix Multiplication II
May 13, 2026Or Ordentlich, Yury PolyanskiyLLM Quantization Rate-Distortion Theory
the Hebrew University of Jerusalem, Israel · MIT, USA
LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss
May 9, 2026Euntae Choi, Sumin Song, Sungjoo YooLLM Quantization Quantization-Aware Training
Seoul National University
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
May 9, 2026Beshr IslamBouli, David JinLLM Quantization 4-Bit Quantization
Massachusetts Institute of Technology, Cambridge, MA, USA.
OSAQ: Outlier Self-Absorption for Accurate Low-bit LLM Quantization
May 6, 2026Zhikai Li, Zhen Dong, Xuewen Liu +2LLM Quantization Weight-Only Quantization
Institute of Automation, Chinese Academy of Sciences · University of California, Santa Barbara.
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Mixed-Precision Quantization LLM Quantization
1Technion—Israel Institute of Technology · 2Intuit · 3Ben-Gurion University of the Negev +1
Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method
Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision Quantization LLM Quantization
Beihang University · Communication University of China · Beijing Jiaotong University
LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization
Date pendingHaoyu Wang, Xingyu Yu, Haiyan Zhao +2LLM Quantization Quantization-Aware Training
Department of Computer Science and Technology, Tsinghua University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · College of Computer Science and Technology, National University of Defense Technology, Hunan, China.

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization

Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu, Baihua He+5

University of Science and Technology of China · University College London · Shanghai University of Finance and Economics+5

LLM Quantization Post-Training Quantization Weight-Only Quantization

Open paper AskPDF arXiv

Weight-only post-training quantization (PTQ) relies heavily on reconstruction loss minimization to preserve model quality at low precision. We show that the weights favored by minimizing this loss need not yield better model performance on new tasks. In fact, we find that lower reconstruction loss can even degrade model performance on the same calibration data. Our analysis further shows that weights with lower reconstruction loss on calibration data can have higher loss than other weights when the distribution of input activations changes. Motivated by these observations and our analysis, we propose Distributionally Robust Quantization (DRQ), a post-hoc refinement process that minimizes worst-case reconstruction loss over a constrained set of input activation distributions. DRQ refines the integer codes representing quantized weights within the existing quantization grid, keeping quantization parameters and inference operators unchanged. Extensive experiments show that DRQ improves models quantized by six representative PTQ methods, including AWQ, GPTQ, and ParoQuant, and delivers gains across both dense and mixture-of-experts large language models. These results establish DRQ as a general post-hoc refinement framework for weight-only PTQ, achieving better downstream performance without adding inference overhead.

Momentum

Related topics

Latest papers 55