Weight-Only Quantization

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 55

All topics

When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization
Oct 8, 2026Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu +6LLM Quantization Post-Training Quantization
University of Science and Technology of China · University College London · Shanghai University of Finance and Economics +5
CurveTQ: Rotation-Free Trellis Quantization of LLM Weights via Curvature-Weighted Search
Oct 6, 2026Guanhua Ding, Zi Wang, Ruichao Li +1LLM Quantization Neural Network Quantization
Black Sesame Technologies
StagQ: Constraint-Driven Multi-Precision Weight Quantization for LLMs
Oct 5, 2026Zhe Wei, Mengqi Guo, Yuan Yuan +3Mixed-Precision Quantization LLM Quantization
Huawei Technologies Ltd.
JARQ: Joint Alternating Refinement for Quantization
Sep 29, 2026Xinyu Wang, Sicheng Lyu, Xiao-Wen ChangLLM Quantization Weight-Only Quantization
McGill University
ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights
Sep 29, 2026Jonathan Mei, Sang Hyub Kim, Oliver Knitter +2LLM Quantization Weight-Only Quantization
Applications R&D IonQ College Park, MD 20740, USA
Security-Enhanced Seed-Based Weight Quantization for Large Language Models
Sep 29, 2026Qiuyu Ren, Sudipta Paria, Aritra Dasgupta +1LLM Quantization LLM Compression
Department of Electrical and Computer Engineering (ECE) University of Florida Gainesville, FL 32611, USA
Tetra: Serving Leech-Lattice Quantized LLMs at 2.7 Bits per Parameter
Sep 28, 2026Pier-Jean MalandrinoLLM Quantization LLM Inference Acceleration
Scub, Bordeaux, France
Fiona: Accelerating FHE Inference with Packing-Aware Ternary Weights
Sep 28, 2026Yiteng Peng, Zhibo Liu, Dongwei Xiao +1Mixed-Precision Quantization Efficient Neural Network Inference
The Hong Kong University of Science and Technology · State Key Laboratory of Novel Software Technology, Nanjing University
EntroPack: Fast and Accurate Entropy-Coded Weight Compression at Arbitrary Bitrates
Sep 28, 2026Hong Zhang, Zhongjie Duan, Yingda ChenEfficient Neural Network Inference Weight-Only Quantization
Alibaba Inc.
PulseQuant: Propagation-Guided Subspace Correction for 4-Bit Video Diffusion Transformers
Sep 27, 2026Yutong Wang, Xingtong Ge, Enhuai Liu +4Video Diffusion Models Diffusion Transformer
The University of Sydney · The Hong Kong University of Science and Technology · The Chinese University of Hong Kong +1
Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures
Sep 24, 2026Se Un Park, Yutae Kim, Junyoung ParkTTS Evaluation Weight-Only Quantization
UX Factory, Inc.
Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement
Sep 22, 2026Akihiro Yoshida, Yuma IchikawaMixed-Precision Quantization LLM Quantization
Fujitsu Limited, Institute of Science Tokyo · Fujitsu Limited, RIKEN Center for AIP
SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models
Sep 21, 2026Kewei Zhang, Zheng Chen, Haotong Qin +1VLM Quantization Efficient VLM Inference
Shanghai Jiao Tong University · The Hong Kong Polytechnic University
Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization
Sep 20, 2026Kuanlin Chen, Chen-Wei Kuo, Cheng-En OuAttention Head Analysis 4-Bit Quantization
Independent Researcher Taoyuan, Taiwan · National Tsing Hua University Hsinchu, Taiwan
Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families
Sep 14, 2026Hyojung HanText Embedding Evaluation Weight-Only Quantization
ThakiCloud
Structured Transforms for Low-Overhead Quantization of Language Models
Sep 11, 2026Daria Cherniuk, Alexander Rudikov, Boris Kashin +1LLM Quantization Weight-Only Quantization
Institute of Numerical Mathematics · Steklov Mathematical Institute
Scaling Post-Training Ternarisation to Qwen3-8B Capability Retention, Reproduction, Lossless Packing, and Packed Execution
Sep 10, 2026Anirudh Malik, M Sparsh Mehra, Poojith DevanLLM Quantization Ternary Quantization
OneBit AI
Quantization Amplifies Determinism, Not Bias: Scale-Dependent Behavioral Effects of Serving-Time Weight Compression
Sep 7, 2026Dachi KurtskhaliaLLM Quantization Weight-Only Quantization
Impel
Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights
Sep 2, 2026Pier-Jean MalandrinoLLM Inference Vector Quantization
Scub
Post-Training Ternarization of Qwen3-4B Capability, Effective Bit Budget, Storage Compression, and Deployment
Sep 2, 2026Anirudh Malik, M Sparsh Mehra, Poojith DevanLLM Quantization Ternary Quantization
OneBit AI
FAMPWQ: Fisher Information-based Adaptive Mixed Precision Weight Quantization for Effective LLM Inference
Aug 24, 2026Gongwei Lee, Ji Liu, Juncheng Jia +1Mixed-Precision Quantization LLM Quantization
School of Computer Science and Technology, Soochow University, Suzhou, China · Hithink Research, Hangzhou, China · Electronic Engineering, Tsinghua University, Beijing, China
Activation-Weighted Seeded Residual Coding for Low-Bit LLM Weight Repair
Aug 24, 2026Zehao Liu, Chuangchuang Fang, Yang RenLLM Quantization Weight-Only Quantization
Huawei Technologies Research & Development (UK) Limited, Edinburgh, United Kingdom
ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization
Aug 11, 2026He-Yen Hsieh, H. T. KungLLM Quantization Weight-Only Quantization
Harvard University
Quantization Degradation in Large Language Models: A Signal-Noise Perspective
Aug 8, 2026Chenxi Zhou, Pengfei Cao, Jinyu Ye +5LLM Quantization Weight-Only Quantization
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China +1
CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights
Aug 7, 2026Xuetian GaoLLM Quantization LLM Inference Acceleration
TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models
Aug 4, 2026Seokho Han, Dongwei Wang, Jinhee Kim +4Mixed-Precision Quantization Diffusion Model Quantization
Department of Electrical and Computer Engineering, Sungkyunkwan University, Korea · Department of Electrical and Computer Engineering, University of Arizona, USA · Department of Electrical and Computer Engineering, Duke University, USA +1
ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads
Aug 3, 2026Şuayp Talha Kocabay, Talha Rüzgar Akkuş, Kamer Ali YukselLLM Quantization LLM Compression
Independent Researcher · aiXplain, Inc.
PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
Jul 17, 2026Yuchen Yang, Yifan Zhao, Anisha Dasgupta +1LLM Quantization Mixture-of-Experts Language Models
University of Illinois Urbana-Champaign
ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level
Jul 15, 2026Chethan Reddy G. PLLM Quantization Ternary Quantization
BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression
Jul 9, 2026Yuantian Shao, Peisong Wang, Zhilei Liu +6LLM Compression Weight-Only Quantization
Nanjing University of Science and Technology, Nanjing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · Huawei, China

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization

Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu, Baihua He+5

University of Science and Technology of China · University College London · Shanghai University of Finance and Economics+5

LLM Quantization Post-Training Quantization Weight-Only Quantization

Open paper AskPDF arXiv

Weight-only post-training quantization (PTQ) relies heavily on reconstruction loss minimization to preserve model quality at low precision. We show that the weights favored by minimizing this loss need not yield better model performance on new tasks. In fact, we find that lower reconstruction loss can even degrade model performance on the same calibration data. Our analysis further shows that weights with lower reconstruction loss on calibration data can have higher loss than other weights when the distribution of input activations changes. Motivated by these observations and our analysis, we propose Distributionally Robust Quantization (DRQ), a post-hoc refinement process that minimizes worst-case reconstruction loss over a constrained set of input activation distributions. DRQ refines the integer codes representing quantized weights within the existing quantization grid, keeping quantization parameters and inference operators unchanged. Extensive experiments show that DRQ improves models quantized by six representative PTQ methods, including AWQ, GPTQ, and ParoQuant, and delivers gains across both dense and mixture-of-experts large language models. These results establish DRQ as a general post-hoc refinement framework for weight-only PTQ, achieving better downstream performance without adding inference overhead.

Momentum

Related topics

Latest papers 55