LLM Quantization

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

33 papers in the last four weeks, up 43% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 251

All topics

Statistically-Lossless Quantization of Large Language Models
May 4, 2026Michael Helcig, Eldar Kurtic, Dan AlistarhLLM Quantization LLM Inference Acceleration
ETH Z̈urich · ISTA
Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels
May 2, 2026Plawan Kumar Rath, Rahul MaliakkalLLM Quantization Social Bias in Language Models
Meta
AGoQ: Activation and Gradient Quantization for Memory-Efficient Distributed Training of LLMs
May 1, 2026Wenxiang Lin, Juntao Huang, Luhan Zhang +5LLM Quantization Communication-Efficient Distributed Training
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen · Huawei Technologies Ltd.
Silicon Showdown: Performance, Efficiency, and Ecosystem Barriers in Consumer-Grade LLM Inference
May 1, 2026Abdurrahman Javat, Allan KazakovLLM Inference Efficiency LLM Quantization
Bahcesehir University, Istanbul, Türkiye
BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs
May 1, 2026Zhixiong Zhao, Zukang Xu, Dawei YangLLM Quantization LLM Inference Acceleration
1Houmo AI
Technical Report: Activation Residual Hessian Quantization (ARHQ) for Low-Bit LLM Quantization
Apr 30, 2026YiFeng Wang, Zhun Sun, Keisuke SakaguchiLLM Quantization Activation Quantization
Graduate School of Information Sciences Tohoku University
CoQuant: Joint Weight-Activation Subspace Projection for Mixed-Precision LLMs
Apr 29, 2026Zhe Ding, Su Pan, Duowei PanMixed-Precision Quantization LLM Quantization
School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing, China · Amazon AGI, Seattle, WA, USA
Density Field State Space Models: 1-Bit Distillation, Efficient Inference, and Knowledge Organization in Mamba-2
Apr 28, 2026Chirag ShindeModel Compression LLM Quantization
Independent Researcher
BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deployment
Apr 27, 2026Md. Ashiq Ul Islam Sajid, Mohammad Sakib Mahmood, Md. Tareq Hasan +3LLM Quantization RL for Language Models
Department of Computer Science BRAC University Dhaka 1212, Bangladesh · Department of Computer Science Missouri State University Springfield, MO, USA · Department of Computer Science & Engineering Prime University Dhaka, Bangladesh +2
Coverage-Based Calibration for Post-Training Quantization via Weighted Set Cover over Outlier Channels
Apr 27, 2026Ibne Farabi Shihab, Sanjeda Akter, Anuj SharmaLLM Quantization Activation Outliers
Department of Computer Science, Iowa State University · Department of Civil, Construction & Environmental Engineering, Iowa State University
QuantClaw: Precision Where It Matters for OpenClaw
Apr 24, 2026Manyi Zhang, Ji-Fu Li, Zhongao Sun +5Mixed-Precision Quantization LLM Quantization
Huawei Technologies · National University of Singapore · University of Science and Technology of China
RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
Apr 22, 2026Fei Zuo, Zikang Zhou, Hao Cong +2LLM Inference Efficiency Mixed-Precision Quantization
1BA TechWorks (BMW Group) · 2National University of Singapore · 3Tsinghua University
From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
Apr 21, 2026Chenxi Zhou, Pengfei Cao, Jiang Li +4LLM Quantization Post-Training Quantization
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences · College of Computer Science, Inner Mongolia University +1
LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation
Apr 21, 2026Siqing Song, Chuang Wang, Yong Lang +2LLM Quantization Activation Quantization
MAIS, Institute of Automation, Chinese Academy of Sciences, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, China · Central Media Technology Institute, Huawei
Benchmarking System Dynamics AI Assistants: Cloud Versus Local LLMs on CLD Extraction and Discussion
Apr 20, 2026Terry LeitchLLM Quantization Dynamical Systems
Ruxton AI
GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling
Apr 20, 2026Alireza Dadgarnia, Soroush Tabesh, Mahdi Nikdan +4LLM Quantization Post-Training Quantization
ISTA · ETH Zürich · ISTA & Red Hat AI +1
Depth Registers Unlock W4A4 on SwiGLU: A Reader/Generator Decomposition
Apr 20, 2026Ziyang LiuLLM Quantization 4-Bit Quantization
Independent Researcher
DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization
Apr 20, 2026Haokun Lin, Xinle Jia, Haobo Xu +7LLM Quantization 4-Bit Quantization
CASIA · CityU · NJU +3
D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
Apr 18, 2026Junlin Li, Shuangyong Song, Guodong Du +6LLM Quantization LLM Compression
Harbin Institute of Technology, Shenzhen, China · TeleAI of China Telecom · The Hong Kong Polytechnic University +1
When Flat Minima Fail: Characterizing INT4 Quantization Collapse After FP32 Convergence
Apr 16, 2026Marcus ArmstrongLLM Quantization 4-Bit Quantization
Department of Computer Science University of Houston Houston, TX 77004
Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate
Apr 15, 2026Jaemin Kim, Sungkyun Kim, Junyeol Lee +1LLM Quantization Language Model Calibration
Seoul National University · Hanyang University
Leech Lattice Vector Quantization for Efficient LLM Compression
Mar 11, 2026Tycho F. A. van der Ouderaa, Mart van Baalen, Paul Whatmough +1LLM Quantization LLM Compression
Qualcomm AI Research
Dissecting Quantization Error: A Concentration-Alignment Perspective
Mar 4, 2026Marco Federici, Boris van Breugel, Paul Whatmough +1LLM Quantization 4-Bit Quantization
Qualcomm AI Research
QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs
Feb 11, 2026Kanghyun Noh, Jinheon Choi, Yulhwa KimLLM Quantization LLM Compression
Department of Semiconductor Convergence Engineering, Sungkyunkwan University · Department of Electrical and Computer Engineering, Sungkyunkwan University · Department of Semiconductor Systems Engineering, Sungkyunkwan University
Investigating Social Bias Changes in Quantized Language Models
Feb 5, 2026Stanley Z. Hua, Sanae Lotfi, Irene Y. ChenLLM Quantization Social Bias in Language Models
UC Berkeley & UCSF · Meta Superintelligence Labs
MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models
Feb 3, 2026Maximilian Kleinegger, Elvir Crnčević, Dan AlistarhLLM Quantization LLM Inference Acceleration
1Vienna University of Technology, Vienna, Austria · Institute of Science & Technology Austria (ISTA), Vienna, Austria · 3Red Hat AI, Boston, USA
Preserving Fairness and Safety in Quantized LLMs Through Critical Weight Protection
Jan 17, 2026Muhammad Alif Al Hakim, Alfan Farizki Wicaksono, Fajri KotoLLM Quantization Multilingual Language Model Evaluation
Faculty of Computer Science, Universitas Indonesia · Mohamed bin Zayed University of Artificial Intelligence
ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs
Jan 12, 2026Haoqian Meng, Yilun Luo, Yafei Zhao +3LLM Quantization 4-Bit Quantization
School of Computer Science and Technology, Tianjin University
When Less is More: 8-bit Quantization Improves Continual Learning in Large Language Models
Dec 22, 2025Michael S. Zhang, Rishi A. Ruia, Arnav Kewalram +3LLM Quantization Continual Learning for LLMs
Algoverse
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Mixed-Precision Quantization LLM Quantization
1Technion—Israel Institute of Technology · 2Intuit · 3Ben-Gurion University of the Negev +1

May 4, 2026, cs.LGJ/K move · Enter open · S save

Statistically-Lossless Quantization of Large Language Models

Michael Helcig, Eldar Kurtic, Dan Alistarh

ETH Z̈urich · ISTA

LLM Quantization LLM Inference Acceleration Post-Training Quantization Low-Bit Quantization Neural Network Quantization

Open paper AskPDF arXiv

Model quantization has become essential for efficient large language model deployment, yet existing approaches involve clear trade-offs: methods such as GPTQ and AWQ achieve practical compression but are lossy, while lossless techniques preserve fidelity but typically do not accelerate inference. This paper explores the middle ground of statistically-lossless compression through three complementary notions of losslessness for quantized LLMs. First, task-lossless compression preserves zero-shot benchmark accuracy within natural sampling variance and remains achievable at aggressive bitwidths. Second, we formalize the stricter notion of distribution-lossless compression, requiring the quantized model's next-token distribution to be practically indistinguishable from the original, and propose the Expected Acceptance Rate (EAR), the maximum token-agreement probability under optimal coupling, as a directly interpretable fidelity metric (for example, EAR >= 0.99 indicates 99% agreement). Third, we prove a gamma-squared variance law showing that symmetric quantization inflates noise variance by gamma squared relative to asymmetric quantization, making asymmetry necessary for distribution-lossless fidelity but not for task-level preservation. Using SLQ, a layer-wise non-uniform method with asymmetric quantization and wide bitwidth search, we achieve task-lossless compression at well below 4 bits per parameter (as low as 3.3 bits depending on the model), distribution-lossless compression at 5 to 6 bits per parameter on average, and inference speedups of 1.7 to 3.6x relative to FP16 with optimized kernels. Source code is available at https://github.com/IST-DASLab/SLQ.

Momentum

Related topics

Latest papers 251