Low-Bit Quantization

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 62% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 155

All topics

Fine-Tuning Low-Bit Models with Gradient in Quantized Code Space
Aug 31, 2026Shiguang Wu, Zhouchen Lin, Quanming YaoFine-Tuning Low-Bit Quantization
Department of Electronic Engineering, Tsinghua University · School of Intelligence Science and Technology, Peking University
RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search
Aug 31, 2026Rastislav Lenhardt, Teodora Dobos, Thomas Vecchiato +2Rotation-Based Quantization Vector Quantization
Google Zurich, Switzerland · Technical University of Munich Munich, Germany · University of Copenhagen Copenhagen, Denmark +1
A Target-Centric Survey of Quantization-Aware Training
Aug 30, 2026Jiamin Song, Mengjie Zhao, Zijing Wang +6LLM Quantization Quantization-Aware Training
1Northeastern University, China · 2Shandong University, China · 3CIS, LMU Munich; MCML, Germany
Activation-Weighted Seeded Residual Coding for Low-Bit LLM Weight Repair
Aug 24, 2026Zehao Liu, Chuangchuang Fang, Yang RenLLM Quantization Weight-Only Quantization
Huawei Technologies Research & Development (UK) Limited, Edinburgh, United Kingdom
Beyond Dense Adam States: Adaptive Log-Space Quantization for Memory-Efficient Optimizers
Aug 23, 2026Yan WangDeep Learning Optimization Memory-Efficient Optimization
Independent Researcher
QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction
Aug 14, 2026Vincent Counathe, Ben Athiwaratkun, Christopher De Sa +1LLM Quantization Quantization-Aware Training
Cornell University · Together AI
HAMP-LIC: Hessian-Aware Mixed-Precision Post-Training Quantization for Learned Image Compression
Aug 12, 2026Yuefeng ZhangMixed-Precision Quantization Learned Image Compression
Beijing Institute of Computer Technology and Application, Beijing, China · School of Elite Engineering, Northwestern Polytechnical University, Shaanxi, China
SoftWater: Class-Aware Rate Allocation for Softmax Quantization
Aug 12, 2026Joao V. Cavalcanti, Ashia C. WilsonLLM Quantization Post-Training Quantization
Department of Electrical Engineering and Computer Science · Massachusetts Institute of Technology · Cambridge, MA 02139, USA
ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization
Aug 11, 2026He-Yen Hsieh, H. T. KungLLM Quantization Weight-Only Quantization
Harvard University
SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features
Aug 11, 2026HyeonJun Lee, Hyeonsik Jo, Jinwoo Chung +1Quantization-Aware Training Self-Supervised Knowledge Distillation
Kookmin University
From Sweep to Seam: Interleaved Cross-Block Post-Training Quantization
Aug 10, 2026Achille Jacquemond, Yuma Ichikawa, Akira SakaiLLM Quantization Post-Training Quantization
Fujitsu Limited · RIKEN Center for AIP · Tokai University
CurveFP: Co-Designing Numerical Representation and Product Arithmetic for Language Models
Aug 8, 2026Ye QiaoLLM Quantization Low-Bit Quantization
Department of Electrical Engineering and Computer Science University of California, Irvine Irvine, CA, USA
CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights
Aug 7, 2026Xuetian GaoLLM Quantization LLM Inference Acceleration
Which Decisions Low-Bit Quantization Breaks, and How to Predict Them
Aug 6, 2026Zekun Wu, Swati Dhiman, Adriano KoshiyamaPost-Training Quantization Low-Bit Quantization
Holistic AI · UCL Centre for Artificial Intelligence, University College London
Low-Dimensional High-Leverage Subspace Optimization: Beyond Full-Parameter Coupled Training for Neural Network Quantization
Aug 4, 2026Peng Xia, Junbiao Pang, Zheng HuangQuantization-Aware Training Post-Training Quantization
School of Information Science and Technology, Beijing University of Technology, Beijing, China
Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms
Aug 4, 2026Samuel Fernández-Menduiña, Amir Ziashahabi, Eduardo Pavez +2KV Caching Vector Quantization
Department of Electrical and Computer Engineering, University of Southern California
Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs
Aug 4, 2026Yu Luo, Bo Dong, Wenhua Cheng +1Mixed-Precision Quantization LLM Quantization
Intel
Output-Aware Rotation for INT2 KV-Cache Quantization
Aug 3, 2026Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4LLM Inference KV Caching
University of Southern California · Seoul National University · Inha University
LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference
Jul 30, 2026Sangjin Kim, Yuseon Choi, Jungjun Oh +2LLM Quantization AI Accelerator Inference
PIM Semiconductor Design Research Center (AI-PIM), Daejeon 34141, South Korea
GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference
Jul 30, 2026Sangjin Kim, Yuseon Choi, Byeongcheol Kim +2AI Accelerator Inference Rotation-Based Quantization
School of Electrical Engineering, Korea Advanced Institue of Science and Technology (KAIST)
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model
Jul 28, 2026Mahendra Singh Rathor, Anagheem AzzamMemory-Efficient Fine-Tuning Small Language Models
Independent Researcher · Jaipur, India · Hamburg, Germany
Research Report on Noise-Shaped One-Bit Coefficients in Discrete Polynomial Fourier Extension
Jul 26, 2026Shengquan WangLow-Bit Quantization
[cs.CL]
When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation
Jul 25, 2026Mojtaba SoltanalianLow-Bit Quantization Neural Network Quantization
Department of Electrical and Computer Engineering University of Illinois Chicago 851 S. Morgan St., MC 154 Chicago, IL 60607, USA
Approximate reservoir computing with a semiconductor laser for reducing energy consumption
Jul 25, 2026Tatsuki Ito, Kazutaka Kanno, Satoshi Kawakami +1Energy-Efficient ML Reservoir Computing
Department of Information and Computer Sciences, Saitama University, 255 Shimokubo, Sakura-ku, Saitama City, Saitama 338-8570, Japan · Faculty of Information Science and Electrical Engineering, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka 819-0395, Japan
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
Jul 22, 2026Miguel P. Bento, João F. SeabraLLM Quantization Rotation-Based Quantization
Independent Researcher · Lisbon, Portugal
Contraction-Gauge Preconditioning for Quantized Matrix Multiplication
Jul 21, 2026Piyush Sao, Narasinga Miniskar, Pedro Valero-Lara +2Post-Training Quantization Low-Bit Quantization
Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, USA
Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models
Jul 16, 2026Ryona NodaLLM Quantization Post-Training Quantization
Ryo Noda
ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMM
Jul 16, 2026Hyunwoo Oh, Suyeon Jang, Hanning Chen +3Efficient Neural Network Inference Hardware-Software Co-Design
University of California, Irvine
Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models
Jul 13, 2026Peng Xia, Junbiao Pang, Muhammad Ayub SabirActivation Quantization Post-Training Quantization
School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China
The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
Jul 9, 2026Baha Rababah, Cuneyt Gurcan Akcora, Carson K. LeungLLM Evaluation Post-Training Quantization
Department of Computer Science, University of Manitoba, Canada · AI Initiative, University of Central Florida, USA

Aug 31, 2026, cs.LGJ/K move · Enter open · S save

Fine-Tuning Low-Bit Models with Gradient in Quantized Code Space

Shiguang Wu, Zhouchen Lin, Quanming Yao

Department of Electronic Engineering, Tsinghua University · School of Intelligence Science and Technology, Peking University

Fine-Tuning Low-Bit Quantization

Open paper AskPDF arXiv

Fine-tuning Low-bit models aims to adapt a quantized model while keeping the final deployed checkpoint in the same low-bit form. This setting is practically important as it reduces memory and inference cost for storage and deployment. Under this constraint, adaptation becomes an optimization problem over quantization codes and scales. Existing continuous low-bit training is efficient, but it can be distorted by straight through estimation error or by post-quantize gap; discrete search is deployment-faithful, but it is often too inefficient under a finite training budget. We propose code surrogate gradient as the first order signal in deployable code space to acceleate optimization, and performing guided search to preserve deployment faithfulness. Experiments across arithmetic reasoning, instruction following, and structured language understanding show that GradCodes consistently improves fine-tuning low-bit models across different quantization datatypes. Code is provided at https://github.com/ovo67/GradCodes.

Momentum

Related topics

Latest papers 155