Quantization-Aware Training

Also known as QAT

Quantization-aware training (QAT) simulates low-precision arithmetic during training so models can learn to withstand rounding and clipping errors at deployment. Common approaches use straight-through estimators, learned or mixed-precision quantizers, progressive bit reduction, and teacher-guided distillation. The methods are applied to settings from ImageNet vision models and edge devices to large language models, where evaluations include accuracy, perplexity, reasoning, and real-device speed. A central challenge is balancing accuracy against bit width and hardware cost: results depend on the numerical format and deployment path, and low-precision training can become unstable or diverge from inference behavior.

Latest papers

89 papers17 papers in the last four weeks, up 89% on the four weeks before. 0.1% of all new papers.

ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning
Oct 26, 2026Rhea Sanjay Sukthanker, Pashmina CameronLLM Quantization Low-Bit Quantization
Microsoft Applied Sciences Group
SQuAT: Spherical Lattice Vector Quantization-Aware Training
Oct 22, 2026Sebastian Loeschcke, Ivan Koryakovskiy, Tycho F. A. van der Ouderaa +4LLM Quantization Vector Quantization
University of Copenhagen · Qualcomm AI Research
Right On Q: Recovering Reasoning under Quantization via Self-Generated Trajectories
Oct 21, 2026Boyuan Zheng, Jing Xiong, Chenwei Wu +8LLM Quantization Quantization-Aware Training
University of Michigan, Ann Arbor · University of California, Santa Barbara · Meta AI
Extreme Low-State Quantized Transformer for Electricity-Load Forecasting
Oct 20, 2026Muhammad Sabbir Alam, Harindra Sandun Mavikumbure, Victor Cobilean +2Low-Bit Quantization Quantization-Aware Training
Department of Mechanical and Nuclear Engineering, Virginia Commonwealth University, Richmond, VA, 23284, USA · Department of Computer Science, Virginia Commonwealth University, Richmond, VA, 23284, USA · Department of Electrical and Computer Engineering, Virginia Commonwealth University, Richmond, VA, 23284, USA
E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks
Oct 20, 2026Xinyu Lei, Hong Huang, Jiebin Zhang +6LLM Quantization Quantization-Aware Training
Peking University · City University of Hong Kong · Tencent
Noise-shaped Sampling and Annealing for Scalable On-the-edge Training of Binary and Ternary AI Models
Oct 15, 2026Zihao Chen, Zhili Xiao, Rui-Jie Zhu +3Binary Neural Networks Ternary Quantization
Department of Electrical and Systems Engineering, Washington University in St. Louis, One Brookings Drive, St. Louis, 63130, MO, USA. · Department of Electrical and Computer Engineering, University of California, Santa Cruz, 1156 High Street, Santa Cruz, 95064, CA, USA. · Department of Computer Science and Engineering, Washington University in St. Louis, One Brookings Drive, St. Louis, 63130, MO, USA.
ShiftQAT: Quantization-Aware Training for the Numerical Contract of a Fixed-Function Edge CNN Accelerator
Oct 15, 2026Oguzhan Buyuksolak, Osman Erman Okman, Mehmet Gorkem UlkarHardware-Aware Quantization Neural Network Quantization
Analog Devices, Inc.
Revisiting Error Feedback for Model Quantization
Oct 13, 2026Wai-Yiu Keung, Chi Cui, Wing-Kin Ma +2Neural Network Quantization LLM Quantization
The Chinese University of Hong Kong, Hong Kong SAR of China · Universidad Carlos III de Madrid, Spain · Aarhus University, Denmark
Recovering Low-Bit Failures in Vision-Language-Action Policies via Quantization-Conditioned Reinforcement Learning
Oct 11, 2026Seongmin Park, Jaewon Lee, Yujun Heo +4VLM Quantization Efficient VLA Model Inference
Hanyang University, Republic of Korea · LG Electronics, Republic of Korea
OnlineQAT: On-Policy Distillation for Ultra-Low-Bit Large Language Models
Oct 7, 2026Wenjun Wang, Heng Li, Yanggan Gu +1LLM Quantization Quantization
The Hong Kong Polytechnic University · Sun Yat-sen University · PolyU-Daya Bay Technology and Innovation Research Institute
Few Bits, One Law: Toward W2A4KV2
Oct 6, 2026Kai Yi, Tarek Elgamal, Sruthikesh Surineni +3Neural Network Quantization LLM Quantization
Meta AI
Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
Oct 6, 2026Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2LLM Quantization Quantization-Aware Training
ISTA · EPFL, ISTA
TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models
Oct 6, 2026Xin Wang, Hao Yu, Zhengyang Zhuge +9LLM Quantization Quantization-Aware Training
Alibaba Token Hub, Alibaba Group · Ohio State University
RPFQ-ViT: Rotated Phase-Frame Quantization for Extremely Low-Bit Weights in Vision Transformers
Oct 3, 2026Mengyuan Fan, Bokai Huang, JiaMing Pan +4Low-Bit Quantization Rotation-Based Quantization
Peking University
LatentQuant: Preserving the Policy-Facing Latent Contract under NVFP4 VAE Quantization
Oct 2, 2026Ziye Deng, Lufang Chen, Shuyu Feng +7Neural Network Quantization 4-Bit Quantization
UC San Diego · XSquare Robot · Sun Yat-Sen University +2
Q-MINO: A Minimal-Norm Method for Quantization-Aware Training
Sep 30, 2026Don LiStraight-Through Estimator Quantization-Aware Training
Department of Mathematics & Statistics Portland State University
QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs
Sep 30, 2026Weili Xu, Jisen Li, Yuqing Jian +8LLM Quantization Quantization-Aware Training
Together AI · University of Illinois Urbana-Champaign · The University of Texas at Austin
Delta-Matching: Closing the Final Gap of Native 8-bit Training for LLMs
Sep 29, 2026Haozhan Tang, Hao Kang, Han Cai +2Low-Precision Training LLM Training
Carnegie Mellon University · NVIDIA · NVIDIA & Massachusetts Institute of Technology
JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization
Sep 27, 2026Kaicheng Yang, Kaisen Yang, Chunyu Liu +8Neural Network Quantization Diffusion Model Quantization
Shanghai Jiao Tong University · Tsinghua University · The Hong Kong Polytechnic University
Pretraining Transformers with Quantized Softmax in Attention
Sep 27, 2026Shangzhen Zhu, Muyan Hu, Tomasz KozlowskiLanguage Model Pretraining Quantization-Aware Training
University of Illinois Urbana-Champaign
Quantization-Aware Pre-Training with Constrained Empirical Weight Distribution
Sep 26, 2026Ningfeng Yang, Tor M. AamodtQuantization-Aware Training LLM Quantization
University of British Columbia
Depth Laws for the Precision Floor of Trained Neural Networks: Amplification, Residual Scaling, and a Quantization-Aware Training Paradox
Sep 25, 2026Ahmad S. TarawnehNeural Network Quantization Post-Training Quantization
Faculty of Information Technology Mutah University Al-Karak, Jordan
AERIAL: Adversarial Evaluation of Robustness in Accuracy-Preserving Low-Precision EEG Decoders
Sep 24, 2026Saim Rehman, Muhammad ShafiqueQuantization-Aware Training EEG Decoding
eBRAIN Lab, Division of Engineering, New York University Abu Dhabi (NYUAD), Abu Dhabi, UAE
Baseline Shape Decides the Verdict: A Controlled Re-Examination of Ternary Language Models at 60K Parameters
Sep 24, 2026Gautam VeldandaTernary Quantization Quantization-Aware Training
Independent Researcher
Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning
Sep 22, 2026Yuanteng Chen, Zhilei Liu, Peisong Wang +7Low-Bit Quantization LLM Quantization
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Zhongguancun Academy +2
Towards Full Pipeline FP8 Reinforcement Learning for LLMs
Sep 19, 2026Fanchao Chen, Ziheng Jiang, Ziyun Wei +5RL for Language Models RL Fine-Tuning
University of Wisconsin–Madison · Work done during internship at ByteDance Seed · ByteDance Seed
DSAQuant: Denoising-Stage-Aligned Quantization-Aware Training for Video Generation
Sep 3, 2026Shuaiting Li, Zelin Gao, Haibin Shen +3Diffusion Model Guidance Video Diffusion Models
1Robbyant · 2ZJU · 4HKUST +1
A Target-Centric Survey of Quantization-Aware Training
Aug 30, 2026Jiamin Song, Mengjie Zhao, Zijing Wang +6LLM Quantization Quantization-Aware Training
1Northeastern University, China · 2Shandong University, China · 3CIS, LMU Munich; MCML, Germany
QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction
Aug 14, 2026Vincent Counathe, Ben Athiwaratkun, Christopher De Sa +1LLM Quantization Quantization-Aware Training
Cornell University · Together AI
FQTree: Fine-grained Quantization and Hardware Generation of Boosted Decision Trees
Aug 12, 2026Zhiqiang Que, Chang Sun, Haiyang Wang +6Gradient-Boosted Decision Trees Quantization-Aware Training
University of Bristol, UK · California Institute of Technology, USA · Imperial College London, UK

Oct 26, 2026J/K move · Enter open · S save

ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning

Rhea Sanjay Sukthanker, Pashmina Cameron

Microsoft Applied Sciences Group

LLM Quantization Low-Bit Quantization Post-Training Quantization Mixed-Precision Quantization Quantization-Aware Training

Open paper AskPDF arXiv

Reasoning language models generate thousands of tokens per answer, and generation speed is limited by memory bandwidth, so storing weights in two bits instead of bfloat16 can potentially make inference 8

\times

cheaper. Two-bit methods are usually judged by perplexity and multiple-choice question-answering accuracy, both of which say little about whether a model can still reason through more difficult problems. We present ThinkBit, a strictly scalar two-bit framework built to preserve reasoning, in which every weight is an integer codeword times a bfloat16 scale shared by a group of weights. We first use a sensitivity metric to assign each layer a ternary, 2-bit, or 3-bit tier to allocate bitwidths optimally among layers. This is followed by a layerwise Gumbel-softmax-based scalar quantization algorithm that searches for optimal codewords block by block. Since a layerwise scheme does not capture how quantized layers interact with each other, we recalibrate scales, with the codewords frozen, using a knowledge-distillation-based loss against the teacher model. We then perform a short quantization-aware training (QAT) stage that lets the model mimic the base model's distribution by reassigning codewords. Finally, a paired reasoning test on training problems selects the final checkpoint between the recalibration stage and the QAT stage. On Qwen3-1.7B at matched size, our recalibrated checkpoint scores 46.7 on MATH-500 against 30.0 for EfficientQAT trained on the same data, and 73.0 after QAT on reasoning traces; at 0.52 GB it is smaller than every published 2-bit GGUF of the model yet more accurate on MATH-500. We show that this framework scales and up to 14B, a 3.74 GB model scores 94.8 on MATH-500, against 97.5 for the 29.5 GB bfloat16 model.