Post-Training Quantization

Also known as PTQ

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

360 papers70 papers in the last four weeks, up 46% on the four weeks before. 0.4% of all new papers.

Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression
May 26, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +3KV-Cache Compression LLM Quantization
Massachusetts Institute of Technology, Cambridge, MA, USA · University of Toronto, Toronto, Canada · IBM Research, Cambridge, MA, USA.
Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V
May 26, 2026Junhao Wu, Dezhong Yao, Hai JinVideo Diffusion Models Diffusion Transformer
School of Computer Science and Technology, Huazhong University of Science and Technology Wuhan, Hubei, China 430074
WINDQuant: Weight-Informed Neural Decision-Making for Global Mixed-Precision LLM Quantization
May 26, 2026Phong Nam Huu Nguyen, Khoi M. Le, Cong-Duy T Nguyen +3Mixed-Precision Quantization LLM Quantization
CAIR, VinUniversity, Vietnam
Tail-Aware HiFloat4: W4A4 Post-Training Quantization for Wan2.2
May 26, 2026Zhanfeng Feng, Shuai Guo, Xin Di +3Video Diffusion Models 4-Bit Quantization
University of Science and Technology of China Hefei, China
QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
May 25, 2026Preetam Sharma, Kacper DobekLLM Quantization Rotation-Based Quantization
Independent Research · Institute of Computing Science Poznan University of Technology, Poznan, Poland
GoQuant: Geometric Orthogonal Residual Projection for Multiplier-Free Power-of-Two Transformer Quantization
May 25, 2026Maoyang Xiang, Tao Luo, Bo WangLLM Quantization Efficient Neural Network Inference
Information Systems Technology and Design Singapore University of Technology and Design Singapore · Institute of High Performance Computing (IHPC) Agency for Science, Technology and Research (A*STAR) Singapore
InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
May 25, 2026Ke Li, Dong An, Xiaoling Zang +6LLM Quantization Activation Quantization
School of Software Technology, Zhejiang University · Ant Group · College of Computer Science and Technology, Zhejiang University of Technology +3
GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
May 21, 2026Jianing Deng, Song Wang, Dongwei Wang +4Mixed-Precision Quantization LLM Quantization
University of Pittsburgh · University of Central Florida · University of Arizona +1
MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantization
May 20, 2026Zhong Wang, Zukang Xu, Xing Hu +1Mixed-Precision Quantization Vision-Language Models
Bauman Moscow State Technical University · Houmo AI
ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models
May 19, 2026Arash Akbari, Arman Akbari, Masih Eskandar +11Neural Network Quantization VLM Quantization
1Northeastern University · University of Georgia · 3Cisco Systems
Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor
May 19, 2026Xiaocan Li, Shiliang Wu, Zheng ShenLLM Quantization 4-Bit Quantization
Huawei Canada
Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models
May 19, 2026Yi Zhong, Haotong Qin, Xindong Zhang +2Vision-Language Models VLM Quantization
VCIP, College of Computer Science, Nankai University · D-ITET, ETH Zürich · OPPO Research Institute +1
Quant.npu: Enabling Efficient Mobile NPU Inference for on-device LLMs via Fully Static Quantization
May 19, 2026Jinghe Zhang, Daliang Xu, Chenghua Wang +5Mixed-Precision Quantization LLM Quantization
TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization
May 19, 2026Zukang Xu, Xing Hu, Dawei YangLLM Quantization Rotation-Based Quantization
1Houmo AI.
Quantized Machine Learning Models for Medical Imaging in Low-Resource Healthcare Settings
May 19, 2026Sumanth Meenan Kanneti, Aryan ShahEfficient Neural Network Inference Medical Image Classification
Georgia State University · University of Southern California
GAMMA: Global Bit Allocation for Mixed-Precision Models under Arbitrary Budgets
May 18, 2026Zhangyang Yao, Haiyan Zhao, Haoyu Wang +3Mixed-Precision Quantization LLM Quantization
Beihang University · Tsinghua University · ByteDance Inc
MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization
May 18, 2026Le Su, Xing Luo, Zhi JinLLM Quantization Post-Training Quantization
School of Intelligent Systems Engineering Shenzhen Campus of Sun Yat-sen University Shenzhen, Guangdong 518107, China · Department of Frontier Research Peng Cheng Laboratory Shenzhen, Guangdong 518055, China
OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
May 18, 2026Zhongzhu Zhou, Donglin Zhuang, Jisen Li +4Rotation-Based Quantization LLM Inference Acceleration
Together AI · University of Sydney · University of Illinois Urbana-Champaign
CAR-SAM: Cross-Attention Reconstruction for Post-Training Quantization of the Segment Anything Model
May 16, 2026Houji Wen, Jiangyong Yu, Jun Li +1Cross-Attention 4-Bit Quantization
1Nanjing University of Science and Technology · 2Houmo AI
E-PMQ: Expert-Guided Post-Merge Quantization with Merged-Weight Anchoring
May 16, 2026Wenjun Wang, Yanggan Gu, Shuo Cai +44-Bit Quantization Post-Training Quantization
The Hong Kong Polytechnic University · PolyU-Daya Bay Technology and Innovation Research Institute
DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
May 16, 2026Sayeh Sharify, Mahsa Salmani, Hesham MostafaDiffusion Transformer Diffusion Model Quantization
d-Matrix Santa Clara, CA 95054
Not All Tasks Quantize Equally: Fisher-Guided Quantization for Visual Geometry Transformer
May 15, 2026Yipu Zhang, Jintao Cheng, Weilun Feng +5Feed-Forward 3D Reconstruction Post-Training Quantization
Department of Electronic and Computer Engineering, HKUST · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences +2
Widening the Gap: Exploiting LLM Quantization via Outlier Injection
May 14, 2026Xiaohua Zhan, Kazuki Egashira, Robin Staab +2LLM Quantization LLM Backdoor Attacks
ETH Zurich
Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution
May 14, 2026Saisab Sadhu, Pratinav Seth, Vinay Kumar SankarapuMachine Unlearning Post-Training Quantization
Lexsi Labs
A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models
May 14, 2026Earl KillianMixed-Precision Quantization LLM Quantization
Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantization
May 14, 2026Peilin Sun, Jianxin WuActivation Outliers Quantization Error Compensation
State Key Laboratory of Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China
XFP: Quality-Targeted Adaptive Codebook Quantization with Sparse Outlier Separation for LLM Inference
May 14, 2026Thomas WittLLM Quantization Vector Quantization
Gemini Stiftung Leipzig
High-Rate Quantized Matrix Multiplication II
May 13, 2026Or Ordentlich, Yury PolyanskiyLLM Quantization Rate-Distortion Theory
the Hebrew University of Jerusalem, Israel · MIT, USA
Search Your Block Floating Point Scales!
May 12, 2026Tanmaey Gupta, Hayden Prairie, Xiaoxia Wu +104-Bit Quantization LLM Inference Acceleration
Cornell University, Ithaca, NY, USA · Together AI, USA · University of California San Diego, La Jolla, CA, USA +1
Grid Games: The Power of Multiple Grids for Quantizing Large Language Models
May 12, 2026Vage Egiazarian, Erik Schultheis, Andrei Panferov +3LLM Quantization 4-Bit Quantization
ISTA · Aril Computer · ETH Zürich +1

May 26, 2026J/K move · Enter open · S save

Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression

Kabir Swain, Sijie Han, Daniel Karl I. Weidele, Mauro Martino+2

Massachusetts Institute of Technology, Cambridge, MA, USA · University of Toronto, Toronto, Canada · IBM Research, Cambridge, MA, USA.

KV-Cache Compression LLM Quantization KV Caching Neural Network Quantization KV-Cache Quantization Post-Training Quantization+1

Open paper AskPDF arXiv

We propose Hurwitz Quaternion Multiplicative Quantization (HQMQ), a calibration-free method for KV cache compression of large language models. HQMQ treats each 4-element chunk of K or V as a quaternion and quantizes its unit direction to the product

q_p \cdot q_s

, where

q_p

ranges over the 24-element Hurwitz group

2T

(the 24 vertices of the 24-cell on

S^3

, pairwise angle

60^\circ

) and

q_s

ranges over a per-(layer, head) secondary codebook of

S

random unit quaternions. The multiplicative composition yields

24S

effective codewords at

S

stored parameters; random initialization suffices because left-multiplication is an

S^3

isometry, so seeded codebooks vary in end-task ppl by

<1.5\%

. A per-batch median-multiplier outlier extraction step (

C{=}3

, no calibration) handles modern outlier-heavy architectures. We evaluate on five modern open models: Mistral-7B (dense MHA), Llama-3-8B and Qwen2.5-7B and Qwen3-8B (dense GQA), and gpt-oss-20b (sparse MoE). On Mistral-7B and Qwen3-8B, HQMQ matches fp16 within

0.02

0.03

ppl points at

\sim

5 bits. On Qwen2.5-7B and Qwen3-8B, where naive int4 collapses to

10^4{+}

ppl, HQMQ + Med3

\times

recovers fp16 quality within

0.02

0.10

ppl points at

\sim

5 bits. HQMQ Pareto-dominates naive int by

3

1900\times

at matched bits across all five models, and downstream zero-shot accuracy matches fp16 at

3.79

bits on Mistral. Against the strongest calibrated KV-quantization baseline, HQMQ at

3.79

bits matches KIVI-4 (

\sim 4.5

bits) within

{\sim}1

pt on CoQA,

0.6

pts on TruthfulQA, and

2.3

pts on GSM8K, at

16\%

fewer bits and without a calibration pass. At the storage level, HQMQ delivers up to

5.05\times

KV compression, shrinking a Llama-3-70B 128k-context cache from 43 GB to 8.5 GB.