Post-Training Quantization

Also known as PTQ

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

361 papers71 papers in the last four weeks, up 48% on the four weeks before. 0.4% of all new papers.

RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models
Sep 21, 2026Zhiping Wu, Dongdong Ren, Yangchengyu Zhou +4VLM Quantization Large Vision-Language Models
State Key Laboratory for Novel Software Technology and the School of Intelligence Science and Technology, Nanjing University, Suzhou Campus, Suzhou 215163, China
VLAQuantBench: Closed-Loop Evaluation of Post-Training Quantization for Vision-Language-Action Models
Sep 21, 2026Jiuyi Xu, Qing Jin, Meida Chen +3Mixed-Precision Quantization Closed-Loop Control of VLA Models
Colorado School of Mines · Independent Researcher · University of Central Florida +2
SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models
Sep 21, 2026Kewei Zhang, Zheng Chen, Haotong Qin +1VLM Quantization Efficient VLM Inference
Shanghai Jiao Tong University · The Hong Kong Polytechnic University
When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs
Sep 21, 2026Yeji Kim, Mi-Young Kim, Randy GoebelLLM Quantization Faithfulness of Language Model Explanations
University of Alberta
FoldQuantVLA: Native Low-Bit Quantization of Vision-Language-Action Models via Consistent Folding
Sep 21, 2026Hung T. Ho, Khanh D. Nguyen, Quang D. Nguyen +5VLM Quantization Efficient VLA Model Inference
VinRobotics, Vietnam · AICV Lab, EECS Department, University of Arkansas · School of Advanced Manufacturing and Robotics, Peking University, China +2
The Undetected Damage of Quantization on Retrieval and How to Fix It
Sep 21, 2026Luca Zhou, Alessandro Zirilli, Daniele Solombrino +2Mixed-Precision Quantization Information Retrieval
Sapienza University of Rome · Not Diamond · Paradigma
ShatterQuant: Breaking Uniform Precision with Block-Wise Mixed-Precision on a Systolic Transformer Hardware Accelerator
Sep 20, 2026Mikolaj Walczak, Edward Humes, Chao Fang +2Mixed-Precision Quantization AI Accelerator Inference
Johns Hopkins University Baltimore, Maryland, USA
Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization
Sep 20, 2026Kuanlin Chen, Chen-Wei Kuo, Cheng-En OuAttention Head Analysis 4-Bit Quantization
Independent Researcher Taoyuan, Taiwan · National Tsing Hua University Hsinchu, Taiwan
Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals
Sep 18, 2026Yamato Narita, Issei SatoPost-Training Quantization Neural Network Quantization
Department of Computer Science, The University of Tokyo
Predict Before You Deploy: Offline Prediction of Quantization-Induced Task Degradation for World Action Models
Sep 16, 2026Jiuyi Xu, Jinjia Guo, Meida Chen +2World Action Models Post-Training Quantization
Robotics Program, Colorado School of Mines, Golden, CO USA. · Department of Civil and Coastal Engineering, University of Florida, Gainesville, FL USA. · University of Southern California - Institute for Creative Technology, Los Angeles, CA USA. +1
The Inference Engineering Pareto Atlas: Which Optimizations Dominate the Cost, Quality, and Latency Frontier?
Sep 15, 2026Srikanta Datta Tumkur, Jay Iyer, Mehar Simhadri +3LLM Inference Efficiency LLM Quantization
Vizuara
Channel-Wise and Token-Aware Post-Training Quantization for Visual State Space Duality
Sep 15, 2026Jonghyeon Lim, Changhoon YimVisual SSMs Activation Quantization
Intelligent Image Processing Laboratory · Department of Computer Science and Engineering, Konkuk University · Seoul, Republic of Korea
Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families
Sep 14, 2026Hyojung HanText Embedding Evaluation Weight-Only Quantization
ThakiCloud
Is INT8 Portable? A Cross-Platform Measurement Study of Quantized Inference on Embedded and Automotive Accelerators
Sep 14, 2026Yuyeong ShinAI Accelerator Inference Neural Network Quantization
Korea Automotive Technology Institute (KATECH)
WaterKron and FlipFlop Hessian: Information-Theoretically Grounded Quantization with Kronecker-factored Hessians
Sep 13, 2026Johann Birnick, Rayan SaabNeural Network Quantization Post-Training Quantization
Department of Mathematics University of California San Diego La Jolla, CA 92093, USA · Department of Mathematics and HDSI University of California San Diego La Jolla, CA 92093, USA
Why Does Post-Training Quantization Work?
Sep 10, 2026Yuxiang Chen, Michael Beyer, Jun Zhu +1LLM Quantization Quantization Robustness
Dept. of Comp. Sci. and Tech., Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint ML Center, Tsinghua University · College of AI, Tsinghua University · Bosch AI Research, Renningen, Germany
Structured Transforms for Low-Overhead Quantization of Language Models
Sep 10, 2026Daria Cherniuk, Alexander Rudikov, Boris Kashin +1LLM Quantization Weight-Only Quantization
Institute of Numerical Mathematics · Steklov Mathematical Institute
OmniKVQuant: KV Cache Quantization for Omni-LLMs
Sep 10, 2026Suho Yoo, Hyunjong Ok, Jongmin Choi +2Efficient Multimodal Inference KV Caching
KAIST · POSTECH
Scaling Post-Training Ternarisation to Qwen3-8B Capability Retention, Reproduction, Lossless Packing, and Packed Execution
Sep 8, 2026Anirudh Malik, M Sparsh Mehra, Poojith DevanLLM Quantization Ternary Quantization
OneBit AI
Accuracy is Not Enough: A Divergence-Based Approach to Evaluate Fidelity Loss in Quantized LLMs
Sep 7, 2026Shahzeb Qamar, Lorenz Sparrenberg, Christian Bauckhage +5LLM Quantization LLM Evaluation
Fraunhofer IAIS, Germany · PhenoRob Cluster of Excellence, University of Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany +4
All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs
Sep 5, 2026Zhixiong Zhao, Zukang Xu, Guangyu Sun +2LLM Quantization Post-Training Quantization
Houmo AI · School of Integrated Circuits, Peking University
When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference
Sep 3, 2026Ismail Erbas, Xavier Intes, Vikas PandeyRecurrent Neural Networks Recurrent-State Quantization
Department of Biomedical Engineering Center for Modeling, Simulation, and Imaging in Medicine Rensselaer Polytechnic Institute Troy, NY 12180
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Sep 3, 2026Sergii Kozyrev, Davyd MaiborodaLLM Quantization 4-Bit Quantization
The Effect of Quantization on Clinical Benchmarks: Accuracy and Safety Across Model Families
Sep 2, 2026Leonard Twagirayezu, Prasenjit MitraLLM Quantization Post-Training Quantization
Kigali, Rwanda
Hardware-Accelerated Instance Segmentation for Resource-Constrained Space Robotics with Criticality Analysis
Sep 2, 2026Siddhant Shete, Hilmi Dogu Kücüker, Udo Frese +1Neural Network Hardware Acceleration Instance Segmentation
Robotics Innovation Center of the German Research Center for Artificial Intelligence GmbH (DFKI), Bremen, Germany · University of Bremen, Germany
Post-Training Ternarization of Qwen3-4B Capability, Effective Bit Budget, Storage Compression, and Deployment
Sep 2, 2026Anirudh Malik, M Sparsh Mehra, Poojith DevanLLM Quantization Ternary Quantization
OneBit AI
SCULPT: Training Edge Vision Models for Post-Training Quantization Readiness
Sep 1, 2026Bharadwaj Kavuri, Sourav Babu-PK, Varadhraj Ellapan +2Edge Computing Activation Quantization
Valeo Vision Systems, Tuam, Ireland · Valeo India Pvt. Ltd, Chennai, India
The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally
Sep 1, 2026Jundong Hu, Shekar RamachandranMixed-Precision Quantization LLM Quantization
PayPal AI
QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization
Aug 31, 2026Yipin Guo, Arun M George, Jie Fu +3LLM Quantization Ternary Quantization
University of Notre Dame
TopGQ: Fast GNN Post-Training Quantization Leveraging Topology Information
Aug 31, 2026Dain Kwon, Kanghyun Choi, Hyeyoon Lee +4Graph Neural Networks Activation Quantization
Seoul National University Seoul, South Korea

Sep 21, 2026J/K move · Enter open · S save

RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models

Zhiping Wu, Dongdong Ren, Yangchengyu Zhou, Zhengjie Zhang+3

State Key Laboratory for Novel Software Technology and the School of Intelligence Science and Technology, Nanjing University, Suzhou Campus, Suzhou 215163, China

VLM Quantization Large Vision-Language Models Post-Training Quantization Low-Bit Quantization

Open paper AskPDF arXiv

Large vision-language models (VLMs) can be efficiently deployed under stringent memory and latency constraints through post training quantization (PTQ). However, most PTQ methods are designed for unimodal large language models (LLMs). These methods treat quantization errors as isotropic perturbations under the Euclidean assumption, which provides weak guidance on directions most sensitive to quantization in VLMs. Consequently, directly adapting unimodal PTQ approaches or solely employing modality-specific scaling often leads to uneven bit-width distribution and inconsistent performance in low-bit settings. To address these challenges, we propose Riemannian Geometry-Sensitive Quantization (RGSQ), which formulates quantization as a reconstruction problem under a unified Fisher-Riemannian metric. RGSQ identifies modality-specific sensitive directions via Riemannian manifold mappings built from modality-partitioned empirical Fisher factors and fused into a modality-aware Kronecker-structured metric. We then apply geometry-aligned rotations to reorient the local tangent frame, steering low-bit perturbations toward loss-insensitive axes. Finally, we apply a whitening transformation that maps the Riemannian objective to an equivalent Euclidean form, enabling standard unimodal PTQ methods to evaluate multimodal quantization error under their original assumptions. Across an extensive and diverse set of mainstream VLM benchmarks, RGSQ achieves the highest accuracy and stability under extremely low-bit settings (W2A8 and W3A8). It outperforms VLM-aware baselines, such as MBQ and MQuant, by up to 5.9% and surpasses single-modality improvements by up to 8.6%.