Post-Training Quantization

Also known as PTQ

Ask about this topic

Overview

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

CardsList

360 papers70 papers in the last four weeks, up 46% on the four weeks before. 0.4% of all new papers.

  1. Holding the FP8 Quality Ceiling at 8-Bit Weights and Activations: INT8 and GGUF Post-Training Quantization of Ideogram 4.0 for Consumer GPUs

    Jun 10, 2026Deep Gandhi, Ali Asaria, Tony SalomoneMixed-Precision QuantizationDiffusion Transformer

    Transformer Lab

  2. PhaseQuant: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics

    Jun 10, 2026Zimo Zhao, Maolin Wang, Bowen Yu +3Mixed-Precision QuantizationLLM Quantization

    City University of Hong Kong Hong Kong, China · Zhejiang University of Technology Hangzhou, China

  3. Optimal Post-Training Quantization Scales and Where to Find Them

    Jun 9, 2026Juan Amboage, Pablo Monteagudo-Lago, Ian Colbert +2LLM QuantizationWeight-Only Quantization

    AMD

  4. UniSVQ: 2-bit Unified Scalar-Vector Quantization

    Jun 9, 2026Haoyu Wang, Haiyan Zhao, Xingyu Yu +4LLM QuantizationVector Quantization

    Department of Computer Science and Technology, Tsinghua University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · School of Physics, Beihang University, Beijing, China

  5. Quality Is Not a Safety Proxy Under Quantization

    Jun 8, 2026Sahil KadadekarLLM QuantizationLLM Refusal Behavior

    Independent Researcher

  6. Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin

    Jun 8, 2026Hanyang Li, Jianhao Ma, Ying CuiNeural Network OptimizationStraight-Through Estimator

    Department of IEOR, University of California, Berkeley · Department of Statistics and Data Science, University of Pennsylvania

  7. Trainable Smooth-Rotation Transforms with Learned Channel Scales for LLM Quantization

    Jun 7, 2026Patrik Czakó, Gábor Kertész, Sándor SzénásiLLM Quantization4-Bit Quantization

    Doctoral School of Applied Informatics and Applied Mathematics, Obuda University Budapest, Hungary · John von Neumann Faculty of Informatics, Obuda University Budapest, Hungary

  8. Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression

    Jun 5, 2026Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi +1Mixed-Precision QuantizationLLM Quantization

    UiT The Arctic University of Norway · University of Oslo, Norway

  9. SPEAR: A System for Post-Quantization Error-Adaptive Recovery Enabling Efficient Low-Bit LLM Serving

    Jun 4, 2026Hongyuan Liu, Yawei Li, Zhiqiang Que +3LLM QuantizationLLM Serving

    University of Electronic Science and Technology of China · University of Bristol · 3ETH Zurich

  10. FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models

    Jun 4, 2026Haoyu Huang, Linlin Yang, Sheng Xu +5LLM QuantizationDiffusion Model Quantization

    National College for Excellent Engineers, Beihang University, Beijing, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China · Independent researcher +4

  11. Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models

    Jun 4, 2026Hancheol Park, Geonho Lee, Tairen Piao +1LLM QuantizationMixture-of-Experts Quantization

    Nota Inc., South Korea.

  12. Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models

    Jun 3, 2026Rayyan Abdalla, Amir Hussein, Min Wu +1LLM QuantizationLLM Inference Acceleration

  13. STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models

    Jun 3, 2026Xin Yan, Aqiang Wang, Zhenglin Wan +2LLM QuantizationDiffusion Model Quantization

    School of Artificial Intelligence, Beijing Normal University, Beijing, China · Department of Computer Science, National University of Singapore, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore

  14. QuBLAST: A Framework for Quantizing Large Language Models with Block-Level Compression Approach and Activation Scaling Strategy

    Jun 3, 2026Pasindu Wickramasinghe, Achyuta Muthuvelan, Rachmad Vidya Wicaksana Putra +2Mixed-Precision QuantizationLLM Quantization

    eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates

  15. MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models

    Jun 3, 2026Yue Wu, Changyuan Wang, Zixuan Wang +2LLM Quantization4-Bit Quantization

  16. Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

    Jun 2, 2026Devleena Das, Rajeev Patwari, Elliott Delaye +1Mixed-Precision QuantizationLLM Quantization

    Advanced Micro Devices, Inc. (AMD)

  17. Perplexity Can Miss SAE Feature Damage Under Quantization

    Jun 2, 2026Evan DuanLLM QuantizationLLM Interpretability

    University of Michigan

  18. Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference

    Jun 1, 2026Chi-Wei Huang, Chia-Chi TsaiLLM QuantizationRotation-Based Quantization

    National Cheng Kung University

  19. Massive Spikes in LLMs are Bias Vectors: Mechanistic Uncovering and Spike-Free Quantization

    Jun 1, 2026Yung-Chin Chen, Chung Peng Lee, Ze-Wei Liou +1LLM QuantizationActivation Outliers

    Princeton University, NJ, USA · EnCharge AI, CA, USA

  20. GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation

    May 31, 2026Shihao Zhang, Rayan SaabLLM QuantizationLow-Rank Adaptation

    Department of Mathematics, University of California San Diego

  21. Boundary-Protection W8A8 HiFloat8 Quantization for Large-Scale Text-to-Video Diffusion Transformers

    May 31, 2026Yiming ZhaoVideo Diffusion ModelsDiffusion Model Quantization

    University of Chinese Academy of Sciences Beijing, China

  22. Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models

    May 30, 2026Jinyang Du, Shenghao Jin, Ziqian Xu +5Video Diffusion ModelsDiffusion Model Quantization

    SenseTime

  23. ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization

    May 30, 2026Li Lin, Xiaojun WanLLM Quantization4-Bit Quantization

    Wangxuan Institute of Computer Technology, Peking University

  24. ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression

    May 30, 2026Wenya Yu, Chao Zhang, Li Wang +2LLM QuantizationPost-Training Quantization

    School of Mathematics and Statistics, Central South University, Changsha, China · Department of Computer Science, Central South University, Changsha, China · CRAN, Université de Lorraine, CNRS, Nancy, France +1

  25. Quantized Reasoning Models Think They Need to Think Longer, but They Do Not

    May 29, 2026Sanae Lotfi, Polina Kirichenko, Steven Li +1Logit AdjustmentOverthinking in Language Models

    1FAIR at Meta · 2Meta AI

  26. QVGGT: Post-Training Quantized Visual Geometry Grounded Transformer

    May 29, 2026Zhizhen Pan, Hesong Wang, Huan WangMixed-Precision Quantization3D ViTs

    Westlake University · Beijing University of Posts and Telecommunications

  27. HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization

    May 28, 2026Artur Zagitov, Gleb Molodtsov, Aleksandr BeznosikovLLM QuantizationRotation-Based Quantization

    1MIRAI · 2BRAIn Lab · 3Innopolis University

  28. LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMs

    May 28, 2026Jung Hyun Lee, June Yong Yang, Jungwook Choi +1LLM QuantizationWeight-Only Quantization

    Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea · LG AI Research, Seoul, South Korea · Hanyang University, Seoul, South Korea +1

  29. Ω-QVLA: Robust Quantization for Vision-Language-Action Models via Composite Rotation and Per-step Scaling

    May 27, 2026Xinyu Wang, Mingze Li, Sicheng Lyu +6Post-Training QuantizationEfficient VLA Model Inference

    McGill University · Université de Montréal · Mila – Quebec AI Institute +2

  30. TARQ: Tail-Aware Reconstruction Quantization for Rare-Word Robust Automatic Speech Recognition

    May 27, 2026Xinyu Wang, Ziyu Zhao, Ke Bai +4Automatic Speech RecognitionPost-Training Quantization

    McGill University · Boson AI · Arizona State University