Post-Training Quantization

Also known as PTQ

Overview

Post-training quantization (PTQ) converts a trained model to lower-precision numbers without retraining it, reducing memory use and enabling inference on constrained hardware. Methods range from rounding and calibration-based scaling to Hessian-aware reconstruction such as GPTQ, rotations and outlier handling, codebooks, and mixed-precision allocation across layers, experts, or caches. Calibration usually uses a small set of representative inputs, while evaluation spans language-model perplexity and reasoning, vision accuracy, multimodal tasks, and real-device latency and memory. The central tension is that moderate precision often retains task quality, while very low bit widths can amplify errors and alter behavior; calibration and hardware-aware kernels help, but accuracy, behavioral fidelity, and deployment speed do not always move together.

Latest papers

CardsList

395 papers49 papers in the last four weeks, down 37% on the four weeks before. 0.3% of all new papers.

  1. Training-Free Acceleration of Multi-View 3D Transformers via Regime-Aware Attention Approximation

    Nov 13, 2026Muhammed Izzet Sezer, Sevval Belkis Dikkaya3D ViTsEfficient ViTs

  2. SignRot: LLM Quantization with Massive Outlier-Aware Sign-Adjusted Rotation

    Nov 12, 2026Jaehun Gim, Jaewoo Kim, Gyuwan Kim +1LLM QuantizationNeural Network Quantization

  3. SoScale: Second-Order Scale Search for FP4 Quantization

    Nov 12, 2026Pengxiang Zhao, Xing Li, Hui-Ling Zhen +3LLM QuantizationNeural Network Quantization

  4. AutoQS: Automated Quantization Strategy Design with Large Language Models

    Nov 9, 2026Zhen Liu, Zhiqin Yang, Yonggang Zhang +3LLM QuantizationQuantization

  5. Beyond Similarity Entropy: A Rank-Guided Data-Free Quantization Framework for Vision Transformers

    Nov 8, 2026Runqing Jiang, Ye Zhang, Pengpeng Yu +4Neural Network QuantizationPost-Training Quantization

  6. When Calibration Data Matter in Sub-4-Bit Language Model Quantization

    Nov 8, 2026Yehong Jiang, Yen-Kuang Chen, Xinmin TianLLM QuantizationPost-Training Quantization

  7. MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations

    Nov 7, 2026Lorenz K. Muller, Philippe Bich, Chiara Boretti +3Post-Training QuantizationNeural Network Quantization

  8. Rotation Is Token-Blind: A Structural Criterion for Post-Training Quantization of Feed-Forward Geometry Transformers

    Nov 5, 2026Huanfei He, Bohao Zhang, Jiaxin Zhang +4Post-Training QuantizationNeural Network Quantization

  9. TileQuant: Tile as the Precision-Allocation Unit for LLM Mixed-Precision Post-Training Quantization

    Nov 4, 2026Hanzhi Zhang, Qiao Zhang, Qinglei Cao +4LLM QuantizationPost-Training Quantization

  10. PTQ4SAM3: Semantics-Preserving Post-Training Quantization for Segment Anything Model 3

    Nov 4, 2026Xi Xiao, Yunbei Zhang, Lin Zhao +7Post-Training QuantizationNeural Network Quantization

  11. LoopQ: Fair, Closed-Loop Evaluation of Quantized Vision-Language-Action Models

    Nov 3, 2026Syed Umer Bukhari, Magnus Almgren, Ahmed Ali-EldinPost-Training QuantizationEfficient VLA Model Inference

  12. SPA-Q: Structure-Preserving Adaptive Post-Training Quantization for Monocular Depth Estimation

    Nov 3, 2026Jaemin Choi, Jincheol Yang, Nahyun Lim +4Post-Training QuantizationMonocular Depth Estimation

  13. Beyond Accuracy: How Quantization Affects Coding Agents

    Nov 2, 2026Areej Mehboob, Muhammad Ali Shafique, Muhammad Usman Qadeer +1AI Coding AgentsQuantization Robustness

  14. MatMoE: Matryoshka Mixture-of-Experts with Dynamic Mixed-Precision Quantization for Efficient Inference

    Nov 1, 2026Renjie Wei, Haochen Huang, Chuyu Qiu +3Mixture-of-Experts InferenceMixture-of-Experts Quantization

  15. TopoQuant: Topological Framework to Characterize Behavioral Shifts in Quantized Large Language Models

    Oct 31, 2026Baha Rababah, Shahzeb Qamar, Lorenz Sparrenberg +4LLM QuantizationQuantization Robustness

  16. Quantization of Large Models with Weight Outliers and Sparsity

    Oct 30, 2026Leonidas Tsigkounakis, Abraham Jaeger Mountain, Yilang Zhang +1LLM QuantizationNeural Network Quantization

  17. Trainable Vector Quantization for Large Language Models

    Oct 30, 2026Anastasiia Prutianova, Alina Bogdanova, Mikhail Shutov +3LLM QuantizationVector Quantization

  18. A Unified Hessian-Guided Framework for Post-Training Quantization

    Oct 29, 2026Mads Toftrup, Marco Federici, Boris van Breugel +2QuantizationLLM Quantization

  19. Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4

    Oct 28, 2026Billal BoumedineAutomatic Speech RecognitionOn-Device Inference

  20. Quantize by Doing: Task-Guided Mixed-Precision Quantization of Vision-Language-Action Models

    Oct 28, 2026Jiuyi Xu, Xiao Hu, Meida Chen +3Mixed-Precision QuantizationVLM Quantization

  21. Quantized Edge Inference of a Self-Supervised SAR-Optical Encoder on Commodity x86 Hardware

    Oct 27, 2026Christian Sanchez-Leon, Dongming ChenEfficient Transformer InferenceEdge Inference

  22. AxisQ: Depth-Adaptive Channel--Frequency Quantization for Camouflaged Object Detection

    Oct 27, 2026Chi Ruan, Tianqi Li, Xin He +3Post-Training QuantizationAdaptive Quantization

  23. ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning

    Oct 26, 2026Rhea Sanjay Sukthanker, Pashmina CameronLLM QuantizationLow-Bit Quantization

  24. ResCom: Activation Residual Compensation for Large Language Model Quantization

    Oct 25, 2026Ning Yang, Bohan Cui, Zhuowen Liu +1LLM QuantizationQuantization Error Compensation