Overview

Activation quantization studies how to represent the intermediate values passed between neural-network layers with fewer bits, reducing memory traffic and enabling faster, smaller deployments. Common approaches include calibration-based clipping and scale selection, smoothing or rotating activations to reduce outliers, mixed-precision allocation, and quantization-aware training; methods target formats such as INT8, INT4, and FP4. Work spans language and vision models, diffusion systems, speech models, and interactive policies, with evaluations using measures such as perplexity, ImageNet accuracy, generation quality, and task success. A central tension is preserving quality across changing activation distributions while delivering real speed and memory gains on the hardware and kernels used in deployment.

Latest papers

CardsList

128 papers10 papers in the last four weeks, down 64% on the four weeks before. 0.1% of all new papers.

  1. Practical FP4 Training for Large-Scale MoE Models on Hopper GPUs

    Mar 3, 2026Wuyue Zhang, Chongdong Huang, Chunbo You +3Low-Precision TrainingLLM Training

  2. FreeAct: Freeing Activations for LLM Quantization

    Mar 2, 2026Xiaohao Liu, Xiaobo Xia, Manyi Zhang +6LLM QuantizationActivation Quantization

  3. S2D: Selective Spectral Decay for Quantization-Friendly Conditioning of Neural Activations

    Feb 16, 2026Arnav Chavan, Nahush Lele, Udbhav Bamba +3Activation OutliersActivation Quantization

  4. Activation Quantization of Vision Encoders Needs Prefixing Registers

    Oct 6, 2025Seunghyeon Kim, Taesun Yeom, Jinho Kim +3Activation OutliersActivation Quantization

  5. GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers

    Jun 13, 2025Guang Liang, Xinyao Liu, Jianxin WuQuantization-Aware Training4-Bit Quantization

  6. FPTQuant: Function-Preserving Transforms for LLM Quantization

    Jun 5, 2025Boris van Breugel, Yelysei Bondarenko, Paul Whatmough +1LLM QuantizationLLM Inference Acceleration

  7. On Orthogonal Recurrent Neural Networks Quantization

    Feb 5, 2024Armand Foucault, Franck Mamalet, François MalgouyresRecurrent Neural NetworksNeural Network Quantization