Overview

Low-bit quantization studies how to represent model weights, activations, attention caches, and training states with fewer bits, reducing memory use and enabling faster or more energy-efficient inference. Post-training methods use calibration data, error reconstruction, Hessian information, rotations, and outlier handling, while quantization-aware training adapts models to the errors they will encounter at deployment. Mixed precision and scalar or vector codebooks allocate limited precision to sensitive components, with specialized kernels and hardware formats needed to turn compression into real speed and energy gains. The central tension is that aggressive compression can damage generation, reasoning, or long-context behavior even when standard accuracy metrics look stable, and improvements in quantization quality do not always translate into faster end-to-end systems.

Latest papers

CardsList

254 papers23 papers in the last four weeks, down 49% on the four weeks before. 0.2% of all new papers.

  1. HADVQ: Hessian Aware Directional Vector Quantization

    Nov 17, 2026Daniil Li, Mars Davletshin, Vadim Klyzhenko +4QuantizationLLM Quantization

  2. MiSTiQ: MicroScaling Trellis Quantization for Extreme KV Cache Compression

    Nov 16, 2026Ofir Gordon, Lior Dikstein, Paul Balanca +3KV-Cache QuantizationLow-Bit Quantization

  3. Planners Exploit Deterministic Error: Dithered Low-Bit World Models

    Nov 9, 2026Yong Hwi Song, Keon Myung LeeWorld Model-Based PlanningLow-Bit Quantization

  4. When Calibration Data Matter in Sub-4-Bit Language Model Quantization

    Nov 8, 2026Yehong Jiang, Yen-Kuang Chen, Xinmin TianLLM QuantizationPost-Training Quantization

  5. Sensitivity-Gated Accumulator Caching for On-Device Learning of Low-Bit Hand Networks

    Nov 7, 2026Soroor Ghandali, Kanika Mamgain, Manjot Kaur +2On-Device TrainingLow-Bit Quantization

  6. CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Low-Precision LRMs

    Nov 6, 2026Xingyu Liu, Xiangyang Yin, Tianhua Xia +2LLM QuantizationNeural Network Quantization

  7. ResQAdam: Residual-Preserving Adaptive Optimization for Memory-Efficient Quantization Aware Training

    Nov 6, 2026Liqun Ma, Shengkun Tang, Liangyu Wang +7Optimizer State CompressionLLM Quantization

  8. StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference

    Nov 5, 2026Jake Hyun, Ziyu Zhao, Jae-sun Seo +1LLM InferenceLLM Inference Efficiency

  9. UniCompress: From Multi-LoRA Experts to One Compact Model for Edge Intelligence

    Nov 4, 2026Mingkang Liu, Peipei Jiao, Weiming Meng +3VLM CompressionVLM Quantization

  10. FlagRot: Selective Rotation for Low-Bit KV Caches

    Nov 2, 2026Kunming Shao, Xijie Huang, Jierun Chen +7KV-Cache QuantizationRotation-Based Quantization

  11. SHq: Learnable Grids for LLM Weight Quantization

    Nov 2, 2026Michael Helcig, Earl Killian, Dan AlistarhLLM QuantizationNeural Network Quantization

  12. A Unified Hessian-Guided Framework for Post-Training Quantization

    Oct 29, 2026Mads Toftrup, Marco Federici, Boris van Breugel +2QuantizationLLM Quantization

  13. CoS-VQ: Coordinated Shared-Scale Vector Quantization for Large Language Models

    Oct 27, 2026Zukang Xu, Zhixiong Zhao, Xing Hu +3LLM QuantizationVector Quantization

  14. ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning

    Oct 26, 2026Rhea Sanjay Sukthanker, Pashmina CameronLLM QuantizationLow-Bit Quantization

  15. ResCom: Activation Residual Compensation for Large Language Model Quantization

    Oct 25, 2026Ning Yang, Bohan Cui, Zhuowen Liu +1LLM QuantizationQuantization Error Compensation

  16. BiLoCo: Binary Low-Rank Corrections for Post-Training Quantization

    Oct 23, 2026David Jin, Beshr IslamBouli, Tarushii Goel +2LLM QuantizationPost-Training Quantization

  17. MPQ: A Message-Passing View of Post-Training Quantization

    Oct 23, 2026Yanshuo Chen, Chenxi Liu, Reza Shirkavand +1QuantizationLLM Quantization

  18. Extreme Low-State Quantized Transformer for Electricity-Load Forecasting

    Oct 20, 2026Muhammad Sabbir Alam, Harindra Sandun Mavikumbure, Victor Cobilean +2Low-Bit QuantizationQuantization-Aware Training

  19. Beyond 4 bits: Data-Centric Post-training Quantization and Recovery

    Oct 20, 2026Rhea Sanjay Sukthanker, Riccardo Grazzi, James Hensman +1LLM QuantizationPost-Training Quantization

  20. E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks

    Oct 20, 2026Xinyu Lei, Hong Huang, Jiebin Zhang +6LLM QuantizationQuantization-Aware Training

  21. RoLaQ: Joint Rotation and Range Calibration for 2-Bit KV Caches in Multi-Head Latent Attention

    Oct 14, 2026Shiyu Ji, Peng Ling, Chen Lin +4KV-Cache QuantizationLow-Bit Quantization