Overview

Attention quantization studies how to compute attention and store key-value caches with fewer bits, reducing memory use and speeding up long-context inference and model training. Methods include FP8, FP4, and integer formats; mixed-precision allocation for sensitive heads or token interactions; rotations and calibration; and fused kernels that compute directly on compressed data. Researchers evaluate these approaches using language-model perplexity, long-context retrieval, and vision or video-generation quality, alongside throughput and memory use on specific hardware. A central challenge is that small quantization errors can be amplified by softmax or accumulate during training, so methods must balance quality, speed, hardware constraints, and reliable error detection.

Latest papers

CardsList

46 papers9 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

  1. BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design

    Nov 6, 2026Dayou Du, Woon Yee Ng, Guangming Sheng +3LLM Inference AccelerationKV-Cache Quantization

  2. FOSCAR: Accelerating Decode Attention over 2- and 4-Bit KV Caches through Affine Folding

    Nov 6, 2026Donglin Zhuang, Pragaash Ponnusamy, Xiaoxia Wu +2LLM Inference AccelerationGPU Kernel Optimization

  3. Fused-Softmax for efficient quantized attention

    Nov 6, 2026Tianxing Wu, Juuso Korhonen, Jesus Carabano BravoSoftmax AttentionGPU Kernel Optimization

  4. ZYFA: Speed and Accuracy of FP8 FlashAttention on AMD CDNA4

    Nov 5, 2026Quentin Anthony, Ganesh Bikshandi, Praneeth Medepalli +1Efficient AttentionGPU Acceleration

  5. TriAttention: Unified Sparse Mixed-Precision Attention for Long-Context LLM Inference

    Nov 5, 2026Hanzhi Zhang, Qiao Zhang, Qinglei Cao +4LLM Inference AccelerationAttention Quantization

  6. SPA-Q: Structure-Preserving Adaptive Post-Training Quantization for Monocular Depth Estimation

    Nov 3, 2026Jaemin Choi, Jincheol Yang, Nahyun Lim +4Post-Training QuantizationMonocular Depth Estimation

  7. Jet-QSparse: Accelerating Agentic LLM Inference with 4-Bit Sparse Attention

    Oct 31, 2026Haozhan Tang, Hao Kang, Meng Lu +7LLM Inference AccelerationKV-Cache Quantization

  8. RAQ-MLA: Role-Aware Quantization for Multi-Head Latent Attention Caches

    Oct 24, 2026Tianqi Chen, Xunchao Li, Linchuan Xie +4LLM QuantizationAttention Quantization

  9. LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization

    Sep 29, 2026Yi Pan, Haocheng Xi, Kan Zhu +10LLM QuantizationLLM Inference Acceleration

  10. Delta-Matching: Closing the Final Gap of Native 8-bit Training for LLMs

    Sep 29, 2026Haozhan Tang, Hao Kang, Han Cai +2Low-Precision TrainingLLM Training

  11. QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching

    Sep 29, 2026Zunhai Su, Yuxuan Sun, Jianchao Tan +5LLM Inference AccelerationAttention Quantization

  12. From Attention Sensitivity to Layer Role: Revisiting Mixed-Precision Quantization of Transformers

    Sep 28, 2026Nafiseh HosseinpourFardi, Negar Alihadi, Mahmoudreza Babaei +2Mixed-Precision QuantizationLLM Quantization

  13. Pretraining Transformers with Quantized Softmax in Attention

    Sep 27, 2026Shangzhen Zhu, Muyan Hu, Tomasz KozlowskiLanguage Model PretrainingQuantization-Aware Training

  14. DraftAttention2: Fast Video Diffusion with Low-Resolution-Guided Mixed-Precision Attention

    Sep 26, 2026Rui Ding, Haopeng Li, Weize Ma +10Video Diffusion ModelsDiffusion Model Inference Acceleration

  15. Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization

    Sep 20, 2026Kuanlin Chen, Chen-Wei Kuo, Cheng-En OuAttention Head Analysis4-Bit Quantization

  16. Attention Quantization for Tabular Foundation Models

    Sep 11, 2026Jonas M. Kübler, Benjamin Jäger, Klemens Flöge +2Efficient Transformer InferenceTabular Foundation Models

  17. EFQ-Softmax: Exp-Free Quantization for Softmax

    Sep 9, 2026Haohui Han, Yuming Wan, Hongni Wang +4Efficient Transformer InferenceTransformer Attention

  18. HyQuant: Hybrid-Precision Quantization for LLM Attention

    Aug 28, 2026Jiatong Ding, Bingxin Xing, Yu Zhang +9Mixed-Precision QuantizationLLM Quantization

  19. KV Cache Compression Through the Lens of Transform Coding

    Aug 14, 2026Hannah Laus, Claudio Mayrink Verdun, Hao Wang +2KV-Cache CompressionKV-Cache Quantization

  20. RotaryQuant: Fitting 120B MoE Models on Consumer Hardware via Fused Compressed-Space Attention

    Aug 8, 2026Anthony. Lui, Mohamed. Elsaied, N. P. SavaniExpert OffloadingMixture-of-Experts Language Models

  21. Output-Aware Rotation for INT2 KV-Cache Quantization

    Aug 3, 2026Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4LLM InferenceKV Caching

  22. Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantization

    Jul 19, 2026Libo Sun, Po-Wei Harn, Zewei Zhang +2KV CachingAttention Quantization

  23. AVQ-Attention: Adaptive Vector-Quantized Attention

    Jul 14, 2026Winfried van den dool, Patrick Forré, Amir Habibian +2Transformer AttentionVector Quantization

  24. Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention

    Jul 5, 2026Siyu Ding, Mingchuan Ma, Jiabo Tong +3Language Model PretrainingMixed-Precision Quantization