Attention Quantization

Ask about this topic

Overview

Attention quantization studies how to compute attention and store key-value caches with fewer bits, reducing memory use and speeding up long-context inference and model training. Methods include FP8, FP4, and integer formats; mixed-precision allocation for sensitive heads or token interactions; rotations and calibration; and fused kernels that compute directly on compressed data. Researchers evaluate these approaches using language-model perplexity, long-context retrieval, and vision or video-generation quality, alongside throughput and memory use on specific hardware. A central challenge is that small quantization errors can be amplified by softmax or accumulate during training, so methods must balance quality, speed, hardware constraints, and reliable error detection.

Latest papers

CardsList

38 papers5 papers in the last four weeks, down 17% on the four weeks before. 0.0% of all new papers.

  1. QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention

    Apr 28, 2026Sehyeon Oh, Yongin Kwon, Jemin LeeSoftmax AttentionGPU Acceleration

    University of Science and Technology, Daejeon, Republic of Korea · Electronics and Telecommunications Research Institute, Daejeon, Republic of Korea · Pusan National University, Busan, Republic of Korea +1

  2. Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Silicon

    Apr 18, 2026Sai VegasenaAI Accelerator InferenceSelf-Attention

    Ensue

  3. Diagonal-Tiled Mixed-Precision Attention for Efficient Low-Bit MXFP Inference

    Apr 5, 2026Yifu Ding, Xinhao Zhang, Jinyang GuoLLM Inference AccelerationGPU Kernel Optimization

    State Key Laboratory of Complex & Critical Software Environment (SKLCCSE), Beihang University · School of Computer Science and Engineering, Beihang University · Beijing Jiaotong University +1

  4. AXELRAM: Quantize Once, Never Dequantize

    Apr 3, 2026Yasushi NishidaAttention QuantizationKV-Cache Quantization

    Axelidea Inc.

  5. BinaryAttention: One-Bit QK-Attention for Vision and Diffusion Transformers

    Mar 10, 2026Chaodong Xiao, Zhengqiang Zhang, Lei ZhangAttention QuantizationAttention Mechanisms

    The Hong Kong Polytechnic University · OPPO Research Institute

  6. Diagnosing FP4 inference: a layer-wise and block-wise sensitivity analysis of NVFP4 and MXFP4

    Mar 5, 2026Musa Cim, Burak Topcu, Mahmut Taylan KandemirQuantization Sensitivity AnalysisLLM Quantization

    Department of Computer Science and Engineering The Pennsylvania State University University Park, PA, USA

  7. SageBwd: A Trainable Low-bit Attention

    Mar 2, 2026Jintao Zhang, Marco Chen, Haoxu Wang +5Attention QuantizationLow-Precision Training

    Tsinghua University, UC Berkeley

  8. Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training

    Feb 21, 2026Seyed Morteza EmadiTransformer AttentionAttention Quantization

    UNC-Chapel Hill, Kenan-Flagler Business School, Chapel Hill, NC, USA