Key-Value Cache Quantization

Follow this topic Ask about this topic

Recent momentum

-25%

3 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Key-Value Cache Quantization.

Period ending 2026-09-07

2 new papers

A weekly snapshot of new work published in Key-Value Cache Quantization.

39 papers

Latest in Key-Value Cache Quantization

Open your feed →

OmniKVQuant: KV Cache Quantization for Omni-LLMs
Sep 11, 2026Suho Yoo, Hyunjong Ok, Jongmin Choi +2Key-Value Cache Quantization Large Language Model Quantization
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Sep 3, 2026Sergii Kozyrev, Davyd MaiborodaLarge Language Model Quantization Fp8
Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generation
Aug 31, 2026Atta Ul Asad, Ahsan Bilal, Muhammad Ali +2Key-Value Cache Quantization Quantization
Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms
Aug 4, 2026Samuel Fernández-Menduiña, Amir Ziashahabi, Eduardo Pavez +2Key-Value Cache Quantization Calibration-Free Low-Bit Vector Quantization
Output-Aware Rotation for INT2 KV-Cache Quantization
Aug 3, 2026Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4Key-Value Cache Quantization Kv-Cache Steering
WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
Jul 30, 2026Fanzhe Wei, Li Liu, Ziyang Wang +1Key-Value Cache Quantization Finite-Sample Certificates
MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel
Jul 21, 2026Lenore Mulin, Gaetan HainsTransformer Attention Key-Value Cache Quantization
Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantization
Jul 19, 2026Libo Sun, Po-Wei Harn, Zewei Zhang +2Key-Value Cache Quantization Kv-Cache Steering
TriRoute: Unified Learned Routing for Joint Adaptive Attention, Experts, and KV-Cache Allocation
Jul 7, 2026Andrii Balashov, Olena PonomarovaMixture-Of-Expert Routing Key-Value Cache Quantization
Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inference
Jul 2, 2026Wenchen Han, Gingfung Matthew Yeung, Marco Barletta +3Key-Value Cache Quantization Key-Value Cache
GSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache
Jul 1, 2026Soosung Kim, Minjae Park, Eui-Young Chung +1Key-Value Cache Quantization Residual Quantization
RoPE-Aware Bit Allocation for KV-Cache Quantization
Jun 23, 2026Fengfeng Liang, Yuechen Zhang, Jiaya JiaKey-Value Cache Quantization Dynamic Sparse Attention
HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models
Jun 22, 2026Yuval Domb, Hadar Sackstein, Tomer SolbergPost-Training Quantization Quantization
QK-Normed MLA: QK normalization without full key caching
Jun 15, 2026Yizhou Han, Yao Zhao, Jun Zhou +2Multi-Head Attention Key-Value Cache Quantization
SpectrumKV: Per-Token Mixed-Precision KV Cache Transfer for Prefill-Decode Disaggregated LLM Serving
Jun 7, 2026Yang PengjuKey-Value Cache Quantization Cache
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks
Jun 2, 2026Lorenz K. Muller, Philippe Bich, Chiara Boretti +3Key-Value Cache Quantization Autoregressive Decoding
Do Transformers Need Three Projections? Systematic Study of QKV Variants
Jun 1, 2026Ali Kayyam, Anusha Madan Gopal, M Anthony LewisKey-Value Cache Quantization Projection
Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
Jun 1, 2026Bruce Changlong Xu, Adarsh Kumarappan, Mu ZhouKey-Value Cache Quantization Large Language Model Quantization
Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression
May 26, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +3Key-Value Cache Quantization Quantization
Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion
May 25, 2026Tuna Tuncer, Felix Becker, Thomas PfeilKey-Value Cache Quantization Video Diffusion Models
OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
May 20, 2026Mark Boss, Vikram Voleti, Simon Donné +1Key-Value Cache Quantization Key-Value Cache Compression
Runtime-Certified Bounded-Error Quantized Attention
May 20, 2026Dean CalverKey-Value Cache Quantization Systematic Fp16 Divergence
OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
May 19, 2026Zunhai Su, Rui Yang, Chao Zhang +11Key-Value Cache Quantization Key-Value Cache Compression
OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
May 18, 2026Zhongzhu Zhou, Donglin Zhuang, Jisen Li +4Key-Value Cache Quantization Quantization
TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Tasks
May 16, 2026Hanzhang Shen, Haoran Wu, Yiren Zhao +1Key-Value Cache Quantization LLM Inference Efficiency
FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression
May 12, 2026Namyoon Lee, Yongjune KimKey-Value Cache Quantization Calibration-Free Low-Bit Vector Quantization
When Quantization Is Free: An int4 KV Cache That Outruns fp16 on Apple Silicon
May 7, 2026Mohamed Amine BergachKey-Value Cache Quantization Systematic Fp16 Divergence
QKVShare: Quantized KV-Cache Handoff for Multi-Agent On-Device LLMs
May 5, 2026Pratik Honavar, Tejpratap GVSLKey-Value Cache Quantization LLM Inference Efficiency
HeadQ: Model-Visible Distortion and Score-Space Correction for KV-Cache Quantization
May 5, 2026Jorge L. Ruiz WilliamsKey-Value Cache Quantization Cache
WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization
May 4, 2026Wei Tao, Xiaoyang Qu, Peiqiang Wang +4Key-Value Cache Quantization Mixed-Precision Quantization
Statistical Inference and Quality Measures of KV Cache Quantisations Inspired by TurboQuant
Apr 27, 2026Paolo D'AlbertoKey-Value Cache Quantization Rate-Distortion Theory
RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
Apr 22, 2026Fei Zuo, Zikang Zhou, Hao Cong +2Key-Value Cache Quantization Quantization
Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experiments
Apr 21, 2026Jianyang Gao, Yutong Gou, Yuexuan Xu +5Key-Value Cache Quantization Quantile Regression
SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
Apr 21, 2026Jinda Jia, Jisen Li, Zhongzhu Zhou +8Key-Value Cache Quantization Key-Value Cache Compression
Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Silicon
Apr 18, 2026Sai VegasenaLLM Inference Efficiency Systematic Fp16 Divergence
The Illusion of Equivalence: Systematic FP16 Divergence in KV-Cached Autoregressive Inference
Apr 16, 2026Ranjith Chodavarapu, Lei XuSystematic Fp16 Divergence Key-Value Cache Quantization
Attention-Weighted Value Projection for KV-Cache Compression
Apr 13, 2026Samuel SalfatiKey-Value Cache Quantization Low-Rank Factorization
NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache
May 23, 2025Donghyun Son, Euntae Choi, Sungjoo YooKey-Value Cache Quantization Calibration-Free Low-Bit Vector Quantization
UltraQuant: 4-bit KV Caching for Context-Heavy Agents
Date pendingInesh Chakrabarti, David Limpus, Aditi Ghai Rana +4Key-Value Cache Quantization Key-Value Cache Compression

Sep 11, 2026 · cs.CVJ/K move · Enter open · S save

OmniKVQuant: KV Cache Quantization for Omni-LLMs

Suho Yoo, Hyunjong Ok, Jongmin Choi, Jihoo Jung+1

As Omni-modal large language models (Omni-LLMs) take in audio, video and text together, their KV cache memory cost grows. KV cache quantization is the de facto approach in text-only LLMs, but its application to Omni-LLMs remains unexplored. In this paper, we analyze how TurboQuant, a representative rotation-based KV cache quantization method, behaves on multimodal caches and identify two critical issues: temporal key drift and heterogeneous value geometry. To address these, we propose OmniKVQuant, a training-free framework that (i) sets the key quantization range over each short window of the input stream; and (ii) rotates values separately per modality. On Qwen2.5-Omni and Qwen3-Omni, OmniKVQuant enables 2-bit KV caches while substantially preserving performance across seven audio-visual benchmarks. We further provide a fused Triton decode kernel that unpacks the 2-bit cache during attention, so no dense FP16 cache is ever built. Code: https://github.com/kaistmm/OmniKVQuant

Key-Value Cache Quantization Large Language Model Quantization Omni-Modal Large Language Models Quantization Cache Large Multimodal Models

Open paper Ask about it PDF