cs.LGNov 7, 2026

CALFA-KV: Joint Calibration and Fisher-Guided Anchors for 1-Bit KV Cache Compression

Authors: Xiancai Tian, Leichao Zhang, Shenhui Zhang, Yi Shan, Hailong Li, Feng Lin, Xiaotian Yu, Aijun Li

Organizations: Shenzhen Intellifusion Technologies Co. Ltd., Shenzhen, China

Abstract

1-bit key-value (KV) cache compression can combine quantizer calibration with selective high-precision storage to reduce reconstruction error. We propose Calibration and Fisher-guided Anchors for KV cache compression (CALFA-KV). Offline, we jointly fit a key center, radial normalization parameters, and K/V vector codebooks to match the frozen model's projected attention output. We also estimate channel-wise Fisher-type sensitivities from squared gradients of a reference next-token cross-entropy loss. Online, we use these sensitivities to weight the squared reconstruction errors of each KV state. Here, a KV state is one token's key-value pair within one layer and KV head. We retain the highest-scoring states at high precision under a per-layer, per-KV-head budget. Under exact restoration, we bound the excess risk of score-based anchor selection relative to the optimum at the same anchor budget, where risk is the conditional mean square of the first-order loss change. With a budget of 56 additional FP16 anchors per layer and KV head, CALFA-KV (d8b8) outperforms all evaluated nominal 1-bit baselines on the mean score across seven benchmarks for each of the three models. At the same anchor budget, CALFA-KV (d8b10) uses a nominal index rate of 1.25 bits per channel and narrows the gap to BF16 to 0.90--2.01 points on the same metric.

Figures & tables

Explore similar work

CardsList
  1. KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

    Sep 21, 2026Sihyeon Ha, Jaeho Lee, Yo-Seb JeonLow-Rank CompressionKV-Cache Compression

  2. AnchorKV: Anchor-Residual KV Cache Compression

    Aug 3, 2026Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2KV CachingLong-Context Language Model Inference

  3. Tailoring the Quantization Space for 1-Bit KV Cache Compression

    Oct 2, 2026Minsoo Cheong, Donghyun Son, Sungjoo YooBinary QuantizationKV-Cache Compression