cs.CVNov 11, 2026

TraceKV: Policy-Preserving Low-Rank KV Cache Compression for Reasoning Vision-Language Models

Authors: Ke Li, Zishan Shao, Yiran Chen

Organizations: Department of Electrical and Computer Engineering, Duke University

Abstract

Reasoning vision-language models (VLMs) impose substantial KV cache demands from both large multimodal contexts and growing reasoning traces. Low-rank KV compression has emerged as a promising paradigm that stores low-dimensional latents of KV states and can be combined with quantization. However, existing low-rank methods often induce growing predictive policy drift from the dense model when compressing reasoning KV states, resulting in reasoning failures. We identify two key causes of this degradation: calibration-state mismatch between static contexts and on-policy reasoning states, and unequal error sensitivity across KV states. To address these challenges, we propose TraceKV, a training-free low-rank KV compression framework that unifies on-policy calibration, sensitivity-aware SVD, and global rank allocation under a single policy-preserving principle, substantially reducing policy-KL divergence during reasoning. Experiments across multiple VLMs and visual reasoning benchmarks demonstrate superior accuracy--memory trade-offs, achieving near-lossless reasoning accuracy with 5.64×\times KV cache compression when combined with quantization. Optimized low-rank attention kernels further deliver up to 1.83×\times end-to-end decoding speedup. Code will be available at https://github.com/AK-DREAM/TraceKV.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVD

    Oct 2, 2026Yiren Zhao, Guanghui Song, Tianrui Qin +3KV-Cache CompressionLow-Rank Compression