SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning

Sanjay Kariyappa; G. Edward Suh

cs.AIFeb 26, 2026

SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning

Authors: Sanjay Kariyappa, G. Edward Suh

Organizations: NVIDIA

Abstract

Long-running agentic tasks, such as deep research, require multi-hop reasoning over information distributed across multiple webpages and documents. In such tasks, the LLM context is dominated by tokens from external retrieval, causing memory usage to grow rapidly and limiting decode performance. While several KV cache compression techniques exist for long-context inputs, we find that existing heuristics fail to support multi-step reasoning models effectively. We address this challenge with SideQuest -- a novel approach that leverages the Large Reasoning Model (LRM) itself to perform KV cache compression by reasoning about the usefulness of tokens in its context. To prevent the tokens associated with this management process from polluting the model's memory, we frame KV cache compression as an auxiliary task executed in parallel to the main reasoning task. Our evaluations, using a model trained with just 215 samples, show that SideQuest reduces peak token usage by up to 65% on agentic tasks with minimal degradation in accuracy, outperforming heuristic-based KV cache compression techniques.

Figures & tables

Figure 1: Walkthrough example of SideQuest. (a) The main thread processes the user request by performing multi-turn reasoning and tool calling. (b) At regular intervals we spawn an auxiliary thread that runs in parallel with the shared context (c) The auxiliary thread reflects on the context and lists the cursors that can be deleted. (d) We clear the messages in the context by invoking a tool, reducing the context size for future turns.

Figure 2: Distribution of ReAct Iterations and token count for FRAMES and BrowseComp with gpt-oss-20b (medium effort).

Figure 3: Efficiency vs. Utility Trade-off. We evaluate Accuracy against Peak Token Usage and KV cache memory reads for gpt-oss-20b with Medium and High reasoning effort, on the FRAMES and BrowseComp benchmarks. The Uncompressed Baseline establishes the upper bound for accuracy but incurs the highest memory cost. SideQuest achieves substantial memory savings—reducing peak token usage by 56-65% compared to the baseline—while providing a better accuracy compared to heuristic based methods.

Figure 4: Non-Completion Rate across benchmarks, categorized by failure type: Unparsable Responses (orange), Context Limits (green), and Turn Limits (purple). SideQuest demonstrates superior reliability, matching the near-zero failure rate of the uncompressed baseline, while other methods suffer from high rates of model collapse.

Figure 5: Serving Performance in SGLang. We compare Sidequest against the uncompressed baseline for gpt-oss-20b (Medium Effort) on the FRAMES benchmark using a single NVIDIA H100 GPU. (Left) Sidequest increases peak throughput by

83.9\%

by enabling larger batch sizes. (Center) Peak KV cache usage is reduced by

53.9\%

, freeing up significant memory headroom. (Right) The combination of higher concurrency and reduced memory movement lowers total benchmark runtime by

36.8\%

Figure 6: The SideQuest framework can be extended to various auxiliary tasks beyond the memory management task.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Examples of memory-management reasoning produced by SideQuest.

KV-Cache Compression

Key-value cache compression reduces the memory and bandwidth needed to retain attention states during long-context inference.

See all 257 papers

StepKV: Step-Aware KV Cache Compression for LLM Agents
Aug 26, 2026Boyu Feng, Jiahong Liu, Yifan Li +7KV Caching KV-Cache Compression
The Chinese University of Hong Kong · Huawei Technologies Co., Ltd
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
Sep 4, 2026Janghyeon Kim, Minsoo Kim, Kyuhong Shim +1Long-Context Reasoning in Language Models KV-Cache Compression
Hanyang University · Sungkyunkwan University
ActKV: Efficient LLM Agents through Action-Guided KV Cache Management
Sep 25, 2026Zihan Wang, Cheng Tang, Lei Gong +4KV-Cache Management KV-Cache Compression
University of Science and Technology of China Hefei, China · Suzhou Institute for Advanced Research, University of Science and Technology of China Suzhou, China

Aug 26, 2026J/K move · Enter open · S save

StepKV: Step-Aware KV Cache Compression for LLM Agents

Boyu Feng, Jiahong Liu, Yifan Li, Wenhao Yu+6

The Chinese University of Hong Kong · Huawei Technologies Co., Ltd

KV Caching KV-Cache Compression Long-Horizon LLM Agents LLM Agent Efficiency KV-Cache Pruning

Open paper AskPDF arXiv

Key-value (KV) caching is essential for efficient autoregressive large language model (LLM) inference, but the cache grows linearly with context length, increasing storage and decoding costs. KV cache compression mitigates this cost by retaining only a subset of cached tokens. This challenge is particularly important for multi-step LLM agents, where a query expands into trajectories of reasoning, tool interactions, and retrieved observations. Existing pruning methods typically treat the cache as a flat token stream and rank tokens by recency or attention saliency. This creates a mismatch between the unit of compression and the unit of reasoning: token-level pruning removes individual entries, whereas useful information in multi-step agents is often organized into reasoning steps with uneven and delayed importance. Consequently, an early observation or intermediate decision may receive little recent attention yet remain essential for later evidence synthesis. We term this failure mode Reasoning Continuity Disruption.These observations motivate KV cache compression that jointly considers token- and reasoning-step-level information. StepKV addresses this goal by treating reasoning steps as first-class retention units. It associates cache entries with their generating steps, estimates step utility from trajectory-derived signals, and combines this utility with token-level saliency. The resulting scores globally rank prunable tokens, from which StepKV retains the top-scoring entries under a target budget. StepKV thus provides a step-centric perspective for agent KV cache compression. Across multi-hop QA and long-horizon web reasoning tasks, StepKV sustains accuracy under low KV budgets where token-level baselines degrade sharply, offering a more robust efficiency-accuracy trade-off for multi-step agent inference.

Abstract

KV-Cache Compression

Explore similar work

StepKV: Step-Aware KV Cache Compression for LLM Agents

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

ActKV: Efficient LLM Agents through Action-Guided KV Cache Management