cs.CLOct 1, 2025

GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness

Authors: Kung-Hsiang Huang, Haoyi Qiu, Yutong Dai, Caiming Xiong, Chien-Sheng Wu

Organizations: Salesforce AI Research · University of California, Los Angeles

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Graphical user interface (GUI) agents built on vision-language models have emerged as a promising approach to automate human-computer workflows. However, they also face the inefficiency challenge as they process long sequences of high-resolution screenshots and solving long-horizon tasks, making inference slow, costly and memory-bound. While key-value (KV) caching can mitigate this, storing the full cache is prohibitive for image-heavy contexts. Existing cache-compression methods are sub-optimal as they do not account for the spatial and temporal redundancy of GUIs. In this work, we first analyze attention patterns in GUI agent workloads and find that, unlike in natural images, attention sparsity is uniformly high across all transformer layers. This insight motivates a simple uniform budget allocation strategy, which we show empirically outperforms more complex layer-varying schemes. Building on this, we introduce GUI-KV, a plug-and-play KV cache compression method for GUI agents that requires no retraining. GUI-KV combines two novel techniques: (i) spatial saliency guidance, which augments attention scores with the L2 norm of hidden states to better preserve semantically important visual tokens, and (ii) temporal redundancy scoring, which projects previous frames' keys onto the current frame's key subspace to preferentially prune redundant history. Across standard GUI agent benchmarks and models, GUI-KV outperforms competitive KV compression baselines, closely matching full-cache accuracy at modest budgets. Notably, in a 5-screenshot setting on the AgentNetBench benchmark, GUI-KV reduces decoding FLOPs by 38.9% while increasing step accuracy by 4.1% over the full-cache baseline. These results demonstrate that exploiting GUI-specific redundancies enables efficient and reliable agent performance.

Figures & tables

Figure 1: Attention sparsity for UI-TARS-1.5-7B across layers for screenshots from ScreenSpot-V2. All scenarios exhibit extremely high sparsity (mostly > 0.99) across all layers.
Figure 2: Performance comparison of KV cache methods on ScreenSpotV2. SnapKV’s uniform allocation outperforms approaches that allocate varying budgets.
Figure 3: Overview of GUI-KV . Spatial Saliency Guidance utilizes the L2 norms ( rir_{i} ) of Step tt visual representations to identify intrinsically important features. Temporal Redundancy Scoring constructs a subspace ( QthQ_{t}^{h} ) from Step tt keys to prune historical tokens that possess low projection residuals ( ρ\rho ), indicating redundancy. The final score combines these signals to retain only spatially salient and temporally unique KV states.
Figure 4: Illustration of our temporal redundancy scoring mechanism. We first perform QR decomposition on the current screenshot at step tt to obtain a subspace spanned by Qth\bm{Q}^{h}_{t} . Then, we project key vectors ki′sk_{i}^{\prime}s of visual tokens from previous frames onto Qth\bm{Q}^{h}_{t} . The magnitude of the residual ρ\rho (the component orthogonal to the subspace) quantifies the visual token’s non-redundancy. In this example, k3k_{3} and k4k_{4} are less redundant than k1k_{1} and k2k_{2} since they have larger residuals orthogonal to Qth\bm{Q}^{h}_{t} . This serves as a qualitative visualization of our pruning criteria: static background elements ( k1k_{1} , k2k_{2} ) are pruned, while semantically critical UI changes ( k3k_{3} , k4k_{4} ) are selected and preserved.
Figure 5
More in this topic

KV-Cache Compression

Key-value cache compression reduces the memory and bandwidth needed to retain attention states during long-context inference.

See all 265 papers

Explore similar work

CardsList
Jun 1, 2026

STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language Models

Vision-language-model-based graphical user interface (GUI) agents have shown broad automation capabilities, yet deployment is bottlenecked by a key-value (KV) cache that grows linearly with interaction steps. For instance, UI-TARS-1.5-7B consumes 76 GB of GPU memory on merely five screenshots, approaching the capacity of mainstream 80 GB accelerators. Existing KV compression methods share two structural assumptions: aggregating visual-token importance into a single shared saliency map, and applying a fixed top-B cutoff to the fused score distribution. Pilot measurements refute both: spatial specialization lives at the attention-subspace level and migrates across layers, while the score distribution drifts in shape along a trajectory. We propose STaR-KV (Spatio-Temporal Adaptive Re-weighting), a training-free KV cache compression framework that calibrates token importance along three axes: (i) subspace-aware scoring driven by online spatial mutual information; (ii) a temporal stability discount that suppresses redundant cache entries from persistently attended subspaces; and (iii) an entropy-derived temperature that adaptively reshapes the score distribution. Across four GUI benchmarks, STaR-KV achieves the strongest average accuracy among state-of-the-art KV compression methods (e.g., GUIKV, SnapKV) at matched budgets, with no compression-stage FLOPs overhead (-0.07%) and cutting peak GPU memory by nearly 40% at a 20% KV-cache budget. Code is available at https://github.com/kawhiiiileo/STaR-KV.

Yuhang Han, Wenzheng Yang, Yujie Chen +4

1EPIC Lab, SJTU · 2HKUST (GZ) · 3The University of Sydney +3

Vision-Language ModelsKV CachingKV-Cache CompressionGUI Automation
AskPDF
Feb 27, 2026

ST-Lite: Training-Free KV Cache Compression with Spatio-Trajectory Guidance for Long-Horizon GUI Agents

Training-free KV cache compression is essential for deploying vision-language GUI agents under memory and latency constraints, yet existing methods are designed for generic language workloads and ignore the distinctive structure of GUI interaction traces. We characterize three GUI-specific workload properties--high inter-frame visual redundancy, extremely small UI-element spatial footprints, and near-uniform cross-layer attention sparsity--that cause existing schemes to retain as few as 39% of oracle-important KV pairs at the standard 20% budget. To address this, we propose ST-Lite, a training-free compression scheme whose three components each target one property: Trajectory-aware Semantic Gating (TSG) filters redundant historical frames, Component-centric Spatial Saliency (CSS) preserves fine-grained element boundaries, and a flat per-layer budget avoids hierarchical misallocation. Across seven GUI benchmarks and two backbones in the deployment-relevant 10%-40% window, ST-Lite consistently outperforms all existing compression baselines, matching or exceeding Full Cache task accuracy on the primary backbone at the 20% budget while delivering up to 2.35x decoding speedup at fivefold compression. The implementation is available at https://github.com/94wen94/ST-Lite.

Bowen Zhou, Zhou Xu, Wanli Li +3

Tsinghua University, Shenzhen, China · Zhejiang University, Hangzhou, China · The Chinese University of Hong Kong, Hong Kong, China

KV-Cache CompressionGUI AgentsVisual Token Pruning
AskPDF
Feb 26, 2026

Spatio-Temporal Token Pruning for Efficient High-Resolution GUI Agents

Pure-vision GUI agents provide universal interaction capabilities but suffer from severe efficiency bottlenecks due to the massive spatiotemporal redundancy inherent in high-resolution screenshots and historical trajectories. We identify two critical misalignments in existing compression paradigms: the temporal mismatch, where uniform history encoding diverges from the agent's "fading memory" attention pattern, and the spatial topology conflict, where unstructured pruning compromises the grid integrity required for precise coordinate grounding, inducing spatial hallucinations. To address these challenges, we introduce GUIPruner, a training-free framework tailored for high-resolution GUI navigation. It synergizes Temporal-Adaptive Resolution (TAR), which eliminates historical redundancy via decay-based resizing, and Stratified Structure-aware Pruning (SSP), which prioritizes interactive foregrounds and semantic anchors while safeguarding global layout. Extensive evaluations across diverse benchmarks demonstrate that GUIPruner consistently achieves state-of-the-art performance, effectively preventing the collapse observed in large-scale models under high compression. Notably, on Qwen2-VL-2B, our method delivers a 3.4x reduction in FLOPs and a 3.3x speedup in vision encoding latency while retaining over 94% of the original performance, enabling real-time, high-precision navigation with minimal resource consumption.

Zhou Xu, Bowen Zhou, Qi Wang +2

Tsinghua University, Shenzhen, China · Xidian University, Xi’an, China · The Chinese University of Hong Kong, Hong Kong, China

GUI AgentsVisual Token PruningVisual Token CompressionEfficient VLM Inference
AskPDF