Graphical user interface (GUI) agents built on vision-language models have emerged as a promising approach to automate human-computer workflows. However, they also face the inefficiency challenge as they process long sequences of high-resolution screenshots and solving long-horizon tasks, making inference slow, costly and memory-bound. While key-value (KV) caching can mitigate this, storing the full cache is prohibitive for image-heavy contexts. Existing cache-compression methods are sub-optimal as they do not account for the spatial and temporal redundancy of GUIs. In this work, we first analyze attention patterns in GUI agent workloads and find that, unlike in natural images, attention sparsity is uniformly high across all transformer layers. This insight motivates a simple uniform budget allocation strategy, which we show empirically outperforms more complex layer-varying schemes. Building on this, we introduce GUI-KV, a plug-and-play KV cache compression method for GUI agents that requires no retraining. GUI-KV combines two novel techniques: (i) spatial saliency guidance, which augments attention scores with the L2 norm of hidden states to better preserve semantically important visual tokens, and (ii) temporal redundancy scoring, which projects previous frames' keys onto the current frame's key subspace to preferentially prune redundant history. Across standard GUI agent benchmarks and models, GUI-KV outperforms competitive KV compression baselines, closely matching full-cache accuracy at modest budgets. Notably, in a 5-screenshot setting on the AgentNetBench benchmark, GUI-KV reduces decoding FLOPs by 38.9% while increasing step accuracy by 4.1% over the full-cache baseline. These results demonstrate that exploiting GUI-specific redundancies enables efficient and reliable agent performance.
Figures & tables
Figure 1: Attention sparsity for UI-TARS-1.5-7B across layers for screenshots from ScreenSpot-V2. All scenarios exhibit extremely high sparsity (mostly > 0.99) across all layers.
Figure 2: Performance comparison of KV cache methods on ScreenSpotV2. SnapKV’s uniform allocation outperforms approaches that allocate varying budgets.
Figure 3: Overview of GUI-KV . Spatial Saliency Guidance utilizes the L2 norms ( ri ) of Step t visual representations to identify intrinsically important features. Temporal Redundancy Scoring constructs a subspace ( Qth ) from Step t keys to prune historical tokens that possess low projection residuals ( ρ ), indicating redundancy. The final score combines these signals to retain only spatially salient and temporally unique KV states.
Figure 4: Illustration of our temporal redundancy scoring mechanism. We first perform QR decomposition on the current screenshot at step t to obtain a subspace spanned by Qth . Then, we project key vectors ki′s of visual tokens from previous frames onto Qth . The magnitude of the residual ρ (the component orthogonal to the subspace) quantifies the visual token’s non-redundancy. In this example, k3 and k4 are less redundant than k1 and k2 since they have larger residuals orthogonal to Qth . This serves as a qualitative visualization of our pruning criteria: static background elements ( k1 , k2 ) are pruned, while semantically critical UI changes ( k3 , k4 ) are selected and preserved.