Overview

Context compression studies how to shrink long prompts, interaction histories, retrieved documents, or model memory without losing information needed for later tasks. Approaches include selecting or pruning text, rewriting it as summaries or structured records, encoding it into learned latent tokens, and adaptively retaining or retrieving history as an agent works. Methods are tested on long-context question answering, retrieval-augmented generation, coding and web agents, and long-horizon tool use, where accuracy, task success, latency, memory, and cost can all matter. The central tension is that aggressive compression can discard critical details or constraints, and fewer tokens do not always mean lower end-to-end cost or more reliable execution.

Latest papers

91 papers1 paper in the last four weeks, down 96% on the four weeks before. 0.0% of all new papers.

May 28, 2025

InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing

Although existing model editing methods perform well in recalling exact edit facts, they often struggle in complex scenarios that require deeper semantic understanding rather than mere knowledge regurgitation. Leveraging the strong contextual reasoning abilities of large language models (LLMs), in-context learning (ICL) becomes a promising editing method by comprehending edit information through context encoding. However, this method is constrained by the limited context window of LLMs, leading to degraded performance and efficiency as the number of edits increases. To overcome this limitation, we propose InComeS, a flexible framework that enhances LLMs' ability to process editing contexts through explicit compression and selection mechanisms. Specifically, InComeS compresses each editing context into the key-value (KV) cache of a special gist token, enabling efficient handling of multiple edits without being restricted by the model's context window. Furthermore, specialized cross-attention modules are added to dynamically select the most relevant information from the gist pools, enabling adaptive and effective utilization of edit information. We conduct experiments on diverse model editing benchmarks with various editing formats, and the results demonstrate the effectiveness and efficiency of our method.