LLM Context Compression

LLM: Large Language Model

Follow this topic Ask about this topic

Latest papers

37 papers4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction
Mar 20, 2026Runsong Zhao, Shilei Liu, Jiwei Tang +8Language Model Performance Prediction LLM Context Compression
Northeastern University, China · Future Living Lab of Alibaba · Tsinghua University
BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection
Mar 20, 2026Zhengpei Hu, Kai Li, Dapeng Fu +4LLM Context Compression Prompt Compression
School of Computer Technology and Application, Qinghai University · Tsinghua University · Ant Group Security and Intelligence Laboratory (SIL)
EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering
Mar 10, 2026Thao Do, Dinh Phu Tran, An Vo +2Efficient Language Model Inference Knowledge-Intensive QA
Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, South Korea
Fast KV Compaction via Attention Matching
Feb 18, 2026Adam Zweiger, Xinghong Fu, Han Guo +1KV-Cache Management LLM Context Compression
Massachusetts Institute of Technology.
Controllably Efficient Language Models
Nov 7, 2025Jatin Prakash, Aahlad Puli, Rajesh RanganathEfficient Transformer Inference Long-Context Language Modeling
New York University
BudgetMem: Training-Free Selective Memory for Cost-Efficient Long-Context Processing in Language Models
Nov 7, 2025Chandra Vamsi Krishna Alla, Harish Naidu Gaddam, Manohar Kommi +1Memory-Efficient Optimization LLM Compression
University of Texas at Arlington
CompLLM: Compression for Long Context Q&A
Sep 23, 2025Gabriele Berton, Jayakrishnan Unnikrishnan, Son Tran +1LLM Compression Long-Context Language Model Inference
Amazon · Center For Research in Computer Vision, University of Central Florida

Mar 20, 2026J/K move · Enter open · S save

PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction

Runsong Zhao, Shilei Liu, Jiwei Tang, Langming Liu+7

Northeastern University, China · Future Living Lab of Alibaba · Tsinghua University

Language Model Performance Prediction LLM Context Compression Context Compression LLM Context Management LLM Inference Efficiency

Open paper AskPDF arXiv

While context compression can mitigate the growing inference costs of Large Language Models (LLMs) by shortening contexts, existing methods that specify a target compression ratio or length suffer from unpredictable performance degradation, hindering their reliable deployment. We introduce a paradigm shift to Performance-oriented Context Compression (PoC), where developers specify an acceptable performance floor instead of a compression ratio. PoC employs a lightweight performance predictor to automatically find the most aggressive compression ratio that satisfies this constraint before steering an off-the-shelf compressor. We design and compare two predictor variants: a simple context-agnostic predictor and a more sophisticated context-aware one that considers the input's inherent compressibility. On both question-answering and summarization benchmarks, the context-aware predictor consistently achieves lower performance prediction error than the context-agnostic predictor, while the resulting context-aware PoC attains a superior overall performance. Our work paves the way for a more reliable, efficient, and performance-aware deployment of context compression for LLMs.