cs.CLMay 29, 2025

Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compression

Authors: Yong Zhang, Heng Li, Yanwen Huang, Ning Cheng, Yang Guo, Yun Zhu, Yanmeng Wang, Shaojun Wang, 

Organizations: Ping An Technology (Shenzhen) Co., Ltd., China · University of Science and Technology of China · University of Electronic Science and Technology of China

Abstract

Retrieved contexts are often long, noisy, and redundant, motivating query-aware compression before answer generation. Native attention provides signals for context selection, but heterogeneous head behaviors complicate their direct aggregation into sentence-utility scores. We introduce Sentinel, a sentence-level compressor that learns to combine these signals through a signed linear readout across layers and heads. Only the readout is trained using existing QA evidence annotations, while the backbone remains frozen and sentence scoring requires no autoregressive generation. Across twelve LongBench tasks, Sentinel with a 0.5B backbone outperforms Full Context on multiple tasks under a 2,000-token budget with each of three readers: Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and DeepSeek V4.1 Flash. Despite English-only QA supervision, the same readout transfers to Chinese tasks without task-specific retraining. Length-stratified profiling further shows lower compression latency and GPU memory usage than LLMLingua-2 and CPC.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering

    Mar 10, 2026Thao Do, Dinh Phu Tran, An Vo +2Efficient Language Model InferenceKnowledge-Intensive QA

  2. SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering

    Jul 31, 2026Maryam Haghifam, Jason Cong, Yizhou SunMemory-Augmented Language ModelsLong-Context QA

  3. TopoCompress: Long Context Compression via Graph-Wired Semantic Trajectories

    Aug 31, 2026Daniel Agyei Asante, Yang LiLLM CompressionLong-Context QA