Read Less, Solve More: Token-Efficient Sparse Reading for AI Agents

Zedong Liu; Jiaan Wu; Xinyang Ma; Le Xu; Kai Wang; Yuanchao Hu; Dingwen Tao; Guangming Tan

cs.AIAug 23, 2026

Read Less, Solve More: Token-Efficient Sparse Reading for AI Agents

Authors: Zedong Liu, Jiaan Wu, Xinyang Ma, Le Xu, Kai Wang, Yuanchao Hu, Dingwen Tao, Guangming Tan

Organizations: University of Chinese Academy of Sciences, Beijing, China · Songshan Lake Materials Laboratory, Dongguan, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

Abstract

Long-horizon agents increasingly rely on repeated access to external artifacts, yet current reading interfaces often expose entire objects even when only sparse evidence is needed. This over-reading increases token and latency costs and can dilute task-relevant evidence, while existing context-reduction methods mainly intervene after broad content has already entered the trajectory. We present SparseRead, a training-free, model-transparent reading layer that controls content admission before unnecessary evidence reaches the model context. SparseRead combines a regime-aware Read Gate, extensible Reader Backends, and a stateful protocol for bounded, source-anchored evidence acquisition with explicit refinement, verification, stopping, and fallback. Across six frontier models, including Claude Opus 5, and five workload scenarios, SparseRead reduces token volume by up to 92.9% and wall time by up to 89.0%, while preserving or improving task quality. Its consistent gains across three agent frameworks further demonstrate broad portability.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 251 papers

Periscope: Extending Frozen Language Models Beyond Their Context Window
Oct 2, 2026Mohamed Eltahir, Anas Obayd, Raed Rashid +5Long-Context Language Model Inference Long-Context Reasoning in Language Models
King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Department of Computer Science, Edge Hill University, Ormskirk, England
Reasoner-Executor-Synthesizer: Scalable Agentic Architecture with Static O(1) Context Window
Mar 23, 2026Ivan DobrovolskyiAgentic RAG LLM Hallucination Mitigation
Walmart Global Tech, Sunnyvale, CA, USA
Not Worth Another Token: Marginal Value Estimation for Efficient Deep Research Agents
Aug 9, 2026Harshitha Kolukuluru, Reshma Ashok, Kirat Arora +7Deep Research Agents AI Agent Evaluation
University of Massachusetts Amherst · The University of Texas at Austin · Adobe Research

Oct 2, 2026J/K move · Enter open · S save

Periscope: Extending Frozen Language Models Beyond Their Context Window

Mohamed Eltahir, Anas Obayd, Raed Rashid, Abdulrahman Alghamdi+4

King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Department of Computer Science, Edge Hill University, Ormskirk, England

Long-Context Language Model Inference Long-Context Reasoning in Language Models Efficient Language Model Inference LLM Context Management Long-Context Retrieval

Open paper AskPDF arXiv

A language model reads long text in one quadratic forward pass, stops at the context window, and loses accuracy with length before reaching it. We ask whether the read can be factorized when deciding over a finite set: which document is relevant, which option is supported, which passage is the evidence. Periscope, a training-free inference method, arranges the

N

chunks of a text on a

K{\times}K

grid with

K{=}\lceil\sqrt{N}\rceil

and asks a frozen model the same question about

K

local spans of consecutive chunks and

K

strided spans that sample the whole text, reading the log-odds of every answer at one token. Each answer takes its best local and strided score, and scoring every chunk by its two spans gives an evidence map at no further cost, whose peak is the chunk behind the answer. Every probe is about

\sqrt{sc}

tokens for a text of

s

tokens and chunk size

c

, so a window of

W

tokens reaches

W^{2}/c

tokens at

s^{1.5}

cost. The map replaces the long read. On LongBench v2, reading only the

K

chunks the map ranks highest, 9k tokens, matches the same model's best window read across windows from 32k to 1M tokens, and on InfiniteBench, where the median context is 150k tokens, it leads the best window read by 5 points. The same map ranks BRIGHT's long-document corpora with the best NDCG@10 of six methods. Each call caches only one probe, so a 27B model reads 4.5M-token contexts on one 80GB GPU, where a single pass would need 296GB of cache. A long read then needs a GPU that holds the model, not one that holds the text.

Abstract

LLM Context Management

Explore similar work

Periscope: Extending Frozen Language Models Beyond Their Context Window

Reasoner-Executor-Synthesizer: Scalable Agentic Architecture with Static O(1) Context Window

Not Worth Another Token: Marginal Value Estimation for Efficient Deep Research Agents