LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Management

Songtao Huang; Zhijie Zhong; Xiaohan He; Shuaiyu Zhang; Zhiyin Yu; Li Kang; Heng Zhou; Yutao Fan

cs.AIOct 9, 2026

LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Management

Authors: Songtao Huang, Zhijie Zhong, Xiaohan He, Shuaiyu Zhang, Zhiyin Yu, Li Kang, Heng Zhou, Yutao Fan,

Organizations: Fudan University · Shanghai Artificial Intelligence Laboratory · Shanghai Jiaotong University · Peking University ·

Abstract

Corpus-level long-context reasoning requires agents to acquire and integrate evidence across document collections that exceed their context windows. Existing agent harnesses enable incremental access to these corpora, but acquired evidence and evolving conclusions often remain entangled in a growing interaction history. This implicit state management can lead to two coupled failure modes: unbounded exploration, where retrieval expands the interaction history without resolving decision-relevant information gaps, and evidence-state drift, where previously acquired evidence or subsequent corrections fail to inform the current conclusion. We present LongHarness, a model-agnostic harness that makes the interaction between acquired evidence and evolving conclusions explicit. Its core mechanism, Evidence-Conclusion Coevolution, uses unresolved conclusions to guide exploration toward missing information, while newly acquired evidence strengthens or revises the conclusions that directed the search. LongHarness couples this process with Evidence Lifecycle Management, which keeps accumulated evidence current and traceable throughout the interaction. Together, these mechanisms focus retrieval on remaining information needs and keep conclusions aligned with current evidence. We evaluate LongHarness on OfficeQA Pro V1 and V2, AA-LCR, and LongBench v2 using GLM-5.3-Flash and Qwen3.8-Flash. Compared with direct reasoning using the same backbone, LongHarness improves GLM-5.3-Flash exact match on OfficeQA Pro from 15.8% to 60.2% on V1 and from 4.4% to 61.1% on V2. Against state-of-the-art agent harnesses, it surpasses Claude Code by 12.0 percentage points on V1 and Codex by 18.9 points on V2. With Qwen3.8-Flash, LongHarness reaches 61.7% on V1, outperforming Claude Code by 8.3 points and Codex by 12.0 points.

Figures & tables

Figure 1: Evidence lifecycle management for corpus-level long-context reasoning. The left panel illustrates how an agent loses an earlier constraint and continues searching, while LongHarness retains the evidence, uses an unresolved conclusion to guide retrieval, and revises its decision when new evidence arrives. The right panel compares exact accuracy (%) on OfficeQA Pro V1 and V2 for direct models and agent harnesses using GLM-5.3-Flash.

Figure 2: Overview of LongHarness . A single agent maintains an EvidenceBank and a Conclusion State while interacting with external documents. Evidence–Conclusion Coevolution uses unresolved conclusions to guide exploration and accepted evidence to revise those conclusions, while Evidence Lifecycle Management governs which statements remain active and preserves their source references throughout the interaction.

Panel of Table 1 — Table 1: Main results (%) across four benchmarks. OfficeQA Pro reports exact match (Ex.) and accuracy within 5% relative error (@5%), while AA-LCR and the LongBench v2 Long subset report Exact accuracy. Direct baselines use BM25 retrieval into a 256K-token window on OfficeQA Pro. “-” denotes direct reasoning without a harness, and “Avg.” averages the Exact scores across benchmarks. Each harness result is from one run per setting. Direct AA-LCR scores are reported by Artificial Analysis; harness AA-LCR scores are evaluated in this work.

Variant	OfficeQA Pro V1		OfficeQA Pro V2
Variant	Ex.	$\Delta$	Ex.	$\Delta$
LongHarness (full)	60.2	–	61.1	–
w/o Persistent States	55.6	4.5	53.3	7.8
w/o Evidence–Conclusion Coevolution	54.9	5.3	51.7	9.4
Naive Evidence Accumulation	56.4	3.8	51.1	10.0
w/o Explicit Conclusion-Guided Exploration	55.6	4.5	53.3	7.8

Table 2: Ablation results with GLM-5.3-Flash on OfficeQA Pro V1 and V2. Ex. denotes exact-match accuracy (%), and

\Delta

denotes the decrease from full LongHarness in percentage points, computed before rounding.

Evidence level	OfficeQA Pro V1			OfficeQA Pro V2
Evidence level	Claude Code	Codex	LongHarness	Claude Code	Codex	LongHarness
L1: Retrieval ( $r_{1}$ )	85.0	72.2	85.0	72.2	76.7	87.8
L2: Evidence editing ( $r_{2}$ )	85.8	83.3	87.6	84.6	78.3	87.3
L3: Transformation ( $r_{3}$ )	66.0	77.5	80.8	58.2	70.4	79.7

Table 3: Success rates (%) for evidence retrieval, editing, and transformation on OfficeQA Pro V1 and V2 with GLM-5.3-Flash. Retrieval measures whether the required evidence is found, editing measures whether retrieved evidence is retained and adopted in the final decision, and transformation measures whether the adopted evidence yields an exactly correct answer. Editing and transformation rates are calculated only over cases that pass the preceding stages. Each harness is evaluated on one run per split.

Figure 3: Distributions of model turns per question on OfficeQA Pro V1 and V2 with GLM-5.3-Flash. Each panel compares the two versions for one harness. Curves show kernel density estimates, and circles indicate medians.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	# Items	Evidence access	Metric
OfficeQA Pro V1	133	697-doc full corpus	Ex., @5%
OfficeQA Pro V2	90	1,435-doc full corpus	Ex., @5%
AA-LCR-1.1	100	234 provided docs	Judged Ex.
LongBench v2 Long	108	132K–1.38M-word context	Choice Ex.

Appendix

Table 4: Benchmark sizes, evidence access, and evaluation metrics. The AA-LCR document count covers all document sets, and LongBench v2 lengths are measured in words from the preprocessed contexts used in our experiments.

Operation	Purpose	Returned observation
search	Locate documents or passages relevant to an information need	Ranked candidate locations and navigation snippets
read	Inspect selected locations under a bounded read extent	Source text and registered source references
EvidenceBank	Commit or revise source-linked statements	Active statements, revision status, and validation feedback
compute	Evaluate a deterministic numerical expression over evidence values	The computed value and integrity warnings
final	Submit the answer with supporting references	Accepted output with resolved references, or validation feedback

Appendix

Table 5: LongHarness interaction interface, showing the purpose and returned observation of each operation.

Operation	Input	Effect
upsert	One or more proposed source-linked statements	Accepts each validated statement as an active EvidenceBank entry
revise	An active evidence identifier and either a replacement statement or a null replacement	Replaces or withdraws the identified active entry

Appendix

Table 6: EvidenceBank operations for admitting, correcting, and retracting source-linked statements.

Setting	Value
Maximum turns	200
Maximum search calls	60
Cumulative read budget	1,000,000 tokens
Maximum tokens per read	16,384

Appendix

Table 7: Interaction and reading limits used in the LongHarness evaluations.

System	Available operations	Constraints
Context window	Direct prompt-based reasoning over the supplied or retrieved context	No tool interaction; 256K-token context window
Claude Code	Workspace file search and reading, local shell commands, and workspace file modification	200 model turns and 200 tool calls
Codex	Workspace file inspection, local shell commands, and workspace file modification	200 model turns and 200 tool calls
RLM	Persistent Python REPL, programmatic document access, and model subcalls	30 root-level iterations
LongHarness	search , bounded read , EvidenceBank upsert and revise , compute , and final	200 turns, 60 search calls, a 1M-token cumulative read budget, and at most 16,384 tokens per read

Appendix

Table 8: Available operations and resource limits for direct reasoning and the agent harnesses in the main comparison.

Harness	Run 1	Run 2	Run 3	Mean	Std.	Max.
OfficeQA Pro V1
Codex	45.86	46.62	47.37	46.62	0.76	47.37
Claude Code	48.12	48.87	51.13	49.37	1.57	51.13
RLM	45.11	42.86	48.12	45.36	2.64	48.12
LongHarness	60.15	60.15	59.40	59.90	0.43	60.15
OfficeQA Pro V2

Appendix

Table 9: Three-run Exact accuracy with GLM-5.3-Flash. Run scores, means, and maxima are percentages. Standard deviations are in percentage points.

LLM Agent Harnesses

LLM agent harnesses are the software and runtime systems that organize an agent’s tools, context, state, execution, and safeguards.

See all 387 papers

HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning
Jun 3, 2026Zeju Li, Ziyang Zheng, Yizhou Zhou +1Agent Memory Multi-Agent LLM Systems
The Chinese University of Hong Kong · Independent Researcher
Harness-Search: Guiding Long-Horizon Search through Multi-Agent Coordination
Oct 4, 2026Shanyong Wang, Zhenwen Ji, Lei Jin +7Multi-Agent Coordination Multi-Agent Systems
Xiaohongshu Inc. · the Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University · University of Illinois at Urbana-Champaign
ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning
Jul 2, 2026Yanjun Zhao, Ruizhong Qiu, Tianxin Wei +6Long-Context Inference Long-Context Reasoning in Language Models
University of Illinois Urbana-Champaign

Jun 3, 2026J/K move · Enter open · S save

HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning

Zeju Li, Ziyang Zheng, Yizhou Zhou, Qiang Xu

The Chinese University of Hong Kong · Independent Researcher

Agent Memory Multi-Agent LLM Systems Long-Context Inference Memory-Augmented Agents

Open paper AskPDF arXiv

Long-context reasoning requires models to access, retrieve, and integrate evidence scattered across documents, dialogues, and accumulated interaction histories. Standard retrieval-augmented generation reduces this problem to top-

K

chunk retrieval, but such passive access can discard relevant evidence before reasoning begins, especially when relevance depends on broader context. We propose HMARS, a hierarchical multi-agent memory system that treats long contexts as managed memory rather than a flat retrieval corpus. Sub-agents maintain grounded access to bounded memory regions, mid-agents manage regional context and provide query-specific coordination, and a frontier model performs final reasoning over retrieved evidence pages. To evaluate this view, we construct two diagnostic benchmarks targeting evidence breadth and context-dependent relevance. Across long-document and multi-turn memory tasks, HMARS achieves the best overall performance against retrieval, reranking, full-context, graph-based, and agentic long-context baselines. Evidence coverage analysis further shows that its gains come from retrieving the required supporting evidence more completely, rather than merely changing the final answer prompt.

Abstract

LLM Agent Harnesses

Explore similar work

HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning

Harness-Search: Guiding Long-Horizon Search through Multi-Agent Coordination

ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning