PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents

Kun Li; Zexuan Qiu; Tianhua Zhang; Irwin King; Helen Meng

cs.CLSep 6, 2026

PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents

Authors: Kun Li, Zexuan Qiu, Tianhua Zhang, Irwin King, Helen Meng

Organizations: The Chinese University of Hong Kong

Abstract

Sequential memory agents process long documents by reading chunks one after another while maintaining a compact memory state, coupling document traversal to reasoning depth. This coupling introduces sensitivity to evidence placement and ties inference latency linearly to document length. We introduce PARSER, which decouples reading from reasoning. A bank of lightweight subagents each bound to a single chunk read the entire document in parallel, while a lead agent reasons in depth through iterative scatter--gather rounds: at each round it broadcasts a query to all subagents, aggregates the returned evidence, and formulates a deeper follow-up query conditioned on what has been found so far. This decoupled design concentrates all learnable behavior in the lead agent, which is optimized with reinforcement learning, while the subagents remain frozen off-the-shelf models. On multi-hop QA with contexts ranging from 7K to 896K tokens, PARSER with a 4B backbone outperforms the strongest sequential memory baseline by 5.7 points on average and by 12.0 points at 896K tokens. Scaling to a 9B backbone, PARSER surpasses DeepSeek-V4-Pro by 6.3 points. Controlled experiments confirm that PARSER is robust to perturbations in evidence position, order, and distance, conditions that cause large accuracy swings in sequential methods, while reducing inference latency by up to 11x.

Figures & tables

Figure 1: Comparison between ParSer and sequential memory methods. (a) Sequential memory traverses

T

chunks through a chain of dependent reading steps. (b) ParSer turns the chunks into parallel width: at each round, all chunks are read concurrently under the same query, while reasoning proceeds across rounds.

(a) Accuracy on HotpotQA (In-Distribution)
Backbone	Method	# Paragraphs (Total Length)								Avg.
Backbone	Method	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)	Avg.
DeepSeek-V4-Pro	Full-context (non-think)	78.12	77.34	76.56	79.69	77.34	75.78	73.44	62.50	75.10
DeepSeek-V4-Pro	Full-context (think-max)	82.03	82.81	80.47	80.47	80.47	81.25	77.34	78.91	80.47
Qwen3.5-4B	Full-context (non-think)	75.78	75.78	71.88	74.22	67.97	61.72	53.13	34.38	64.36
	Full-context (think)	80.47	78.91	78.12	76.56	74.22	60.94	46.09	31.25	65.82

Table 1: Long-context QA results on HotpotQA ( Yang et al., 2018 ) and 2WikiMultiHopQA ( Ho et al., 2020 ) . Values are accuracy (Sub_EM, %). Within each Qwen backbone block, the best result in each column is bolded .

Figure 2: Performance under controlled evidence distributions. (a) Evidence position: supporting evidence is placed within different percentile ranges of 894K-token documents. (b) Evidence order: evidence-bearing paragraphs in 894K-token documents either follow or reverse their logical dependency order. (c) Evidence distance: the number of intervening paragraphs between two pieces of supporting evidence is varied.

Concurrency	Method	# Paragraphs (Total Length)
Concurrency	Method	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
1	Full-context (non-think)	1.16	1.86	2.11	3.52	6.25	18.36	51.86	185.42
	Full-context (think)	12.33	17.78	18.93	31.89	42.73	68.19	162.97	375.32
	MemAgent	10.56	15.99	30.40	58.49	112.59	222.86	438.61	876.20
	ParSer	5.33	5.84	6.83	8.71	13.30	22.43	42.19	78.22
16	Full-context (non-think)	0.63	0.91	1.11	2.55	9.46	28.00	94.22	–

Table 2: Amortized wall-clock inference time per sample (seconds), computed as the total subset wall-clock time divided by the size of the subset (128), across document lengths under concurrency of 1, 16, and 32. Full-context methods fail to finish 896K-token evaluation at high concurrency due to GPU memory limits.

Figure 3: Inference step counts of MemAgent and ParSer across lengths.

Subagent	# Paragraphs (Total Length)								Avg.
Subagent	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)	Avg.
Qwen3.5-2B	82.55	84.38	82.29	84.11	77.87	76.56	69.01	69.27	78.26
Qwen3.5-4B (default)	85.68	84.64	83.60	85.68	85.42	83.07	83.07	85.42	84.57
Qwen3.5-9B	82.81	85.16	87.50	84.38	84.38	85.16	82.81	85.94	84.77

Table 3: Effect of subagent size on HotpotQA (Sub_EM, %). The lead agent is fixed as Qwen3.5-4B. Best results in each column are bolded .

Chunk size	# Paragraphs (Total Length)								Avg.
Chunk size	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)	Avg.
4096	85.68	84.64	83.60	85.68	85.42	83.07	83.07	85.42	84.57
Full	83.85	84.64	82.29	83.86	74.48	71.10	56.77	53.13	73.76
16,384	82.03	82.29	85.94	82.81	82.81	83.86	85.42	83.59	83.59
65,536	84.64	82.03	80.73	80.73	78.91	80.99	83.59	83.08	81.84
131,072	83.07	83.59	80.99	80.21	75.78	80.21	79.69	79.17	80.34

Table 4: Effect of chunk size on HotpotQA (Sub_EM, %). All variants are based on Qwen3.5-4B. For the Full-document setting, YaRN-based RoPE scaling is enabled on the 3200 (448K) and 6400 (896K) subsets to accommodate contexts beyond the model’s native window.

Agent	# Paragraphs (Total Length)								Avg.
Agent	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)	Avg.
DCI agent	73.96	75.78	74.74	76.56	74.48	75.52	77.60	76.30	75.61
Lead agent + DCI subagents	86.72	88.28	89.06	86.72	85.94	80.47	81.25	78.91	84.67
Full-context (think)	80.47	78.91	78.12	76.56	74.22	60.94	46.09	31.25	65.82
Lead agent + thinking subagents	85.16	86.72	87.50	82.03	85.94	85.94	87.50	83.59	85.55

Table 5: Comparison of alternative subagent implementations with their full-context counterparts on HotpotQA (Sub_EM, %). All agents are built on Qwen3.5-4B.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

	# Paragraphs (Total Length)
	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
MemAgent (# tokens)	2211	3379	6353	12334	23805	47174	93806	187058
ParSer (# tokens by Lead + Subagents)	534+312	533+348	525+379	485+392	523+474	522+515	546+586	541+804
ParSer /MemAgent (Decode Computation)	13.9%	10.6%	6.1%	3.2%	2.0%	1.1%	0.6%	0.4%

Appendix

Table 6: Average output token counts per sample under different document lengths. Both methods are implemented based on Qwen3.5-4B. MemAgent entries report the total output tokens. For ParSer , entries report the combined output-token counts of the lead agent and all subagents. The final row reports the ratio of ParSer to MemAgent theoretical decoding computation, calculated from Equations ( 4 ) and ( 7 ) with

n/T=5000

and

K=4

, under the assumption of sufficient GPU memory to retain all chunk KV caches.

(a) Accuracy on HotpotQA (In-Distribution)
Backbone	Method		# Paragraphs (Total Length)								Avg.
Backbone	Method	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)		Avg.
Qwen3.5-4B	MemAgent	w/o RL	69.53	62.76	58.86	50.78	40.62	32.29	25.78	19.79	45.05
		w/ RL	81.25	81.51	83.59	77.86	77.86	79.69	73.96	72.92	78.58
	ReMemR1	w/o RL	66.41	63.28	59.90	48.44	44.53	34.38	27.87	20.58	45.67
		w/ RL	82.03	79.95	82.03	78.91	77.86	79.95	77.08	73.44	78.91

Appendix

Table 7: Effect of reinforcement learning on HotpotQA ( Yang et al., 2018 ) and 2WikiMultiHopQA ( Ho et al., 2020 ) . Values are accuracy (Sub_EM, %).

Figure 4: Out-of-distribution performance of the 4B and 9B ParSer on ten RULER tasks across context lengths from 8K to 512K tokens. Each cell reports Sub_EM (%).

Split	# Samples	# Paragraphs per sample
Training (HotpotQA)	$32{,}768$	$200$
Eval (HotpotQA)	$128\times 8$	$\{50,100,200,400,800,1600,3200,6400\}$
Eval (2WikiMultiHopQA)	$128\times 8$	$\{50,100,200,400,800,1600,3200,6400\}$

Appendix

Table 8: Statistics of training and evaluation data. Training contexts use

200

paragraphs (

\sim 28

K tokens). Evaluation contexts range from

50

6400

paragraphs (

\sim 7

K–

896

K tokens), with the same

128

questions reused across the eight length settings.

Category	Hyperparameter	Value
Actor optimization	Optimizer	Adam
	Learning rate; warm-up steps; schedule	$1\times 10^{-6}$ ; $15$ ; constant
	Adam $(\beta_{1},\beta_{2})$ ; weight decay	$(0.9,0.999)$ ; $0.01$
	Gradient clipping	$1.0$
	PPO mini-batch size	$128$
	PPO clip range; entropy coefficient	$[0.2,0.2]$ ; $0$

Appendix

Table 9: Training hyperparameters for ParSer .

Figure 5: Training dynamics of the Qwen3.5-4B and Qwen3.5-9B lead agents over 180 reinforcement-learning steps. Thin curves show per-step measurements, while thick curves show 10-step moving averages. Logging for tool calls per turn and subagent replies per query was introduced after the 4B run had completed; consequently, these metrics are available only for the 9B run.

Panel of Figure 9 — Figure 9: Success case 4 ( ParSer -9B; HotpotQA, 6400-paragraph).

Long-Context Inference

Long-context inference studies how language models retrieve, retain, and reason over information spanning very large inputs.

See all 52 papers

PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory
Aug 4, 2026Dawei Liu, Haixu Song, Shuang Cheng +9Memory-Augmented Language Models Long-Context QA
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Tsinghua University +3
PAIR: Memory-Efficient and Fast Inference with Reading--Reasoning Fusion
Oct 12, 2026Sejun Park, Haocheng Xia, Yongjoo ParkMemory-Efficient Inference Long-Context Language Model Inference
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA
Trace Only What You Need: Structure-Aware On-Demand Hypergraph Memory for Long-Document Question Answering
Jun 9, 2026Xiangjun Zai, Xingyu Tan, Chen Chen +2Retrieval-Augmented Generation Graph Retrieval-Augmented Generation
University of New South Wales, Australia · CSIRO, Australia · University of Wollongong, Australia

Aug 4, 2026J/K move · Enter open · S save

PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory

Dawei Liu, Haixu Song, Shuang Cheng, Shijie Wang+8

Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Tsinghua University+3

Memory-Augmented Language Models Long-Context QA LLM Inference Acceleration

Open paper AskPDF arXiv

Long-context reasoning remains a critical bottleneck for large language models, as recent recurrent-memory approaches face two inherent challenges: sequential chunk-wise updates can overwrite early critical evidence with later irrelevant content, and serial inter-chunk dependencies limit parallelism and cause latency to increase with context length. To address these issues, we propose PI-Mem (Parallel-Iterative Memory), a mechanism that processes all chunks in parallel and iteratively refines a shared memory over a bounded number of turns. In each turn, PI-Mem reads all chunks in parallel conditioned on the current memory, selects new or complementary evidence from each chunk, and merges the selected evidence into a compact shared memory for the next turn. To discourage redundant turns, we optimize the workflow through reinforcement learning with an auxiliary turn-efficiency reward, enabling the model to adaptively exit once sufficient evidence has been accumulated. We evaluate PI-Mem with Qwen3.5-35B-A3B and Qwen2.5-7B on the HotpotQA benchmark across context lengths up to 3.6 million tokens and find that it outperforms the recurrent-memory baseline by +6.25 and +7.81 absolute points while achieving 6.1

\times

and 2.1

\times

inference speedups, respectively. These results demonstrate that PI-Mem breaks the accuracy--efficiency trade-off in long-context reasoning and provides a scalable approach to complex multi-hop question answering over extremely long documents.

Abstract

Long-Context Inference

Explore similar work

PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory

PAIR: Memory-Efficient and Fast Inference with Reading--Reasoning Fusion

Trace Only What You Need: Structure-Aware On-Demand Hypergraph Memory for Long-Document Question Answering