Deep research requires connecting multiple steps of search, reasoning, and computation, with intermediate findings guiding what to investigate next. As investigations grow, agents must maintain research state across model calls and check findings before errors propagate into subsequent work. We introduce PANINI-DR, a multi-agent harness that separates coordination of the overall investigation from the research and verification of individual subquestions. A shared workspace, built on the Generative Semantic Workspace representation, records subquestions, their dependencies, findings, and supporting evidence. Researchers investigate individual subquestions, while a separate verification agent checks proposed answers against those subquestions and their evidence. Verification feedback guides further investigation, and recorded findings inform subsequent assignments. We evaluate PANINI-DR on BrowseComp-Plus, FRAMES, and MoNaCo against seven state-of-the-art deep-research harnesses, all using a shared language-model backbone and frozen corpora. PANINI-DR achieves the strongest average performance across the three benchmarks. On BrowseComp-Plus, it achieves 73.3% accuracy, outperforming the next-best harness by 16.0 points. Its advantage is strongest on hard questions, where it answers nearly twice as many as the next-best harness. Ablations further show that structured planning and persistent state improve performance, as does checking each finding against its cited evidence before it is reused. Because a model can answer correctly from memory alone, we also analyze the evidence retained by each harness. PANINI-DR achieves the highest answer recoverability from saved evidence on all three benchmarks, while complementary claim-support audits show stronger evidence-backed answers. Accounting for this evidence changes how harnesses compare and further strengthens PANINI-DR's advantage.
Figures & tables
Figure 1: PANINI-DR illustrated on a sample query. The schematic stages separate the subquestions for explanation, rather than reproduce the recorded dispatch timing. Each Wk=(P,Sk) pairs the plan with per-node research records; ∅ denotes that no findings have yet been recorded. Dark green marks additions, dashed boxes remain unresolved, and orange outlines indicate readiness. The expanded stage shows each Researcher’s initial retrieval step (labelled “typed resolver”) and, when needed, ReAct agent, followed by verification and admission; the Reagan rejection is hypothetical. Findings supply subsequent subquestions and the date calculation that produces 51. The recorded execution investigated all four factual subquestions in one dispatch.
BCP CF-150
FRAMES
MoNaCo
Avg.
Method
Easy
Med.
Hard
Overall
Accuracy
F1
PyRAG
56
20
6
27.3
61.9
0.446
44.6
Re-TRAC
70
32
16
39.3
64.9
0.544
52.9
Omni-Decision
84
48
10
47.3
64.6
0.359
49.3
HyMem
82
52
12
48.7
70.8
0.612
60.2
Open Deep Search
82
50
28
53.3
76.0
0.571
62.1
Table 1: Official accuracy (%) on BCP and FRAMES and mean F1 on MoNaCo. Avg. equally weights BCP Overall, FRAMES, and MoNaCo, with F1 rescaled to 0–100. BCP groups contain 50 questions each. Bold: best; underline: second best, including ties.
Method
BCP
FRAMES
MoNaCo
Avg.
PyRAG
26.0
46.6
0.332
35.3
Re-TRAC
14.0
46.1
0.396
33.2
Omni-Decision
45.3
53.9
0.275
42.2
HyMem
46.7
36.7
0.388
40.7
Open Deep Search
51.3
47.6
0.325
43.8
IterResearch
51.3
45.9
0.390
45.4
Table 2: Observed support (Obs.): official answer credit recoverable from saved evidence, as defined in Section 3.2 . BCP and FRAMES report percentages; MoNaCo reports mean F1. Avg. equally weights the three benchmarks, with F1 rescaled to 0–100. Best scores are bold; second-best scores are underlined, including ties.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
System
Protocol
PyRAG
1 executable retrieval program; no read tool.
Re-TRAC
8 recursive search cycles, each compressed into the next cycle’s starting state.
IterResearch
25 turns; sampling temperature is hard-coded at 0.6.
Omni-Decision
15 iterations behind an evidence-state readiness gate.
Published default of 20 turns and retrieval top- k=3 ; maintains ranked hypotheses.
Appendix
Table 3: Baseline protocols used for the reported runs.
System
Primary grader
Second grader
Δ
Agreement
κ
PANINI-DR (cap 5)
110 (73.3)
111 (74.0)
+0.7
99.3
0.983
SLEUTH
86 (57.3)
86 (57.3)
0.0
100
1.000
HyMem
73 (48.7)
73 (48.7)
0.0
100
1.000
Omni-Decision
71 (47.3)
71 (47.3)
0.0
100
1.000
Re-TRAC
59 (39.3)
59 (39.3)
0.0
100
1.000
Appendix
Table 4: Grader sensitivity on BCP CF-150 (150 questions): gpt-5.6-luna (primary) versus gpt-5.6-terra , with the same answers, official prompt, and 3-pass majority. Scores are correct-answer counts with accuracy (%) in parentheses; Δ is in percentage points. Agreement is the percentage of matching verdicts; κ is Cohen’s kappa.
Method
BCP-150
FRAMES-824
MoNaCo-1315
Accuracy (%)
Accuracy (%)
Mean-F1
HyMem
48.0
43.6
0.352
SLEUTH
55.3
67.1
0.417
PANINI-DR
72.0
73.2
0.443
Appendix
Table 5: Answer quality after a common model auditor filters unsupported claims. Rebuilt answers are re-graded under each benchmark’s official protocol. Every question is included, with empty answers scoring zero. Bold marks the best score.
BCP-150 (%)
FRAMES-824 (%)
MoNaCo-1315 (F1)
Evidence source
Present
Uncertain
Present
Uncertain
Present
Uncertain
PANINI-DR
72.0
62.7
76.7
69.4
0.601
0.547
SLEUTH
59.3
53.3
70.0
59.3
0.506
0.476
HyMem
54.7
50.0
52.2
43.9
0.470
0.419
Omni-Decision
58.7
55.3
71.2
62.5
0.538
0.515
Appendix
Table 6: Answer quality of a common evaluation reader using each system’s saved evidence. Present assumes the answer is available; Uncertain allows abstention. All questions are included; empty answers score zero. Bold marks the best score.
FRAMES-824
MoNaCo-1315
Arithmetic/Boolean target records
192
335
Passing calculations
118
184
Answers receiving additional credit
51
55
Obs.
66.1
0.458
Obs-D
72.3
0.498
Appendix
Table 7: PANINI-DR -only calculation audit. FRAMES scores are percentages; MoNaCo scores are mean-F1. All questions remain in the denominator. Only official credit not already retained by Obs. is added.
Benchmark
n
Found recall
Cited recall
Cited precision (%)
BCP
150
.860 / .455
.485 / .248
43.3 / 37.0
FRAMES
823
.945 / .820
.774 / .586
91.9 / 84.4
MoNaCo
1,315
.515 / .368
.281 / .250
73.3 / 67.0
Appendix
Table 8: Source coverage: PANINI-DR / SLEUTH. Recall is the fraction of annotated pages found or recorded as support; precision is the annotated share of support pages. Values average per-question fractions, with zero precision when no citation is recorded. Source measures omit one FRAMES question without annotations; answer scores retain all 824. SLEUTH source titles are normalized and matched to recorded pages.
Benchmark
Recovery answers
Official
Obs.
BCP
60
66.7
63.3
FRAMES
103
60.2
52.4
MoNaCo
250
0.430
0.358
Appendix
Table 9: Scores over all identified recovery answers, including incorrect ones: percentages on BCP/FRAMES and mean F1 on MoNaCo. Finalizer outputs number 50/89/241 and stored-candidate fallback outputs 10/14/9, respectively. One additional MoNaCo output has an unclassified origin.
BCP
FRAMES
MoNaCo
Easy
Med.
Hard
All
Acc.
F1
PANINI-DR
92
76
52
73.3
79.1
0.549
Annotated-source reference
96
94
90
93.3
79.5
0.550
Appendix
Table 10: Official scores with research or supplied sources: accuracy (%) on BCP/FRAMES and mean F1 on MoNaCo. Denominators are 150/824/1,315; each BCP difficulty group contains 50 questions.
Recovery off
Recovery on
Prompt (M)
Arm
Official
Obs.
Official
Obs.
All roles, mean
Off : prose memory
41.3
39.3
56.7
53.3
1.38
On : planning/state
58.0
56.0
73.3
68.7
1.78
Full system
46.7
45.3
73.3
70.7
1.43
Appendix
Table 11: Planning/state and common-recovery controls on all 150 BCP questions; scores are percentages. Prompt means cover the recorded reference runs: they exclude post-hoc Finalizer calls on the controlled arms and include live recovery in the full system. Full-system rows restore multiple components and are not isolated planning/state controls.
Benchmark
Verification
Official
Obs.
Prompt (k)
BCP (150)
On
73.3
70.7
1,042
Off
57.3
54.7
453
FRAMES (200)
On
79.5
69.0
125
Off
69.5
57.0
35
MoNaCo (200)
On
0.583
0.479
288
Off
0.517
0.412
47
Appendix
Table 12: Verification ablation on matched questions. Off removes LLM review and source-evidence checks but retains assignment and type checks. Scores are percentages on BCP/FRAMES and mean F1 on MoNaCo. Prompt use is the per-question median in thousands, using trajectory accounting.
Cap
Acc.
Obs.
Hard / 50
Warned / resolved
Prompt (M)
1
67.3
65.3
19
128 / 773
1.249 (0.817)
2
72.0
68.7
23
27 / 636
1.147 (0.854)
5
73.3
70.7
26
2 / 666
1.200 (1.042)
10
66.0
63.3
20
0 / 673
1.317 (1.115)
Appendix
Table 13: Shared review-cap sensitivity on BCP CF-150. Accuracy and Obs. are percentages; Hard counts correct answers out of 50. Warning counts use resolved bindings in the final workspace snapshots, including computed bindings. Prompt use is the per-question mean in millions, with the median in parentheses, using the trajectory accounting in Table 14 . These are benchmark averages for the supplied executions, not averages across research repetitions.
Method
Prompt (M)
Completion (k)
PyRAG
0.002 (0.017)
3.2 (6.5)
SLEUTH
0.037 (0.074)
2.3 (7.3)
Open Deep Search
0.10 (0.11)
10.0 (10.9)
Omni-Decision
0.13 (0.16)
9.0 (10.1)
IterResearch
0.17 (0.17)
9.8 (10.6)
HyMem
0.29 (0.31)
51.0 (65.9)
Appendix
Table 14: BCP CF-150 token use: per-question medians (means), with prompt tokens in millions and completion in thousands. All rows use gpt-oss-120b . Counting details appear in Appendix D.3.1 .
BCP
FRAMES
MoNaCo
Questions
150
824
1,315
Reported trajectory prompt total (M)
180.04
270.38
775.12
Sum of 10 recorded roles (M prompt tokens)
213.94
340.79
1,040.13
Role sum / trajectory total
1.188
1.260
1.342
Selected roles: mean prompt tokens per question, k (share of role sum)
Orchestrator
58.4 (4.1%)
23.1 (5.6%)
32.8 (4.1%)
Appendix
Table 15: PANINI-DR cap-5 role accounting from saved traces. Role telemetry covers every question; shares use summed role prompt tokens across all 10 roles, including the 6 not listed individually. ReAct denotes the Researcher’s second stage, excluding the typed resolver. k and M denote thousands and millions of tokens. Combined shares use unrounded counts. Table 14 instead reports trajectory totals.
Harness
Input (k)
Cache (%)
Uncached (k)
Generated (k)
PFLOP
PANINI-DR
1,422
66.5
477
78
5.68
SLEUTH
74
7.3
69
7
0.78
Omni-Decision
157
7.1
146
10
1.59
HyMem
309
45.0
170
66
2.41
Appendix
Table 16: Engine measurements over all 150 BCP questions. Tokens and analytically estimated model-core PFLOPs are per-question means; cache percentage is the share of submitted input reused.
Runs
PFLOP/q
vs. ours
Vote acc.
Any correct
Vote Obs.
1
0.76
0.13
61.3
61.3
58.7
2
1.66
0.29
62.0
66.0
59.3
4
3.22
0.57
63.3
69.3
60.7
6
4.79
0.84
64.7
69.3
61.3
7
5.51
0.97
65.3
70.0
62.7
8
6.34
1.12
64.7
70.0
62.0
Appendix
Table 17: Repeated SLEUTH research runs on all 150 BCP questions at cumulative model-core computation. Vote selects the most frequent non-empty answer; “Any correct” credits a question if at least one run answers correctly. Scores are percentages. Individual runs average 55.5% accuracy versus the greedy reference’s 57.3%.
Allowance
Accuracy
Obs.
Mean prompt (M)
Empty
PANINI-DR : turns per Researcher, review cap 5
20 (default)
73.3
70.7
1.20
4
50
74.7
71.3
2.54
2
80
76.0
74.0
3.33
6
PANINI-DR : shared question budget, review cap 5
20
31.3
28.7
0.115
71
Appendix
Table 18: Separate allowance sweeps on all 150 BCP questions, including unanswered cases. Scores are percentages; prompt means use trajectory accounting. Turn limits govern different activity across panels and are not matched computation. Each row is a separate execution, not a continuation of the shorter run.
Method
Table 1
Run 2
Run 3
Mean ± SD
Mean (%)
PANINI-DR
110
107
110
109.0±1.7
72.7
SLEUTH
86
91
84
87.0±3.6
58.0
Open Deep Search
80
88
93
87.0±6.6
58.0
IterResearch
82
83
80
81.7±1.5
54.4
HyMem
73
73
74
73.3±0.6
48.9
Omni-Decision
71
68
59
66.0±6.2
44.0
Appendix
Table 19: BCP stability across three research runs per method, each on the same 150 questions under the headline settings. Run columns and mean ± sample SD count correct answers; the final column converts the mean count to accuracy. The first run is the one reported in Table 1 .
Single item (720)
Collection (595)
Harness
F1
Obs.
P
R
F1
Obs.
PANINI-DR
0.593
0.446
0.690
0.462
0.495
0.474
SLEUTH
0.579
0.338
0.678
0.503
0.528
0.467
Open Deep Search
0.598
0.269
0.652
0.519
0.538
0.394
IterResearch
0.583
0.317
0.617
0.546
0.540
0.479
Omni-Decision
0.410
0.282
0.399
0.281
0.298
0.267
Appendix
Table 20: MoNaCo answer structure for the main-table executions. Columns average over all questions in each group using the same maximum-F1 pass. Mean F1 is not the harmonic mean of aggregate precision and recall. Bold marks the largest value.
Official
Obs.
Dataset
Comparator
n
Δ
95% CI
Δ
95% CI
Headline comparisons: PANINI-DR (cap 5) minus baseline
BCP
SLEUTH
150
+16.0
[+8.0,+24.0]
+16.7
[+8.7,+24.7]
FRAMES
SLEUTH
824
+1.9
[−1.0,+4.9]
+10.9
[+7.8,+14.1]
MoNaCo
SLEUTH
1,315
−0.007
[−0.029,+0.015]
+0.062
[+0.041,+0.083]
BCP
HyMem
150
+24.7
[+16.0,+34.0]
+24.0
[+14.7,+32.7]
Appendix
Table 21: Paired score differences on the matched question sets, with bootstrap 95% confidence intervals from 10,000 resamples paired by question. Positive differences favor the first-named arm in each subtraction. Values are percentage points on BCP and FRAMES and mean-F1 units on MoNaCo; n is the number of matched questions. Differences use unrounded scores. Each comparison resamples the same question set in both arms, so an interval describes the difference rather than either score separately.
System
Implementation or adaptation
SLEUTH
Paper/prompt-based: published reasoning, memory, commitment, and top-3 retrieval rules; dense and BM25 routes.
Omni-Decision
Paper-based: 15 iterations, top-6 retrieval, zero-novelty termination after 3 iterations.
HyMem
Paper-based: 4 tools, published tool wording, and per-layer budgets.
Open Deep Search
Released code: search timeout raised from 10 to 300 s; local-link gateway.
PyRAG
Output limit raised from 1,024 to 4,096 tokens; JSON-schema output.
IterResearch
Endpoint/return-value compatibility; assistant prefill, XML repair, JSON lift, action seeding, answer extraction; authors’ hard-coded temperature 0.6.
Appendix
Table 22: Baseline implementations and changes for the shared evaluation environment. Model-facing changes are listed separately from interface compatibility within each row.
Deep research agents have achieved remarkable progress on complex information seeking tasks. Even long ReAct style rollouts explore only a single trajectory, while recent state of the art systems scale inference time compute via parallel search and aggregation. Yet deep research answers are composed of complementary pieces of evidence, which parallel rollouts often duplicate rather than complete, yielding diminishing returns while pushing the aggregation context toward the model's limit. We propose Argus, an agentic system in which a Searcher and a Navigator cooperate to treat deep research as assembling a jigsaw from complementary evidence pieces, rather than brute forcing the whole answer in parallel. The Searcher collects evidence traces for a given sub-query through ReAct-style interaction. The Navigator maintains a shared evidence graph, verifying which pieces are still missing, dispatching Searchers to gather them, and reasoning over the completed graph to produce a source-traced final answer. We train the Navigator with reinforcement learning to verify, dispatch, and synthesize, while independently training the Searcher to remain a standard ReAct agent. The resulting Navigator supports rollouts with a single Searcher or many in parallel without retraining. With both Searcher and Navigator built on a 35B-A3B MoE backbone, Argus gains 5.5 points with a single Searcher and 12.7 points with 8 parallel Searchers, averaged over eight benchmarks. With 64 Searchers it reaches 86.2 on BrowseComp, surpassing every proprietary agent we benchmark, while the Navigator's reasoning context stays under 21.5K tokens.
Deep research agents autonomously conduct open-ended investigations, integrating complex information retrieval with multi-step reasoning across diverse sources to solve real-world problems. To sustain this capability on long-horizon tasks, reliable verification is critical during both training and inference. A major bottleneck in existing paradigms stems from the lack of explicit verification mechanisms in QA data synthesis, trajectory construction, and test-time scaling. Errors introduced at each stage propagate downstream and degrade the overall agent performance. To address this, we present Marco DeepResearch, a deep research agent optimized with a verification-centric framework design at three levels: (1)~QA Data Synthesis: We introduce verification mechanisms to graph-based and agent-based QA synthesis to control question difficulty while ensuring answers are unique and correct; (2)~Trajectory Construction: We design a verification-driven trajectory synthesis method that injects explicit verification patterns into training trajectories; and (3)~Test-time scaling: We use Marco DeepResearch itself as a verifier at inference time and effectively improve performance on challenging questions. Extensive experimental results demonstrate that our proposed Marco DeepResearch agent significantly outperforms 8B-scale deep research agents on most challenging benchmarks, such as BrowseComp and BrowseComp-ZH. Crucially, under a maximum budget of 600 tool calls, Marco DeepResearch even surpasses or approaches several 30B-scale agents, like Tongyi DeepResearch-30B.
Deep Research Agents (DRA) aim to help users search the web, synthesize information, and deliver comprehensive investigative reports. Prior benchmarks often either under-evaluate a system's ability to produce meaningful insights and high-quality writing, or adopt coarse or LLM-defined criteria that are hard to verify and can diverge from human expert judgment. To address these issues, we introduce Deep Research Bench II, a new benchmark for evaluating DRAs. It contains 132 grounded research tasks across 22 domains; for each task, an agent must produce a research report that is evaluated by a set of 9,430 fine-grained binary rubrics in total, covering three dimensions: information recall, analysis, and presentation. All rubrics are derived from carefully selected expert-written investigative articles and are constructed through a four-stage LLM+human pipeline that combines automatic extraction with over 400 human-hours of expert review, ensuring that the criteria are verifiable and aligned with human expert judgment. We evaluate several state-of-the-art deep-research agents on Deep Research Bench II and find that even the strongest models satisfy fewer than 50% of the rubrics, revealing a substantial gap between current DRAs and human experts. We release the benchmark, evaluation scripts, and all rubrics at https://github.com/imlrz/DeepResearch-Bench-II to facilitate future research on deep-rearch agents.
Ruizhe Li, Mingxuan Du, Benfeng Xu +3
University of Science and Technology of China · Metastone Technology, Beijing, China