Organizations: National University of Singapore · TCG CREST · Indian Institute of Technology Kharagpur · Singapore Institute of Technology · Indian Institute of Technology Bombay
We present Crase, a bounded and inspectable alternative to deep research agents for scholarly search. Instead of an open-ended search loop, Crase queries a search engine once for seed papers, expands them along their 1.5-hop citation neighborhood, prunes citation edges whose claims lack entailment support, and ranks the remaining papers with a recency-aware random walk. This makes the candidate set, the reason each paper is kept, and the stopping condition explicit and fixed before inference. On LitSearch and one further benchmarks over a 500K-paper arXiv corpus, Crase outperforms deep research agents built on proprietary models by up to 3× recall@50 at roughly a third of the cost.
Figures & tables
Figure 1: Unlike deep research agents that repeatedly search until the model decides to stop, Crase performs a seeded search, constructs a 1.5-hop citation graph, prunes unsupported edges, and ranks papers with a recency-aware walk.
Dataset
Method
Recall@K
MAP@K
@5
@10
@20
@30
@40
@50
@5
@10
@20
@30
@40
@50
ACL
DeepResearch-GPT
0.0335
0.0696
0.1289
0.1572
0.1881
0.2294
0.0107
0.0170
0.0234
0.0253
0.0267
0.0279
DeepResearch-Claude
0.0309
0.0696
0.1160
0.1366
0.1804
0.2294
0.0085
0.0150
0.0196
0.0203
0.0223
0.0242
Crase -Llama-PPR
0.0730
0.1236
0.1882
0.2584
0.3258
0.3848
0.0453
0.0530
0.0602
0.0656
0.0697
0.0730
Spector2-Deepwalk
0.0284
0.0747
0.1263
0.1830
0.2242
0.2629
0.0170
0.0248
0.0303
0.0341
0.0366
0.0380
ICLR
DeepResearch-GPT
0.0183
0.0427
0.0610
0.0732
0.1098
0.1220
0.0082
0.0106
0.0119
0.0123
0.0150
0.0155
Table 1: Main results. Recall@K and MAP@K across the three datasets and retrieval setups. Best results bold-colored; second-best underlined. We observe that Crase -Llama-PPR consistently performs best on ACL and ICLR and remains competitive or best across most settings on arXiv .
Dataset
Variant
Recall@K
MAP@K
@5
@10
@20
@30
@40
@50
@5
@10
@20
@30
@40
@50
ACL
Crase -Qwen-SALSA
0.0335
0.0747
0.1263
0.1675
0.2113
0.2552
0.0200
0.0258
0.0289
0.0317
0.0336
0.0359
Recency
0.0335
0.0619
0.1521
0.2165
0.2655
0.3015
0.0160
0.0209
0.0281
0.0333
0.0360
0.0376
Crase -Qwen-PPR
0.0309
0.0722
0.1572
0.2191
0.2706
0.3144
0.0148
0.0212
0.0284
0.0336
0.0362
0.0382
Crase -Llama-SALSA
0.0393
0.0787
0.1433
0.1938
0.2584
0.3146
0.0177
0.0269
0.0332
0.0359
0.0398
0.0428
ICLR
Crase -Qwen-SALSA
0.0549
0.0793
0.1280
0.1768
0.2073
0.2195
0.0197
0.0234
0.0280
0.0314
0.0325
0.0329
Table 2: Ablations. Alternative claim extraction models ( Qwen / Llama ) and ranking strategies ( SALSA / PPR ) and the Recency baseline. Best per column is bold-colored; second-best is underlined. We observe that Crase -Llama-SALSA gives the strongest overall performance, while Crase -Qwen-PPR is competitive in Recall@K.
Recall@ K
MAP@ K
Dataset
Seeds
5
10
20
30
40
50
5
10
20
30
40
50
Query yr.
0.0295
0.0516
0.1204
0.1622
0.2187
0.2654
0.0117
0.0158
0.0223
0.0253
0.0283
0.0311
ACL
Current yr.
0.0309
0.0722
0.1572
0.2191
0.2706
0.3144
0.0148
0.0212
0.0284
0.0336
0.0362
0.0382
Δ
+.0014
+.0206
+.0368
+.0569
+.0519
+.0490
+.0031
+.0054
+.0061
+.0083
+.0079
+.0071
Query yr.
0.0606
0.1138
0.2036
0.2216
0.2515
0.2994
0.0278
0.0338
0.0441
0.0454
0.0463
0.0483
ICLR
Current yr.
0.0488
0.1220
0.2195
0.2561
0.3171
0.3659
0.0245
0.0372
0.0497
0.0531
0.0561
0.0580
Table 3: Seed selection ablation. Query-year vs. current-year seeds (shaded; used in main experiments); bold marks the best result and Δ the absolute change. Current-year seeds improve nearly all Recall@K and MAP@K scores, with the only drop at Recall@5 for ICLR.
Method
Calls ↓
Tokens ↓
Time ↓
Cost ↓
R@50 ↑
(K)
(s)
($)
DeepResearch-GPT
18
620
272
1.76
0.1220
DeepResearch-Claude
17
560
249
2.06
0.1220
Crase -Llama-PPR
5
235
104
0.37
0.3659
Table 4: Retrieval utility and interaction cost on ICLR. We report median per-query resource consumption together with recall@ 50 . Crase achieves substantially higher retrieval recall while requiring fewer external interactions and lower inference cost. ↓ indicates lower is better and ↑ indicates higher is better.
Measure
Score
Crase –human agreement
84.0%
Retained-edge precision
88.0%
Pruned-edge correctness
80.0%
Unanimous human agreement
68.0%
Fleiss’ κ
0.71
Table 5: Human validation of evidence-graph decisions. Three domain experts independently assess 50 citation edges while being blinded to Crase ’s decision. Crase ’s retain/prune decisions are compared against the majority human judgment.
Model
MSciNLI
SciNLI
Qwen2.5-1.5B-Instruct
0.75
0.77
Qwen2.5-3B-Instruct
0.80
0.83
Table 6: F1 score on claim entailment. Best per dataset in bold . Qwen2.5-3B consistently outperforms Qwen2.5-1.5B on both datasets.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Category
#
Category
#
cs.LG
135,116
cs.NE
6,972
cs.CV
150,055
cs.MA
3,890
cs.AI
36,148
stat.ML
18,549
cs.CL
82,273
cs.IR
14,081
Appendix
Table 7: Per-category paper counts in C . Papers with multiple tags are counted once per category.
Setting
Value
Base model
Qwen2.5-3B-Instruct
Training stage
Supervised fine-tuning (SFT)
Fine-tuning type
Full parameters
Optimization backend
DeepSpeed ZeRO-3
Prompt template
qwen
Max sequence length
2048
Appendix
Table 8: Fine-tuning configuration for the Crase entailment model.
Graph
R@50
∣V∣
Entailment calls
1-hop
0.331
642
1,286
1.5-hop
0.372
642
3,947
2-hop
0.386
4,781
28,614
Appendix
Table 9: Effect of exploration radius on ICLR. The 1-hop and 1.5-hop constructions contain the same nodes; 1.5-hop additionally retains relations among retrieved neighbors.
m
R@50
Median ∣V∣
Relative cost
5
0.326
371
0.61×
10
0.372
655
1.00×
15
0.381
911
1.42×
20
0.386
1,174
1.83×
Appendix
Table 10: Effect of the seed budget on ICLR.
ID
Type
Extracted claim
Source phrase
Paper 1: A Review of Cooperation in Multi-agent Learning
C1
CONTEXTUAL
Cooperation in multi-agent learning intersects with game theory, economics, social sciences, and evolutionary biology.
“Cooperation in multi-agent learning (MAL) is a topic at the intersection of numerous disciplines”
C2
CONTEXTUAL
Research in multi-agent learning aims to understand how agents can coordinate effectively when goals are aligned.
“Research in this area aims to understand both how agents can coordinate effectively when goals are aligned”
C3
CONTEXTUAL
Research in multi-agent learning also aims to understand how agents may cooperate in settings with potential for conflict.
“how they may cooperate in settings where gains from working together are possible but possibilities for conflict abound”
C4
METHODOLOGICAL
The paper provides an overview of fundamental concepts, problem settings, and algorithms in multi-agent learning.
“we provide an overview of the fundamental concepts, problem settings and algorithms of multi-agent learning”
C5
CONTEXTUAL
The paper discusses open challenges in multi-agent learning to inspire new research avenues.
“Finally we discuss open challenges in the field with the aim of inspiring new avenues for research”
Appendix
Table 11: Outputs of the claim-extraction stage. Each paper is represented by a small set of atomic, typed claims together with the source phrase from which the claim was extracted. These source anchors make the intermediate representation directly inspectable before claim-level entailment is computed.
Figure 2: Recall@ K (left) and MAP@ K (right) under seed corruption. Lexically similar distractors degrade retrieval gracefully— Crase retains ∼56% of clean Recall@50 and ∼81% of clean MAP@50, because claim-level pruning suppresses the unsupported edges the distractor introduces. Random irrelevant seeds collapse to zero: an off-manifold seed induces a neighborhood disconnected from the ground truth, which Crase does not attempt to recover through further search.
Stage
Model contribution
Structural control
What can be inspected?
Planning
Generates focused sub-queries
Exactly L external search calls
Research plan and retrieved seeds
Graph construction
Extracts atomic claims
Candidate graph fixed after seed construction
Nodes, citation edges, extracted claims
Evidence scoring
Predicts local claim support
Each citation edge scored independently
Entailment matrix and Δ(u,v)
Pruning
Provides support scores
Fixed threshold determines edge fate
Why each edge is retained or removed
Ranking
No new evidence is generated
Deterministic PPR/SALSA over the weighted graph
Edge weights and final ranking scores
Stopping
None
Graph scoring completes and ranking converges
Explicit stopping condition
Appendix
Table 12: Where does model autonomy remain in Crase ? Model judgments are restricted to local, inspectable operations, while exploration, pruning, ranking, and termination remain structurally constrained after seed construction.
Seed paper
Fate
ρ1 “faithful explicit helpfulness measurement”
A resource theory of quantum memories and their faithful verification with minimal assumptions
✗
Pretty good measurement for bosonic Gaussian ensembles
✗
Topology-faithful nonparametric estimation and tracking of bulk interface networks
✗
ρ2 “human explanations language models”
Sparse Autoencoders Find Highly Interpretable Features in Language Models
✓
Appendix
Table 13: The seed set is noisy by design,and it does not matter. The complete seed set S ( ∣S∣=29 ) for the query of Trace 1, grouped by contributing sub-query ρℓ . ✓ = seed survives Stage 2 claim-level pruning and enters the ranking walk; ✗ = every incident edge receives coverage affinity Δ=0 and the seed is removed from the evidence graph. Keyword search over a 500K-paper corpus inevitably retrieves lexical-match artifacts (quantum measurement, galaxy catalogs, retail image classification); in a search-and-synthesis loop these would enter the context of every subsequent step, whereas in Crase the evidence graph , not the search engine , enforces relevance.
Deep research agents have achieved remarkable progress on complex information seeking tasks. Even long ReAct style rollouts explore only a single trajectory, while recent state of the art systems scale inference time compute via parallel search and aggregation. Yet deep research answers are composed of complementary pieces of evidence, which parallel rollouts often duplicate rather than complete, yielding diminishing returns while pushing the aggregation context toward the model's limit. We propose Argus, an agentic system in which a Searcher and a Navigator cooperate to treat deep research as assembling a jigsaw from complementary evidence pieces, rather than brute forcing the whole answer in parallel. The Searcher collects evidence traces for a given sub-query through ReAct-style interaction. The Navigator maintains a shared evidence graph, verifying which pieces are still missing, dispatching Searchers to gather them, and reasoning over the completed graph to produce a source-traced final answer. We train the Navigator with reinforcement learning to verify, dispatch, and synthesize, while independently training the Searcher to remain a standard ReAct agent. The resulting Navigator supports rollouts with a single Searcher or many in parallel without retraining. With both Searcher and Navigator built on a 35B-A3B MoE backbone, Argus gains 5.5 points with a single Searcher and 12.7 points with 8 parallel Searchers, averaged over eight benchmarks. With 64 Searchers it reaches 86.2 on BrowseComp, surpassing every proprietary agent we benchmark, while the Navigator's reasoning context stays under 21.5K tokens.
Deep research agents are often trained on expensive, environment-grounded tool-use trajectories that require repeated retrieval, document inspection, and report evaluation. We introduce Deep Research Pretraining (DRP), an offline framework that derives predictive navigation supervision from naturally occurring evidence structures. Given a citation-bearing or hyperlinked passage, DRP constructs a proxy research objective, recovers linked evidence and graph-related alternatives, and converts them into search-open-write trajectories. This teaches models what to search for, which documents to inspect, and how to synthesize evidence, without a live retrieval environment or executed policy rollout. We instantiate DRP on scholarly citation graphs (DRP-Paper) and Wikipedia hyperlinks (DRP-Web), continually pretrain separate Qwen3-14B-Base models on 1B tokens, and fine-tune them on controlled fractions of 13K agent trajectories. Across five independently sampled subsets at each low-data budget, both variants consistently outperform matched no-DRP models on DeepResearch Bench. With one quarter of the SFT data, DRP-Web even surpasses a fixed no-DRP full-data checkpoint, with gains transferring to ResearchQA, WebWalkerQA, and SimpleQA. Starting from matched low-data SFT checkpoints, the DRP-Web advantage also persists through subsequent agentic RL. Source-matched and evidence-mismatch controls indicate that these improvements arise from evidence-conditioned navigation rather than domain exposure or agent-format imitation. DRP thus provides a promising complementary approach to trajectory-based agent training.
Academic paper search is a core step in scientific research, and LLM-based search agents are emerging as a promising paradigm for iterative, intent-driven literature exploration. However, existing benchmarks offer limited coverage of research intents and insufficient support for scalable answer construction and reproducible multi-turn evaluation. We propose ScholarQuest, a taxonomy-guided benchmark constructed from over 1,000 computer science topics and organized around four research intents. To construct answer sets at scale, we combine multi-source retrieval, citation expansion, and multi-stage relevance adjudication. We further build ScholarBase, a million-scale literature environment with unified retrieval and citation-expansion interfaces, together with a shared multi-turn search harness. We benchmark complete literature search systems and compare language models within this harness, evaluating retrieval coverage, search efficiency, and robustness across research intents. Results show that agentic methods outperform single-shot retrieval baselines, yet the best-performing search system achieves only 0.312 Recall@100 and 0.352 Recall@All. Comparisons under the shared harness reveal intent-dependent model strengths, with higher Planner token consumption not consistently yielding higher recall. We release the benchmark, retrieval backend, and evaluation harness for reproducible research
Tingyue Pan, Mingyue Cheng, Daoyu Wang +4
State Key Lab of Cognitive Intelligence, University of Science and Technology of China