Finding training data in million-scale machine learning repositories is a Web search problem where text relevance is not enough. A query couples a semantic training goal with constraints such as task, license, and training stage, and the metadata that would verify them are often missing. Text retrieval ranks dataset cards by similarity but cannot enforce hard constraints or follow the relations that reach a sparsely described dataset. Free-form tool-calling agents are more flexible, but the same model that picks a dataset also declares it a match, and nothing checks that the query's constraints are actually met. Neither approach establishes that the data it returns will improve model training. To address it, we propose AgenticDS, a contract-governed agentic framework in which every language model decision instantiates a typed object before deterministic code consumes it. The framework compiles a query into a typed search plan, executes it through admissible deterministic operators over hybrid retrieval and bounded graph expansion, and verifies the provenance-grounded evidence before accepting a ranking or emitting a typed diagnosis that permits only local repairs. On HuDS-Bench, a new benchmark of 200 queries with 24,908 graded judgments pooled over 15 systems, AgenticDS improves NDCG@10 by 25.7% over the strongest dense--sparse hybrid and by 18.9% over a commercial LLM agent. A controlled search-to-train study further shows that the datasets it retrieves give the best downstream accuracy in mathematics, code, and medicine. Code and data are publicly available at https://github.com/nju-websoft/AgenticDS.
Figures & tables
Figure 1. Overview of AgenticDS. A Planner constructs complementary query branches. An Engineer allocates bounded resources and deterministically executes typed retrieval, directional expansion, branch fusion, and Critic-pool construction. The Critic independently scores evidence and diagnoses local repairs.
Query type
Count
Primary grounding
Typical requirement
General
90
Metadata + task tags
Discover data satisfying task, language, modality, or scale constraints
Fine-tuning
40
Metadata + training stage
Find instruction, dialogue, preference, or domain-adaptation data
Paper reference
40
Paper/model relations
Locate data associated with a specific paper or training lineage
Entity neighbor
30
Dataset/collection relations
Find datasets reachable through multi-hop collection, model, or task paths
Table 1. Composition of HuDS-Bench.
NDCG
MAP
Method
@5
@10
@20
@5
@10
@20
Platform search
GoogleDS
.0301 †
.0217 †
.0165 †
.0262 †
.0146 †
.0091 †
HFFullText
.2786 †
.2438 †
.2100 †
.2989 †
.2301 †
.1661 †
Sparse
BM25
.4719 †
.4717 †
.4797 †
.5912 †
.5575 †
.5252 †
Table 2. Retrieval quality on HuDS-Bench. Bold marks the best system and underline the second best. † marks a cell where AgenticDS (Qwen3-Emb) is significantly higher than that system ( p<0.05 ).
Figure 2. Search-pipeline ablation. Each stem is a configuration’s distance from the full system (vertical line). Lollipop chart of ten ablated configurations, each a horizontal stem running back to a vertical reference line at the full system's 0.6148 NDCG@10, with single-module removals in blue and two-module removals in orange, ranging from -0.017 for sample evidence to -0.156 for removing both the adaptive budget and the Critic.
Math (EM)
Code (pass@1)
Medicine (acc.)
Acquisition method
GSM8K
MATH
AMC23
OCWCourses
Avg
HumanEval
BigCodeBench
Avg
MedQA
MMLU-Med
Avg
All
No fine-tuning (zero-shot)
54.54
42.91
27.50
16.42
35.34
55.08
29.04
42.06
51.40
54.84
53.12
43.51
Qwen3-Emb+BM25
79.78
52.73
34.17
17.16
45.96
66.06
32.37
49.21
49.36
58.82
54.09
49.75
monoT5 (OOD)+BM25
69.34
48.87
29.17
17.15
41.13
57.93
31.78
44.85
52.58
63.66
58.12
48.03
Claude Code
77.05
45.25
30.00
17.16
42.37
64.02
29.97
47.00
55.77
61.25
58.51
49.29
Codex
65.56
50.82
30.83
17.16
41.09
62.60
29.73
46.17
53.44
61.86
57.65
48.30
Table 3. Training utility on eight benchmarks from three domains. Each block averages the three base models. Avg is the within-domain mean and All the mean of the three domain averages. Bold marks the best system.
Search
Training
Sign
Component removed
Δ NDCG@10
mean
s.e.
agrees
Critic
− 0.149
− 1.85
1.36
yes
Planner
− 0.086
− 0.60
0.92
yes
KG expansion
− 0.084
+ 0.10
0.56
no
Feedback loop
− 0.036
—
—
—
Adaptive budget
− 0.031
+ 0.43
0.85
no
Table 4. Search-end and training-end ablation compared. Training entries are means over the nine (domain, base model) blocks, and s.e. is the standard error of that mean.
Figure 3. Cost over the 200 HuDS-Bench queries. (a) billed tokens by type, (b) cost in USD against mean latency, and (c) AgenticDS token and latency share by stage. Three panels. Left: horizontal stacked bars of billed tokens for AgenticDS (12.3M), Claude Code (111.4M, 87 percent cached) and Codex (280.9M, 92 percent cached). Middle: a scatter of cost in US dollars against mean seconds per query, with AgenticDS cheapest at 19.86 dollars and Codex both slowest and most expensive at 112.40 dollars. Right: two stacked strips over the same five pipeline stages, one for token share and one for latency share, showing the Critic at 83 percent of tokens but 28 percent of time, the two Planner calls at 13 percent of tokens but 50 percent of time, and the Engineer's deterministic execution at 0 percent of tokens and 7 percent of time.
Failure code/severity; affected intents; evidence and confidence; permitted local plan repair within protocol limits.
Appendix
Table 5. Typed objects exchanged between the three agents.
Query type
Pairs
Label 0
Label 1
Label 2
General
10,730
2,990
6,388
1,352
Fine-tuning
4,968
1,134
3,210
624
Paper reference
5,049
4,659
1
389
Entity neighbor
4,161
3,842
164
155
Total
24,908
12,625
9,763
2,520
Appendix
Table 6. Graded relevance judgments by query type.
Answer reachable from
Query type
Queries
Δ NDCG@10
Card text
General
90
+ 0.075
Fine-tuning
40
+ 0.105
Relations only
Paper reference
40
+ 0.206
Entity neighbor
30
+ 0.197
Appendix
Table 7. NDCG@10 gain over the Qwen3-Emb+BM25 hybrid by query type. All four survive Holm correction.
Backbone
NDCG@5
NDCG@10
MAP@10
GLM-5.2
0.6297
0.6148
0.6973
Qwen3.5-35B-A3B
0.5417
0.5244
0.5702
GLM-4-9B
0.4809
0.4706
0.5750
Appendix
Table 8. Sensitivity to the agent backbone.
Domain
Benchmark
Items
Metric
Math
GSM8K
1,319
exact match
MATH
5,000
exact match
AMC23
40
exact match
OCWCourses
272
exact match
Code
HumanEval
164
pass@1
BigCodeBench
1,140
pass@1
Appendix
Table 9. Evaluation split of every benchmark.
Figure 4. Component ablation under the search-to-train protocol, all base models and domains. Each block is the change in domain accuracy when a component is removed. The full system’s score is printed beneath each column. Heatmap of five ablation rows by nine base-model-and-domain columns, coloured on a blue-to-red diverging scale centred at zero, with the numeric delta printed in every block and a separated nine-block mean column at the right.
Domain
Model
Qwen3-Emb+BM25
monoT5 (OOD)
Claude Code
Codex
AgenticDS
Math
Qwen
49.63
46.40
39.60
45.84
56.90
Llama
29.34
21.79
29.20
19.45
32.64
Olmo
58.90
55.21
58.30
57.98
59.93
Code
Qwen
56.58
52.79
55.48
53.56
56.80
Llama
36.86
36.69
34.32
32.23
37.55
Olmo
54.20
45.09
51.19
52.71
54.50
Appendix
Table 10. Per-model domain scores. Bold: best per row.
Billed tokens (M)
Cost (USD)
System
s/query
in, unc.
in, cached
output
total
per query
AgenticDS (Qwen3-Emb)
150.5
10.6
—
1.7
19.86
0.099
AgenticDS (bge-m3)
212.7
13.1
—
2.1
24.35
0.122
Claude Code
139.3
13.1
97.2
1.0
48.78
0.244
Codex
868.5
20.0
258.1
2.8
112.40
0.562
Appendix
Table 11. End-to-end cost of the 200-query evaluation. A dash marks no cached-prompt reuse, so AgenticDS is billed entirely at the uncached rate and its cost is an upper bound.