Training environments remain a major bottleneck for agents in data-driven scientific discovery. Existing collections of executable and verifiable training environments are small, and their construction remains difficult to scale. To address this limitation, we introduce Poseidon, an automated pipeline for generating diverse, verifiable training environments from scientific code repositories at scale. From the same repository budget, Poseidon extracts nearly 10 times as many training environments as a prior approach by decomposing scientific workflows into four complementary task types: Reproduction, Subflow, Completion, and Correction, where each type targets a distinct capability for scientific research. A recursive improvement loop further refines failed candidates and recovers otherwise discarded environments. Using Poseidon, we construct Pearl, a collection of 97,659 executable and verifiable environments from 7,110 public repositories spanning 10 scientific disciplines. To the best of our knowledge, Pearl is the largest and most broadly multidisciplinary collection of scientific-agent training environments to date. Qwen3.5 models (9B, 27B, and 35B-A3B) trained on Pearl achieve consistent gains across six challenging scientific-agent benchmarks. Overall, the scale and diversity of Pearl offer opportunities for training scientific discovery agents.
Figures & tables
Figure 1: Scaling scientific-agent training environments and improving downstream performance. Left: Cumulative number of environments synthesized as the number of source repositories increases. Under a matched repository budget of 306 repositories, Poseidon produces 5,370 environments, compared with 575 produced by D3-Gym—a 9.3 × higher yield. Right: Success rates of Qwen3.5-27B before and after training on Pearl across six scientific-discovery benchmarks.
Figure 2: An example task from Pearl and the corresponding scientific-agent workflow.
Figure 3: Overview of Poseidon
Figure 4: Distribution of the collected scientific repositories across disciplines.
Benchmark
Size
Disciplines
Open-ended
Evaluation
SciCode
65
Phys., Math., Mat., Chem., Bio.
✗
Rule-based
BioCoder
154
Bioinfo.
✗
Rule-based
MedAgentBench
300
Clin.M̃ed.
✗
Rule-based
BixBench
205
Bioinfo.
✓
LLM-as-Judge
BioMysteryBench
90
Bioinfo.
✓
LLM-as-Judge
ScienceAgentBench
102
Psych., Bioinfo., GIS, Comp.C̃hem.
✓
LLM-as-Judge
Table 1: Comparison of scientific agent benchmarks. We summarize benchmark scale, covered disciplines, task format, and evaluation protocol.
Table 2: Overall comparison across scientific discovery benchmarks. MA-bench and SA-bench denote MedAgentBench and ScienceAgentBench, respectively. Scores are success rates (%), and Avg is the mean over all benchmark columns. Within each size group (9B, A3B, 27B), Pearl is trained on the corresponding Qwen3.5 backbone;
Difficulty
Success (%)
Avg. Steps
D1
94.0
6.2
D2
86.0
8.1
D3
57.0
15.1
D4
41.0
16.4
Table 3: Solvability across environment difficulty levels
Model
# Traj.
BioCoder
SciCode
MA-Bench
SA-Bench
BixBench
BioMystery
Avg
Main
Sub
MCQ
Open
Qwen3.5-27B
–
57.1
12.3
42.7
75.7
34.3
61.5
34.1
36.7
44.3
+ MedAgentGym
2,137
60.4 ↑ 3.3
13.8 ↑ 1.5
42.7
79.0 ↑ 3.3
36.2 ↑ 1.9
53.4 ↓ 8.1
28.4 ↓ 5.7
32.2 ↓ 4.5
43.3 ↓ 1.0
+ D3-Gym
2,153
59.1 ↑ 2.0
10.8 ↓ 1.5
38.9 ↓ 3.8
73.7 ↓ 2.0
36.2 ↑ 1.9
62.0 ↑ 0.5
25.9 ↓ 8.2
28.9 ↓ 7.8
41.9 ↓ 2.4
+ Pearl (2k)
2,000
62.3 ↑ 5.2
13.8 ↑ 1.5
43.4 ↑ 0.7
77.7 ↑ 2.0
38.1 ↑ 3.8
63.9 ↑ 2.4
37.6 ↑ 3.5
34.4 ↓ 2.3
46.4 ↑ 2.1
Table 4: Data-scale-controlled comparison against scientific-agent training baselines. We subsample 2,000 training trajectories from Pearl to obtain a data budget comparable to baselines.
Figure 5: Scaling repository diversity and environment difficulty under a fixed trajectory budget. The top row varies repository coverage, whereas the bottom row varies the average difficulty of the training environments. The dashed line marks the untrained Qwen3.5-35B-A3B backbone.
Model Variant
BioCoder
MedAgentBench
BixBench (MCQ)
ScienceAgentBench
Avg.
Qwen3.5-35B (+PEARL)
60.4
78.3
69.3
47.1
63.8
w/o Reproduction
54.5 ↓ 5.9
70.7 ↓ 7.6
63.9 ↓ 5.4
44.1 ↓ 3.0
58.3 ↓ 5.5
w/o Subflow
55.8 ↓ 4.6
73.0 ↓ 5.3
60.5 ↓ 8.8
41.2 ↓ 5.9
57.6 ↓ 6.2
w/o Completion
59.7 ↓ 0.7
71.7 ↓ 6.6
62.0 ↓ 7.3
43.1 ↓ 4.0
59.1 ↓ 4.7
w/o Correction
55.2 ↓ 5.2
72.0 ↓ 6.3
62.0 ↓ 7.3
46.1 ↓ 1.0
58.8 ↓ 5.0
Table 5: Ablation study on removing individual workflow-derived task types.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11
Dataset
Length
Diversity
Q. Len.
Resp. Len.
Iter.
Vendi ↑
Cent. Dist. ↑
MedAgentGym
293
2,393
2.3
43.57±1.55
0.4272±0.0032
D3-Gym
208
10,073
12.6
62.24±0.27
0.4415±0.0009
Pearl
912
14,262
13.1
92.03±0.62
0.4674±0.0017
Appendix
Table 6: Comparison of dataset length and semantic diversity.
Figure 8: Difficulty distribution across scientific discovery training environment.