Training environments remain a major bottleneck for agents in data-driven scientific discovery. Existing collections of executable and verifiable training environments are small, and their construction remains difficult to scale. To address this limitation, we introduce Poseidon, an automated pipeline for generating diverse, verifiable training environments from scientific code repositories at scale. From the same repository budget, Poseidon extracts nearly 10 times as many training environments as a prior approach by decomposing scientific workflows into four complementary task types: Reproduction, Subflow, Completion, and Correction, where each type targets a distinct capability for scientific research. A recursive improvement loop further refines failed candidates and recovers otherwise discarded environments. Using Poseidon, we construct Pearl, a collection of 97,659 executable and verifiable environments from 7,110 public repositories spanning 10 scientific disciplines. To the best of our knowledge, Pearl is the largest and most broadly multidisciplinary collection of scientific-agent training environments to date. Qwen3.5 models (9B, 27B, and 35B-A3B) trained on Pearl achieve consistent gains across six challenging scientific-agent benchmarks. Overall, the scale and diversity of Pearl offer opportunities for training scientific discovery agents.
Figures & tables
Figure 1: Scaling scientific-agent training environments and improving downstream performance. Left: Cumulative number of environments synthesized as the number of source repositories increases. Under a matched repository budget of 306 repositories, Poseidon produces 5,370 environments, compared with 575 produced by D3-Gym—a 9.3 × higher yield. Right: Success rates of Qwen3.5-27B before and after training on Pearl across six scientific-discovery benchmarks.
Figure 2: An example task from Pearl and the corresponding scientific-agent workflow.
Figure 3: Overview of Poseidon
Figure 4: Distribution of the collected scientific repositories across disciplines.
Benchmark
Size
Disciplines
Open-ended
Evaluation
SciCode
65
Phys., Math., Mat., Chem., Bio.
✗
Rule-based
BioCoder
154
Bioinfo.
✗
Rule-based
MedAgentBench
300
Clin.M̃ed.
✗
Rule-based
BixBench
205
Bioinfo.
✓
LLM-as-Judge
BioMysteryBench
90
Bioinfo.
✓
LLM-as-Judge
ScienceAgentBench
102
Psych., Bioinfo., GIS, Comp.C̃hem.
✓
LLM-as-Judge
Table 1: Comparison of scientific agent benchmarks. We summarize benchmark scale, covered disciplines, task format, and evaluation protocol.
Table 2: Overall comparison across scientific discovery benchmarks. MA-bench and SA-bench denote MedAgentBench and ScienceAgentBench, respectively. Scores are success rates (%), and Avg is the mean over all benchmark columns. Within each size group (9B, A3B, 27B), Pearl is trained on the corresponding Qwen3.5 backbone;
Difficulty
Success (%)
Avg. Steps
D1
94.0
6.2
D2
86.0
8.1
D3
57.0
15.1
D4
41.0
16.4
Table 3: Solvability across environment difficulty levels
Model
# Traj.
BioCoder
SciCode
MA-Bench
SA-Bench
BixBench
BioMystery
Avg
Main
Sub
MCQ
Open
Qwen3.5-27B
–
57.1
12.3
42.7
75.7
34.3
61.5
34.1
36.7
44.3
+ MedAgentGym
2,137
60.4 ↑ 3.3
13.8 ↑ 1.5
42.7
79.0 ↑ 3.3
36.2 ↑ 1.9
53.4 ↓ 8.1
28.4 ↓ 5.7
32.2 ↓ 4.5
43.3 ↓ 1.0
+ D3-Gym
2,153
59.1 ↑ 2.0
10.8 ↓ 1.5
38.9 ↓ 3.8
73.7 ↓ 2.0
36.2 ↑ 1.9
62.0 ↑ 0.5
25.9 ↓ 8.2
28.9 ↓ 7.8
41.9 ↓ 2.4
+ Pearl (2k)
2,000
62.3 ↑ 5.2
13.8 ↑ 1.5
43.4 ↑ 0.7
77.7 ↑ 2.0
38.1 ↑ 3.8
63.9 ↑ 2.4
37.6 ↑ 3.5
34.4 ↓ 2.3
46.4 ↑ 2.1
Table 4: Data-scale-controlled comparison against scientific-agent training baselines. We subsample 2,000 training trajectories from Pearl to obtain a data budget comparable to baselines.
Figure 5: Scaling repository diversity and environment difficulty under a fixed trajectory budget. The top row varies repository coverage, whereas the bottom row varies the average difficulty of the training environments. The dashed line marks the untrained Qwen3.5-35B-A3B backbone.
Model Variant
BioCoder
MedAgentBench
BixBench (MCQ)
ScienceAgentBench
Avg.
Qwen3.5-35B (+PEARL)
60.4
78.3
69.3
47.1
63.8
w/o Reproduction
54.5 ↓ 5.9
70.7 ↓ 7.6
63.9 ↓ 5.4
44.1 ↓ 3.0
58.3 ↓ 5.5
w/o Subflow
55.8 ↓ 4.6
73.0 ↓ 5.3
60.5 ↓ 8.8
41.2 ↓ 5.9
57.6 ↓ 6.2
w/o Completion
59.7 ↓ 0.7
71.7 ↓ 6.6
62.0 ↓ 7.3
43.1 ↓ 4.0
59.1 ↓ 4.7
w/o Correction
55.2 ↓ 5.2
72.0 ↓ 6.3
62.0 ↓ 7.3
46.1 ↓ 1.0
58.8 ↓ 5.0
Table 5: Ablation study on removing individual workflow-derived task types.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11
Dataset
Length
Diversity
Q. Len.
Resp. Len.
Iter.
Vendi ↑
Cent. Dist. ↑
MedAgentGym
293
2,393
2.3
43.57±1.55
0.4272±0.0032
D3-Gym
208
10,073
12.6
62.24±0.27
0.4415±0.0009
Pearl
912
14,262
13.1
92.03±0.62
0.4674±0.0017
Appendix
Table 6: Comparison of dataset length and semantic diversity.
Figure 8: Difficulty distribution across scientific discovery training environment.
Scientific code repositories encode decades of human knowledge in executable models, methods, and tools. Yet fragmented toolchains, implicit domain conventions, and specialized correctness criteria make this knowledge difficult to convert into reliable learning experience-a challenge we call the scientific experience bottleneck. We introduce ScienceIDE, infrastructure for turning the world's scientific code into programmable environments for scientific agents. Guided by expert-defined scientific cases and acceptance criteria, agents transform repositories into executable environments that support task generation, execution, and scientific verification. These environments provide a shared foundation for supervised fine-tuning, reinforcement learning, and evaluation. Using verified interaction trajectories, we train PhAI-IDE-72B, PhAI-IDE-9B, and PhAI-IDE-4B. The model family shows gains in held-out scientific-code repair and across selected general-purpose benchmarks in code, reasoning, and knowledge, providing evidence of positive transfer from scientific experience to broader capabilities. ScienceIDE lays the foundation for an integrated workspace for agent learning and scientific practice, making humanity's scientific software a shared substrate for developing scientific intelligence. Code: https://github.com/aitofound/ScienceIDE
Despite recent progress in language models and agents for scientific data-driven discovery, advancing their capabilities is held back by the absence of verifiable environments representing real-world scientific tasks. To fill this gap, we introduce D3-Gym, the first automatically constructed dataset with verifiable environments for scientific Data-Driven Discovery. D3-Gym comprises 565 tasks from 239 real scientific repositories across four disciplines, each with a natural language instruction, an executable environment with pre-installed dependencies, dataset previews, a reference solution, and an automatically synthesized evaluation script. Our evaluation scripts achieve 87.5% agreement with human-annotated gold standards and strong alignment in domain-specific evaluation logic. Training on trajectories sampled from D3-Gym yields consistent gains across Qwen3 models on ScienceAgentBench, boosting Qwen3-32B by 7.8 absolute points and shrinking the gap with strong proprietary models. We further illustrate, through case studies, how D3-Gym environments can serve as a testbed for studying agentic optimization loops such as Autoresearch on real scientific workflows. We open-source D3-Gym, its creation workflow, sampled trajectories, and training scripts at https://github.com/OSU-NLP-Group/D3-Gym.
Large language model agents have shown promising capabilities in data-driven scientific discovery tasks, where an agent interacts with an execution environment and produces a statistical claim. Long-horizon scientific analysis remains constrained by the lack of process supervised environments over real-world scientific data. This paper introduces SciDisco, a scalable framework for training Scientific Discovery agents in process-verifiable environments. SciThèque compiles hypotheses, datasets, hidden evidence graphs, and verifiers into task environments where analytical progress can be checked during interaction. DAG-grounded trajectory synthesis uses these environments to construct verifier-filtered multi-turn demonstrations. DiscoPO then uses the environment as the source of training signal, assigning turn-level credit to actions that produce verifiable analytical evidence. Experiments show that SciDisco-14B reaches state-of-the-art on hypothesis-driven scientific data analysis benchmarks.