Large Language Model (LLM)-based agent systems are increasingly used for scientific tasks, yet their practical capability remains constrained by the narrow scope of manually curated tools they can invoke. Much scientific computational functionality already exists in open-source code repositories, but these resources remain difficult to standardize, operationalize, and invoke reliably for agent use. Here we present ToolRosella, a framework that automatically transforms heterogeneous scientific code repositories into standardized, agent-invocable tools. ToolRosella combines repository analysis, tool interface construction, execution testing, and iterative repair to address the problem of repository-to-tool standardization. Across 122 GitHub repositories spanning 35 subdisciplines in six domains, ToolRosella reaches a 61.5% repository conversion success rate after iterative repair, with a 4.4 speedup over human engineers. The resulting 1,580 callable tools support a downstream task success rate of 84.0% and improve performance when integrated into other agent frameworks, particularly on tasks whose required tools are absent from fixed, curated inventories.
Figures & tables
Figure 1: Overview of ToolRosella: a , ToolRosella’s strength lies in how to automatically transform heterogeneous scientific resources into callable tools, bridging existing repositories and agent execution. Unlike fixed, manually curated tool libraries, it focuses on repository-to-tool standardization for reusable tool construction. b , The pipeline for using ToolRosella to complete scientific tasks.
Figure 2: Overview of the ToolRosella ecosystem. Each node represents a GitHub repository automatically converted into standardized tool services, organized into five major scientific areas (Physical Sciences, Earth & Environmental Sciences, Biological Sciences, Health Sciences, and Scientific Community & Society) and Computer Science. Node groupings reflect sub-domain categorization within each area.
Figure 3: Automated tool conversion performance evaluation. a , Radial bar chart of average completion time per subdiscipline (in minutes) for ToolRosella, human experts, and the LLM-only baseline. b , Repository conversion success rates across 35 subdisciplines within six domains, comparing all three methods. c , Joint visualization of success rate versus completion time, where each point represents one subdiscipline. d , Repair-focused ablation showing cumulative improvement from successive rounds of the Review-Revise-Fix (RRF) mechanism across six domains. e , Error types and their resolution statistics after three RRF rounds.
Figure 4: Downstream task evaluation results. a , Execution success rate and verified success rate of ToolRosella and five baseline systems (SciToolAgent, ChemCrow, RepoMaster, OpenAgents, and OpenClaw) across the six domains. b , Verified task completion success rates across 35 subdisciplines spanning the six categories. Stars denote 23 out-of-distribution (OOD) subdisciplines that are not covered by curated baseline tool sets. c , Verified performance gains when integrating ToolRosella-converted tools into three existing agent frameworks (OpenAgents, RepoMaster, and OpenClaw), with percentage annotations indicating improvement per domain.
Figure 5: Multi-domain case studies of ToolRosella for scientific workflow assistance. The framework converts user queries into executable MCP-compatible toolchains, enabling coordinated orchestration of task-specific analytical modules. Case 1 illustrates a stroke-analysis workflow integrating unsupervised, predictive, and causal inference tools, together with feature-importance summaries and treatment-effect estimates. Case 2 illustrates sequence-analysis workflows combining validation, compositional analysis, and database-driven species prediction. Case 3 illustrates a perovskite material design-and-evaluation workflow spanning literature mining, candidate generation, virtual screening, and mechanistic interpretation.
Figure 6: Vulnerable vs. Secure workflow, illustrated with representative cases of (i) data exfiltration in biomedical pipelines, (ii) command injection in legacy tool invocation, and (iii) resource exhaustion in High-Performance Computing (HPC) environments.
Scientific code repositories encode decades of human knowledge in executable models, methods, and tools. Yet fragmented toolchains, implicit domain conventions, and specialized correctness criteria make this knowledge difficult to convert into reliable learning experience-a challenge we call the scientific experience bottleneck. We introduce ScienceIDE, infrastructure for turning the world's scientific code into programmable environments for scientific agents. Guided by expert-defined scientific cases and acceptance criteria, agents transform repositories into executable environments that support task generation, execution, and scientific verification. These environments provide a shared foundation for supervised fine-tuning, reinforcement learning, and evaluation. Using verified interaction trajectories, we train PhAI-IDE-72B, PhAI-IDE-9B, and PhAI-IDE-4B. The model family shows gains in held-out scientific-code repair and across selected general-purpose benchmarks in code, reasoning, and knowledge, providing evidence of positive transfer from scientific experience to broader capabilities. ScienceIDE lays the foundation for an integrated workspace for agent learning and scientific practice, making humanity's scientific software a shared substrate for developing scientific intelligence. Code: https://github.com/aitofound/ScienceIDE
Tool-integrated LLMs retrieve information, perform computations, and take real-world actions, but their reliability depends on both tool-use accuracy and intrinsic tool accuracy, including tool correctness, stability, and safety. While prior work primarily emphasizes tool use, intrinsic tool accuracy remains underexamined. We introduce OpenTools, a community-driven and maintainable toolbox for discovering, using, evaluating, and contributing open-source tools. OpenTools standardizes tool interfaces, converts documented Python functions into reviewable bundles, supports maintainer-triggered evaluation, and combines non-executing risk inspection with optional advisory LLM review. A public web demo allows users to run tools and agents, inspect evidence, contribute tests, and submit tools for maintainer review, while MCP enables controlled access from external applications. Experiments show that community-contributed, task-specific tools yield relative gains of 6% to 22% over an existing toolbox across multiple agent architectures, highlighting the importance of intrinsic tool accuracy.
Hy Dang, Quang Dao, Meng Jiang
University of Notre Dame · Rose-Hulman Institute of Technology
Training environments remain a major bottleneck for agents in data-driven scientific discovery. Existing collections of executable and verifiable training environments are small, and their construction remains difficult to scale. To address this limitation, we introduce Poseidon, an automated pipeline for generating diverse, verifiable training environments from scientific code repositories at scale. From the same repository budget, Poseidon extracts nearly 10 times as many training environments as a prior approach by decomposing scientific workflows into four complementary task types: Reproduction, Subflow, Completion, and Correction, where each type targets a distinct capability for scientific research. A recursive improvement loop further refines failed candidates and recovers otherwise discarded environments. Using Poseidon, we construct Pearl, a collection of 97,659 executable and verifiable environments from 7,110 public repositories spanning 10 scientific disciplines. To the best of our knowledge, Pearl is the largest and most broadly multidisciplinary collection of scientific-agent training environments to date. Qwen3.5 models (9B, 27B, and 35B-A3B) trained on Pearl achieve consistent gains across six challenging scientific-agent benchmarks. Overall, the scale and diversity of Pearl offer opportunities for training scientific discovery agents.
Qifan Zhang, Yuhan Li, Nuo Chen +8
The Hong Kong University of Science and Technology (Guangzhou) · Hunyuan Team, Tencent