cs.AISep 11, 2026
SaveLabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI Agents
Organizations: Department of Biostatistics, Yale University · Department of Computer Science, Northeastern University · EWSC Center, Broad Institute of MIT and Harvard · Department of Computer Science, Yale University ·
Abstract
Scientific research is a continuous process that emphasizes inheritance. Methods developed by predecessors are often expanded upon by new researchers to explore more novel and in-depth scientific questions. However, the change of lab staff, such as student graduation, leads to a lack of personnel capable of replicating methods. Methods that have been developed with significant effort and resources cannot be continued. To address these limitations, we propose LabAgent, a reproduce and discovery harness tailored for a lab's continuous work. LabAgent employs two mechanisms to guarantee that all skills can be executed and verified and to record the corrective methods and experiences, allowing for direct correction or avoidance of similar errors. We applied LabAgent to drug property prediction, biomedical problem analysis, protein variant effect prediction, and statistical genetics in life science domains. LabAgent ranks first over commercial generalist agents in every domain, and demonstrates accurate reproduction of a published figure. Overall, these results demonstrate that LabAgent can effectively integrate and reasonably expand laboratory knowledge.
Figures & tables
Figure 1: The LabAgent framework. (a) The framework of LabAgent. A target reaches the explore agent as a task configuration or as a paper with its repository, dataset, metric and budget. That agent searches the literature and code and deposits an executable skill in the library. The reproduce agent routes the target to one skill, loads its lessons, and works it through six roles from a machine probe to a failure analysis. It compares the result with the published value and returns verified metrics with the scripts, logs and configurations behind them, and an improved skill. The dashed path replays a verified skill. A persistent memory holds the skills, their evidence, the logs and the diagnoses that pair a failure with its fix, and every stage reads and writes it. (b) Downstream applications. Five settings of the Results, from drug property prediction on public leaderboards to fine-mapping with no released figure code. (c) How a skill is made. The agent reads a repository when the target names one and searches for the code when it does not. Both routes reach one draft, and a second model call checks it against the rubrics. Only a skill that runs end to end enters the library.
Figure 2: Reproducing published ADMET leaderboard values. (a)-(e) One representative dataset from each ADMET category, with the three top-ranked entries of that leaderboard. Grey bars give the published value and its standard deviation, and the coloured bars give what LabAgent, Claude Code and Codex obtained. (a) Absorption. (b) Distribution. (c) Metabolism. (d) Excretion. (e) Toxicity. (f) Rank of each of the four values on all 65 entries, with the count at each rank and the mean rank above the bar. (g) One reproduction trajectory over 67 steps, with the failures the agent met, the diagnoses it made and the fixes it applied. The run ends at an AUPRC of 0.455 against a published 0.474 0.025. (h) Entries that fall inside the published standard deviation against cost per entry. (i) Cost per entry in US dollars.
Figure 3: Open biomedical data analysis under a process-level rubric. (a) Mean rubric score across the 50 public tasks for LabAgent, Claude Code, Codex and the arm with the explore agent removed. (b) Mean score against cost per task. (c) Per-task difference in score between LabAgent and each other arm, with the 50 tasks ranked by the size of the gap. A value above zero marks a task where LabAgent leads. (d) Mean score by task type for the three systems, on a scale from 0 to 1, with the number of tasks beside each type. (e) The same view for LabAgent and the arm with the explore agent removed. (f) The same scores as (d) on the rubric scale, with the value above each bar.
Figure 4: Open analysis across dependent decision points. (a) Problems solved outright, of 10. (b) Partial credit on each of the 10 problems. (c) Mean partial credit against cost per problem. (d)-(m) The ambient-state eQTL problem. (d)-(g) The 588 cells in the first two principal components, for the reference estimator and then for each system. Cells the panel calls activated carry its colour and the rest stay grey. (h)-(k) The same cells in a UMAP embedding, in the same order. (l) Ambient contamination for each of the 24 donors against its cis genotype dosage. The shaded band holds no donor and separates the two technical groups. (m) The estimated effect on CXCL10 in activated cells with its 95% confidence interval, for each system as submitted, with the technical group withheld from that system’s own pipeline, and with the group restored. The vertical line and the band around it give the published value and the tolerance the grader allows.
Figure 5: Variant effect prediction across 13 deep mutational scans. (a) Spearman correlation on each assay for the four arms. The dashed line above each group gives the best published value for that assay. (b) The 13 values for each arm, with the mean as a solid line and the median as a dashed line. (c) Mean Spearman correlation against cost per assay. (d)-(i) Six of the scanned proteins. The cartoon carries the mean agreement between the reproduction and the measurement over each secondary-structure element, on one scale shared by the six panels that runs from to , and grey marks residues the scan did not measure. Sticks mark the positions of highest agreement that lie together in the fold, from four to six in each protein, and the inset enlarges that group. A glycine carries no side chain and appears in no panel, so panels (f) and (g) each draw one stick fewer than the text names. (d) CARD11 CARD domain. (e) SARS-CoV-2 spike receptor-binding domain. (f) SRC kinase domain. (g) HLA-A. (h) Bacillus subtilis lipase A. (i) Photosystem I subunit PsaE.
Figure 6: Rebuilding a published simulation figure. Five fine-mapping methods across five heritability settings, in two causal architectures. (a)-(c) The architecture in which every causal variant is also an eQTL of a risk gene, at eight causal variants per locus. (d)-(f) The architecture that adds two causal variants outside the risk genes, at ten per locus. (a),(d) Power, the share of causal variants that at least one credible set captures. (b),(e) Coverage, the share of credible sets that hold a causal variant, with the nominal 0.95 as a dashed line. (c),(f) Average credible set size. Points are means over 50 replicates per cell against the 100 the article used, error bars give the standard error across those replicates, and points sit offset horizontally within each heritability so that overlapping intervals stay visible. The SuSiE and SuSiE2 lines are the agent’s own implementations of the published algorithms, and the fastPAINTOR line is the supplied PAINTOR binary. The mvSuSiE and flashfm lines are surrogates the agent wrote in place of the published packages, with parameters it chose to match the reported behaviour of those methods, and we show them for completeness and not as reproductions. PAINTOR ran with a maximum of two enumerated causal variants. That bounds its power at 0.250 and 0.200.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Name | Role | Focus |
|---|---|---|
| methodology_expert | A computational methods expert | Algorithms, benchmark comparisons and methodological innovation. |
| domain_scientist | A domain scientist in the task area | Biological context, domain challenges and experimental validation. |
| ml_engineer | A machine learning engineer | Architectures, training strategy, preprocessing and scalability. |
| benchmarker | An evaluation specialist | Metrics, benchmark datasets, reproducibility and fair comparison. |
| tool_developer | A bioinformatics tool developer | Software, libraries, databases and practical workflows. |
| review_author | The author of a review article | Surveys, systematic reviews and tutorials over a whole area. |
Supplementary Tab. 1: The seven perspectives a sub-scout can search from. Each perspective fixes a role, a focus and a query style, and a supervisor picks one when it delegates a topic. Supplementary Data 1 gives each entry in full.
| Repository | Leaderboards |
|---|---|
| NilavoBoral/Therapeutics-Data-Commons | 21 |
| BioSystemsUM/deepmol_case_studies | 20 |
| euclia/public-models | 20 |
| maplightrx/MapLight-TDC | 18 |
| partex-nv-opensource/tdc-submission | 16 |
| ersilia-os/zaira-chem-tdc-benchmark | 9 |
Supplementary Tab. 2: The repositories behind the drug property skills. Every public repository named by a leaderboard entry that a processor alone can run, with the number of the 22 leaderboards it serves. We read the benchmark group page on 2026-06-07. maplightrx/MapLight-TDC publishes a processor variant and an accelerator variant of the same model, and the registry records that the skill must build the former.
Supplementary Fig. 1: Every absorption leaderboard. One panel per leaderboard, in that leaderboard’s own metric. Nothing is normalised and nothing is averaged across panels. AUROC, AUPRC and Spearman improve upward and MAE improves downward. The grey bar gives the published value and its error bar gives the published standard deviation. Error bars on the other arms give the spread across the five training seeds of that run, which is the quantity the published deviation also measures. They do not measure agent variability. Each entry was attempted once, so how far a number would move on a second run is not captured here, and a short bar is not evidence of robustness.
Supplementary Fig. 2: Every distribution leaderboard. Drawn as in Fig. 1 .
Supplementary Fig. 3: Every metabolism leaderboard. Drawn as in Fig. 1 .
Supplementary Fig. 4: Every excretion leaderboard. Drawn as in Fig. 1 .
Supplementary Fig. 5: Every toxicity leaderboard. Drawn as in Fig. 1 .
| Problem | Partial score | Verdict |
|---|---|---|
| wf_selection | 100 | ✓ |
| statgen_scrna_ambient_state_eqtl | 100 | ✓ |
| structural_inversion_subhap_expression_risk | 56 | |
| crispri_casrx_transcript_vs_locus | 33 | |
| txr1_mtb_causal_sv | 25 | |
| multiparent_qtl_hmm_lmm | 0 |
Supplementary Tab. 3: GeneBench-Pro problem by problem. Partial credit and verdict for LabAgent on each of the 10 problems. The partial score runs from 0 to 100 over the quantities a problem asks for, and the verdict is the benchmark’s own binary test against its recoverable target. The mean of 31.4 sits between two groups rather than among the problems, since the agent either recovers a problem outright or returns nothing that scores.
| Assay | Lab- Agent | Claude Code | Codex | Claude Science | Lib- rary | Pub- lished |
|---|---|---|---|---|---|---|
| AACC1_PSEAI_Dandage_2018 | 0.361 | 0.428 | 0.274 | 0.430 | 0.525 | 0.543 |
| CAR11_HUMAN_Meitlis_2020_gof | 0.407 | 0.333 | 0.259 | 0.326 | 0.408 | 0.408 |
| ESTA_BACSU_Nutschel_2020 | 0.334 | 0.318 | 0.251 | 0.324 | 0.557 | 0.557 |
| HSP82_YEAST_Cote-Hammarlof_2020_growth-H2O2 | 0.253 | 0.292 | 0.023 | 0.312 | 0.437 | 0.453 |
| I6TAH8_I68A0_Doud_2015 | 0.294 | 0.134 | 0.274 | 0.300 | 0.401 | 0.401 |
| OXDA_RHOTO_Vanella_2023_activity | 0.352 | 0.390 | 0.122 | 0.328 | 0.429 | 0.466 |
Supplementary Tab. 4: Every assay of the protein benchmark. Spearman correlation between the score an arm assigns a variant and the effect the assay measured for it, over all 13 assays and all four arms. The last two columns give the context we read from the leaderboard, which are the best score any skill in our own library reaches on that assay and the best score any published method reaches. An arm that returned no usable prediction is marked none rather than scored.
| Assay | Posi- tions | Median RSA | Below 0.15 | Median percentile | Chain length |
|---|---|---|---|---|---|
| CAR11_HUMAN_Meitlis_2020_gof | 6 | 0.080 | 4 | 12 | 1154 |
| ESTA_BACSU_Nutschel_2020 | 5 | 0.536 | 0 | 80 | 212 |
| OXDA_RHOTO_Vanella_2023_activity | 4 | 0.121 | 2 | 35 | 364 |
| PSAE_PICP2_Tsuboyama_2023_1PSE | 6 | 0.481 | 0 | 60 | 68 |
| Q53Z42_HUMAN_McShan_2019_binding-TAPBPR | 6 | 0.245 | 3 | 37 | 365 |
| SPIKE_SARS2_Starr_2020_expression | 6 | 0.008 | 5 | 8 | 1273 |
Supplementary Tab. 5: Burial of every marked position. Relative solvent accessibility at each position the main text marks, for all eight assays that ship a predicted structure. We compute accessibility on the whole chain with hydrogens stripped and divide by the theoretical maximum for that residue ( 52 ) . The percentile places the median marked position among all residues of the same chain. Main-text Figure 5 draws six of these assays, so D-amino-acid oxidase and HIV-1 Tat appear only here.
| Arch. | Method | Power | Coverage | Size | Sets | |
|---|---|---|---|---|---|---|
| a | 0.02 | SuSiE | 0.3325 | 0.9852 | 6.215 | 135 |
| a | 0.02 | SuSiE2 | 0.4850 | 0.9949 | 2.128 | 195 |
| a | 0.02 | mvSuSiE | 0.4450 | 0.9780 | 1.665 | 182 |
| a | 0.02 | flashfm | 0.4775 | 0.9745 | 4.908 | 196 |
| a | 0.02 | fastPAINTOR | 0.2550 | 0.6711 | 3.178 | 152 |
| a | 0.04 | SuSiE | 0.5125 | 0.9807 | 5.870 | 207 |
Supplementary Tab. 6: The reproduced fine-mapping grid. Every cell behind main-text Figure 6, one row per causal architecture, heritability and method. Power is the share of causal variants at least one credible set captures, coverage the share of credible sets holding a causal variant against a nominal 0.95, size the average variants in a set, and sets how many the run found. Every row averages 50 replicates against the article’s 100.
| Task | Turns | Cost (USD) | One command (s) | Whole run (s) |
|---|---|---|---|---|
| Agent default | 40 | 5 | 120 | none |
| TDC ADMET | 100 | 15 | 1800 | 3600 |
| BiomniBench-DA | 80 | 10 | 3600 | 3600 |
| GeneBench-Pro | 150 | 25 | 7200 | 7200 |
| ProteinGym | 60 | 15 | 1800 | 14400 |
| Fine-mapping figure | 150 | 40 | 3600 | none |
Supplementary Tab. 7: The ceilings on a single run. Four quantities bound a run and whichever binds first ends it. Turns and cost are enforced inside the agent loop, the command limit by the shell the agent acts through, and the run limit by the scheduler that launches it. The first row gives the defaults of the agent itself and every other row gives what the reported runs of that task used. We raise a ceiling only after a run has died against it. Seven earlier attempts at the fine-mapping figure had all ended against one, at 5.09, 5.02, 5.05, 6.77, 5.16, 20.31 and 25.08 US dollars, and the run we report raised both the turn and the cost ceiling. The fine-mapping run was launched on its own rather than through a scheduler array, so no run limit applied to it.
| Claim | Group | Weight | Full credit requires |
|---|---|---|---|
| SuSiE2 leads on power | Relations | 30 | SuSiE2 above every other method in every cell of the grid |
| Power rises with heritability | Relations | 5 | Power increases at every step of the grid |
| The first architecture is easier | Relations | 5 | Scenario a above scenario b for every method |
| Gain over the alternatives | Quantities | 10 | 15% to 40% over SuSiE and about 5% over mvSuSiE |
| Coverage | Quantities | 15 | SuSiE, SuSiE2 and flashfm near nominal, fastPAINTOR below it, mvSuSiE under it at low heritability |
| Credible set size | Quantities | 15 | SuSiE2 and mvSuSiE about a third smaller than SuSiE, flashfm barely smaller, fastPAINTOR smallest |
Supplementary Tab. 8: Rubric weights for the fine-mapping figure. A hundred points across eight claims in three groups. Forty go to the relations the figure asserts, forty to the quantities the article states beside it, and twenty to whether the work exists at all. Each claim returns a fraction of its own weight, and Supplementary Note 9 gives the test each one applies.
| Dataset | Released by | Version used here | Terms |
|---|---|---|---|
| TDC ADMET Benchmark Group | Therapeutics Data Commons | Leaderboards read 2026-06-07 | Open |
| ProteinGym | OATML-Markslab | Baseline repositories checked 2026-08-17 | Open |
| BiomniBench-DA | Its own public release | Rubric and tasks as released | Open |
| GeneBench-Pro | Its own public release | Problems as released | Open |
| Chromosome 1 genotype panel | An earlier study of our own laboratory ( 45 ) | 93,246 common variants, 10,000 samples | Not redistributable |
Supplementary Tab. 9: Data availability. Every dataset the work uses, with the project that releases it, the version used here and the terms it comes under. The genotype panel was assembled from UK Biobank data for an earlier study of our own laboratory and we stage it unchanged. Individual-level UK Biobank data cannot be redistributed and reach approved researchers through the UK Biobank Access Management System, so the panel carries no accession number here.
More in this topic
LLM Agent Harnesses
LLM agent harnesses are the software and runtime systems that organize an agent’s tools, context, state, execution, and safeguards.
See all 387 papers