LabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI Agents
Authors: Lei Liu, Yikun Zhang, Jialin Chen, Wanjia Zhao, Rex Ying, Wengong Jin, Hua Xu, James Zou, Tianyu Liu, Hongyu Zhao
Organizations: Department of Biostatistics, Yale University · Department of Computer Science, Northeastern University · EWSC Center, Broad Institute of MIT and Harvard · Department of Computer Science, Yale University · Department of Computer Science, Stanford University · Interdepartmental Program in Computational Biology and Bioinformatics, Yale University · Department of Biomedical Informatics and Data Science, Yale University · Department of Biomedical Data Science, Stanford University
Scientific research is a continuous process that emphasizes inheritance. Methods developed by predecessors are often expanded upon by new researchers to explore more novel and in-depth scientific questions. However, the change of lab staff, such as student graduation, leads to a lack of personnel capable of replicating methods. Methods that have been developed with significant effort and resources cannot be continued. To address these limitations, we propose LabAgent, a reproduce and discovery harness tailored for a lab's continuous work. LabAgent employs two mechanisms to guarantee that all skills can be executed and verified and to record the corrective methods and experiences, allowing for direct correction or avoidance of similar errors. We applied LabAgent to drug property prediction, biomedical problem analysis, protein variant effect prediction, and statistical genetics in life science domains. LabAgent ranks first over commercial generalist agents in every domain, and demonstrates accurate reproduction of a published figure. Overall, these results demonstrate that LabAgent can effectively integrate and reasonably expand laboratory knowledge.
Figures & tables
Figure 1: The LabAgent framework. (a) The framework of LabAgent. A target reaches the explore agent as a task configuration or as a paper with its repository, dataset, metric and budget. That agent searches the literature and code and deposits an executable skill in the library. The reproduce agent routes the target to one skill, loads its lessons, and works it through six roles from a machine probe to a failure analysis. It compares the result with the published value and returns verified metrics with the scripts, logs and configurations behind them, and an improved skill. The dashed path replays a verified skill. A persistent memory holds the skills, their evidence, the logs and the diagnoses that pair a failure with its fix, and every stage reads and writes it. (b) Downstream applications. Five settings of the Results, from drug property prediction on public leaderboards to fine-mapping with no released figure code. (c) How a skill is made. The agent reads a repository when the target names one and searches for the code when it does not. Both routes reach one draft, and a second model call checks it against the rubrics. Only a skill that runs end to end enters the library.
Figure 2: Reproducing published ADMET leaderboard values. (a)-(e) One representative dataset from each ADMET category, with the three top-ranked entries of that leaderboard. Grey bars give the published value and its standard deviation, and the coloured bars give what LabAgent, Claude Code and Codex obtained. (a) Absorption. (b) Distribution. (c) Metabolism. (d) Excretion. (e) Toxicity. (f) Rank of each of the four values on all 65 entries, with the count at each rank and the mean rank above the bar. (g) One reproduction trajectory over 67 steps, with the failures the agent met, the diagnoses it made and the fixes it applied. The run ends at an AUPRC of 0.455 against a published 0.474 ± 0.025. (h) Entries that fall inside the published standard deviation against cost per entry. (i) Cost per entry in US dollars.
Figure 3: Open biomedical data analysis under a process-level rubric. (a) Mean rubric score across the 50 public tasks for LabAgent, Claude Code, Codex and the arm with the explore agent removed. (b) Mean score against cost per task. (c) Per-task difference in score between LabAgent and each other arm, with the 50 tasks ranked by the size of the gap. A value above zero marks a task where LabAgent leads. (d) Mean score by task type for the three systems, on a scale from 0 to 1, with the number of tasks beside each type. (e) The same view for LabAgent and the arm with the explore agent removed. (f) The same scores as (d) on the rubric scale, with the value above each bar.
Figure 4: Open analysis across dependent decision points. (a) Problems solved outright, of 10. (b) Partial credit on each of the 10 problems. (c) Mean partial credit against cost per problem. (d)-(m) The ambient-state eQTL problem. (d)-(g) The 588 cells in the first two principal components, for the reference estimator and then for each system. Cells the panel calls activated carry its colour and the rest stay grey. (h)-(k) The same cells in a UMAP embedding, in the same order. (l) Ambient contamination for each of the 24 donors against its cis genotype dosage. The shaded band holds no donor and separates the two technical groups. (m) The estimated effect on CXCL10 in activated cells with its 95% confidence interval, for each system as submitted, with the technical group withheld from that system’s own pipeline, and with the group restored. The vertical line and the band around it give the published value and the tolerance the grader allows.
Figure 5: Variant effect prediction across 13 deep mutational scans. (a) Spearman correlation on each assay for the four arms. The dashed line above each group gives the best published value for that assay. (b) The 13 values for each arm, with the mean as a solid line and the median as a dashed line. (c) Mean Spearman correlation against cost per assay. (d)-(i) Six of the scanned proteins. The cartoon carries the mean agreement between the reproduction and the measurement over each secondary-structure element, on one scale shared by the six panels that runs from −0.35 to 0.70 , and grey marks residues the scan did not measure. Sticks mark the positions of highest agreement that lie together in the fold, from four to six in each protein, and the inset enlarges that group. A glycine carries no side chain and appears in no panel, so panels (f) and (g) each draw one stick fewer than the text names. (d) CARD11 CARD domain. (e) SARS-CoV-2 spike receptor-binding domain. (f) SRC kinase domain. (g) HLA-A. (h) Bacillus subtilis lipase A. (i) Photosystem I subunit PsaE.
Figure 6: Rebuilding a published simulation figure. Five fine-mapping methods across five heritability settings, in two causal architectures. (a)-(c) The architecture in which every causal variant is also an eQTL of a risk gene, at eight causal variants per locus. (d)-(f) The architecture that adds two causal variants outside the risk genes, at ten per locus. (a),(d) Power, the share of causal variants that at least one credible set captures. (b),(e) Coverage, the share of credible sets that hold a causal variant, with the nominal 0.95 as a dashed line. (c),(f) Average credible set size. Points are means over 50 replicates per cell against the 100 the article used, error bars give the standard error across those replicates, and points sit offset horizontally within each heritability so that overlapping intervals stay visible. The SuSiE and SuSiE2 lines are the agent’s own implementations of the published algorithms, and the fastPAINTOR line is the supplied PAINTOR binary. The mvSuSiE and flashfm lines are surrogates the agent wrote in place of the published packages, with parameters it chose to match the reported behaviour of those methods, and we show them for completeness and not as reproductions. PAINTOR ran with a maximum of two enumerated causal variants. That bounds its power at 0.250 and 0.200.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Name
Role
Focus
methodology_expert
A computational methods expert
Algorithms, benchmark comparisons and methodological innovation.
domain_scientist
A domain scientist in the task area
Biological context, domain challenges and experimental validation.
ml_engineer
A machine learning engineer
Architectures, training strategy, preprocessing and scalability.
benchmarker
An evaluation specialist
Metrics, benchmark datasets, reproducibility and fair comparison.
tool_developer
A bioinformatics tool developer
Software, libraries, databases and practical workflows.
review_author
The author of a review article
Surveys, systematic reviews and tutorials over a whole area.
Appendix
Supplementary Tab. 1: The seven perspectives a sub-scout can search from. Each perspective fixes a role, a focus and a query style, and a supervisor picks one when it delegates a topic. Supplementary Data 1 gives each entry in full.
Repository
Leaderboards
NilavoBoral/Therapeutics-Data-Commons
21
BioSystemsUM/deepmol_case_studies
20
euclia/public-models
20
maplightrx/MapLight-TDC
18
partex-nv-opensource/tdc-submission
16
ersilia-os/zaira-chem-tdc-benchmark
9
Appendix
Supplementary Tab. 2: The repositories behind the drug property skills. Every public repository named by a leaderboard entry that a processor alone can run, with the number of the 22 leaderboards it serves. We read the benchmark group page on 2026-06-07. maplightrx/MapLight-TDC publishes a processor variant and an accelerator variant of the same model, and the registry records that the skill must build the former.
Supplementary Fig. 1: Every absorption leaderboard. One panel per leaderboard, in that leaderboard’s own metric. Nothing is normalised and nothing is averaged across panels. AUROC, AUPRC and Spearman improve upward and MAE improves downward. The grey bar gives the published value and its error bar gives the published standard deviation. Error bars on the other arms give the spread across the five training seeds of that run, which is the quantity the published deviation also measures. They do not measure agent variability. Each entry was attempted once, so how far a number would move on a second run is not captured here, and a short bar is not evidence of robustness.
Supplementary Fig. 2: Every distribution leaderboard. Drawn as in Fig. 1 .
Supplementary Fig. 3: Every metabolism leaderboard. Drawn as in Fig. 1 .
Supplementary Fig. 4: Every excretion leaderboard. Drawn as in Fig. 1 .
Supplementary Fig. 5: Every toxicity leaderboard. Drawn as in Fig. 1 .
Problem
Partial score
Verdict
wf_selection
100
✓
statgen_scrna_ambient_state_eqtl
100
✓
structural_inversion_subhap_expression_risk
56
crispri_casrx_transcript_vs_locus
33
txr1_mtb_causal_sv
25
multiparent_qtl_hmm_lmm
0
Appendix
Supplementary Tab. 3: GeneBench-Pro problem by problem. Partial credit and verdict for LabAgent on each of the 10 problems. The partial score runs from 0 to 100 over the quantities a problem asks for, and the verdict is the benchmark’s own binary test against its recoverable target. The mean of 31.4 sits between two groups rather than among the problems, since the agent either recovers a problem outright or returns nothing that scores.
Assay
Lab- Agent
Claude Code
Codex
Claude Science
Lib- rary
Pub- lished
AACC1_PSEAI_Dandage_2018
0.361
0.428
0.274
0.430
0.525
0.543
CAR11_HUMAN_Meitlis_2020_gof
0.407
0.333
0.259
0.326
0.408
0.408
ESTA_BACSU_Nutschel_2020
0.334
0.318
0.251
0.324
0.557
0.557
HSP82_YEAST_Cote-Hammarlof_2020_growth-H2O2
0.253
0.292
0.023
0.312
0.437
0.453
I6TAH8_I68A0_Doud_2015
0.294
0.134
0.274
0.300
0.401
0.401
OXDA_RHOTO_Vanella_2023_activity
0.352
0.390
0.122
0.328
0.429
0.466
Appendix
Supplementary Tab. 4: Every assay of the protein benchmark. Spearman correlation between the score an arm assigns a variant and the effect the assay measured for it, over all 13 assays and all four arms. The last two columns give the context we read from the leaderboard, which are the best score any skill in our own library reaches on that assay and the best score any published method reaches. An arm that returned no usable prediction is marked none rather than scored.
Assay
Posi- tions
Median RSA
Below 0.15
Median percentile
Chain length
CAR11_HUMAN_Meitlis_2020_gof
6
0.080
4
12
1154
ESTA_BACSU_Nutschel_2020
5
0.536
0
80
212
OXDA_RHOTO_Vanella_2023_activity
4
0.121
2
35
364
PSAE_PICP2_Tsuboyama_2023_1PSE
6
0.481
0
60
68
Q53Z42_HUMAN_McShan_2019_binding-TAPBPR
6
0.245
3
37
365
SPIKE_SARS2_Starr_2020_expression
6
0.008
5
8
1273
Appendix
Supplementary Tab. 5: Burial of every marked position. Relative solvent accessibility at each position the main text marks, for all eight assays that ship a predicted structure. We compute accessibility on the whole chain with hydrogens stripped and divide by the theoretical maximum for that residue ( 52 ) . The percentile places the median marked position among all residues of the same chain. Main-text Figure 5 draws six of these assays, so D-amino-acid oxidase and HIV-1 Tat appear only here.
Arch.
h2
Method
Power
Coverage
Size
Sets
a
0.02
SuSiE
0.3325
0.9852
6.215
135
a
0.02
SuSiE2
0.4850
0.9949
2.128
195
a
0.02
mvSuSiE
0.4450
0.9780
1.665
182
a
0.02
flashfm
0.4775
0.9745
4.908
196
a
0.02
fastPAINTOR
0.2550
0.6711
3.178
152
a
0.04
SuSiE
0.5125
0.9807
5.870
207
Appendix
Supplementary Tab. 6: The reproduced fine-mapping grid. Every cell behind main-text Figure 6, one row per causal architecture, heritability and method. Power is the share of causal variants at least one credible set captures, coverage the share of credible sets holding a causal variant against a nominal 0.95, size the average variants in a set, and sets how many the run found. Every row averages 50 replicates against the article’s 100.
Task
Turns
Cost (USD)
One command (s)
Whole run (s)
Agent default
40
5
120
none
TDC ADMET
100
15
1800
3600
BiomniBench-DA
80
10
3600
3600
GeneBench-Pro
150
25
7200
7200
ProteinGym
60
15
1800
14400
Fine-mapping figure
150
40
3600
none
Appendix
Supplementary Tab. 7: The ceilings on a single run. Four quantities bound a run and whichever binds first ends it. Turns and cost are enforced inside the agent loop, the command limit by the shell the agent acts through, and the run limit by the scheduler that launches it. The first row gives the defaults of the agent itself and every other row gives what the reported runs of that task used. We raise a ceiling only after a run has died against it. Seven earlier attempts at the fine-mapping figure had all ended against one, at 5.09, 5.02, 5.05, 6.77, 5.16, 20.31 and 25.08 US dollars, and the run we report raised both the turn and the cost ceiling. The fine-mapping run was launched on its own rather than through a scheduler array, so no run limit applied to it.
Claim
Group
Weight
Full credit requires
SuSiE2 leads on power
Relations
30
SuSiE2 above every other method in every cell of the grid
Power rises with heritability
Relations
5
Power increases at every step of the grid
The first architecture is easier
Relations
5
Scenario a above scenario b for every method
Gain over the alternatives
Quantities
10
15% to 40% over SuSiE and about 5% over mvSuSiE
Coverage
Quantities
15
SuSiE, SuSiE2 and flashfm near nominal, fastPAINTOR below it, mvSuSiE under it at low heritability
Credible set size
Quantities
15
SuSiE2 and mvSuSiE about a third smaller than SuSiE, flashfm barely smaller, fastPAINTOR smallest
Appendix
Supplementary Tab. 8: Rubric weights for the fine-mapping figure. A hundred points across eight claims in three groups. Forty go to the relations the figure asserts, forty to the quantities the article states beside it, and twenty to whether the work exists at all. Each claim returns a fraction of its own weight, and Supplementary Note 9 gives the test each one applies.
Dataset
Released by
Version used here
Terms
TDC ADMET Benchmark Group
Therapeutics Data Commons
Leaderboards read 2026-06-07
Open
ProteinGym
OATML-Markslab
Baseline repositories checked 2026-08-17
Open
BiomniBench-DA
Its own public release
Rubric and tasks as released
Open
GeneBench-Pro
Its own public release
Problems as released
Open
Chromosome 1 genotype panel
An earlier study of our own laboratory ( 45 )
93,246 common variants, 10,000 samples
Not redistributable
Appendix
Supplementary Tab. 9: Data availability. Every dataset the work uses, with the project that releases it, the version used here and the terms it comes under. The genotype panel was assembled from UK Biobank data for an earlier study of our own laboratory and we stage it unchanged. Individual-level UK Biobank data cannot be redistributed and reach approved researchers through the UK Biobank Access Management System, so the panel carries no accession number here.
Scientific discovery workflows rely heavily on lab notes, where researchers record observations, interpret uncertain results, and plan follow-up experiments. Unlike polished publications, lab notes preserve evolving scientific reasoning, tacit scientific knowledge, and author uncertainty, giving AI agents access to the process of science. However, most prior work on scientific text focuses on papers, protocols, or structured databases, leaving informal laboratory notes underexplored as inputs to AI agents for science. This gap matters because tacit knowledge in lab notes is not written as a clean set of instructions: validated observations, tentative judgments, and possible experimental next steps often appear in the same passage. If these signals are conflated, an AI agent may mistake uncertain scientific judgments for confirmed conclusions or executable actions. To this end, we present Notes2Skills, a two-stage framework for turning lab notebooks into verifiable skills for scientific AI agents while preserving the author's certainty. Across three wet-lab sessions and seven original downstream configurations, Notes2Skills is the only tested configuration that avoids both observed regime-level failures: over-acting on uncertain notes and missing firm directives. We show that certainty preservation is a key missing piece between lab notebooks and reliable agent skills, opening a path toward safer AI co-scientist systems.
Shi Liu, Jiayao Chen, Chengwei Qin +3
Southern University of Science and Technology, Shenzhen, China · Shenzhen Loop Area Institute, Shenzhen, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China +1
The replicability of papers is a cornerstone of scientific knowledge, ensuring the reliability of existing results and providing a base for further experiments. The act of replication typically illuminates details that were previously underspecified, and thus requires similar hypothesis-driven exploration to open-ended research. In this work, we develop Replica, a scalable task space for paper replication. To provide reward signal, we introduce an auto-generated rubric-based judge that has low noise and agrees with human assessment of replication quality. We post-train Faraday, a 27B-parameter "AI Scientist" agent that leverages coding agents as tools, surpassing the performance of Claude Opus 4.8 and GPT-5.5 on held-out replication tasks. Qualitative analysis of individual rollouts reveals that Faraday adopts a more scientifically-principled approach. We believe that our results provide a stepping stone towards AI agents capable of long-horizon scientific innovation without requiring complex harnesses.
Scientific research proceeds through iterative cycles of hypothesis generation, experiment design, execution, and revision. AI agents can automate parts of this process, but existing approaches typically follow a single research trajectory or coordinate through a central planner with fixed objectives. As a result, they struggle to sustain parallel exploration, adapt as experimental evidence changes, or preserve knowledge of failed directions over long-running experiments. We introduce AutoScientists, a decentralized team of AI agents for long-running computational scientific experimentation. Agents interpret a shared experimental state, self-organize into teams around promising hypotheses, critique proposals before using experimental compute, and share successes and failures to reduce redundant exploration. Under matched experimental budgets, AutoScientists improves over prior AI agents across biomedical machine learning, language-model training optimization, and protein fitness prediction. On BioML-Bench, spanning biomedical imaging, protein engineering, single-cell omics, and drug discovery, AutoScientists achieves a mean leaderboard percentile of 74.4% across 24 tasks, improving over the strongest AI agent by +8.33%. On GPT training optimization, AutoScientists reaches a target validation bits-per-byte 1.9x faster than Autoresearch and continues discovering improvements from a starting champion where the single-agent approach finds none (7 vs. 0 accepted improvements). On ProteinGym fitness prediction, AutoScientists discovers a method for ACE2-Spike binding that improves over the current state-of-the-art model by +12.5% in Spearman correlation. Applied without modification across all 217 ProteinGym assays, the same method improves over the prior state of the art by +6.5% (Spearman correlation).