Organizations: State Key Lab. of Chemo & Biosensing, Coll. of Comp. Sci. & Electron. Eng., Hunan University · State Key Laboratory for Novel Software Technology, Sch. of Comput. Sci., Nanjing University · The Chinese University of Hong Kong · Zhejiang University · Yale University · Wuhan University of Science and Technology · South China Normal University Affiliated High School · Southeast University · Stanford University
Scientific LLM agents have shown promise in literature reasoning, tool use, and experiment planning, but it remains unclear whether they can autonomously improve large, tightly coupled scientific machine-learning systems through executable code changes and computationally expensive validation. We study this question in protein folding, where progress requires coordinated architectural modifications, multi-objective evaluation, and domain-aware interpretation. We present AgentFold, a multi-agent framework that formulates folding-model development as a closed-loop search over executable code variants. Starting from ESMFold, AgentFold proposes hypotheses, implements and debugs code-level modifications, evaluates model variants, analyzes experimental outcomes, and stores both successful and failed interventions in structured memory. An MCTS-style policy allocates computational resources across high-scoring search branches. On an engineering-scale protein-folding codebase comprising more than 2,000 lines of code, AgentFold explores approximately 80 model variants using approximately 5,000 GPU-hours and 170 million LLM tokens. Under a matched computational budget, AgentFold improves the best lDDT by 7.5% over independent Codex proposals and outperforms a random-search control. Beyond model improvement, the resulting intervention traces reveal recurring empirical design patterns: stable gains tend to arise from early, soft, learnable priors and gated refinement, whereas direct geometric perturbations and geometry-conditioned feedback often destabilize training. The code and experimental resources are publicly available at https://github.com/lmqfly/AgentFold.
Figures & tables
Figure 1: AgentFold system overview. We cast model improvement as MCTS-style search over a code-variant tree, coupling an inner loop (sample → evolve → run → analyze) with a database-backed memory, and an outer periodic update that re-scores candidates to refine the search policy.
Figure 2: MCTS-style tree evolution. Each node is a sampled variant scored by average lDDT ( lddt_mean ). Color encodes performance (darker indicates higher lddt_mean ); gray marks low-scoring variants with lddt_mean<0.1 .
Method
Best lDDT
Top-5 lDDT
Best NWRS
Top-5 NWRS
AgentFold
0.285
0.267
0.526
0.516
Codex proposals
0.265
0.257
0.512
0.509
Random controller
0.260
0.242
0.510
0.506
Table 1: Matched comparison at 36 evaluations; Top-5 by NWRS.
Variant
NWRS ↑
bb_lddt ↑
lddt ↑
oligo_gdtts ↑
rmsd ↓
tm_score ↑
esmfold
0.500
0.644/0.651
0.232/0.220
0.564/0.570
7.380/5.358
0.648/0.693
esmfold_struct_enhanced_v4
+0.026
+0.009/+0.010
+0.053 / +0.059
+0.005/-0.003
+0.082/-0.038
+0.004/-0.012
esmfold_struct_local_context_v1
+0.020
+0.002/+0.006
+0.049 /+0.044
-0.001/-0.005
+0.176/ -0.129
+0.001/-0.012
esmfold_struct_dist_aware_v1
+0.018
+0.011 / +0.014
+0.027/+0.024
+0.011 / +0.020
-0.088/ -0.134
+0.011 /-0.009
esmfold_struct_enhanced_multiscale_v2
+0.017
+0.007/ +0.014
+0.045/ +0.049
+0.004/+0.006
+0.261/+0.338
+0.001/-0.019
esmfold_net_conformal_geometric_attention
+0.017
-0.006/-0.008
+0.043/+0.046
-0.006/ +0.011
-0.063/+0.240
-0.005/-0.001
Table 2: CAMEO2022 development-benchmark performance for representative variants. We show the top NWRS variants and variants used in later targeted analyses. The ESMFold row reports absolute mean/median values; other rows report deltas relative to ESMFold. Bold and underline mark the largest and second-largest favorable changes among displayed variants.
Variant
Aspect
Loop
Physical
Contact
loop lDDT ↑
loop bb-lDDT ↑
MolProbity ↓
Prec 12-24↑
F1 12-24↑
esmfold
Base
0.162
0.613
3.773
0.599
0.606
esmfold_struct_enhanced_v4
L/P/C
+0.060
+0.008
-0.157
+0.019
+0.010
esmfold_struct_local_context_v1
L
+0.063
+0.002
–
–
–
esmfold_struct_enhanced_v1_dup2
L/C
+0.031
+0.007
–
+0.020
+0.013
esmfold_struct_attn_frame_v1
L/P
+0.025
+0.001
-0.049
–
–
Table 3: Targeted evaluation summary by motivation aspect. The ESMFold row reports absolute means; other rows report changes relative to ESMFold. L/P/C denote loop-quality, physical-plausibility, and contact-modeling motivations. Bold indicates the largest improvement, and underline indicates the second largest.
Figure 3: Selected variant subtree used in the evolutionary analysis. Colors distinguish high-lDDT variants, collapse cases, and ancestor/context nodes; each node reports mean lDDT.
Figure 4: Robustness under repeated runs and deeper Folding Trunks. Matched repeated-run settings are used; RMSD is omitted due to its different scale. Error bars denote standard deviations.
Figure 5: Component ablation of esmfold_struct_enhanced_v4 . Each panel shows one metric; RMSD is omitted due to its different scale. Error bars denote standard deviations.
Figure 6: Loop-region case studies on four CAMEO targets. Superpositions of ground truth (green), ESMFold (magenta), and esmfold_struct_enhanced_v4 (cyan) are shown for 7wj0_A, 7vd7_B, 7ycv_A, and 8d08_D. Text in each panel reports loop lDDT, loop backbone lDDT, and loop RMSD for ESMFold and esmfold_struct_enhanced_v4 . esmfold_struct_enhanced_v4 improves loop placement and backbone alignment in most cases, while 7vd7_B illustrates a residual metric trade-off.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Main Goal
Specific Objective
Count
Global Improvement
pLDDT/LDDT metric
41
RMSD reduction
21
Loop Quality
Loop region prediction
42
Physical Plausibility
Regularization constraints
31
Torsion angle constraints
22
Long-range Contact
Long-range contact modeling
28
Appendix
Table 4: Motivation taxonomy of proposed variants. We group each variant’s stated goal into five high-level categories (global improvement, loop quality, physical plausibility, long-range contact, and long-sequence quality), and further refine each category by its specific objective. Categories are not mutually exclusive; counts indicate how many variant motivations fall into each objective.
Pattern
Representative variants
Evidence from stored reports
Outcome signal
P1: Bias before geometry
#28 dist_aware_v1 ; #47 local_context_v1
Reports describe these variants as injecting information into the initial pair representation or IPA logits before coordinates are produced. The reports attribute gains to making residue-pair relations more learnable without directly changing rigid-frame updates.
#28 improves backbone/global metrics; #47 achieves the largest loop-lDDT gain in Table 3 .
P1 failure contrast
#11 frame_reg_v4 ; #18 enhanced_frame_pred_v1
Reports note that fixed frame/torsion biases are added directly to BackboneUpdate or combined with attention biases after structural refinement is already coupled. The stored analyses describe collapse or destructive interaction despite plausible motivations.
Both variants have near-zero lDDT and high RMSD in the search logs, indicating failed folding behavior.
P2: Multiplicative refinement
#36 enhanced_v4 ; #48 adaptive_backbone_v1
Reports describe sigmoid or confidence-related scaling of backbone updates. These mechanisms modulate update magnitude rather than adding a fixed displacement, allowing uncertain regions to be dampened.
#36 is the highest-NWRS variant; #48 improves backbone lDDT and RMSD relative to ESMFold in the stored report.
P3: Avoid geometry-to-attention feedback
#18 enhanced_frame_pred ; #57 geom_alg_phys
Reports describe variants that feed structure/geometric features into attention or replace IPA with geometry-heavy attention modules. The reports emphasize that such feedback can introduce conflicting optimization signals when geometric features are immature or noisy.
#18 collapses; #57 is much larger but does not dominate compact variants and shows weaker all-atom/local performance than #36.
Hard geometric perturbation failure
#60 differential_geometry
The report states that curvature/torsion features are computed from sequence features and then fed back into the standard IPA path, while the intended geometric mechanism is not realized. The stored evaluation records a catastrophic failure.
bb-lDDT =0.015 , lDDT =0.000 , TM-score =0.091 in the stored test record.
Appendix
Table 5: Representative agent-report evidence used to derive the empirical pattern set P in Section 4.3.1 . The evidence is summarized from the stored intervention reports in my_tree_data_dup.json ; it is descriptive rather than causal proof.
Variant
Aspect
loop bb-lDDT ↑
loop lDDT ↑
loop RMSD ↓
esmfold
Base
0.613
0.162
5.433
esmfold_struct_enhanced_v4
L/P/C
+0.008
+0.060
+0.041
esmfold_struct_local_context_v1
L
+0.002
+0.063
+0.046
esmfold_struct_enhanced_v1_dup2
L/C
+0.007
+0.031
+0.022
esmfold_struct_attn_frame_v1
L/P
+0.001
+0.025
-0.007
esmfold_struct_enhanced_multiscale_v2
L/P/C
+0.002
+0.056
+0.129
Appendix
Table 6: Complete loop-region metrics for the motivation-aspect summary. The ESMFold row reports absolute means; other rows report changes relative to ESMFold.
Variant
Aspect
Ram. out. ↓
Ram. fav. ↑
Rot. out. ↓
C β dev. ↓
Clashscore ↓
RMS bonds ↓
RMS angles ↓
MolProbity ↓
esmfold
Base
5.238
86.597
4.678
0.000
186.953
0.091
6.099
3.773
esmfold_struct_enhanced_v4
L/P/C
-1.435
+3.240
-0.518
+0.000
-18.169
-0.014
-1.023
-0.157
esmfold_struct_local_context_v1
L
–
–
–
–
–
–
–
–
esmfold_struct_enhanced_v1_dup2
L/C
–
–
–
–
–
–
–
–
esmfold_struct_attn_frame_v1
L/P
-0.752
+1.808
-0.109
+0.000
-3.748
-0.007
-0.488
-0.049
esmfold_struct_enhanced_multiscale_v2
L/P/C
-1.034
+2.562
+0.469
+0.000
-8.692
-0.012
-0.852
-0.043
Appendix
Table 7: Complete physical-plausibility metrics for the motivation-aspect summary. The ESMFold row reports absolute means; other rows report changes relative to ESMFold.
Variant
Aspect
Prec 0-6↑
Prec 6-12↑
Prec 12-24↑
Prec ≥24 ↑
F1 0-6↑
F1 6-12↑
F1 12-24↑
F1 ≥24 ↑
esmfold
Base
0.944
0.621
0.599
0.537
0.952
0.617
0.606
0.521
esmfold_struct_enhanced_v4
L/P/C
+0.003
+0.032
+0.019
-0.010
+0.001
+0.024
+0.010
-0.007
esmfold_struct_local_context_v1
L
–
–
–
–
–
–
–
–
esmfold_struct_enhanced_v1_dup2
L/C
+0.000
+0.025
+0.020
+0.003
+0.000
+0.023
+0.013
+0.001
esmfold_struct_attn_frame_v1
L/P
–
–
–
–
–
–
–
–
esmfold_struct_enhanced_multiscale_v2
L/P/C
-0.000
+0.009
+0.009
-0.017
-0.002
+0.008
+0.007
-0.021
Appendix
Table 8: Complete contact metrics for the motivation-aspect summary. The ESMFold row reports absolute means; other rows report changes relative to ESMFold.
Index
Variant Name
Parameters (M)
1
esmfold
22.606659
2
esmfold_struct_enhanced_v1
22.606659
3
esmfold_struct_dynamic_head_weights
22.608207
4
esmfold_struct_sequence_distance_bias_v2
22.607595
5
esmfold_struct_attention_bias_v1
22.606660
6
esmfold_struct_residue_type_bias
22.606659
Appendix
Table 9: List of variants with their corresponding indices and parameter counts.
Figure 7: Structure module comparison. Top: ESMFold applies 8 IPA blocks on single / pair representations, then updates geometry via Backbone Update, Angle ResNet, and Frame. Bottom: Our variant adds a residue-index-conditioned bias MLP before IPA; the remaining geometric heads are unchanged.
Large language models (LLMs) are capable of emulating reasoning and using tools, creating opportunities for autonomous agents that execute complex scientific tasks. Protein design provides a natural testbed: although machine learning (ML) methods achieve strong results, these are largely restricted to canonical amino acids and narrow objectives, leaving unfilled need for a generalist tool for broad design pipelines. We introduce Agent Rosetta, an LLM agent paired with a structured environment for operating Rosetta, the leading physics-based heteropolymer design software, capable of modeling non-canonical building blocks and geometries. Agent Rosetta iteratively refines designs to achieve user-defined objectives, combining LLM reasoning with Rosetta's generality. We evaluate Agent Rosetta on design with canonical amino acids, matching specialized models and expert baselines, and with non-canonical residues -- where ML approaches fail -- achieving comparable performance. Critically, prompt engineering alone often fails to generate Rosetta actions, demonstrating that environment design is essential for integrating LLM agents with specialized software. Our results show that properly designed environments enable LLM agents to make scientific software accessible while matching specialized tools and human experts.
Jacopo Teneggi, S. M. Bargeen A. Turzo, Tanya Marwah +4
Polymathic AI · Johns Hopkins University · Center for Computational Biology, Flatiron Institute +3
Protein language models (PLMs) are passive oracles: they generate sequences in a single forward pass with no mechanism to consult external biophysical feedback or redirect generation when a candidate violates thermodynamic or structural constraints. We introduce AgentPLM, which addresses this by equipping a pre-trained PLM with i) Reasoning-Augmented Decoding (RAD), which interleaves autoregressive generation with tool calls (ESMFold, FoldX, AutoDock Vina), and ii) Contrastive Agent Policy Optimisation (CAPO), a trajectory-level extension of direct preference optimisation that trains the policy end-to-end to learn when oracle feedback is informative rather than merely imitating high-fitness sequences. We evaluate AgentPLM on benchmark tasks spanning de novo enzyme design, antibody optimisation, thermostability, PPI interface design, and zero-shot fitness prediction with standardised oracle APIs and controlled sequence-identity splits. AgentPLM achieves state-of-the-art results with a gain in antibody top-10% hit rate over the strongest passive baseline, providing mechanistic evidence of online error correction without explicit backtracking.
Sahil Rahman, Maxx Richard Rahman
Bedford College, London, United Kingdom · Saarland University, Saarbrücken, Germany
We introduce ABLE, a benchmark for evaluating LLM agents' ability to use biological AI models (BAIMs), such as ProteinMPNN and AlphaFold3, in dual-use protein design workflows. ABLE assesses agent performance through a set of tasks spanning structure retrieval, sequence generation, and design validation. We evaluate 15 frontier models and find that seven refuse all tasks, while the remaining models exhibit substantial performance differences. Claude Sonnet 4 and Gemini 3 Pro achieve the highest scores across information retrieval, tool selection, and tool use. We further compare model performance on a subset of tasks against an expert human baseline. Our results suggest that current LLMs can substantially lower barriers to protein design, but remain inconsistent in planning, strategy generation, and integrating biological knowledge with tool use.