Natural-Language-Guided Generator-Agnostic Shortlisting for Protein Binder Design
Authors: Gyubok Lee, Kiwoong Yoo, Jimin Seo, Jiyoun Kim, Kyunghoon Hur, Edward Choi
Organizations: Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea · LG AI Research, Seoul, South Korea · Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea · Korea Electronics Technology Institute (KETI), Seongnam, South Korea
Modern de novo design workflows generate many candidate protein binders, but wet-lab validation capacity remains limited, making shortlisting a major bottleneck. We study whether LLMs can generate multi-metric ranking policies from precomputed structural-confidence and interface-quality proxy scores. Rather than proposing a new protein binder design pipeline, we focus on post-generation binder shortlisting: selecting the final top-K candidates from already generated binder pools using a shared panel of precomputed proxy scores. On the 10-target held-out split, averaging performance over five sampled global iterative gpt-4o policies reaches 0.589 Recall@10, modestly improving over the strongest single-feature fixed baseline, Protenix binder ipTM, which reaches 0.571 Recall@10. On the 3-target held-out subset comprising Nipah, RBX1, and TREM2, target-conditioned iterative gpt-5.4 policies reach the strongest LLM performance, with 0.519 Recall@10 and 0.583 NDCG@10. These results suggest that LLM-generated ranking policies can act as an interpretable post-generation decision layer for combining heterogeneous proxy metrics to prioritize binders from large candidate pools.
Figures & tables
Figure 1: Overview of the post-generation binder shortlisting task. For each target protein, a fixed pool of candidate binders is generated in advance by upstream design workflows. A shared proxy-score panel is then computed for all candidates using AF2-Multimer, Boltz-2, Protenix/PXDesign, and interface descriptors such as Rosetta interface ΔG . Shortlisting methods, including fixed heuristics, supervised ML models, global LLM policies, and target-conditioned LLM policies, rank the candidates and return a top- K shortlist for experimental testing. Global LLM policies apply a single shared ranking rule to all held-out targets, whereas target-conditioned LLM policies generate a target-specific ranking rule.
Mixed participant-submitted methods, including Adaptyv EGFR competition submissions and BindCraft EGFR revalidation
EGFR
605
68
1
pMHC minibinders
RFdiffusion + ProteinMPNN + AlphaFold2 filtering
NY-ESO-1 pMHC; RVTDESILSY pMHC
137
3
2
Nipah release
Mixed participant-submitted methods
Nipah Virus Glycoprotein G (NiV-G)
1,030
103
1
GEM/Adaptyv
Mixed participant-submitted methods
RBX1
321
9
1
Table 1: Composition of the development and held-out evaluation datasets after target-level merging and exact sequence de-duplication. Counts report designs, experimentally confirmed binders, and targets for each source. For pMHC targets, the table reports short target names. The full peptide–HLA pairs are SLLMWITQC–HLA-A02:01 and RVTDESILSY–HLA-A01:01.
Family
Features
Description
AF2-Multimer
ipTM, pTM, binder pLDDT, interface PAE
Model-native complex confidence, binder local confidence, and interface PAE from AF2-Multimer ( Evans et al., 2021 ; Mirdita et al., 2022 ) ; lower interface PAE is better.
Boltz-2
ipTM, pTM, binder pLDDT, ipSAE, pDockQ2
Model-native confidence scores from Boltz-2 ( Passaro et al., 2025 ) , ipSAE-style interface confidence ( Dunbrack Jr, 2025 ) , and pDockQ2 ( Zhu et al., 2023 ) , reported as metric-wise best summaries across target-MSA diffusion samples.
Confidence fields from Protenix/PXDesign ( Team et al., 2025a ; Team et al., 2025b ) covering complex, binder-chain, and binder-target confidence under our two-chain target-binder convention.
Rosetta interface descriptors
interface ΔG , buried SASA, shape complementarity
Rosetta descriptors ( Stranges and Kuhlman, 2013 ; Lawrence and Colman, 1993 ) are computed on Boltz-2 target-MSA predicted complexes, not experimental structures.
Table 2: Candidate-level proxy score panel (17 active features). Feature definitions, monotonic directions, and extraction details are in Appendix C .
10-target held-out split
3-target held-out subset
Method
Recall@10
Hit@10
NDCG@10
Recall@10
Hit@10
NDCG@10
Fixed ranking heuristics
AF2 ipTM
0.437
0.360
0.417
0.433
0.433
0.487
AF2 interface PAE
0.417
0.290
0.337
0.300
0.300
0.298
Boltz-2 ipTM
0.403
0.290
0.377
0.133
0.133
0.166
Boltz-2 pDockQ2
0.515
0.340
0.383
0.400
0.400
0.416
Table 3: Held-out evaluation ( K=10 ), grouped by method type. Recall@10 uses denominator min(K,nbinders) ; Hit@10 is equivalent to Precision@10. Results are shown for the 10-target held-out split and for the 3-target held-out subset containing Nipah, RBX1, and TREM2. Dashes mark model/split combinations not reported. LLM results average five sampled policies.
Figure 2: Feature weights in target-conditioned iterative gpt-4o ranking policies over the full 17-feature panel. The vertical axis lists held-out targets, and the horizontal axis lists all proxy-score features. Colored cells indicate features selected by the policies; the color scale shows the average selected weight across five sampled policies. Near-zero background cells indicate features not included in the ranking policy. For lower-is-better features, positive plotted weights apply to the direction-corrected normalized score used by the executor.
Removed group
TC iter. gpt-4o 3-target
TC iter. gpt-5.4 3-target
No removal
0.497
0.519
AF2
0.436
0.499
Boltz-2
0.517
0.545
Protenix
0.340
0.453
Rosetta
0.455
0.423
Table 4: Proxy-score group ablation of target-conditioned iterative policies on the shared 3-target held-out subset. Values are mean Recall@10 after removing all selected features from one group. No removal denotes the same ablation policy set before feature removal. Boltz-2 includes Boltz-2-derived pDockQ2; Rosetta denotes Rosetta interface descriptors.
Table 5: Example RBX1 ranking rules. Each x^ is a within-target 0 to 1 normalized feature; + rewards higher raw values and − rewards lower raw values. For LLM policies, coefficients are average policy weights across five sampled policies, with unselected features contributing weight 0 to the average; therefore, an averaged rule can include more nonzero terms than any single sampled policy. Global policies use one averaged rule for every target, whereas target-conditioned policies are averaged for RBX1 specifically. Candidates are sorted by the resulting rank score in descending order, and the top K candidates are selected.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Model / run
Documented cutoff
Use in paper
gpt-4o-2024-11-20
2023-10-01
Reported on the 10-target held-out split and the 3-target held-out subset
gpt-5.4
2025-08-31
Reported on the 3-target held-out subset: Nipah, RBX1, and TREM2
Appendix
Table 6: Model cutoff dates used for the temporal-leakage audit.
Source / pool
Targets used
Public date
After gpt-4o ?
After gpt-5.4 ?
BindCraft1 revalidation
spCas9, Der f 21, IFNAR2, Der f 7, EGFR entries
2025-10-06
yes
yes
Adaptyv EGFR R1
EGFR
2024-10-18
yes
no
pMHC minibinders
NY-ESO-1 pMHC, RVTDESILSY pMHC
2024-12-03
yes
no
Adaptyv EGFR R2
EGFR
2025-01-15
yes
no
BoltzGen validated-label dataset
development targets only
2025-10-27
yes
yes
GEM/Adaptyv RBX1
RBX1
2026-04-26
yes
yes
Appendix
Table 7: Dataset-level temporal-leakage audit. Dates report conservative public availability of candidate-level experimental labels, or stable public data-package availability where explicitly noted. Development and transfer-baseline entries are shown for provenance but are not held-out evaluation labels. For the gpt-5.4 comparison, Nipah, TREM2, and RBX1 are the held-out targets whose source pools and experimental labels became public after the documented cutoff.
Feature
Direction
Extraction summary
AF2-Multimer ipTM
higher
Mean AF2-Multimer ipTM across aggregated target-MSA complex models.
AF2-Multimer pTM
higher
Mean AF2-Multimer pTM across aggregated target-MSA complex models.
AF2 binder pLDDT
higher
Mean binder-chain AF2-Multimer pLDDT across aggregated target-MSA complex models.
AF2 interface PAE
lower
Mean cross-chain AF2-Multimer PAE over target-binder interface residue pairs, averaged across models.
Boltz-2 ipTM
higher
Best Boltz-2 ipTM summary across target-MSA diffusion samples.
Boltz-2 pTM
higher
Best Boltz-2 pTM summary across target-MSA diffusion samples.
Appendix
Table 8: Active features in the 17-feature policy panel. Directions indicate the monotonic orientation used by the deterministic executor. Target-MSA complex predictions use the target-chain MSA while designed binder chains remain single-sequence.
Prompt aspect
Global LLM
Target-conditioned LLM
Policy emitted
One policy shared by all held-out targets
One policy generated separately for each held-out target
Feature panel
Same 17 feature descriptions and fixed directions
Same 17 feature descriptions and fixed directions
Development information
Per-development-target score distributions and target-averaged single-feature performance
Same development calibration context as the global prompt
Held-out information
No held-out target identifiers, source context, score distributions, or labels
Current held-out target identifier and unlabeled score distributions for that target pool
Labels available to prompt
Development labels only through aggregate single-feature performance
Development labels only through aggregate single-feature performance; no held-out labels
Executor
Same deterministic executor: fixed directions, within-target normalization, weighted sum, top 10
Same deterministic executor: fixed directions, within-target normalization, weighted sum, top 10
Appendix
Table 9: Information contrast between global and target-conditioned single-turn LLM policies. Iterative variants keep the same held-out-label restriction and executor, but add aggregate development-search feedback.
Target
n
nb
Fixed
Global
TC
Iter. global
Iter. TC
spCas9
20
14
0.700
0.800
0.900
0.860
0.840
Der f 21
16
4
0.500
0.250
0.500
0.550
0.600
Nipah G
1030
103
0.700
0.600
0.800
0.760
0.720
IFNAR2
20
8
0.500
0.500
0.500
0.725
0.725
NY-ESO-1 pMHC
41
1
1.000
1.000
0.200
1.000
1.000
Der f 7
20
5
0.800
0.200
0.600
0.760
0.800
Appendix
Table 10: Per-target Recall@10 on the 10-target held-out split for gpt-4o . Fixed is Protenix binder ipTM, the strongest single-feature fixed baseline in Table 3 . Global and TC denote single-turn global and target-conditioned matched-prompt policies; Iter. global and Iter. TC denote the corresponding iterative policies with development-feedback memory. LLM entries are sampled-policy averages where applicable.
Metric
Target
Global
TC
Iter. global
Iter. TC
Recall@10
Nipah G
0.800
0.850
0.580
0.640
Recall@10
RBX1
0.000
0.056
0.111
0.156
Recall@10
TREM2
0.600
0.600
0.720
0.760
Recall@10
mean
0.467
0.502
0.470
0.519
Hit@10
Nipah G
0.800
0.850
0.580
0.640
Hit@10
RBX1
0.000
0.050
0.100
0.140
Appendix
Table 11: Per-target gpt-5.4 performance on the 3-target held-out subset. Global and TC denote single-turn global and target-conditioned matched-prompt policies; Iter. global and Iter. TC denote the corresponding iterative policies with development-feedback memory. Values are reported only for Nipah, RBX1, and TREM2, the post-cutoff subset used for gpt-5.4 evaluation.
We introduce ABLE, a benchmark for evaluating LLM agents' ability to use biological AI models (BAIMs), such as ProteinMPNN and AlphaFold3, in dual-use protein design workflows. ABLE assesses agent performance through a set of tasks spanning structure retrieval, sequence generation, and design validation. We evaluate 15 frontier models and find that seven refuse all tasks, while the remaining models exhibit substantial performance differences. Claude Sonnet 4 and Gemini 3 Pro achieve the highest scores across information retrieval, tool selection, and tool use. We further compare model performance on a subset of tasks against an expert human baseline. Our results suggest that current LLMs can substantially lower barriers to protein design, but remain inconsistent in planning, strategy generation, and integrating biological knowledge with tool use.
Protein interaction modeling is central to protein design, which has been transformed by machine learning with applications in drug discovery and beyond. In this landscape, structure-based de novo binder design is cast as either conditional generative modeling or sequence optimization via structure predictors ("hallucination"). We argue that this is a false dichotomy and propose Proteina-Complexa, a novel fully atomistic binder generation method unifying both paradigms. We extend recent flow-based latent protein generation architectures and leverage the domain-domain interactions of monomeric computationally predicted protein structures to construct Teddymer, a new large-scale dataset of synthetic binder-target pairs for pretraining. Combined with high-quality experimental multimers, this enables training a strong base model. We then perform inference-time optimization with this generative prior, unifying the strengths of previously distinct generative and hallucination methods. Proteina-Complexa sets a new state of the art in computational binder design benchmarks: it delivers markedly higher in-silico success rates than existing generative approaches, and our novel test-time optimization strategies greatly outperform previous hallucination methods under normalized compute budgets. We also demonstrate interface hydrogen bond optimization, fold class-guided binder generation, and extensions to small molecule targets and enzyme design tasks, again surpassing prior methods. Code, models and new data will be publicly released.
Kieran Didi, Zuobai Zhang, Guoqing Zhou +11
NVIDIA · University of Oxford · Mila - Québec AI Institute +8
A central challenge in de novo protein design is generating proteins with desired structural and functional properties. Existing design models typically require a dedicated input representation for each property, limiting the design objectives they support. We develop Prompteina, the first all-atom structure generation model that supports new and broader design objectives specified in natural language, analogous to text-to-image generation. To train Prompteina, we construct Prompteina-Data, a paired text--protein structure dataset of 1.8M structures and 50M+ captions describing global and residue-specific structural and functional properties. We adapt Proteina-Complexa, a pretrained structure generation model, through text-conditioned pretraining and finetuning that aligns residues with their corresponding text. Prompteina outperforms baselines on 15 of 18 structural properties and generates proteins that more closely match descriptions of their intended functions. In binder design, it leads baselines on at least 13 of 20 targets and improves interface confidence by 14% over the strongest baseline. With iterative in silico feedback, Prompteina's gain over non-iterative generation is 57% larger than the next-best baseline on the hardest targets. Finally, we experimentally validate Prompteina designs against GM2A, a target with no reported de novo binders. We identify nine binders, with the strongest at an apparent KD of 107 nM. These results establish natural language as an effective and powerful conditioning interface for protein structure generation.
Joseph Lee, Yanjing Li, Bo Qiang +7
University of Washington · National University of Singapore · Allen Institute for AI (Ai2) +1