Natural-Language-Guided Generator-Agnostic Shortlisting for Protein Binder Design

Gyubok Lee; Kiwoong Yoo; Jimin Seo; Jiyoun Kim; Kyunghoon Hur; Edward Choi

cs.AIAug 21, 2026

Natural-Language-Guided Generator-Agnostic Shortlisting for Protein Binder Design

Authors: Gyubok Lee, Kiwoong Yoo, Jimin Seo, Jiyoun Kim, Kyunghoon Hur, Edward Choi

Organizations: Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea · LG AI Research, Seoul, South Korea · Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea · Korea Electronics Technology Institute (KETI), Seongnam, South Korea

Abstract

Modern de novo design workflows generate many candidate protein binders, but wet-lab validation capacity remains limited, making shortlisting a major bottleneck. We study whether LLMs can generate multi-metric ranking policies from precomputed structural-confidence and interface-quality proxy scores. Rather than proposing a new protein binder design pipeline, we focus on post-generation binder shortlisting: selecting the final top-K candidates from already generated binder pools using a shared panel of precomputed proxy scores. On the 10-target held-out split, averaging performance over five sampled global iterative gpt-4o policies reaches 0.589 Recall@10, modestly improving over the strongest single-feature fixed baseline, Protenix binder ipTM, which reaches 0.571 Recall@10. On the 3-target held-out subset comprising Nipah, RBX1, and TREM2, target-conditioned iterative gpt-5.4 policies reach the strongest LLM performance, with 0.519 Recall@10 and 0.583 NDCG@10. These results suggest that LLM-generated ranking policies can act as an interpretable post-generation decision layer for combining heterogeneous proxy metrics to prioritize binders from large candidate pools.

Figures & tables

Figure 1: Overview of the post-generation binder shortlisting task. For each target protein, a fixed pool of candidate binders is generated in advance by upstream design workflows. A shared proxy-score panel is then computed for all candidates using AF2-Multimer, Boltz-2, Protenix/PXDesign, and interface descriptors such as Rosetta interface $\Delta G$ . Shortlisting methods, including fixed heuristics, supervised ML models, global LLM policies, and target-conditioned LLM policies, rank the candidates and return a top- $K$ shortlist for experimental testing. Global LLM policies apply a single shared ranking rule to all held-out targets, whereas target-conditioned LLM policies generate a target-specific ranking rule.

Split	Source	Candidate origin	Target names	Designs	Binders	Targets
Development	BoltzGen	BoltzGen	AMBP, HNMT, IDI2, IL-7Ra, Insulin Receptor, MZB1/PERP1, PDGFR Beta, PD-L1, PHYH, PMVK, RFK	327	103	11
Held-out eval.	BindCraft1 revalidation	BindCraft	IFNAR2, spCas9, Der f 21, Der f 7	76	31	4
	Merged EGFR	Mixed participant-submitted methods, including Adaptyv EGFR competition submissions and BindCraft EGFR revalidation	EGFR	605	68	1
	pMHC minibinders	RFdiffusion + ProteinMPNN + AlphaFold2 filtering	NY-ESO-1 pMHC; RVTDESILSY pMHC	137	3	2
	Nipah release	Mixed participant-submitted methods	Nipah Virus Glycoprotein G (NiV-G)	1,030	103	1
	GEM/Adaptyv	Mixed participant-submitted methods	RBX1	321	9	1

Table 1: Composition of the development and held-out evaluation datasets after target-level merging and exact sequence de-duplication. Counts report designs, experimentally confirmed binders, and targets for each source. For pMHC targets, the table reports short target names. The full peptide–HLA pairs are SLLMWITQC–HLA-A02:01 and RVTDESILSY–HLA-A01:01.

Family	Features	Description
AF2-Multimer	ipTM, pTM, binder pLDDT, interface PAE	Model-native complex confidence, binder local confidence, and interface PAE from AF2-Multimer ( Evans et al., 2021 ; Mirdita et al., 2022 ) ; lower interface PAE is better.
Boltz-2	ipTM, pTM, binder pLDDT, ipSAE, pDockQ2	Model-native confidence scores from Boltz-2 ( Passaro et al., 2025 ) , ipSAE-style interface confidence ( Dunbrack Jr, 2025 ) , and pDockQ2 ( Zhu et al., 2023 ) , reported as metric-wise best summaries across target-MSA diffusion samples.
Protenix/PXDesign	pair ipTM, complex pTM, binder ipTM, binder pTM, binder pLDDT	Confidence fields from Protenix/PXDesign ( Team et al., 2025a ; Team et al., 2025b ) covering complex, binder-chain, and binder-target confidence under our two-chain target-binder convention.
Rosetta interface descriptors	interface $\Delta G$ , buried SASA, shape complementarity	Rosetta descriptors ( Stranges and Kuhlman, 2013 ; Lawrence and Colman, 1993 ) are computed on Boltz-2 target-MSA predicted complexes, not experimental structures.

Table 2: Candidate-level proxy score panel (17 active features). Feature definitions, monotonic directions, and extraction details are in Appendix C .

Fixed ranking heuristics
	10-target held-out split			3-target held-out subset
Method	Recall@10	Hit@10	NDCG@10	Recall@10	Hit@10	NDCG@10
AF2 ipTM	0.437	0.360	0.417	0.433	0.433	0.487
AF2 interface PAE	0.417	0.290	0.337	0.300	0.300	0.298
Boltz-2 ipTM	0.403	0.290	0.377	0.133	0.133	0.166
Boltz-2 pDockQ2	0.515	0.340	0.383	0.400	0.400	0.416

Table 3: Held-out evaluation (

K=10

), grouped by method type. Recall@10 uses denominator

\min(K,n_{\text{binders}})

; Hit@10 is equivalent to Precision@10. Results are shown for the 10-target held-out split and for the 3-target held-out subset containing Nipah, RBX1, and TREM2. Dashes mark model/split combinations not reported. LLM results average five sampled policies.

Figure 2: Feature weights in target-conditioned iterative gpt-4o ranking policies over the full 17-feature panel. The vertical axis lists held-out targets, and the horizontal axis lists all proxy-score features. Colored cells indicate features selected by the policies; the color scale shows the average selected weight across five sampled policies. Near-zero background cells indicate features not included in the ranking policy. For lower-is-better features, positive plotted weights apply to the direction-corrected normalized score used by the executor.

Removed group	TC iter. gpt-4o 3-target	TC iter. gpt-5.4 3-target
No removal	0.497	0.519
AF2	0.436	0.499
Boltz-2	0.517	0.545
Protenix	0.340	0.453
Rosetta	0.455	0.423

Table 4: Proxy-score group ablation of target-conditioned iterative policies on the shared 3-target held-out subset. Values are mean Recall@10 after removing all selected features from one group. No removal denotes the same ablation policy set before feature removal. Boltz-2 includes Boltz-2-derived pDockQ2; Rosetta denotes Rosetta interface descriptors.

Policy	Rank score
Fixed Protenix binder ipTM, RBX1	$\displaystyle\hat{x}_{\mathrm{Protenix\ binder\ ipTM}}$
Fixed Boltz-2 pDockQ2, RBX1	$\displaystyle\hat{x}_{\mathrm{Boltz\ pDockQ2}}$
Global iterative gpt-4o	$\displaystyle 2.2\hat{x}_{\mathrm{Boltz\ ipTM}}+2.0\hat{x}_{\mathrm{AF2\ ipTM}}+2.0\hat{x}_{\mathrm{Protenix\ pair\ ipTM}}+1.0\hat{x}_{\mathrm{Rosetta\ SC}}+0.2\hat{x}_{\mathrm{Boltz\ pDockQ2}}$
Target-conditioned iterative gpt-4o , RBX1	$\displaystyle 2.6\hat{x}_{\mathrm{Boltz\ ipTM}}+2.0\hat{x}_{\mathrm{Protenix\ pair\ ipTM}}+1.2\hat{x}_{\mathrm{AF2\ ipTM}}+0.8\hat{x}_{\mathrm{Rosetta\ SC}}$
Global iterative gpt-5.4	$\displaystyle 2.4\hat{x}_{\mathrm{Protenix\ pair\ ipTM}}+1.8\hat{x}_{\mathrm{AF2\ ipTM}}+1.2\hat{x}_{\mathrm{Boltz\ pDockQ2}}+1.2\hat{x}_{\mathrm{Rosetta\ SC}}+0.8\hat{x}_{\mathrm{Rosetta\ buried\ SASA}}-0.6\hat{x}_{\mathrm{Rosetta\ interface\ dG}}+0.4\hat{x}_{\mathrm{AF2\ pTM}}-0.4\hat{x}_{\mathrm{AF2\ interface\ PAE}}+0.4\hat{x}_{\mathrm{Boltz\ ipTM}}+0.2\hat{x}_{\mathrm{Protenix\ binder\ pLDDT}}+0.2\hat{x}_{\mathrm{Protenix\ complex\ pTM}}$
Target-conditioned iterative gpt-5.4 , RBX1	$\displaystyle 2.0\hat{x}_{\mathrm{Rosetta\ buried\ SASA}}+1.8\hat{x}_{\mathrm{Protenix\ pair\ ipTM}}+1.6\hat{x}_{\mathrm{Rosetta\ SC}}+1.4\hat{x}_{\mathrm{Boltz\ ipSAE}}+0.6\hat{x}_{\mathrm{Boltz\ pDockQ2}}-0.6\hat{x}_{\mathrm{AF2\ interface\ PAE}}+0.6\hat{x}_{\mathrm{Boltz\ ipTM}}+0.4\hat{x}_{\mathrm{Boltz\ binder\ pLDDT}}+0.4\hat{x}_{\mathrm{AF2\ ipTM}}+0.2\hat{x}_{\mathrm{AF2\ pTM}}$

Table 5: Example RBX1 ranking rules. Each

\hat{x}

is a within-target 0 to 1 normalized feature;

+

rewards higher raw values and

-

rewards lower raw values. For LLM policies, coefficients are average policy weights across five sampled policies, with unselected features contributing weight 0 to the average; therefore, an averaged rule can include more nonzero terms than any single sampled policy. Global policies use one averaged rule for every target, whereas target-conditioned policies are averaged for RBX1 specifically. Candidates are sorted by the resulting rank score in descending order, and the top

K

candidates are selected.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Model / run	Documented cutoff	Use in paper
gpt-4o-2024-11-20	2023-10-01	Reported on the 10-target held-out split and the 3-target held-out subset
gpt-5.4	2025-08-31	Reported on the 3-target held-out subset: Nipah, RBX1, and TREM2

Appendix

Table 6: Model cutoff dates used for the temporal-leakage audit.

Source / pool	Targets used	Public date	After gpt-4o ?	After gpt-5.4 ?
BindCraft1 revalidation	spCas9, Der f 21, IFNAR2, Der f 7, EGFR entries	2025-10-06	yes	yes
Adaptyv EGFR R1	EGFR	2024-10-18	yes	no
pMHC minibinders	NY-ESO-1 pMHC, RVTDESILSY pMHC	2024-12-03	yes	no
Adaptyv EGFR R2	EGFR	2025-01-15	yes	no
BoltzGen validated-label dataset	development targets only	2025-10-27	yes	yes
GEM/Adaptyv RBX1	RBX1	2026-04-26	yes	yes

Appendix

Table 7: Dataset-level temporal-leakage audit. Dates report conservative public availability of candidate-level experimental labels, or stable public data-package availability where explicitly noted. Development and transfer-baseline entries are shown for provenance but are not held-out evaluation labels. For the gpt-5.4 comparison, Nipah, TREM2, and RBX1 are the held-out targets whose source pools and experimental labels became public after the documented cutoff.

Feature	Direction	Extraction summary
AF2-Multimer ipTM	higher	Mean AF2-Multimer ipTM across aggregated target-MSA complex models.
AF2-Multimer pTM	higher	Mean AF2-Multimer pTM across aggregated target-MSA complex models.
AF2 binder pLDDT	higher	Mean binder-chain AF2-Multimer pLDDT across aggregated target-MSA complex models.
AF2 interface PAE	lower	Mean cross-chain AF2-Multimer PAE over target-binder interface residue pairs, averaged across models.
Boltz-2 ipTM	higher	Best Boltz-2 ipTM summary across target-MSA diffusion samples.
Boltz-2 pTM	higher	Best Boltz-2 pTM summary across target-MSA diffusion samples.

Appendix

Table 8: Active features in the 17-feature policy panel. Directions indicate the monotonic orientation used by the deterministic executor. Target-MSA complex predictions use the target-chain MSA while designed binder chains remain single-sequence.

Prompt aspect	Global LLM	Target-conditioned LLM
Policy emitted	One policy shared by all held-out targets	One policy generated separately for each held-out target
Feature panel	Same 17 feature descriptions and fixed directions	Same 17 feature descriptions and fixed directions
Development information	Per-development-target score distributions and target-averaged single-feature performance	Same development calibration context as the global prompt
Held-out information	No held-out target identifiers, source context, score distributions, or labels	Current held-out target identifier and unlabeled score distributions for that target pool
Labels available to prompt	Development labels only through aggregate single-feature performance	Development labels only through aggregate single-feature performance; no held-out labels
Executor	Same deterministic executor: fixed directions, within-target normalization, weighted sum, top 10	Same deterministic executor: fixed directions, within-target normalization, weighted sum, top 10

Appendix

Table 9: Information contrast between global and target-conditioned single-turn LLM policies. Iterative variants keep the same held-out-label restriction and executor, but add aggregate development-search feedback.

Target	$n$	$n_{b}$	Fixed	Global	TC	Iter. global	Iter. TC
spCas9	20	14	0.700	0.800	0.900	0.860	0.840
Der f 21	16	4	0.500	0.250	0.500	0.550	0.600
Nipah G	1030	103	0.700	0.600	0.800	0.760	0.720
IFNAR2	20	8	0.500	0.500	0.500	0.725	0.725
NY-ESO-1 pMHC	41	1	1.000	1.000	0.200	1.000	1.000
Der f 7	20	5	0.800	0.200	0.600	0.760	0.800

Appendix

Table 10: Per-target Recall@10 on the 10-target held-out split for gpt-4o . Fixed is Protenix binder ipTM, the strongest single-feature fixed baseline in Table 3 . Global and TC denote single-turn global and target-conditioned matched-prompt policies; Iter. global and Iter. TC denote the corresponding iterative policies with development-feedback memory. LLM entries are sampled-policy averages where applicable.

Metric	Target	Global	TC	Iter. global	Iter. TC
Recall@10	Nipah G	0.800	0.850	0.580	0.640
Recall@10	RBX1	0.000	0.056	0.111	0.156
Recall@10	TREM2	0.600	0.600	0.720	0.760
Recall@10	mean	0.467	0.502	0.470	0.519
Hit@10	Nipah G	0.800	0.850	0.580	0.640
Hit@10	RBX1	0.000	0.050	0.100	0.140

Appendix

Table 11: Per-target gpt-5.4 performance on the 3-target held-out subset. Global and TC denote single-turn global and target-conditioned matched-prompt policies; Iter. global and Iter. TC denote the corresponding iterative policies with development-feedback memory. Values are reported only for Nipah, RBX1, and TREM2, the post-cutoff subset used for gpt-5.4 evaluation.

LLM Reranking

Large language model reranking uses LLMs to reorder retrieved documents or candidate outputs by relevance, usefulness, or task-specific criteria.

See all 130 papers

Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools
Sep 5, 2026Bryce Cai, Geetha Jeyapragasan, Samira Nedungadi +2LLM Tool Use LLM Agent Evaluation
SecureBio, Cambridge, MA, USA.
Scaling Atomistic Protein Binder Design with Generative Pretraining and Test-Time Compute
Mar 30, 2026Kieran Didi, Zuobai Zhang, Guoqing Zhou +11Protein Binder Design Protein Design
NVIDIA · University of Oxford · Mila - Québec AI Institute +8
Designing Protein Structures from Natural Language
Oct 9, 2026Joseph Lee, Yanjing Li, Bo Qiang +7Protein Structure Generation Protein Design
University of Washington · National University of Singapore · Allen Institute for AI (Ai2) +1

Sep 5, 2026J/K move · Enter open · S save

Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools

Bryce Cai, Geetha Jeyapragasan, Samira Nedungadi, Jake Yukich+1

SecureBio, Cambridge, MA, USA.

LLM Tool Use LLM Agent Evaluation AI Agent Benchmarks AI Agents for Scientific Discovery Protein Design

Open paper AskPDF arXiv

We introduce ABLE, a benchmark for evaluating LLM agents' ability to use biological AI models (BAIMs), such as ProteinMPNN and AlphaFold3, in dual-use protein design workflows. ABLE assesses agent performance through a set of tasks spanning structure retrieval, sequence generation, and design validation. We evaluate 15 frontier models and find that seven refuse all tasks, while the remaining models exhibit substantial performance differences. Claude Sonnet 4 and Gemini 3 Pro achieve the highest scores across information retrieval, tool selection, and tool use. We further compare model performance on a subset of tasks against an expert human baseline. Our results suggest that current LLMs can substantially lower barriers to protein design, but remain inconsistent in planning, strategy generation, and integrating biological knowledge with tool use.

Abstract

LLM Reranking

Explore similar work

Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools

Scaling Atomistic Protein Binder Design with Generative Pretraining and Test-Time Compute

Designing Protein Structures from Natural Language