Confident and Wrong: Structure Confidence Is a Poor Selection Filter for Protein Function
Organizations: Aether Biomachines
Abstract
Structure-guided protein design can generate plausible candidates, but structural confidence does not predict function. We trained a function-guided model on measured metal-binding affinity with protein sequence, ligand identity, and assay conditions, and used it to design rare earth element (REE) binding variants of lanmodulin, a lanthanide-binding protein with four EF-hand domains. We screened 297 designs from three strategies against terbium and dysprosium, adjacent lanthanides among the hardest REE pairs to separate. Selection on structural confidence inverted the outcome it was meant to improve. The Structure-Confidence-Selected panel carried the highest pTM, complex ipLDDT, and composite confidence scores, and expressed at 97%, yet only 16.7% of its designs bound terbium (95% CI 10.4 to 25.7), against 83.3% (74.6 to 89.5) for the function-guided panel, whose confidence scores were the lowest. The odds of binding differed 25-fold between the two panels (Fisher exact ). No structural-confidence metric separated binders from non-binders consistently across panels, whereas the function-trained model's predicted pKd gave the highest ROC AUC in every panel, 0.79 to 0.92, and ranked measured affinity among function-guided binders at Spearman . On the design objective, Tb/Dy selectivity, the function-guided designs also exceeded the natural lanmodulin family, outranking a randomly drawn ortholog 84% of the time across 149 orthologs. Structural confidence is a plausibility signal, not a selection criterion for function; direct supervision on measured function is what ranks designs.
Figures & tables
| Selectivity | vs. Function-Selected | vs. Lanmodulin Orthologs | ||||||
|---|---|---|---|---|---|---|---|---|
| Model/cohort | n | Median | U | BH | (magnitude) | U | BH | (magnitude) |
| Function-Selected | 76 | 1.70 | — | — | — | 9500.0 | +0.678 (large) | |
| Co-Generation-Selected | 40 | 1.38 | 770.0 | (medium) | 3695.0 | 0.024 | +0.240 (small) | |
| Structure-Confidence-Selected | 15 | 1.42 | 263.0 | 0.002 | (large) | 1319.0 | 0.251 | +0.180 (small) |
| Lanmodulin Orthologs | 149 | 1.26 | — | — | — | — | — | — |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Function-Selected | Co-Generation-Selected | Structure-Confidence-Selected | ||||
| Metric | Non-binding | Binding | Non-binding | Binding | Non-binding | Binding |
| Total proteins screened | 100 | 104 | 93 | |||
| Proteins successfully expressed | 96 | 76 | 90 | |||
| Proteins in binding class | 20 | 80 | 57 | 47 | 78 | 15 |
| Boltz-2 structural-confidence metrics | ||||||
| pTM | 0.859 | 0.866 | 0.880 | 0.883 | 0.904 | 0.906 |
| Metric | Function-Selected | Co-Generation-Selected | Structure-Confidence-Selected |
|---|---|---|---|
| Total proteins | 100 | 104 | 93 |
| Expressing proteins eligible for ROC | 96 | 76 | 90 |
| Binders | 80 | 47 | 15 |
| Non-expressing proteins (excluded) | 4 | 28 | 3 |
| Boltz-2 structural-confidence metrics | |||
| pTM | 0.591 | 0.520 | 0.533 |
| Metric | Function-Selected | Co-Generation-Selected | Structure-Confidence-Selected |
|---|---|---|---|
| Measured Binders | 75 | 42 | 15 |
| Boltz-2 structural-confidence metrics | |||
| pTM | 0.003 | 0.017 | 0.034 |
| Ligand ipTM | 0.345 | 0.073 | 0.200 |
| Complex ipLDDT | 0.106 | 0.082 | 0.401 |
| Complex iPDE (Å; lower is better) | 0.276 | 0.076 | 0.212 |
Protein Design
Protein design uses computational and experimental methods to create or modify protein sequences and structures for desired functions.
See all 114 papers