Emergent Biological Capabilities in a Foundation Model for Molecular Interactions
Organizations: Output Labs
Abstract
Foundation models trained on large unlabeled corpora develop emergent capabilities, tasks that go beyond the training objective. We extend this paradigm to multimodal biology with Bonbon, a foundation model for molecular interactions trained on approximately 3.2 trillion tokens of paired protein-ligand sequences. From a single representation space, Bonbon exhibits three zero-shot emergent capabilities at hierarchical levels of resolution: functional (mechanism of action), residue (binding site localization, including orthosteric/allosteric discrimination), and bond (covalent versus non-covalent engagement). All three capabilities generalize to out-of-distribution complexes. Bonbon was trained on protein-ligand sequences alone, without any task-specific supervision and without structural coordinates. We conducted controlled scaling experiments across six model scales from 42M to 1.6B parameters; every capability evaluated shows a sharp jump at the largest scale on continuous metrics, even as pretraining loss decreases smoothly. This demonstrates that loss-based scaling laws alone are insufficient to predict when zero-shot emergent capabilities become available. The capabilities reported here emerge after training at a 1:2,000 parameter-to-token ratio to accommodate the sparsity of paired biological interaction data. The same frozen representations achieve state-of-the-art binding and affinity prediction at five orders of magnitude greater speed than co-folding methods. In prospective wet-lab validation across nine disease targets spanning seven protein families, Bonbon achieves a 61.5% dose-response hit rate on de novo compounds, yielding structurally novel molecules not previously reported to inhibit these proteins, including for out-of-distribution targets.
Figures & tables
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Domain | Params | Tokens | Param:Token | FLOPs | Emergent capabilities |
|---|---|---|---|---|---|---|
| GPT-2 | Language | 1.5B | 10B | 1:7 | Coherent generation | |
| ESM-2 | Proteins | 650M | 65B | 1:100 | Weak structure | |
| ESM-2 | Proteins | 15B | 65B | 1:4 | Structure prediction | |
| Chinchilla | Language | 70B | 1.4T | 1:20 | Few-shot reasoning | |
| GPT-3 | Language | 175B | 300B | 1:2 | In-context learning | |
| Evo | DNA/RNA | 7B | 300B | 1:43 | Gene function |
| Class | Train | Test | Total |
|---|---|---|---|
| Inhibitor | 27,050 99 | 2,950 99 | 30,000 |
| Substrate | 22,905 318 | 1,144 318 | 24,049 |
| Product | 21,244 265 | 1,078 265 | 22,322 |
| Agonist | 12,144 116 | 991 116 | 13,135 |
| Antagonist | 9,218 108 | 1,278 108 | 10,495 |
| Activator | 5,506 68 | 1,520 68 | 7,026 |
| Class | Precision | SD | 95% CI |
|---|---|---|---|
| Activator | 0.987 | 0.010 | [0.980, 0.994] |
| Inhibitor | 0.879 | 0.020 | [0.865, 0.893] |
| Antagonist | 0.829 | 0.020 | [0.815, 0.843] |
| Agonist | 0.792 | 0.023 | [0.776, 0.808] |
| Substrate | 0.691 | 0.093 | [0.625, 0.758] |
| Product | 0.686 | 0.069 | [0.637, 0.735] |
| Train | Test | Val | Total | |
|---|---|---|---|---|
| Agonist | 12,352 | 1,543 | 1,543 | 15,438 |
| Antagonist | 10,152 | 1,268 | 1,268 | 12,688 |
| Total | 22,504 | 2,811 | 2,811 | 28,126 |
| Bonbon | |
|---|---|
| Balanced accuracy | |
| Agonist precision | |
| Antagonist precision |
| Family | ID | OOD | |
|---|---|---|---|
| Kinase | 213 | 0.381 0.090 | 0.370 0.091 |
| Protease | 180 | 0.410 0.163 | 0.403 0.154 |
| GPCR | 156 | 0.404 0.078 | 0.393 0.102 |
| Oxidoreductase | 100 | 0.350 0.155 | 0.378 0.113 |
| Epigenetic | 61 | 0.392 0.122 | 0.409 0.069 |
| Ion channel | 56 | 0.274 0.131 | 0.252 0.123 |
| Method | F1 |
|---|---|
| Bonbon | 0.497 |
| Fpocket [ 13 ] | 0.45 |
| P2Rank [ 14 ] | 0.44 |
| ESMBind [ 29 ] | 0.32 |
| Evaluation subset | Proteins | Complexes | Signature | MFP |
|---|---|---|---|---|
| All OOD | 141 | 6,032 | 0.834 | 0.589 |
| GPCR OOD | 60 | 3,684 | 0.759 | 0.487 |
| Enzyme OOD | 61 | 1,480 | 0.746 | 0.646 |
| TF OOD | 5 | 445 | 0.669 | 0.320 |
| Both-class OOD | 18 | 1,860 | 0.821 | 0.609 |
| GPCR BC OOD | 9 | 1,229 | 0.788 | 0.660 |
| Group | Bond type | Electrophile | Description | |
|---|---|---|---|---|
| A | Covalent | Present | 731 | Warhead engages the target |
| B | Covalent | Absent | 239 | Enzymatic covalent mechanism |
| C | Non-covalent | Present | 131 | Warhead present but does not react |
| D | Non-covalent | Absent | 389 | No electrophilic groups |
| Target residue | AUROC | positive (OOD) |
|---|---|---|
| Serine | 0.879 | 327 |
| Threonine | 0.861 | 151 |
| Lysine | 0.618 | 60 |
| Cysteine | 0.561 | 149 |
| Residue | Original AUROC | 0.5 Å matched | 0.25 Å matched | (original) |
|---|---|---|---|---|
| Serine | 0.902 | 0.877 | 0.725 | 412 positive |
| Cysteine | 0.817 | 0.796 | 0.758 | 179 positive |
| Lysine | 0.701 | 0.698 | 0.500 | 72 positive |
| Target class | Proteins | Molecules | Mean coverage |
|---|---|---|---|
| Enzyme | 7 | 31,159 | 82.9% |
| GPCR | 36 | 80,270 | 85.9% |
| Ion channel | 37 | 40,584 | 82.4% |
| Nuclear receptor | 36 | 26,312 | 85.3% |
| RTK | 29 | 62,710 | 85.0% |
| All | 145 | 241,035 | 84.6% |
| Scale | Median boundary coverage | Sharp ( 0.59) | Soft (0.50–0.59) | Bleeding ( 0.50) |
|---|---|---|---|---|
| 42M | 0.501 | 1/90 | 57/90 | 32/90 |
| 315M | 0.501 | 1/90 | 66/90 | 23/90 |
| 800M | 0.500 | 1/90 | 59/90 | 30/90 |
| 1B | 0.501 | 1/90 | 60/90 | 29/90 |
| 1.3B | 0.502 | 1/90 | 75/90 | 14/90 |
| 1.6B | 0.904 | 113/119 | 1/119 | 5/119 |
| Benchmark | Metric | Bonbon | Boltz-2 ‡ | IntFold ‡ | SPRINT † | DrugCLIP † | Docking ‡ |
|---|---|---|---|---|---|---|---|
| Davis | AUPR | 0.825 | 0.46 | 0.58 | 0.51 | — | — |
| BindingDB | AUPR | 0.752 | 0.53 | 0.67 | 0.72 | — | — |
| MF-PCBA | AUPR | 0.081 | 0.025 | — | — | — | 0.016 |
| LIT-PCBA* | EF1% | 11.4 | — | — | 10.8** | 5.5 | 3.4–4.6 |
| Method | EF1% |
|---|---|
| Bonbon | 16.6 |
| Boltz-2 | 9.3 |
| Protenix | 7.6 |
| Glide | 6.8 |
| ConPlex | 4.3 |
| Gnina | 2.8 |
| Method | Type | CASP16 | FEP+4 | OpenFE |
|---|---|---|---|---|
| FEP+ | Physics ( 1h) | — | 0.78 | 0.72 |
| ABFE | Physics ( 1h) | — | 0.75 | 0.73 |
| OpenFE-calc | Physics ( 1h) | — | 0.66 | — |
| FMO | Physics ( 1h) | — | 0.55 | — |
| MM/PBSA | Physics ( 1h) | — | 0.18 | — |
| Chemgauss4 | Docking | — | 0.26 | — |
| Task | Pairs | Mode | Bonbon | Boltz-2 | Speedup |
|---|---|---|---|---|---|
| FEP+4 | 87 | On-the-fly | 0.08 s | 3,867 s | |
| MF-PCBA (10 targets) | 500K | On-the-fly | 5 min | months* | |
| LIT-PCBA | 2.65M | On-the-fly | 18 min | years* | |
| Proteome DrugBank | 1.66B | Cached | 11 hours** | GPU-years** |
| Target | Tested | Confirmed | IC 50 range | Tanimoto |
|---|---|---|---|---|
| TYK2 | 24 | 23 | 146 nM–40 M | 0.342–0.567 |
| PRMT5 | 24 | 16 | 186 nM–143 M | 0.479–0.597 |
| JAK1 | 24 | 21 | 3 nM–28 M | 0.385–0.684 |
| CDK1 | 22 | 22 | 114 nM–23 M | 0.279–0.706 |
| HDAC6 | 24 | 15 | 115 nM–22 M | 0.278–0.625 |
| MNK2 | 24 | 19 | 39 nM–38 M | 0.269–0.492 |