PeptSol: A Translational Benchmark for Protected-Peptide Solubility in Drug Manufacturing
Organizations: BCG X AI Science Institute
Abstract
Liquid-phase peptide synthesis (LPPS) offers a scalable route to peptide manufacturing, but route development still depends on empirical solvent screens because each protected intermediate must be soluble for reaction and recoverable for purification. The protecting groups, terminal modifications, and noncanonical residues that govern this behaviour are poorly represented in existing peptide property benchmarks, limiting the ability to develop and rigorously evaluate molecular representation models for LPPS-relevant prediction tasks. We introduce PeptSol, an open benchmark of 1388 hand-curated protected-peptide solubility measurements with chemistry-aware splits. The measurements span only 84 parent sequences and 101 protected structures, and 92.5% come from a single historical programme. Across seven frozen molecular encoders, the best macro-F1 is 0.843 on binary solubility for the historical subset under a peptide-held-out split; the harder three-class ordinal task reaches 0.614 in that cell. No foundation model consistently outperforms a baseline combining chemical fingerprints with explicit molecular descriptors, and model rankings vary across prediction tasks and data splits. The dataset, evaluation protocol, and baselines establish a standard for measuring progress on molecular representations for synthesis-aware peptide developability.
Figures & tables
| Model | N-O | N-B | A-O | A-B |
|---|---|---|---|---|
| ChemBERTa | .561 | .765 | .599 | .786 |
| PeptideCLM-2 | .614 | .843 | .628 | .824 |
| PepLand | .586 | .791 | .631 | .816 |
| ECFP-XGB | .521 | .700 | .521 | .718 |
| ECFP+Desc | .627 | .812 | .609 | .785 |
| Model | Nar.-Sc | Nar.-GK | Aug.-Sc | Aug.-GK |
|---|---|---|---|---|
| ChemBERTa | .727 | .765 | .723 | .786 |
| PeptideCLM-2 | .739 | .843 | .715 | .824 |
| PepLand | .726 | .791 | .708 | .816 |
| ESM-2 | .731 | .797 | .724 | .741 |
| ECFP+Desc | .692 | .812 | .697 | .785 |
| Claim | Status | Present evidence | Required next test |
|---|---|---|---|
| Labels are learnable | OOF performance above baseline | Replication on new sources | |
| Transfer to new peptides | Peptide-GroupKFold | Prospectively held-out series | |
| Transfer to new solvents | All aqueous records fail (App. C.4 ) | Solvent-family holdout | |
| Transfer across labs | % from one programme | External laboratory and protocol | |
| Transfer across eras | Splits are not temporal | Historical-to-patent holdout | |
| Value beyond known chemistry | ECFP and descriptor controls | Pre-registered paired comparison |
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Regime | Years | Src. | Rec. | Pep. | Solv. |
|---|---|---|---|---|---|
| Historical literature | 1984–89 | 8 | 1284 | 67 | 25 |
| Process patents | 2018–25 | 3 | 104 | 17 | 29 |
| Full release | 1984–2025 | 11 | 1388 | 84 | 43 |
| Corpus | Insoluble | Partially soluble | Soluble |
|---|---|---|---|
| Historical ( ) | 366 | 82 | 836 |
| Augmented ( ) | 388 | 87 | 913 |
| Family | Hist. | Aug. | Family | Hist. | Aug. |
|---|---|---|---|---|---|
| Polar aprotic | 454 | 480 | Ether | 72 | 91 |
| Other | 264 | 265 | Aromatic | 50 | 55 |
| Protic | 208 | 237 | Aqueous mixture | 0 | 8 |
| Chlorinated | 159 | 160 | Aqueous | 0 | 4 |
| Ester | 77 | 88 |
| Verbatim | Harmonized class | Assay | Censoring | |
|---|---|---|---|---|
| A | Soluble | 25 | Lower bound | 697 |
| B | Soluble | 80 | Lower bound | 139 |
| C | Partially soluble | 80 | Uncensored | 82 |
| D | Insoluble | 80 | Uncensored | 366 |
| Verbatim annotation | Harmonized class | Sources | |
|---|---|---|---|
| clear solution | Soluble | 63 | St., Li., Ja. |
| soluble | Soluble | 10 | St. |
| clear yellow solution | Soluble | 4 | Li. |
| gel | Partially soluble | 3 | Li. |
| hazy solution | Partially soluble | 2 | Li. |
| poorly soluble | Insoluble | 9 | St. |
| Recorded annotation | Rows |
|---|---|
| Direct transcription, no annotation | 866 |
| Label shared across a grouped solvent column | 270 |
| Assay temperature not pinned in source | 89 |
| Footnote overrides the base label | 28 |
| Printed cell carries two letters | 23 |
| Other curation note | 8 |
| Regime | Records | Censored | Quant. | Both |
|---|---|---|---|---|
| Historical literature | 1284 | 836 | 0 | 0 |
| Process patents | 104 | 65 | 87 | 61 |
| Full release | 1388 | 901 | 87 | 61 |
| Encoder | Input | Pooling | Max length | Embedding |
|---|---|---|---|---|
| ChemBERTa-77M | SMILES | mean | 512 | 384 |
| PeptideCLM-2 | SMILES | mean | 512 | 512 |
| MoLFormer-XL | SMILES | mean | 202 | 768 |
| SELFormer | SELFIES | mean | 512 | 768 |
| PepLand | graph | node mean | — | 300 |
| ESM-2 | AA sequence | mean | 1024 | 640 |
| Sub-block | Dimensions | Contents |
|---|---|---|
| Hansen parameters | 3 | |
| Hydrogen bonding | 2 | donors and acceptors |
| RDKit descriptors | 7 | LogP, TPSA, mass, and related features |
| Morgan fingerprint | 256 | radius two |
| Total | 268 |
| Hyperparameter | Candidate values |
|---|---|
| Maximum depth | |
| Learning rate | |
| Trees | |
| Class weighting | none, balanced, inverse |
| Model | N-O | N-B | A-O | A-B |
|---|---|---|---|---|
| ChemBERTa | .561 | .765 | .599 | .786 |
| PeptideCLM-2 | .614 | .843 | .628 | .824 |
| MoLFormer | .574 | .753 | .597 | .730 |
| SELFormer | .506 | .675 | .547 | .701 |
| PepLand | .586 | .791 | .631 | .816 |
| ESM-2 | .603 | .797 | .579 | .741 |
| Corpus | Task | Encoder | % CI | |
|---|---|---|---|---|
| Narita | Ordinal | PeptideCLM-2 | ||
| Narita | Binary | PeptideCLM-2 | ||
| Augmented | Ordinal | PepLand | ||
| Augmented | Binary | PeptideCLM-2 |
| Corpus | Ordinal | Binary |
|---|---|---|
| Narita | .263 | .394 |
| Augmented | .265 | .397 |
| Encoder | N-O | N-B | A-O | A-B |
|---|---|---|---|---|
| (a) Solvent block only: no peptide representation | ||||
| (no encoder) | .421 | .620 | .380 | .631 |
| (b) Peptide block only: no solvent features | ||||
| ChemBERTa | .454 | .703 | .486 | .713 |
| PeptideCLM-2 | .444 | .724 | .495 | .707 |
| MoLFormer | .425 | .672 | .463 | .642 |
| Insoluble | Partially | Soluble | Recall | |
|---|---|---|---|---|
| Insoluble | 289 | 6 | 93 | .745 |
| Partially soluble | 21 | 13 | 53 | .149 |
| Soluble | 56 | 8 | 849 | .930 |
| Stratum | Error rate | |
|---|---|---|
| Solvent family | ||
| Polar aprotic | 480 | .179 |
| Other | 265 | .196 |
| Protic | 237 | .114 |
| Chlorinated | 160 | .137 |
| Ether | 91 | .132 |
Molecular Property Prediction
Molecular property prediction estimates chemical, physical, and biological properties from molecular structure and context.
See all 224 papers