PepBenchmark: A Standardized Benchmark for Peptide Machine Learning

Jiahui Zhang; Rouyi Wang; Kuangqi Zhou; Tianshu Xiao; Lingyan Zhu; Yaosen Min; Yang Wang

cs.LGApr 12, 2026

PepBenchmark: A Standardized Benchmark for Peptide Machine Learning

Authors: Jiahui Zhang, Rouyi Wang, Kuangqi Zhou, Tianshu Xiao, Lingyan Zhu, Yaosen Min, Yang Wang

Organizations: Zhongguancun Academy · University of Science and Technology of China · Nankai University · Tianjin University

Abstract

Peptide therapeutics are widely regarded as the "third generation" of drugs, yet progress in peptide Machine Learning (ML) are hindered by the absence of standardized benchmarks. Here we present PepBenchmark, which unifies datasets, preprocessing, and evaluation protocols for peptide drug discovery. PepBenchmark comprises three components: (1) PepBenchData, a well-curated collection comprising 29 canonical-peptide and 6 non-canonical-peptide datasets across 7 groups, systematically covering key aspects of peptide drug development, representing, to the best of our knowledge, the most comprehensive AI-ready dataset resource to date; (2) PepBenchPipeline, a standardized preprocessing pipeline that ensures consistent dataset cleaning, construction, splitting, and feature transformation, mitigating quality issues common in ad hoc pipelines; and (3) PepBenchLeaderboard, a unified evaluation protocol and leaderboard with strong baselines across 4 major methodological families: Fingerprint-based, GNN-based, PLM-based, and SMILES-based models. Together, PepBenchmark provides the first standardized and comparable foundation for peptide drug discovery, facilitating methodological advances and translation into real-world applications. The data and code are publicly available at https://github.com/ZGCI-AI4S-Pep/PepBenchmark/.

Figures & tables

Figure 1: Overview of PepBenchmark, illustrating its three core components: PepBenchData, PepBenchPipeline, and PepBenchLeaderboard.

Application	Dataset Name	Task Type	Peptide Type	Input Type	Unit	Size	Split
ADME	nonfouling	bc	ca	single		7200	Hybrid
	cpp	bc	ca	single		2296	ECFP
	bbp	bc	ca	single		665	Hybrid
	nc-cpp_pampa	reg	n-ca	single	log(cm/s)	6970	Hybrid
AMP	antimicrobial	bc	ca	single		52941	Hybrid
	antibacterial	bc	ca	single		28591	Hybrid

Table 1: Overview of datasets in PepBenchData . Here, “bc” denotes binary classification and “reg” denotes regression; “ca” denotes canonical and “n-ca” denotes non-canonical. Detailed data sources are provided in Table 6 .

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

PLM-based models

XGBoost

LightGBM

GIN

Pepland

ChemBERTa

PepDoRA

DPLM-150M

ESM2-8M

ESM2-35M

ESM2-150M

ESM2-650M

ESM2-8M-S

ESM2-150M-F

ace_inhibitory

3537

\mathbf{82.2_{\pm 1.3}}

80.4_{\pm 1.8}

\pagecolor{lightgray}{81.1_{\pm 1.8}}

78.8_{\pm 1.4}

72.4_{\pm 3.6}

77.3_{\pm 2.5}

70.3_{\pm 2.0}

79.7_{\pm 1.0}

79.2_{\pm 1.4}

80.7_{\pm 1.9}

80.2_{\pm 0.5}

79.6_{\pm 1.9}

79.1_{\pm 1.6}

80.2_{\pm 2.3}

allergen

2538

84.0_{\pm 4.2}

85.6_{\pm 3.4}

86.1_{\pm 3.6}

77.5_{\pm 1.7}

58.7_{\pm 3.6}

81.5_{\pm 3.8}

61.8_{\pm 1.5}

88.3_{\pm 1.4}

\pagecolor{lightgray}{88.3_{\pm 2.6}}

87.2_{\pm 3.6}

\mathbf{90.2_{\pm 1.2}}

87.4_{\pm 3.8}

83.2_{\pm 2.5}

86.8_{\pm 1.8}

antiaging

548

\pagecolor{lightgray}{66.5_{\pm 1.9}}

63.3_{\pm 5.6}

65.3_{\pm 2.4}

53.2_{\pm 11.7}

64.0_{\pm 12.2}

50.6_{\pm 6.2}

55.6_{\pm 3.5}

64.0_{\pm 7.1}

61.4_{\pm 4.6}

\mathbf{67.2_{\pm 5.2}}

62.5_{\pm 3.3}

63.6_{\pm 5.8}

58.1_{\pm 2.2}

61.9_{\pm 3.9}

antibacterial

28591

88.4_{\pm 0.9}

88.3_{\pm 0.8}

87.6_{\pm 0.7}

82.2_{\pm 1.8}

62.7_{\pm 3.4}

88.9_{\pm 2.2}

73.8_{\pm 2.6}

92.3_{\pm 1.2}

92.0_{\pm 0.9}

92.5_{\pm 0.8}

\pagecolor{lightgray}{93.0_{\pm 1.2}}

93.0_{\pm 0.6}

90.8_{\pm 0.6}

\mathbf{93.2_{\pm 0.5}}

anticancer

12013

87.7_{\pm 1.0}

87.6_{\pm 0.9}

87.1_{\pm 0.6}

78.9_{\pm 2.6}

69.5_{\pm 4.6}

87.5_{\pm 1.2}

71.4_{\pm 2.8}

92.2_{\pm 0.9}

\pagecolor{lightgray}{92.3_{\pm 0.9}}

92.3_{\pm 0.9}

92.3_{\pm 0.8}

\mathbf{92.4_{\pm 0.6}}

90.7_{\pm 0.8}

92.2_{\pm 0.6}

Table 2: Performance of models on canonical peptide classification (ROC-AUC

\uparrow

, %) with hybrid-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Dataset	Size	FP-based models			GNN-based models		SMILES-based models		PLM-based models
Dataset	Size	RF	XGBoost	LightGBM	GIN	Pepland	ChemBERTa	PepDoRA	DPLM-150M	ESM2-8M	ESM2-35M	ESM2-150M	ESM2-650M	ESM2-8M-S	ESM2-150M-F
E.coli_mic	3204	$0.593_{\pm 0.015}$	$0.604_{\pm 0.019}$	$0.584_{\pm 0.019}$	$0.631_{\pm 0.015}$	$0.650_{\pm 0.062}$	$0.603_{\pm 0.007}$	$0.650_{\pm 0.013}$	$0.525_{\pm 0.006}$	$0.539_{\pm 0.023}$	$0.527_{\pm 0.004}$	$\pagecolor{lightgray}{0.509_{\pm 0.010}}$	$\mathbf{0.488_{\pm 0.008}}$	$0.548_{\pm 0.024}$	$0.517_{\pm 0.023}$
P.aeruginosa_mic	1490	$0.540_{\pm 0.030}$	$0.559_{\pm 0.025}$	$0.550_{\pm 0.026}$	$0.537_{\pm 0.032}$	$0.566_{\pm 0.037}$	$0.520_{\pm 0.041}$	$0.546_{\pm 0.029}$	$0.524_{\pm 0.036}$	$0.506_{\pm 0.046}$	$0.497_{\pm 0.042}$	$0.496_{\pm 0.046}$	$\mathbf{0.471_{\pm 0.043}}$	$0.516_{\pm 0.031}$	$\pagecolor{lightgray}{0.483_{\pm 0.024}}$
S.aureus_mic	2822	$0.569_{\pm 0.035}$	$0.588_{\pm 0.022}$	$0.572_{\pm 0.025}$	$0.635_{\pm 0.032}$	$0.648_{\pm 0.016}$	$0.627_{\pm 0.011}$	$0.657_{\pm 0.029}$	$0.563_{\pm 0.023}$	$0.545_{\pm 0.030}$	$0.549_{\pm 0.022}$	$\pagecolor{lightgray}{0.527_{\pm 0.032}}$	$\mathbf{0.522_{\pm 0.022}}$	$0.585_{\pm 0.016}$	$0.533_{\pm 0.028}$
hemolytic_hc50	1926	$0.517_{\pm 0.042}$	$0.527_{\pm 0.038}$	$0.523_{\pm 0.035}$	$0.528_{\pm 0.036}$	$0.516_{\pm 0.033}$	$0.498_{\pm 0.040}$	$0.535_{\pm 0.027}$	$0.404_{\pm 0.028}$	$0.422_{\pm 0.042}$	$\pagecolor{lightgray}{0.400_{\pm 0.021}}$	$0.413_{\pm 0.047}$	$\mathbf{0.394_{\pm 0.017}}$	$0.463_{\pm 0.011}$	$0.412_{\pm 0.020}$
avg	-	$0.555$	$0.569$	$0.557$	$0.583$	$0.595$	$0.562$	$0.597$	$0.504$	$0.503$	$0.493$	$0.486$	$\mathbf{0.469}$	$0.528$	$\pagecolor{lightgray}{0.486}$

Table 3: Performance of models on canonical peptide regression (MAE

\downarrow

, %) with hybrid-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Task	Metric	Dataset	Size	FP-based models					GNN-based models				SMILES-based models
Task	Metric	Dataset	Size	RF	XGBoost	LightGBM	GradBoost	AdaBoost	GCN	GAT	GIN	Pepland	ChemBERTa	PeptideCLM	PepDoRA
cls	AUC ROC	nc-antibacterial	1668	$\mathbf{94.4_{\pm 1.4}}$	$93.6_{\pm 2.2}$	$\pagecolor{lightgray}{93.8_{\pm 1.8}}$	$92.5_{\pm 2.8}$	$90.9_{\pm 3.4}$	$93.0_{\pm 2.6}$	$81.5_{\pm 3.5}$	$90.7_{\pm 4.9}$	$84.9_{\pm 3.9}$	$91.9_{\pm 2.7}$	$71.4_{\pm 1.8}$	$72.4_{\pm 7.4}$
		nc-antifungal	407	$\pagecolor{lightgray}{95.4_{\pm 3.2}}$	$\mathbf{96.5_{\pm 2.9}}$	$94.8_{\pm 3.1}$	$95.2_{\pm 3.8}$	$95.4_{\pm 2.4}$	$70.1_{\pm 12.0}$	$78.5_{\pm 6.2}$	$86.1_{\pm 2.8}$	$83.5_{\pm 4.4}$	$78.0_{\pm 18.3}$	$65.4_{\pm 7.0}$	$65.9_{\pm 12.0}$
		nc-antimicrobial	2465	$97.6_{\pm 0.9}$	$\pagecolor{lightgray}{97.7_{\pm 0.9}}$	$\mathbf{97.8_{\pm 0.6}}$	$97.3_{\pm 0.7}$	$95.3_{\pm 1.8}$	$94.9_{\pm 1.6}$	$79.2_{\pm 4.4}$	$91.5_{\pm 2.3}$	$88.0_{\pm 0.9}$	$95.2_{\pm 1.9}$	$68.3_{\pm 4.7}$	$80.0_{\pm 1.9}$
		nc-hemolytic	3971	$96.1_{\pm 1.1}$	$\pagecolor{lightgray}{96.2_{\pm 0.5}}$	$\mathbf{96.3_{\pm 0.7}}$	$95.3_{\pm 0.8}$	$93.7_{\pm 1.6}$	$89.5_{\pm 3.7}$	$85.7_{\pm 2.6}$	$89.0_{\pm 4.1}$	$82.8_{\pm 2.1}$	$91.4_{\pm 1.5}$	$76.1_{\pm 4.5}$	$72.1_{\pm 7.3}$
		avg	-	$\pagecolor{lightgray}{95.9}$	$\mathbf{96.0}$	$95.7$	$95.1$	$93.8$	$86.9$	$81.2$	$89.3$	$84.8$	$89.1$	$70.3$	$72.6$

Table 4: Performance of models on non-canonical peptide classification (ROC-AUC

\uparrow

, %) and regression (MAE

\downarrow

, %) with ECFP-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Task

Metric

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

PLM-based models

ECFP4

ECFP6

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

ESM2-8M

ESM2-35M

ESM2-150M

ESM2-650M

ESM2-150M-F

cls

AUC ROC

ppi

44148

54.4_{\pm 2.3}

53.7_{\pm 2.3}

\mathbf{61.3_{\pm 7.2}}

59.6_{\pm 2.7}

52.0_{\pm 4.4}

51.4_{\pm 3.6}

59.4_{\pm 2.8}

60.2_{\pm 6.3}

57.6_{\pm 7.2}

55.4_{\pm 3.5}

51.9_{\pm 2.4}

56.0_{\pm 3.1}

reg

MAE

ppi_ba

1433

1.043_{\pm 0.050}

1.043_{\pm 0.060}

1.189_{\pm 0.140}

1.176_{\pm 0.152}

\mathbf{1.034_{\pm 0.013}}

1.128_{\pm 0.042}

1.084_{\pm 0.044}

1.061_{\pm 0.043}

1.056_{\pm 0.092}

1.079_{\pm 0.038}

1.051_{\pm 0.072}

1.038_{\pm 0.030}

nc_ppi_ba

278

1.665_{\pm 0.260}

1.647_{\pm 0.286}

1.741_{\pm 0.359}

1.705_{\pm 0.366}

1.613_{\pm 0.189}

1.580_{\pm 0.142}

\mathbf{1.465_{\pm 0.234}}

Table 5: Performance of models on PepPI classification (ROC-AUC

\uparrow

, %) and regression (MAE

\downarrow

, %) with cold-start-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Appendix figures & tables43 assets

Supplementary material from the paper’s appendix.

Appendix

Application	Dataset Name	Data Source
ADME	nonfouling	Guntuboina et al. (2023)
	cpp	Zhang et al. (2025) ; Peptipedia ( Cabas-Mora et al., 2024 )
	bbp	Dai et al. (2021) ; Peptipedia ( Cabas-Mora et al., 2024 )
	nc-cpp_pampa	CycPeptMPDB ( Li et al., 2023 )
AMP	antimicrobial	Wang et al. (2025c) ; Peptipedia ( Cabas-Mora et al., 2024 )
	antibacterial	Pinacho-Castellanos et al. (2021) ; Peptipedia ( Cabas-Mora et al., 2024 )

Appendix

Table 6: Dataset sources.

Application	Dataset Name	Task Type	Peptide Type	Origin_pos	Filt_pos	Exp_neg	Filt_neg	Total_Size	min_len	max_len	mean_len	Len $>$ 50%
ADME	bbp	bc	ca	358	336	13	13	672	2	82	14.14	1.04
	cpp	bc	ca	1318	1162	0	-	2324	3	61	17.24	1.20
	nonfouling	bc	ca	3600	3600	0	-	7200	5	11	6.06	0.00
	solubility	bc	ca	5096	5096	5384	5384	10480	19	150	112.34	97.86
AMP	antibacterial	bc	ca	21220	15838	0	-	31676	2	150	25.33	9.74
	antifungal	bc	ca	11087	8349	0	-	16698	2	148	34.17	22.82

Appendix

Table 7: Overview of classification datasets.

Application	Dataset Name	Peptide Type	Unit	Total_Size	min_len	max_len	mean_len	Len $>$ 50%
ADME	nc-cpp_pampa	n-ca	log(cm/s)	6970	2	15	8.04	0
AMP	E.coli_mic	ca	lg( $\mu M$ )	3312	2	140	23.24	3.23
	P.aeruginosa_mic	ca	lg( $\mu M$ )	1531	2	140	22.01	2.61
	S.aureus_mic	ca	lg( $\mu M$ )	2900	2	140	22.70	2.66
Metabolic	ace_inhibitory_ic50	ca	lg( $\mu M$ )	337	2	3	2.61	0.00
PepPI	nc-PpI_ba	n-ca	-lg(M)	277	5	19	8.93	0

Appendix

Table 8: Overview of regression datasets.

Dataset Name	Positive Samples	Positive Sample Source	Negative Sampling Method	Used in Articles
ace_inhibitory_raw	1053	Manavalan et al. (2019)	Bioactive peptides	APML Fernández-Díaz et al. (2024)
antibacterial_raw	6010	Pinacho-Castellanos et al. (2021)	Bioactive peptides	APML Fernández-Díaz et al. (2024)
anticancer_raw	861	Agrawal et al. (2021)	Bioactive peptides	APML Fernández-Díaz et al. (2024)
antidiabetic_raw	418	Yue et al. (2024)	Random peptides	Yue et al. (2024)
antifungal_raw	993	Pinacho-Castellanos et al. (2021)	Bioactive peptides	APML Fernández-Díaz et al. (2024)
antimicrobial_raw	22176	Wang et al. (2025c)	Inactive peptides	Wang et al. (2025c)

Appendix

Table 9: Summary of the datasets constructed in prior studies employed in analysis.

Figure 102: Overview of the construction pipeline for peptide classification datasets

Dataset	Deduped_ratio	ROC–AUC	de_ROC–AUC	Perf_drop (%)
hemolytic_raw	0.470	0.805 (0.022)	0.665 (0.037)	17.391
dppiv_inhibitors_raw	0.050	0.830 (0.049)	0.799 (0.024)	3.735
antibacterial_raw	0.290	0.889 (0.006)	0.858 (0.017)	3.487
cpp_raw	0.270	0.929 (0.005)	0.904 (0.018)	2.691
quorum_sensing_raw	0.140	0.930 (0.024)	0.913 (0.026)	1.828
anticancer_raw	0.190	0.938 (0.017)	0.918 (0.010)	2.132

Appendix

Table 10: Dataset redundancy and model performance before and after deduplication. Deduped_ratio: fraction of redundant positive sequences removed; ROC–AUC: model performance on the original dataset; de_ROC–AUC: performance on the deduplicated dataset; Perf_drop: relative decrease in ROC–AUC after deduplication.

Parameter	Value	Meaning / Rationale
--min-seq-id	0.90	Minimum sequence identity (90%); removes peptides differing by $>$ 10%.
-c	0.9	Coverage threshold (90%).
--cov-mode	0	Coverage relative to longer sequence; avoids clustering of short fragments into longer peptides.
--mask	0	Disable low-complexity masking; retain functional motifs (e.g., Cys- and Lys-rich).
--alignment-mode	3	Full alignment with start, end, and identity reported.
--seq-id-mode	2	Identity based on aligned region without terminal gaps; robust to local indels.

Appendix

Table 11: MMseqs2 clustering parameters for redundancy removal in peptide datasets.

Dataset Name	Origin_pos	New_pos	Filt_ratio	Exp_neg	New_exp_neg	Filt_ratio
ace_inhibitory	1833	1780	0.029	–	–	–
allergen	2405	1677	0.303	–	–	–
antiaging	282	279	0.011	–	–	–
antibacterial	21220	15838	0.254	–	–	–
anticancer	9022	6926	0.232	–	–	–
antidiabetic	1599	1514	0.053	75	75	0.000

Appendix

Table 12: Dataset Statistics: Original and Filtered Sample Counts

Figure 103: Dataset sequence overlap heatmap. The color intensity represents the proportion of shared sequences relative to the smaller of the two datasets.

Dataset	Length_js	Charge_js	Hydrophobicity_js	1mers_js	2mers_js
ace_inhibitory	0.115	0.077	0.075	0.026	0.069
allergen	0.249	0.075	0.054	0.014	0.039
antiaging	0.122	0.111	0.074	0.004	0.072
antibacterial	0.082	0.100	0.041	0.012	0.025
anticancer	0.082	0.049	0.041	0.011	0.025
antidiabetic	0.104	0.095	0.063	0.004	0.031

Appendix

Table 13: Jensen-Shannon divergence between positive and negative samples after negative sampling for each dataset. Length_js measures differences in sequence length distributions; Charge_js quantifies differences in net charge distributions; Hydrophobicity_js evaluates differences in hydrophobicity distributions; 1mers_js assesses differences in amino acid composition; and 2mers_js measures differences in dipeptide composition.

Figure 104: Distributional comparison of positive and negative samples before and after sampling.

Dataset	Models
	Random Split							MMseqs2 Split
	LightGBM	RF	XGBoost	ESM2_150M	ESM2_650M	DPLM_150M	DPLM_650M	LGB	RF	XGB	ESM2_150M	ESM2_650M	DPLM_150M	DPLM_650M
anticancer	0.930	0.938	0.931	0.919	0.919	0.927	0.929	0.827	0.825	0.815	0.795	0.815	0.795	0.815
antifungal	0.937	0.955	0.933	0.960	0.974	0.959	0.975	0.686	0.670	0.705	0.798	0.807	0.759	0.788
antimicrobial	0.971	0.977	0.974	0.986	0.987	0.984	0.986	0.951	0.953	0.952	0.962	0.965	0.961	0.967
antiviral	0.923	0.956	0.929	0.955	0.955	0.956	0.959	0.785	0.814	0.775	0.838	0.857	0.843	0.864

Appendix

Table 14: Model Performance Comparison

Figure 105: Isolated Sequences Ratio vs MMseqs2 Identity Threshold (from 0.1 to 1.0) Across classification Datasets.

Figure 106: Overview of the construction pipeline for peptide regression datasets

Dataset	Sample_count	Outlier_remove	Unique_sample_count
S.aureus_mic	5,070	1,167	2,900
E.coli_mic	5,465	1,231	3,312
P.aeruginosa_mic	2,523	500	1,618
PpI_ba_X	1,806	0	1,709
nc-PpI_ba_X	286	0	277

Appendix

Table 15: Outlier removal statistics for regression datasets. Outlier_remove: number of data points identified as outliers.

Figure 107: Isolated Sequences Ratio vs MMseqs2 Identity Threshold (from 0.1 to 1.0) Across Regression Datasets.

Dataset	Size
PpI_X	8570
PpI	7358
PpI_ba	1433
PpI_ba_X	1709

Appendix

Table 16: Statistics of datasets with special amino acids

Figure 108: Schematic diagram of the classification dataset construction.

Parameter	Value	Description
epochs	50	Maximum number of training epochs
batch_size	64	Training batch size
learning_rate	5e-5	Learning rate
early_stopping_patience	5	Stop if validation loss stagnates for 5 epochs
weight_decay	0.0	No regularization

Appendix

Table 17: Hyperparameters for PLM-based and SMILES-based models

Parameter	Value	Description
epochs	50	Maximum number of training epochs
batch_size	64	Training batch size
num_layers	3	Number of GNN layers
emb_dim	300	Dimension of hidden embeddings
learning_rate	0.001	Learning rate

Appendix

Table 18: Hyperparameters for GNN-based models

Parameter	Value	Description
epochs	50	Maximum number of training epochs
batch_size	16	Smaller batch size due to memory usage
learning_rate	1e-4	Learning rate

Appendix

Table 19: Hyperparameters for PPI task

Figure 109: Length distribution of the uniref50_50 dataset.

Figure 110: Amino acid distribution of the uniref50_50 dataset.

Parameter	Value	Description
learning_rate	4e-4	Learning rate
num_train_epochs	500	Training epochs
per_device_train_batch_size	512	Batch size per device
gradient_accumulation_steps	1	Gradient accumulation steps
warmup_steps	2000	Learning rate warm-up steps
weight_decay	0.01	Weight decay

Appendix

Table 20: Hyperparameters for fine-tuning ESM-150M (ESM-150M-F)

Figure 111: Perplexity of ESM-150M (ESM2) and fine-tuned ESM-150M-F (ours) across peptide lengths.

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

XGBoost

LightGBM

GradBoost

AdaBoost

GCN

GAT

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

nonfouling

7200

76.4_{\pm 1.5}

75.4_{\pm 2.1}

76.4_{\pm 1.4}

77.0_{\pm 1.4}

76.3_{\pm 1.5}

76.8_{\pm 1.0}

76.1_{\pm 0.9}

76.7_{\pm 1.4}

75.3_{\pm 2.3}

76.2_{\pm 1.0}

75.6_{\pm 2.1}

70.7_{\pm 2.1}

cpp

2296

82.1_{\pm 2.9}

82.3_{\pm 4.9}

82.1_{\pm 3.4}

79.3_{\pm 3.8}

74.7_{\pm 3.4}

59.7_{\pm 5.4}

53.6_{\pm 5.2}

57.7_{\pm 5.5}

68.2_{\pm 7.2}

72.2_{\pm 8.2}

58.5_{\pm 3.8}

58.9_{\pm 4.3}

bbp

665

\pagecolor{lightgray}{69.3_{\pm 6.4}}

68.2_{\pm 8.4}

69.2_{\pm 7.5}

65.5_{\pm 10.5}

60.9_{\pm 5.0}

62.5_{\pm 5.6}

62.8_{\pm 5.0}

61.2_{\pm 3.4}

56.1_{\pm 9.3}

61.2_{\pm 7.1}

61.9_{\pm 6.9}

54.7_{\pm 7.2}

antimicrobial

52941

87.6_{\pm 0.4}

87.8_{\pm 0.5}

87.5_{\pm 0.6}

85.2_{\pm 0.6}

82.8_{\pm 0.8}

81.7_{\pm 0.4}

73.2_{\pm 1.1}

80.5_{\pm 1.9}

67.5_{\pm 1.6}

89.9_{\pm 0.8}

83.3_{\pm 1.6}

75.4_{\pm 0.9}

antibacterial

28591

88.4_{\pm 0.9}

88.3_{\pm 0.8}

87.6_{\pm 0.7}

85.6_{\pm 0.8}

84.1_{\pm 0.9}

83.2_{\pm 1.0}

75.9_{\pm 1.3}

82.2_{\pm 1.8}

62.7_{\pm 3.4}

88.9_{\pm 2.2}

77.0_{\pm 5.4}

73.8_{\pm 2.6}

Appendix

Table 21: Performance of models on canonical peptide classification (ROC-AUC

\uparrow

, %) with hybrid-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

XGBoost

LightGBM

GradBoost

AdaBoost

GCN

GAT

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

nonfouling

7200

77.0_{\pm 1.4}

76.9_{\pm 1.6}

77.4_{\pm 1.2}

77.6_{\pm 1.4}

77.0_{\pm 1.9}

77.7_{\pm 1.7}

76.6_{\pm 2.5}

78.4_{\pm 1.5}

77.0_{\pm 2.5}

77.2_{\pm 1.9}

75.6_{\pm 2.6}

70.9_{\pm 2.0}

cpp

2296

\mathbf{85.8_{\pm 3.1}}

83.9_{\pm 3.2}

84.0_{\pm 1.9}

81.2_{\pm 1.7}

76.5_{\pm 2.4}

60.8_{\pm 7.1}

54.2_{\pm 3.4}

64.5_{\pm 2.4}

70.3_{\pm 2.9}

74.5_{\pm 4.1}

60.1_{\pm 3.4}

59.5_{\pm 4.5}

bbp

665

\pagecolor{lightgray}{69.8_{\pm 11.1}}

67.4_{\pm 9.0}

\mathbf{69.8_{\pm 9.2}}

62.8_{\pm 4.8}

59.3_{\pm 5.4}

59.3_{\pm 8.3}

59.3_{\pm 5.8}

58.6_{\pm 10.7}

69.1_{\pm 7.0}

51.5_{\pm 11.4}

59.3_{\pm 6.9}

47.5_{\pm 5.0}

antimicrobial

52941

87.8_{\pm 0.1}

87.9_{\pm 0.3}

87.5_{\pm 0.2}

85.3_{\pm 0.3}

82.8_{\pm 0.5}

81.9_{\pm 0.7}

73.5_{\pm 1.3}

81.8_{\pm 0.3}

68.8_{\pm 2.2}

90.2_{\pm 0.6}

83.9_{\pm 1.1}

76.1_{\pm 2.0}

antibacterial

28591

89.6_{\pm 1.0}

89.3_{\pm 1.1}

88.7_{\pm 0.9}

86.3_{\pm 0.9}

84.5_{\pm 0.8}

82.6_{\pm 1.9}

77.3_{\pm 1.2}

83.8_{\pm 0.8}

63.5_{\pm 3.7}

89.9_{\pm 0.7}

83.2_{\pm 0.9}

75.6_{\pm 1.7}

Appendix

Table 22: Performance of models on canonical peptide classification (ROC-AUC

\uparrow

, %) with kmer-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

XGBoost

LightGBM

GradBoost

AdaBoost

GCN

GAT

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

nonfouling

7200

77.7_{\pm 1.4}

76.4_{\pm 2.4}

77.3_{\pm 2.0}

77.7_{\pm 1.8}

76.8_{\pm 1.8}

77.9_{\pm 1.9}

76.7_{\pm 1.5}

77.8_{\pm 1.8}

77.8_{\pm 1.6}

77.6_{\pm 1.8}

75.4_{\pm 1.9}

71.5_{\pm 2.7}

cpp

2296

85.8_{\pm 2.2}

84.3_{\pm 2.8}

84.2_{\pm 2.4}

81.2_{\pm 1.3}

78.1_{\pm 1.3}

69.0_{\pm 3.0}

63.5_{\pm 2.7}

67.2_{\pm 5.8}

69.7_{\pm 9.8}

80.1_{\pm 2.8}

68.3_{\pm 2.1}

68.0_{\pm 1.8}

bbp

665

\mathbf{73.5_{\pm 5.8}}

70.7_{\pm 5.4}

71.6_{\pm 5.5}

71.0_{\pm 7.0}

63.9_{\pm 3.9}

65.4_{\pm 4.5}

63.4_{\pm 4.4}

65.9_{\pm 6.4}

67.6_{\pm 4.5}

58.2_{\pm 7.8}

64.1_{\pm 6.8}

58.9_{\pm 7.4}

antimicrobial

52941

91.9_{\pm 0.4}

91.6_{\pm 0.5}

91.1_{\pm 0.5}

88.8_{\pm 0.5}

86.0_{\pm 0.7}

85.9_{\pm 0.9}

78.8_{\pm 1.1}

85.6_{\pm 0.8}

72.0_{\pm 5.5}

92.9_{\pm 0.5}

87.1_{\pm 1.8}

83.7_{\pm 1.2}

antibacterial

28591

93.3_{\pm 0.3}

92.6_{\pm 0.3}

92.2_{\pm 0.3}

90.2_{\pm 0.4}

88.2_{\pm 0.5}

88.3_{\pm 0.4}

83.6_{\pm 0.6}

88.3_{\pm 0.6}

70.1_{\pm 4.2}

93.6_{\pm 0.5}

87.0_{\pm 0.9}

82.4_{\pm 2.1}

Appendix

Table 23: Performance of models on canonical peptide classification (ROC-AUC

\uparrow

, %) with MMseqs2-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

XGBoost

LightGBM

GradBoost

AdaBoost

GCN

GAT

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

nonfouling

7200

77.1_{\pm 1.5}

76.9_{\pm 1.6}

77.4_{\pm 1.2}

77.6_{\pm 1.5}

77.0_{\pm 1.9}

78.1_{\pm 1.7}

76.7_{\pm 1.7}

78.4_{\pm 1.6}

77.5_{\pm 2.7}

77.3_{\pm 1.7}

75.6_{\pm 2.5}

70.9_{\pm 2.0}

cpp

2296

91.9_{\pm 2.8}

91.7_{\pm 3.0}

91.7_{\pm 2.9}

88.4_{\pm 2.4}

81.8_{\pm 3.2}

78.6_{\pm 3.4}

71.3_{\pm 5.0}

73.7_{\pm 7.8}

77.0_{\pm 5.4}

85.4_{\pm 2.6}

66.4_{\pm 5.3}

71.8_{\pm 2.6}

bbp

665

\mathbf{76.5_{\pm 4.3}}

\pagecolor{lightgray}{75.5_{\pm 4.4}}

75.0_{\pm 3.1}

71.2_{\pm 7.1}

59.7_{\pm 6.2}

59.7_{\pm 6.8}

58.8_{\pm 7.3}

60.0_{\pm 7.3}

62.3_{\pm 7.5}

54.9_{\pm 5.3}

60.8_{\pm 7.7}

47.1_{\pm 6.9}

antimicrobial

52941

93.9_{\pm 0.4}

93.2_{\pm 0.3}

92.6_{\pm 0.4}

90.4_{\pm 0.5}

87.5_{\pm 0.7}

88.1_{\pm 0.6}

80.5_{\pm 0.9}

88.0_{\pm 0.7}

76.2_{\pm 0.7}

94.3_{\pm 0.4}

88.5_{\pm 0.8}

85.0_{\pm 0.6}

antibacterial

28591

95.0_{\pm 0.4}

94.5_{\pm 0.4}

93.9_{\pm 0.4}

91.7_{\pm 0.4}

89.3_{\pm 0.7}

89.5_{\pm 0.5}

83.8_{\pm 1.3}

89.7_{\pm 0.6}

70.2_{\pm 3.1}

95.2_{\pm 0.2}

88.8_{\pm 0.6}

84.0_{\pm 1.0}

Appendix

Table 24: Performance of models on canonical peptide classification (ROC-AUC

\uparrow

, %) with random-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

XGBoost

LightGBM

GradBoost

AdaBoost

GCN

GAT

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

E.coli_mic

3204

0.593_{\pm 0.015}

0.604_{\pm 0.019}

0.584_{\pm 0.019}

0.629_{\pm 0.010}

0.677_{\pm 0.013}

0.670_{\pm 0.039}

0.665_{\pm 0.023}

0.631_{\pm 0.015}

0.650_{\pm 0.062}

0.603_{\pm 0.007}

0.621_{\pm 0.012}

0.650_{\pm 0.013}

S.aureus_mic

2822

0.569_{\pm 0.035}

0.588_{\pm 0.022}

0.572_{\pm 0.025}

0.613_{\pm 0.029}

0.651_{\pm 0.022}

0.643_{\pm 0.017}

0.639_{\pm 0.028}

0.635_{\pm 0.032}

0.648_{\pm 0.016}

0.627_{\pm 0.011}

0.659_{\pm 0.045}

0.657_{\pm 0.029}

P.aeruginosa_mic

1490

0.540_{\pm 0.030}

0.559_{\pm 0.025}

0.550_{\pm 0.026}

0.555_{\pm 0.019}

0.586_{\pm 0.006}

0.612_{\pm 0.047}

0.579_{\pm 0.027}

0.537_{\pm 0.032}

0.566_{\pm 0.037}

0.520_{\pm 0.041}

0.529_{\pm 0.035}

0.546_{\pm 0.029}

hemolytic_hc50

1926

0.517_{\pm 0.042}

0.527_{\pm 0.038}

0.523_{\pm 0.035}

0.546_{\pm 0.036}

0.619_{\pm 0.048}

0.603_{\pm 0.077}

0.539_{\pm 0.023}

0.528_{\pm 0.036}

0.516_{\pm 0.033}

0.498_{\pm 0.040}

0.539_{\pm 0.020}

0.535_{\pm 0.027}

Average

0.555

0.569

0.557

0.586

0.633

0.632

0.605

0.583

0.595

0.562

0.587

0.597

Appendix

Table 25: Performance of models on canonical peptide regression (MAE

\downarrow

, %) with hybrid-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

XGBoost

LightGBM

GradBoost

AdaBoost

GCN

GAT

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

E.coli_mic

3204

0.465_{\pm 0.022}

0.484_{\pm 0.014}

0.476_{\pm 0.016}

0.532_{\pm 0.023}

0.597_{\pm 0.027}

0.593_{\pm 0.038}

0.607_{\pm 0.025}

0.577_{\pm 0.011}

0.557_{\pm 0.026}

0.577_{\pm 0.026}

0.582_{\pm 0.021}

0.635_{\pm 0.021}

S.aureus_mic

2822

0.450_{\pm 0.025}

0.467_{\pm 0.022}

0.450_{\pm 0.014}

0.513_{\pm 0.029}

0.575_{\pm 0.036}

0.572_{\pm 0.016}

0.573_{\pm 0.029}

0.559_{\pm 0.020}

0.576_{\pm 0.027}

0.551_{\pm 0.029}

0.554_{\pm 0.022}

0.586_{\pm 0.033}

P.aeruginosa_mic

1490

0.435_{\pm 0.018}

0.440_{\pm 0.032}

0.439_{\pm 0.024}

0.466_{\pm 0.020}

0.510_{\pm 0.015}

0.517_{\pm 0.024}

0.539_{\pm 0.024}

0.504_{\pm 0.014}

0.521_{\pm 0.017}

0.493_{\pm 0.032}

0.511_{\pm 0.016}

0.533_{\pm 0.029}

hemolytic_hc50

1926

0.383_{\pm 0.024}

0.392_{\pm 0.029}

0.391_{\pm 0.029}

0.428_{\pm 0.018}

0.512_{\pm 0.035}

0.561_{\pm 0.098}

0.483_{\pm 0.028}

0.475_{\pm 0.009}

0.454_{\pm 0.021}

0.439_{\pm 0.025}

0.501_{\pm 0.039}

0.490_{\pm 0.025}

Average

0.433

0.446

0.439

0.485

0.548

0.561

0.550

0.529

0.527

0.515

0.537

0.561

Appendix

Table 26: Performance of models on canonical peptide regression (MAE

\downarrow

, %) with kmer-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

XGBoost

LightGBM

GradBoost

AdaBoost

GCN

GAT

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

E.coli_mic

3204

0.573_{\pm 0.035}

0.583_{\pm 0.036}

0.578_{\pm 0.034}

0.620_{\pm 0.027}

0.676_{\pm 0.031}

0.662_{\pm 0.040}

0.639_{\pm 0.027}

0.618_{\pm 0.043}

0.639_{\pm 0.042}

0.583_{\pm 0.013}

0.608_{\pm 0.027}

0.630_{\pm 0.021}

S.aureus_mic

2822

0.568_{\pm 0.026}

0.592_{\pm 0.024}

0.577_{\pm 0.026}

0.620_{\pm 0.022}

0.644_{\pm 0.026}

0.631_{\pm 0.012}

0.624_{\pm 0.025}

0.616_{\pm 0.022}

0.638_{\pm 0.043}

0.615_{\pm 0.023}

0.636_{\pm 0.027}

0.644_{\pm 0.033}

P.aeruginosa_mic

1490

0.518_{\pm 0.030}

0.523_{\pm 0.032}

0.526_{\pm 0.023}

0.532_{\pm 0.020}

0.577_{\pm 0.023}

0.575_{\pm 0.049}

0.565_{\pm 0.019}

0.558_{\pm 0.036}

0.561_{\pm 0.013}

0.547_{\pm 0.036}

0.561_{\pm 0.042}

0.568_{\pm 0.028}

hemolytic_hc50

1926

0.547_{\pm 0.034}

0.554_{\pm 0.047}

0.543_{\pm 0.039}

0.561_{\pm 0.038}

0.627_{\pm 0.032}

0.617_{\pm 0.075}

0.548_{\pm 0.034}

0.532_{\pm 0.028}

0.540_{\pm 0.017}

0.532_{\pm 0.045}

0.556_{\pm 0.023}

0.538_{\pm 0.031}

Average

0.552

0.563

0.556

0.583

0.631

0.621

0.594

0.581

0.594

0.569

0.590

0.595

Appendix

Table 27: Performance of models on canonical peptide regression (MAE

\downarrow

, %) with MMseqs2-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

XGBoost

LightGBM

GradBoost

AdaBoost

GCN

GAT

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

E.coli_mic

3204

0.468_{\pm 0.022}

0.484_{\pm 0.014}

0.476_{\pm 0.016}

0.532_{\pm 0.023}

0.593_{\pm 0.025}

0.587_{\pm 0.028}

0.613_{\pm 0.025}

0.570_{\pm 0.025}

0.565_{\pm 0.017}

0.551_{\pm 0.045}

0.577_{\pm 0.028}

0.640_{\pm 0.021}

S.aureus_mic

2822

0.448_{\pm 0.020}

0.467_{\pm 0.022}

0.450_{\pm 0.014}

0.513_{\pm 0.029}

0.577_{\pm 0.036}

0.572_{\pm 0.028}

0.573_{\pm 0.036}

0.554_{\pm 0.021}

0.581_{\pm 0.021}

0.558_{\pm 0.040}

0.552_{\pm 0.018}

0.587_{\pm 0.033}

P.aeruginosa_mic

1490

0.433_{\pm 0.021}

0.440_{\pm 0.032}

0.439_{\pm 0.024}

0.465_{\pm 0.020}

0.508_{\pm 0.020}

0.532_{\pm 0.037}

0.529_{\pm 0.021}

0.499_{\pm 0.015}

0.513_{\pm 0.021}

0.491_{\pm 0.033}

0.513_{\pm 0.032}

0.534_{\pm 0.032}

hemolytic_hc50

1926

0.377_{\pm 0.026}

0.392_{\pm 0.029}

0.391_{\pm 0.029}

0.428_{\pm 0.018}

0.519_{\pm 0.022}

0.515_{\pm 0.058}

0.488_{\pm 0.035}

0.468_{\pm 0.021}

0.454_{\pm 0.023}

0.433_{\pm 0.019}

0.494_{\pm 0.034}

0.490_{\pm 0.025}

Average

0.432

0.446

0.439

0.485

0.549

0.551

0.523

0.528

0.508

0.534

0.563

Appendix

Table 28: Performance of models on canonical peptide regression (MAE

\downarrow

, %) with random-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Task	Metric	Dataset	Size	FP-based models					GNN-based models				SMILES-based models
Task	Metric	Dataset	Size	RF	XGBoost	LightGBM	GradBoost	AdaBoost	GCN	GAT	GIN	Pepland	ChemBERTa	PeptideCLM	PepDoRA
cls	AUC ROC	nc-antibacterial	1668	$\mathbf{94.4_{\pm 1.4}}$	$93.6_{\pm 2.2}$	$\pagecolor{lightgray}{93.8_{\pm 1.8}}$	$92.5_{\pm 2.8}$	$90.9_{\pm 3.4}$	$93.0_{\pm 2.6}$	$81.5_{\pm 3.5}$	$90.7_{\pm 4.9}$	$84.9_{\pm 3.9}$	$91.9_{\pm 2.7}$	$71.4_{\pm 1.8}$	$72.4_{\pm 7.4}$
		nc-antifungal	407	$\pagecolor{lightgray}{95.4_{\pm 3.2}}$	$\mathbf{96.5_{\pm 2.9}}$	$94.8_{\pm 3.1}$	$95.2_{\pm 3.8}$	$95.4_{\pm 2.4}$	$70.1_{\pm 12.0}$	$78.5_{\pm 6.2}$	$86.1_{\pm 2.8}$	$83.5_{\pm 4.4}$	$78.0_{\pm 18.3}$	$65.4_{\pm 7.0}$	$65.9_{\pm 12.0}$
		nc-antimicrobial	2465	$97.6_{\pm 0.9}$	$\pagecolor{lightgray}{97.7_{\pm 0.9}}$	$\mathbf{97.8_{\pm 0.6}}$	$97.3_{\pm 0.7}$	$95.3_{\pm 1.8}$	$94.9_{\pm 1.6}$	$79.2_{\pm 4.4}$	$91.5_{\pm 2.3}$	$88.0_{\pm 0.9}$	$95.2_{\pm 1.9}$	$68.3_{\pm 4.7}$	$80.0_{\pm 1.9}$
		nc-hemolytic	3971	$96.1_{\pm 1.1}$	$\pagecolor{lightgray}{96.2_{\pm 0.5}}$	$\mathbf{96.3_{\pm 0.7}}$	$95.3_{\pm 0.8}$	$93.7_{\pm 1.6}$	$89.5_{\pm 3.7}$	$85.7_{\pm 2.6}$	$89.0_{\pm 4.1}$	$82.8_{\pm 2.1}$	$91.4_{\pm 1.5}$	$76.1_{\pm 4.5}$	$72.1_{\pm 7.3}$
		avg	-	$\pagecolor{lightgray}{95.9}$	$\mathbf{96.0}$	$95.7$	$95.1$	$93.8$	$86.9$	$81.2$	$89.3$	$84.8$	$89.1$	$70.3$	$72.6$

Appendix

Table 29: Performance of models on non-canonical peptide classification (ROC-AUC

\uparrow

, %) and regression (MAE

\downarrow

, %) with ECFP-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Task	Metric	Dataset	Size	FP-based models					GNN-based models				SMILES-based models
Task	Metric	Dataset	Size	RF	XGBoost	LightGBM	GradBoost	AdaBoost	GCN	GAT	GIN	Pepland	ChemBERTa	PeptideCLM	PepDoRA
cls	AUC ROC	nc-antibacterial	1668	$\mathbf{94.1_{\pm 1.4}}$	$93.8_{\pm 1.9}$	$\pagecolor{lightgray}{94.0_{\pm 2.2}}$	$\pagecolor{lightgray}{94.0_{\pm 2.4}}$	$91.8_{\pm 2.6}$	$88.1_{\pm 6.6}$	$82.0_{\pm 4.0}$	$89.7_{\pm 1.9}$	$82.9_{\pm 1.6}$	$92.2_{\pm 1.8}$	$71.9_{\pm 4.5}$	$75.1_{\pm 8.8}$
		nc-antifungal	407	$\mathbf{97.3_{\pm 1.1}}$	$\pagecolor{lightgray}{96.0_{\pm 1.9}}$	$96.0_{\pm 1.6}$	$94.9_{\pm 1.1}$	$96.0_{\pm 1.9}$	$70.5_{\pm 2.9}$	$77.4_{\pm 9.8}$	$81.5_{\pm 8.5}$	$84.5_{\pm 2.4}$	$87.3_{\pm 6.9}$	$69.8_{\pm 8.2}$	$61.8_{\pm 14.1}$
		nc-antimicrobial	2465	$96.8_{\pm 1.3}$	$\pagecolor{lightgray}{96.9_{\pm 0.7}}$	$\mathbf{97.0_{\pm 0.9}}$	$96.7_{\pm 0.4}$	$95.6_{\pm 1.2}$	$91.8_{\pm 1.3}$	$83.5_{\pm 2.7}$	$89.2_{\pm 3.1}$	$86.9_{\pm 2.0}$	$95.2_{\pm 0.8}$	$75.4_{\pm 2.6}$	$80.2_{\pm 4.6}$
		nc-hemolytic	3971	$\pagecolor{lightgray}{96.5_{\pm 1.3}}$	$96.0_{\pm 1.4}$	$\mathbf{96.8_{\pm 1.2}}$	$95.7_{\pm 1.7}$	$93.8_{\pm 2.2}$	$90.9_{\pm 3.8}$	$83.9_{\pm 5.4}$	$91.2_{\pm 2.8}$	$84.3_{\pm 4.8}$	$92.0_{\pm 2.5}$	$76.1_{\pm 9.5}$	$76.8_{\pm 7.3}$
		avg	-	$\mathbf{96.2}$	$95.7$	$\pagecolor{lightgray}{95.9}$	$95.3$	$94.3$	$85.3$	$81.7$	$87.9$	$84.7$	$91.7$	$73.3$	$73.5$

Appendix

Table 30: Performance of models on non-canonical peptide classification (ROC-AUC

\uparrow

, %) and regression (MAE

\downarrow

, %) with hybrid-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Task	Metric	Dataset	Size	FP-based models					GNN-based models				SMILES-based models
Task	Metric	Dataset	Size	RF	XGBoost	LightGBM	GradBoost	AdaBoost	GCN	GAT	GIN	Pepland	ChemBERTa	PeptideCLM	PepDoRA
cls	AUC ROC	nc-antibacterial	1668	$94.7_{\pm 0.9}$	$\pagecolor{lightgray}{94.8_{\pm 0.9}}$	$\mathbf{95.2_{\pm 1.3}}$	$94.4_{\pm 0.8}$	$91.6_{\pm 2.3}$	$92.2_{\pm 3.0}$	$80.6_{\pm 7.4}$	$85.6_{\pm 12.0}$	$83.6_{\pm 4.9}$	$91.9_{\pm 2.8}$	$72.2_{\pm 5.7}$	$73.2_{\pm 5.6}$
		nc-antifungal	407	$\mathbf{98.9_{\pm 1.1}}$	$98.7_{\pm 1.0}$	$\pagecolor{lightgray}{98.9_{\pm 0.6}}$	$98.3_{\pm 0.9}$	$98.1_{\pm 0.9}$	$71.8_{\pm 12.8}$	$77.7_{\pm 9.8}$	$89.6_{\pm 5.3}$	$77.3_{\pm 20.1}$	$91.4_{\pm 14.6}$	$65.9_{\pm 9.3}$	$57.2_{\pm 10.7}$
		nc-antimicrobial	2465	$\mathbf{97.9_{\pm 0.9}}$	$97.4_{\pm 0.9}$	$\pagecolor{lightgray}{97.6_{\pm 0.8}}$	$97.6_{\pm 0.8}$	$95.7_{\pm 1.4}$	$93.5_{\pm 1.8}$	$82.6_{\pm 5.8}$	$92.2_{\pm 3.5}$	$87.8_{\pm 1.5}$	$96.1_{\pm 0.8}$	$77.9_{\pm 3.0}$	$81.1_{\pm 1.7}$
		nc-hemolytic	3971	$97.1_{\pm 0.9}$	$\pagecolor{lightgray}{97.4_{\pm 0.9}}$	$\mathbf{97.8_{\pm 0.7}}$	$96.7_{\pm 0.9}$	$94.0_{\pm 0.8}$	$86.9_{\pm 4.2}$	$81.8_{\pm 0.9}$	$88.3_{\pm 3.7}$	$83.2_{\pm 2.7}$	$93.2_{\pm 1.2}$	$75.1_{\pm 3.8}$	$77.9_{\pm 3.1}$
		avg	-	$\pagecolor{lightgray}{97.2}$	$97.1$	$\mathbf{97.4}$	$96.7$	$94.9$	$86.1$	$80.7$	$88.9$	$83.0$	$93.1$	$72.8$	$72.4$

Appendix

Table 31: Performance of models on non-canonical peptide classification (ROC-AUC

\uparrow

, %) and regression (MAE

\downarrow

, %) with kmer-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Task	Metric	Dataset	Size	FP-based models					GNN-based models				SMILES-based models
Task	Metric	Dataset	Size	RF	XGBoost	LightGBM	GradBoost	AdaBoost	GCN	GAT	GIN	Pepland	ChemBERTa	PeptideCLM	PepDoRA
cls	AUC ROC	nc-antibacterial	1668	$\mathbf{97.9_{\pm 1.2}}$	$97.7_{\pm 0.7}$	$\pagecolor{lightgray}{97.8_{\pm 1.1}}$	$97.1_{\pm 1.1}$	$95.7_{\pm 1.4}$	$96.2_{\pm 1.6}$	$89.9_{\pm 3.9}$	$95.8_{\pm 3.5}$	$90.7_{\pm 2.3}$	$96.9_{\pm 1.6}$	$84.5_{\pm 2.0}$	$86.4_{\pm 3.5}$
		nc-antifungal	407	$\pagecolor{lightgray}{95.4_{\pm 3.4}}$	$93.5_{\pm 2.9}$	$93.6_{\pm 3.4}$	$93.2_{\pm 4.6}$	$\mathbf{95.4_{\pm 2.5}}$	$73.8_{\pm 1.7}$	$79.8_{\pm 5.6}$	$85.2_{\pm 4.3}$	$84.3_{\pm 5.9}$	$92.2_{\pm 7.0}$	$75.3_{\pm 5.0}$	$61.5_{\pm 5.1}$
		nc-antimicrobial	2465	$\mathbf{98.0_{\pm 0.4}}$	$97.8_{\pm 0.4}$	$\pagecolor{lightgray}{97.9_{\pm 0.5}}$	$97.8_{\pm 0.5}$	$96.4_{\pm 0.6}$	$95.8_{\pm 1.1}$	$87.4_{\pm 3.6}$	$95.6_{\pm 1.8}$	$89.4_{\pm 1.1}$	$96.7_{\pm 1.1}$	$82.1_{\pm 1.3}$	$87.5_{\pm 2.1}$
		nc-hemolytic	3971	$\pagecolor{lightgray}{98.2_{\pm 0.4}}$	$98.1_{\pm 0.6}$	$\mathbf{98.3_{\pm 0.5}}$	$97.5_{\pm 0.7}$	$95.7_{\pm 0.7}$	$94.4_{\pm 1.1}$	$89.8_{\pm 2.9}$	$94.9_{\pm 1.1}$	$90.8_{\pm 0.7}$	$96.2_{\pm 1.1}$	$85.4_{\pm 2.5}$	$87.9_{\pm 1.2}$
		avg	-	$\mathbf{97.4}$	$96.8$	$\pagecolor{lightgray}{96.9}$	$96.4$	$95.8$	$90.1$	$86.7$	$92.9$	$88.8$	$95.5$	$81.8$	$80.8$

Appendix

Table 32: Performance of models on non-canonical peptide classification (ROC-AUC

\uparrow

, %) and regression (MAE

\downarrow

, %) with MMseqs2-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Task	Metric	Dataset	Size	FP-based models					GNN-based models				SMILES-based models
Task	Metric	Dataset	Size	RF	XGBoost	LightGBM	GradBoost	AdaBoost	GCN	GAT	GIN	Pepland	ChemBERTa	PeptideCLM	PepDoRA
cls	AUC ROC	nc-antibacterial	1668	$\pagecolor{lightgray}{98.4_{\pm 1.1}}$	$98.4_{\pm 1.0}$	$\mathbf{98.7_{\pm 0.9}}$	$98.1_{\pm 1.0}$	$97.0_{\pm 1.1}$	$97.1_{\pm 0.6}$	$94.1_{\pm 1.5}$	$96.6_{\pm 2.0}$	$92.5_{\pm 2.8}$	$97.8_{\pm 1.1}$	$85.5_{\pm 4.3}$	$86.0_{\pm 3.7}$
		nc-antifungal	407	$\mathbf{97.5_{\pm 2.5}}$	$\pagecolor{lightgray}{96.6_{\pm 2.6}}$	$96.2_{\pm 3.2}$	$96.2_{\pm 3.1}$	$95.1_{\pm 3.6}$	$85.1_{\pm 11.6}$	$83.4_{\pm 5.8}$	$85.1_{\pm 8.5}$	$85.8_{\pm 9.7}$	$95.2_{\pm 4.9}$	$79.1_{\pm 6.4}$	$72.2_{\pm 11.6}$
		nc-antimicrobial	2465	$\mathbf{99.1_{\pm 0.4}}$	$98.9_{\pm 0.3}$	$\pagecolor{lightgray}{99.0_{\pm 0.3}}$	$98.6_{\pm 0.3}$	$97.7_{\pm 0.6}$	$96.2_{\pm 1.7}$	$91.6_{\pm 1.9}$	$97.0_{\pm 1.0}$	$92.7_{\pm 1.7}$	$97.9_{\pm 0.8}$	$88.2_{\pm 1.5}$	$85.9_{\pm 2.7}$
		-hemolytic	3971	$\mathbf{99.0_{\pm 0.2}}$	$98.7_{\pm 0.2}$	$\pagecolor{lightgray}{98.9_{\pm 0.2}}$	$98.2_{\pm 0.5}$	$96.6_{\pm 0.8}$	$95.7_{\pm 0.9}$	$89.1_{\pm 1.9}$	$94.3_{\pm 3.0}$	$89.1_{\pm 1.5}$	$97.6_{\pm 0.8}$	$83.6_{\pm 2.2}$	$87.2_{\pm 2.1}$
		avg	-	$\mathbf{98.5}$	$98.1$	$\pagecolor{lightgray}{98.2}$	$97.8$	$96.6$	$93.5$	$89.5$	$93.3$	$90.0$	$97.1$	$84.1$	$82.9$

Appendix

Table 33: Performance of models on non-canonical peptide classification (ROC-AUC

\uparrow

, %) and regression (MAE

\downarrow

, %) with random-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Task

Metric

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

PLM-based models

ECFP4

ECFP6

GCN

GAT

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

ESM2-8M

ESM2-35M

ESM2-150M

ESM2-650M

ESM2-150M-F

cls

AUC ROC

ppi

44148

48.7_{\pm 3.2}

46.7_{\pm 3.9}

47.5_{\pm 2.3}

49.5_{\pm 4.3}

51.1_{\pm 3.9}

50.5_{\pm 3.5}

51.9_{\pm 4.4}

52.4_{\pm 2.9}

51.0_{\pm 2.5}

50.6_{\pm 6.6}

51.8_{\pm 4.4}

52.6_{\pm 2.3}

\mathbf{56.0_{\pm 3.1}}

\pagecolor{lightgray}{55.0_{\pm 2.9}}

reg

MAE

ppi_ba

1433

\pagecolor{lightgray}{1.040_{\pm 0.052}}

1.052_{\pm 0.063}

1.103_{\pm 0.032}

1.128_{\pm 0.030}

1.112_{\pm 0.030}

1.091_{\pm 0.048}

1.053_{\pm 0.025}

1.124_{\pm 0.104}

1.125_{\pm 0.031}

1.059_{\pm 0.043}

1.052_{\pm 0.062}

1.062_{\pm 0.060}

1.082_{\pm 0.013}

\mathbf{1.026_{\pm 0.043}}

nc_ppi_ba

278

\pagecolor{lightgray}{1.425_{\pm 0.249}}

1.462_{\pm 0.177}

1.468_{\pm 0.183}

1.451_{\pm 0.218}

1.489_{\pm 0.208}

1.584_{\pm 0.144}

1.473_{\pm 0.217}

1.474_{\pm 0.129}

\mathbf{1.422_{\pm 0.100}}

Appendix

Table 34: Performance of models on PepPI classification (ROC-AUC

\uparrow

, %) and regression (MAE

\downarrow

, %) with cold-start-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Task

Metric

Dataset

Size

FP-based models

GNN-based models

SMILES-based models

PLM-based models

ECFP4

ECFP6

GIN

Pepland

ChemBERTa

PeptideCLM

PepDoRA

ESM2-8M

ESM2-35M

ESM2-150M

ESM2-650M

ESM2-150M-F

cls

AUC ROC

ppi

44148

54.4_{\pm 2.3}

53.7_{\pm 2.3}

61.3_{\pm 7.2}

59.6_{\pm 2.7}

52.0_{\pm 4.4}

51.4_{\pm 3.6}

59.4_{\pm 2.8}

\mathbf{60.2_{\pm 6.3}}

57.6_{\pm 7.2}

55.4_{\pm 3.5}

51.9_{\pm 2.4}

56.0_{\pm 3.1}

reg

MAE

ppi_ba

1433

1.043_{\pm 0.050}

1.043_{\pm 0.060}

1.189_{\pm 0.140}

1.176_{\pm 0.152}

\mathbf{1.034_{\pm 0.013}}

1.128_{\pm 0.042}

1.084_{\pm 0.044}

1.061_{\pm 0.043}

1.056_{\pm 0.092}

1.079_{\pm 0.038}

1.051_{\pm 0.072}

1.038_{\pm 0.030}

nc_ppi_ba

278

1.665_{\pm 0.260}

1.647_{\pm 0.286}

1.741_{\pm 0.359}

1.705_{\pm 0.366}

1.613_{\pm 0.189}

1.580_{\pm 0.142}

\mathbf{1.465_{\pm 0.234}}

Appendix

Table 35: Performance of models on PepPI classification (ROC-AUC

\uparrow

, %) and regression (MAE

\downarrow

, %) with cold-start-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Dataset	Size	FP-based models					GNN-based models
Dataset	Size	RF	XGBoost	LightGBM	GradBoost	AdaBoost	GCN	GAT	GIN	Pepland
antimicrobial	52941	$88.0_{\pm 0.3}$	$87.5_{\pm 0.5}$	$87.3_{\pm 0.4}$	$85.1_{\pm 0.3}$	$82.5_{\pm 0.4}$	$81.4_{\pm 0.7}$	$72.3_{\pm 1.5}$	$80.2_{\pm 1.6}$	$80.2_{\pm 1.3}$
antifungal	12887	$87.0_{\pm 1.0}$	$86.0_{\pm 0.7}$	$85.6_{\pm 0.8}$	$83.8_{\pm 0.6}$	$82.0_{\pm 0.8}$	$76.0_{\pm 4.4}$	$73.5_{\pm 1.6}$	$78.9_{\pm 2.9}$	$75.2_{\pm 2.4}$
antiparasitic	6755	$86.8_{\pm 1.2}$	$86.2_{\pm 1.7}$	$86.5_{\pm 0.8}$	$84.5_{\pm 1.1}$	$82.8_{\pm 1.6}$	$75.6_{\pm 1.5}$	$75.8_{\pm 1.4}$	$74.4_{\pm 2.6}$	$74.8_{\pm 2.3}$
anticancer	12013	$86.9_{\pm 0.8}$	$86.4_{\pm 0.6}$	$86.0_{\pm 0.5}$	$83.3_{\pm 0.8}$	$81.0_{\pm 0.8}$	$75.0_{\pm 4.1}$	$73.7_{\pm 1.0}$	$77.3_{\pm 4.3}$	$78.1_{\pm 1.2}$
allergen	2538	$76.5_{\pm 2.9}$	$82.2_{\pm 2.0}$	$82.2_{\pm 1.4}$	$79.4_{\pm 1.7}$	$66.8_{\pm 2.8}$	$63.3_{\pm 1.6}$	$59.9_{\pm 4.7}$	$67.2_{\pm 2.7}$	$68.8_{\pm 2.5}$

Appendix

Table 36: Models’ classification (ROC-AUC

\uparrow

, %) performance on the canonical peptide dataset (length

>

150) using hybrid-split. Dataset sizes are shown separately; results are mean ±std . Best and second-best scores per row are in bold and gray shadow.

Category	Metric	Value
Validity	BLIN $\rightarrow$ SMILES Success Rate	89.7%
	Chemical Validity Rate	100%
	Overall Success Rate	89.7%
Closeness to Ground Truth	Exact Match Rate	21.9%
Closeness to Ground Truth	Avg. Tanimoto Similarity	87.9%
Diversity and Novelty	Internal Diversity	66.7%

Appendix

Table 37: Comprehensive evaluation of generated molecules across multiple dimensions.

Panel of Figure 112 — Figure 112: Comparison of physicochemical property distributions between generated molecules and the training set.

Drug Discovery

Drug discovery identifies, designs, and tests molecules that can safely affect disease biology.

See all 164 papers

Feb 27, 2026

CoPeP: Benchmarking Continual Pretraining for Protein Language Models

Protein language models (pLMs) have recently gained significant attention for their ability to uncover relationships between sequence, structure, and function from evolutionary statistics, thereby accelerating therapeutic drug discovery. These models learn from large protein databases that are continuously updated by the biology community and whose dynamic nature motivates the application of continual learning, not only to keep up with the ever-growing data, but also as an opportunity to take advantage of the temporal meta-information that is created during this process. As a result, we introduce the Continual Pretraining of Protein Language Models (CoPeP) benchmark, a novel benchmark for evaluating continual learning approaches on pLMs. Specifically, we curate a sequence of protein datasets derived from the UniProt Knowledgebase spanning a decade and define metrics to assess pLM performance across 31 protein understanding tasks. We evaluate several methods from the continual learning literature, including replay, unlearning, and plasticity-based methods, some of which have never been applied to models and data of this scale. Our findings reveal that incorporating temporal meta-information improves perplexity by up to 7% even when compared to training on data from all tasks jointly. Moreover, even at scale, several continual learning methods outperform naive continual pretraining. The CoPeP benchmark offers an exciting opportunity to study these methods at scale in an impactful real-world application.

Darshan Patil, Pranshu Malviya, Mathieu Reymond +2

Mila - Quebec AI Institute · Polytechnique Montréal · Chandar Research Lab +1

Protein Language Models Replay-Based Continual Learning

Ask PDF

May 3, 2026

PepSpecBench: A Unified Evaluation Benchmark for Peptide Tandem Mass Spectrometry Prediction

Tandem mass spectrometry provides a high-throughput framework for identifying and quantifying proteins in complex biological samples. In computational proteomics, predicting peptide MS/MS spectra is a critical task, enabling downstream applications such as large-scale peptide identification and quantification. While deep learning architectures have substantially improved prediction accuracy, three evaluation challenges obscure the true progress of the field. First, inconsistent data preprocessing and incompatible model output spaces hinder fair model comparison. Second, flawed data splitting strategies can permit hidden sequence leakage and inflate reported performance. Third, existing evaluations typically lack comprehensive cross-species benchmarking and systematic assessment of model robustness to influential experimental conditions. To address these challenges, we propose PepSpecBench, a unified benchmark for peptide MS/MS spectrum prediction. PepSpecBench standardizes data preprocessing across complementary public datasets, enforces a strict backbone-disjoint splitting strategy to eliminate sequence leakage, and evaluates diverse architectures within a shared fragment-ion representation space. It further introduces a comprehensive multi-species evaluation suite and physically grounded metadata perturbation probes to assess model robustness and instrument awareness. We uncover previously unrecognized performance discrepancies and robustness limitations across six representative models, providing actionable insights for future model design, evaluation and practical deployment.

Zhiwen Yang, Pan Liu, Yifan Li +2

The Hong Kong University of Science and Technology (Guangzhou). · Yangzhou University. · The Hong Kong University of Science and Technology.

Benchmark Design Benchmark Contamination Mass Spectrometry

Ask PDF

Jul 28, 2026

AMPBench-MT: A Homology-Controlled Benchmark for Antimicrobial Peptide Potency, Spectrum, and Safety Prediction

Computational AMP discovery is often evaluated through AMP/non-AMP recognition, yet follow-up decisions depend on assay-derived evidence such as target-species potency, hemolysis, toxicity, and selectivity. Existing AMP and peptide benchmarks cover binary recognition, multilabel annotation, assay regression, or broader peptide-model comparison, but they do not jointly place AMP recognition, species-conditioned potency, spectrum, safety-facing proxy endpoints, and cross-endpoint behavior within one sequence-homology-controlled protocol. To address this problem, we introduce AMPBench-MT, a provenance-preserving benchmark that standardizes canonical peptide records and organizes them into binary recognition, species-conditioned pMIC regression, and endpoint-specific potency and safety-facing readouts. Across 161 endpoint-specific model evaluations, high binary performance does not reliably indicate assay-endpoint behavior. Frozen protein-language-model embeddings form the leading pMIC error cluster, while graph and classical regressors remain close. Spectrum labels further reveal that PR-oriented metrics can be misleading under scarce observed negatives, whereas low-toxicity, HC50 hemolysis, and selectivity expose smaller but more assay-facing signals. AMPBench-MT shows that AMP evaluation should move beyond recognition leaderboards toward endpoint-aware evidence auditing. Our proposed benchmark is available at https://huggingface.co/datasets/ZihengZhou06/AMPBench-MT.

Ziheng Zhou, Huiyu Luo, Xiaohu Zhu +4

Shanghai Ocean University, Shanghai, China · Tongji University, Shanghai, China · Center for Safe AGI, Shanghai, China +1

Benchmark Design Toxicity Detection Molecular Property Prediction

Ask PDF

Abstract

Drug Discovery

Explore similar work

CoPeP: Benchmarking Continual Pretraining for Protein Language Models

PepSpecBench: A Unified Evaluation Benchmark for Peptide Tandem Mass Spectrometry Prediction

AMPBench-MT: A Homology-Controlled Benchmark for Antimicrobial Peptide Potency, Spectrum, and Safety Prediction