Limitations of Sequence-Based Protein Representations for Parkinson's Disease Classification: A Leakage-Free Benchmark

César Jesús Núñez-Prado; Grigori Sidorov; Liliana Chanona-Hernández

q-bio.QMApr 13, 2026

Limitations of Sequence-Based Protein Representations for Parkinson's Disease Classification: A Leakage-Free Benchmark

Authors: César Jesús Núñez-Prado, Grigori Sidorov, Liliana Chanona-Hernández

Organizations: Higher School of Mechanical and Electrical Engineering, Instituto Politécnico Nacional, Mexico City, Mexico · Research Center for Computing, Instituto Politécnico Nacional, Mexico City, Mexico

Abstract

The identification of reliable molecular biomarkers for Parkinson's disease remains challenging due to its multifactorial nature. Although protein sequences constitute a fundamental and widely available source of biological information, their standalone discriminative capacity for complex disease classification remains unclear. In this work, we present a controlled and leakage-free evaluation of multiple representations derived exclusively from protein primary sequences, including amino acid composition, k-mers, physicochemical descriptors, hybrid representations, and embeddings from protein language models, all assessed under a nested stratified cross-validation framework to ensure unbiased performance estimation. The best-performing configuration (ProtBERT + MLP) achieves an F1-score of 0.704 +/- 0.028 and ROC-AUC of 0.748 +/- 0.047, indicating only moderate discriminative performance. Classical representations such as k-mers reach comparable F1 values (up to approximately 0.667), but exhibit highly imbalanced behavior, with recall close to 0.98 and precision around 0.50, reflecting a strong bias toward positive predictions. Across representations, performance differences remain within a narrow range (F1 between 0.60 and 0.70), while unsupervised analyses reveal no intrinsic structure aligned with class labels, and statistical testing (Friedman test, p = 0.1749) does not indicate significant differences across models. These results demonstrate substantial overlap between classes and indicate that primary sequence information alone provides limited discriminative power for Parkinson's disease classification. This work establishes a reproducible baseline and provides empirical evidence that more informative biological features, such as structural, functional, or interaction-based descriptors, are required for robust disease modeling.

Figures & tables

Study	Parkinson-focused	Sequence-based	Primary contribution	Gap relevance
Zarkali et al. [ 22 ]	Yes	No	Neuroimaging and fluid biomarkers for PD	Disease focus without sequence benchmarking
Mei et al. [ 13 ]	Yes	No	Machine learning for PD diagnosis	Disease modeling with non-sequence data
Rabie and Akhloufi [ 16 ]	Yes	No	ML/DL methods for PD detection	Disease focus with heterogeneous modalities
Asgari and Mofrad [ 2 ]	No	Yes	Distributed biological sequence embeddings	Sequence modeling without PD focus
Rao et al. [ 18 ]	No	Yes	Protein transfer learning benchmark	Sequence benchmarking without PD focus
Rives et al. [ 19 ]	No	Yes	Large-scale protein language modeling	Advanced sequence modeling without PD focus

Table 1: Representative literature highlighting the gap between Parkinson’s disease modeling studies and sequence-based protein representation research. Categories are derived from the scope explicitly indicated by the cited works.

Panel of Figure 1 — Figure 1: Experimental pipeline used in this work.

Feature	Value
Total proteins	304
Parkinson-associated proteins	152
Control proteins	152
Organism	Homo sapiens
Average length (aa)	521
Standard deviation (aa)	496

Table 2: Dataset summary

ID	Description	Length	Sequence fragment
A7MD48 (SRRM4)	Serine/arginine repetitive matrix protein 4	611	MASVQQGEKQLFEKFWRTGTFKA VATPRP
O00217 (NDUFS8)	NADH dehydrogenase iron-sulfur protein 8	210	MRCLTTPMLLRALAQAARAGPGG RSLHSS
O00592 (PODXL)	Podocalyxin	558	MRCALALSALLLLLSTPPLLPSP SPSPSP
P49279 (NRAMP1)	Natural resistance-associated macrophage protein 1	550	MITGDKGPQRLSGSSYGSISSPT SPSPGQ
Q60568 (PLOD3)	Multifunctional procollagen lysine hydroxylase 3	738	MTSSGPGPRFLLLLLLPLLPPAA SASDRP
Q96L34 (MARK4)	MAP/microtubule affinity-regulating kinase 4	752	MSSRTVLAPGNDRNSDTHGTLGS GRSSDK

Table 3: Representative examples of dataset records. Only initial fragments of protein sequences are shown for illustration purposes.

Panel of Figure 2 — Figure 2: Nested cross-validation scheme. The outer loop is used for performance estimation, while the inner loop performs hyperparameter tuning and model selection. All data-dependent transformations, including preprocessing and feature selection, are fitted exclusively on the training data within each fold, ensuring strict separation between training and evaluation and preventing data leakage.

Representation	Dimension
Classical descriptors
Length (original + log)	2
Amino acid composition	20
5 most frequent amino acids (per sequence)	5
Engineered representations
k -mers ( $k=2$ )	400

Table 4: Feature representations considered in this study.

Model	Selected hyperparameters (range observed)
KNN	$k\in\{3,5,7,9\}$ , weights $\in\{\text{uniform},\text{distance}\}$
Logistic Regression	$C\in\{0.1,1,10\}$ , class_weight $\in\{\text{balanced},\text{None}\}$
SVM (RBF)	$C\in\{0.1,1,10\}$ , $\gamma\in\{\text{scale},0.1,0.01\}$
Random Forest	$n\_estimators\in\{100,200\}$ , max_depth $\in\{\text{None},10\}$ ,
	min_samples_leaf $\in\{1,2\}$

Table 5: Summary of optimal hyperparameters identified during nested cross-validation.

Statistic	Parkinson	Control
Mean	603.178	441.007
Std	662.172	214.780
Min	26.000	111.000
Max	4678.000	956.000

Table 6: Protein length statistics by class.

Figure 3: Distribution of protein length by class.

Figure 4: Histogram of protein length by class.

Figure 5: Average amino acid composition by class.

Panel of Figure 6 — Figure 6: PCA projections for different feature representations, showing substantial overlap between Parkinson-associated and control proteins across all cases.

Representation	Method	Silhouette	ARI	NMI	Accuracy	F1	Specificity
k-mers	K-Means	0.0173	0.0181	0.0185	0.5724	0.6389	0.3882
k-mers	Agglomerative	0.4925	0.0000	0.0064	0.5033	0.0131	1.0000
Amino acid composition	K-Means	0.1218	0.0143	0.0140	0.5658	0.6185	0.4276
Amino acid composition	Agglomerative	0.1127	0.0146	0.0161	0.5658	0.6393	0.3618
Physicochemical properties	K-Means	0.1743	0.0092	0.0090	0.5559	0.5603	0.5461
Physicochemical properties	Agglomerative	0.1575	0.0031	0.0049	0.5395	0.5882	0.4211

Table 7: Clustering performance across representations.

Representation	Model	Accuracy	Precision	Recall	F1-score	ROC-AUC	PR-AUC
k -mers ( $k=2$ )	KNN	0.5066 $\pm$ 0.0223	0.5036 $\pm$ 0.0147	0.9802 $\pm$ 0.0162	0.6652 $\pm$ 0.0144	0.5382 $\pm$ 0.0359	0.5215 $\pm$ 0.0258
Amino acid composition	KNN	0.6050 $\pm$ 0.0451	0.5879 $\pm$ 0.0354	0.6899 $\pm$ 0.0772	0.6343 $\pm$ 0.0529	0.6057 $\pm$ 0.0363	0.5801 $\pm$ 0.0312
Length (original + log)	MLP (shallow)	0.6349 $\pm$ 0.0543	0.6726 $\pm$ 0.0933	0.5527 $\pm$ 0.0240	0.6041 $\pm$ 0.0407	0.6561 $\pm$ 0.0637	0.7186 $\pm$ 0.0470
Physicochemical properties	Logistic Regression	0.5954 $\pm$ 0.0581	0.5941 $\pm$ 0.0548	0.6049 $\pm$ 0.0701	0.5985 $\pm$ 0.0584	0.6338 $\pm$ 0.0564	0.6706 $\pm$ 0.0429

Table 8: Best-performing models for base representations (without hyperparameter optimization). Results are reported as mean

\pm

standard deviation across folds.

Representation	Model	Accuracy	Precision	Recall	F1-score	ROC-AUC	PR-AUC
k -mers ( $k=2$ )	KNN	0.5099 $\pm$ 0.0330	0.5053 $\pm$ 0.0211	0.9800 $\pm$ 0.0183	0.6667 $\pm$ 0.0222	0.5341 $\pm$ 0.0449	0.5203 $\pm$ 0.0299
Amino acid composition	KNN	0.5755 $\pm$ 0.0471	0.5597 $\pm$ 0.0381	0.6897 $\pm$ 0.0932	0.6170 $\pm$ 0.0590	0.6085 $\pm$ 0.0669	0.5991 $\pm$ 0.0595
Physicochemical properties	SVM	0.5493 $\pm$ 0.0902	0.5523 $\pm$ 0.0863	0.7249 $\pm$ 0.2490	0.6057 $\pm$ 0.0980	0.5120 $\pm$ 0.1869	0.5711 $\pm$ 0.1397
Length (original + log)	Logistic Regression	0.6283 $\pm$ 0.0633	0.6888 $\pm$ 0.1053	0.4800 $\pm$ 0.0462	0.5647 $\pm$ 0.0640	0.6603 $\pm$ 0.0563	0.7199 $\pm$ 0.0494

Table 9: Best-performing models for base representations under nested cross-validation. Results are reported as mean

\pm

standard deviation across folds.

Representation	Model	Accuracy	Precision	Recall	F1-score	ROC-AUC	PR-AUC
5 most frequent amino acids	SVM (RBF)	0.5460 $\pm$ 0.0699	0.5535 $\pm$ 0.0860	0.6923 $\pm$ 0.1790	0.6002 $\pm$ 0.0633	0.5330 $\pm$ 0.1399	0.5793 $\pm$ 0.0985
Hybrid representation	KNN	0.5263 $\pm$ 0.0369	0.5143 $\pm$ 0.0242	0.9469 $\pm$ 0.0382	0.6665 $\pm$ 0.0283	0.5796 $\pm$ 0.0677	0.5643 $\pm$ 0.0520
k -mers + GA	KNN	0.5001 $\pm$ 0.0844	0.4976 $\pm$ 0.0603	0.8209 $\pm$ 0.1598	0.6177 $\pm$ 0.0873	0.5403 $\pm$ 0.0857	0.5438 $\pm$ 0.0674
ProtBERT embeddings	KNN	0.6646 $\pm$ 0.0282	0.6446 $\pm$ 0.0276	0.7366 $\pm$ 0.0417	0.6870 $\pm$ 0.0254	0.7383 $\pm$ 0.0281	0.7329 $\pm$ 0.0609
ProtBERT embeddings	MLP (shallow)	0.7041 $\pm$ 0.0313	0.7048 $\pm$ 0.0386	0.7041 $\pm$ 0.0202	0.7043 $\pm$ 0.0284	0.7480 $\pm$ 0.0474	0.7573 $\pm$ 0.0262

Table 10: Best-performing models for advanced and refined representations under nested cross-validation. Each row reports the best model identified for the corresponding representation. Results are reported as mean

\pm

standard deviation across folds.

Panel of Figure 7 — Figure 7: Confusion matrices for representative models across different feature representations.

Figure 8: Confusion matrix for KNN with amino acid composition.

Figure 9: Distribution of sequence length by prediction type (TP, FP, FN, TN) for the ProtBERT + MLP configuration.

Parkinson's Disease Classification

See all 19 papers

MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data
Sep 21, 2026Jian Sun, Kingshuk Ghosh, Lilianna Houston +1Protein Language Models Multi-Task Learning
Department of Computer Science, Ritchie School of Engineering and Computer Science, University of Denver, 2155 E Wesley Ave, Denver, CO 80210 · Department of Surgery, David Geffen School of Medicine at the University of California, Los Angeles, 10833 Le Conte Ave, Los Angeles, CA 90095 · Department of Physics and Astronomy, at the University of Denver, CO 80210 +1
Detecting and explaining clinical-omics inconsistencies to improve patient cohort stratification: an application to Parkinson's disease
Jul 4, 2025José A. Pardo-Pérez, Tomás Bernal, Jaime Ñiguez +5Medical Diagnosis Unsupervised Anomaly Detection
Department of Information and Communications Engineering, Faculty of Computer Science, University of Murcia, Murcia, Spain. · Department of Clinical and Movement Neurosciences, UCL Queen Square Institute of Neurology, London, UK. · Department of Neurology, Washington University School of Medicine, Saint Louis, MO, 63110, USA. +1
Interpretable machine learning predicts Parkinson's disease severity using motion-corrected QSM MRI and multiband multiecho fMRI features
Jun 26, 2026Aixa X. AndradeClinical Outcome Prediction Explainable Medical Image Analysis
Lyda Hill Department of Bioinformatics · Department of Biomedical Engineering · University of Texas Southwestern Medical Center, Dallas, Texas, USA

Sep 21, 2026J/K move · Enter open · S save

MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data

Jian Sun, Kingshuk Ghosh, Lilianna Houston, Mohammad H. Mahoor

Department of Computer Science, Ritchie School of Engineering and Computer Science, University of Denver, 2155 E Wesley Ave, Denver, CO 80210 · Department of Surgery, David Geffen School of Medicine at the University of California, Los Angeles, 10833 Le Conte Ave, Los Angeles, CA 90095 · Department of Physics and Astronomy, at the University of Denver, CO 80210+1

Protein Language Models Multi-Task Learning Molecular Property Prediction Protein Representation Learning

Open paper AskPDF arXiv

Intrinsically disordered proteins (IDPs) differ from folded proteins in that they are dynamic, lack a stable three-dimensional conformation, and have low sequence similarity between similar proteins. The conformational heterogeneity of IDPs - while beneficial for their diverse functions - limits the use of traditional experimental tools to determine their conformation. The experimental difficulty, along with low sequence similarity, results in data scarcity, and makes it difficult to classify/detect IDPs that are similar or dissimilar, a task relevant to understand biology and evolution. We address this challenge using Multi-task ProtBERT (MT-ProtBERT), a multi-task extension of ProtBERT tailored for low-data regimes. MT-ProtBERT integrates Dynamic Window Masking, a Multi-Scale 1D Convolutional classifier (MS-Conv1D), and auxiliary objectives that jointly optimize masked language modeling and biochemistry-informed tasks. We evaluate this framework on two tasks under limited data: (i) phosphorylation site prediction (S/T/Y) in short sequences and small datasets, and (ii) protein compaction prediction on two small datasets (684 and 530 sequences), including sequences comparable in length to typical disordered regions. MT-ProtBERT consistently outperforms PARROT, an RNN-based IDP-specific model, across all tasks. These results demonstrate that combining self-supervised and biochemistry-informed tasks, and multi-scale learning enables robust modeling of unstructured proteins under data scarcity.

Abstract

Parkinson's Disease Classification

Explore similar work

MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data

Detecting and explaining clinical-omics inconsistencies to improve patient cohort stratification: an application to Parkinson's disease

Interpretable machine learning predicts Parkinson's disease severity using motion-corrected QSM MRI and multiband multiecho fMRI features