When Participant Disjoint Validation Is Not Enough to Establish Generalization in Parkinson's Gait Classification

Md. Sifat; Sania Akter; Akif Islam; Md. Ekramul Hamid

cs.LGOct 12, 2026

When Participant Disjoint Validation Is Not Enough to Establish Generalization in Parkinson's Gait Classification

Authors: Md. Sifat, Sania Akter, Akif Islam, Md. Ekramul Hamid

Organizations: Department of Computer Science and Engineering, University of Rajshahi, Motihar, Rajshahi, 6205, Bangladesh

Abstract

Participant-disjoint cross-validation prevents direct participant overlap, but it may still produce optimistic estimates when acquisition site and diagnosis are structurally dependent. We examined Parkinson's disease classification using 159 pressure-walkway summary features from 133 WearGait-PD participants, including 65 participants with Parkinson's disease and 68 controls. The cohort was non-factorial: two sites contained only one diagnostic group, whereas VA-Seattle was the only site containing both groups. In repeated participant-level nested cross-validation, site alone achieved an area under the receiver operating characteristic curve (AUC) of 0.955 plus or minus 0.003, exceeding the gait-feature model (0.882 plus or minus 0.014). Under the evaluated logistic-regression pipelines, adding gait features to site, age, and sex produced no measurable improvement in pooled discrimination. More site-controlled evaluations produced lower and imprecise estimates: VA-Seattle same-site AUC was 0.668 (95% bootstrap confidence interval 0.477 to 0.836), and the JHOC + Bayview-to-VA-Seattle transport stress test achieved 0.576 (0.375 to 0.770). Supporting within-diagnosis site-prediction and site-conditioned permutation analyses showed that the gait representation retained substantial acquisition-site structure. These findings indicate that participant-disjoint pooled validation did not isolate disease-related discrimination from site-associated variation. They encourage careful examination of site-label dependence and, where possible, support for pooled results through within-site or independent validation before broader claims are made about generalization across institutions.

Figures & tables

Site	Group	$n$	Age, years	Female
JHOC	PD	49	$65.8\pm 8.9$ (48)	19/48
Bayview	Control	49	$78.9\pm 6.0$ (46)	32/46
VA-Seattle	PD	16	$70.3\pm 5.8$ (16)	4/16
VA-Seattle	Control	19	$67.8\pm 9.6$ (19)	7/19

Table 1: Self-paced cohort composition and basic demographics.

Evaluation	Model input	AUC and uncertainty
Pooled nested CV	Site only	$0.955\pm 0.003$
Pooled nested CV	Age and sex	$0.755\pm 0.009$
Pooled nested CV	Site, age, and sex	$0.953\pm 0.005$
Pooled nested CV	159 gait features	$0.882\pm 0.014$
Pooled nested CV	Site, age, sex, and gait	$0.950\pm 0.011$
VA participant-level OOF AUC	159 gait features	0.668 (0.477–0.836)

Table 2: Primary diagnosis-classification estimates.

Figure 3: Pooled and site-aware diagnosis-classification performance. Error bars show repeat-level standard deviation or 95% bootstrap confidence intervals; the dashed line marks chance.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Count
Complete clinical release	185
No PKMAS export row	52
At least one PKMAS export row	133
Exact SelfPace record	133
SelfPace_doorpat excluded (additional rows)	2
Participants with repeated exact SelfPace records	2

Appendix

Table 3: Participant-flow accounting for the primary self-paced (SelfPace) analysis.

Item	Count
Candidate PKMAS parameters	193
Internal “#Samples” columns removed	31
Variables removed for $>30\%$ missingness	3
Retained gait features	159
Completely observed retained features	157
Primary SelfPace participants	133

Appendix

Table 4: Feature preparation in the primary self-paced analysis.

Diagnostic group (site comparison)	Age + sex	Gait	Age + sex + gait
Controls (Bayview vs VA-Seattle)	$0.848\pm 0.012$	$0.883\pm 0.041$	$0.880\pm 0.048$
PD (JHOC vs VA-Seattle)	$0.749\pm 0.041$	$0.956\pm 0.018$	$0.956\pm 0.018$

Appendix

Table 5: Within-diagnosis acquisition-site prediction. Values are mean

\pm

standard deviation from the corresponding participant-level validation analysis.

Analysis	Purpose	Result
Missingness indicators only	Test whether measurement availability identified diagnosis	AUC 0.486
157 complete features	Remove any contribution from feature imputation	AUC 0.895
Average repeated eligible rows	Test deterministic first-record selection	AUC 0.893
Joint site–diagnosis stratification	Preserve the four observed composition groups across folds	AUC 0.897
Ridge logistic regression: gait	Test dependence on L1 regularization	AUC 0.838
Ridge logistic regression: site, age, sex, and gait	Alternative combined estimator	AUC 0.909

Appendix

Table 6: Sensitivity analyses for the primary self-paced (SelfPace) cohort. Values support qualitative interpretation and are not independent confirmatory tests.

Permutation scheme	Mean $\pm$ SD	2.5th–97.5th
		percentiles
Global label permutation	$0.498\pm 0.064$	0.369–0.625
Site-conditioned permutation	$0.867\pm 0.031$	0.805–0.926

Appendix

Table 7: Global and site-conditioned permutation reference summaries.

Figure 4: Global and site-conditioned permutation reference distributions. Markers indicate the mean AUC across 200 permutations per scheme; horizontal intervals show the 2.5th–97.5th percentiles, and the solid vertical line marks the observed fixed-

C

reference AUC of 0.908.

Evaluation	Model input	AUC and uncertainty
Pooled nested cross-validation	159 gait features	$0.835\pm 0.020$
VA-Seattle same-site nested CV	159 gait features	$0.506\pm 0.072$
JHOC + Bayview-to-VA-Seattle stress test	159 gait features	0.487 (0.296–0.684)

Appendix

Table 8: Hurried-paced sensitivity results. Nested-cross-validation values are mean

\pm

standard deviation across repeat-level AUCs. The value in parentheses is a 95% participant-bootstrap confidence interval.

Parkinson's Disease Classification

See all 19 papers

Topological descriptors of foot clearance gait dynamics improve differential diagnosis of Parkinsonism
Mar 6, 2026Jhonathan Barrios, Wolfram Erlhagen, Miguel F. Gago +2Parkinson's Disease Classification Differential Diagnosis
Centre of Mathematics (CMAT), School of Sciences, University of Minho, Portugal · Neurology Departmet, Hospital da Senhora da Oliveira, Portugal · ICVS, School of Medicine, University of Minho, Portugal +3
A Compact Stance-Indexed Anterior-Posterior COP Representation for Parkinson's Disease Classification from Plantar VGRF
Sep 19, 2026Md. Sifat, Sania Akter, Akif Islam +1Parkinson's Disease Classification Gait Analysis
Department of Computer Science and Engineering, University of Rajshahi, Rajshahi, Bangladesh
Aggregate, Don't Adapt: Subject-Level Posterior Aggregation and Transductive Calibration for Cross-Site Parkinsonian Gait Severity
Aug 20, 2026Junlong ShenGait Analysis Parkinson's Disease
University of Alberta, Edmonton, Canada

Mar 6, 2026J/K move · Enter open · S save

Topological descriptors of foot clearance gait dynamics improve differential diagnosis of Parkinsonism

Jhonathan Barrios, Wolfram Erlhagen, Miguel F. Gago, Estela Bicho+1

Centre of Mathematics (CMAT), School of Sciences, University of Minho, Portugal · Neurology Departmet, Hospital da Senhora da Oliveira, Portugal · ICVS, School of Medicine, University of Minho, Portugal+3

Parkinson's Disease Classification Differential Diagnosis Parkinson's Disease Gait Analysis Classification Topological Data Analysis

Open paper AskPDF arXiv

Differential diagnosis among parkinsonian syndromes remains a clinical challenge due to overlapping motor symptoms and subtle gait abnormalities. Accurate differentiation is crucial for treatment planning and prognosis. While gait analysis is a well established approach for assessing motor impairments, conventional methods often overlook hidden nonlinear and structural features embedded in foot clearance patterns. We evaluated Topological Data Analysis (TDA) as a complementary tool for Parkinsonism classification using foot clearance time series. Persistent homology produced Betti curves, persistence landscapes, and silhouettes, which were used as features for a Random Forest classifier. The dataset comprised 15 controls (CO), 15 idiopathic Parkinson's disease (IPD), and 14 vascular Parkinsonism (VaP). Models were assessed with leave-one-out cross-validation (LOOCV). Betti-curve descriptors consistently yielded the strongest results. For IPD vs VaP, foot clearance variables minimum toe clearance, maximum toe late swing, and maximum heel clearance achieved 83% accuracy and AUC=0.89 under LOOCV in the medicated (On) state. Performance improved in the On state and further when both Off and On states were considered, indicating sensitivity of the topological features to levodopa related gait changes. These findings support integrating TDA with machine learning to improve clinical gait analysis and aid differential diagnosis across parkinsonian disorders.

Abstract

Parkinson's Disease Classification

Explore similar work

Topological descriptors of foot clearance gait dynamics improve differential diagnosis of Parkinsonism

A Compact Stance-Indexed Anterior-Posterior COP Representation for Parkinson's Disease Classification from Plantar VGRF

Aggregate, Don't Adapt: Subject-Level Posterior Aggregation and Transductive Calibration for Cross-Site Parkinsonian Gait Severity