When Participant Disjoint Validation Is Not Enough to Establish Generalization in Parkinson's Gait Classification

Md. Sifat; Sania Akter; Akif Islam; Md. Ekramul Hamid

cs.LGOct 12, 2026

When Participant Disjoint Validation Is Not Enough to Establish Generalization in Parkinson's Gait Classification

Authors: Md. Sifat, Sania Akter, Akif Islam, Md. Ekramul Hamid

Organizations: Department of Computer Science and Engineering, University of Rajshahi, Motihar, Rajshahi, 6205, Bangladesh

Abstract

Participant-disjoint cross-validation prevents direct participant overlap, but it may still produce optimistic estimates when acquisition site and diagnosis are structurally dependent. We examined Parkinson's disease classification using 159 pressure-walkway summary features from 133 WearGait-PD participants, including 65 participants with Parkinson's disease and 68 controls. The cohort was non-factorial: two sites contained only one diagnostic group, whereas VA-Seattle was the only site containing both groups. In repeated participant-level nested cross-validation, site alone achieved an area under the receiver operating characteristic curve (AUC) of 0.955 plus or minus 0.003, exceeding the gait-feature model (0.882 plus or minus 0.014). Under the evaluated logistic-regression pipelines, adding gait features to site, age, and sex produced no measurable improvement in pooled discrimination. More site-controlled evaluations produced lower and imprecise estimates: VA-Seattle same-site AUC was 0.668 (95% bootstrap confidence interval 0.477 to 0.836), and the JHOC + Bayview-to-VA-Seattle transport stress test achieved 0.576 (0.375 to 0.770). Supporting within-diagnosis site-prediction and site-conditioned permutation analyses showed that the gait representation retained substantial acquisition-site structure. These findings indicate that participant-disjoint pooled validation did not isolate disease-related discrimination from site-associated variation. They encourage careful examination of site-label dependence and, where possible, support for pooled results through within-site or independent validation before broader claims are made about generalization across institutions.

Figures & tables

Site	Group	$n$	Age, years	Female
JHOC	PD	49	$65.8\pm 8.9$ (48)	19/48
Bayview	Control	49	$78.9\pm 6.0$ (46)	32/46
VA-Seattle	PD	16	$70.3\pm 5.8$ (16)	4/16
VA-Seattle	Control	19	$67.8\pm 9.6$ (19)	7/19

Table 1: Self-paced cohort composition and basic demographics.

Evaluation	Model input	AUC and uncertainty
Pooled nested CV	Site only	$0.955\pm 0.003$
Pooled nested CV	Age and sex	$0.755\pm 0.009$
Pooled nested CV	Site, age, and sex	$0.953\pm 0.005$
Pooled nested CV	159 gait features	$0.882\pm 0.014$
Pooled nested CV	Site, age, sex, and gait	$0.950\pm 0.011$
VA participant-level OOF AUC	159 gait features	0.668 (0.477–0.836)

Table 2: Primary diagnosis-classification estimates.

Figure 3: Pooled and site-aware diagnosis-classification performance. Error bars show repeat-level standard deviation or 95% bootstrap confidence intervals; the dashed line marks chance.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Count
Complete clinical release	185
No PKMAS export row	52
At least one PKMAS export row	133
Exact SelfPace record	133
SelfPace_doorpat excluded (additional rows)	2
Participants with repeated exact SelfPace records	2

Appendix

Table 3: Participant-flow accounting for the primary self-paced (SelfPace) analysis.

Item	Count
Candidate PKMAS parameters	193
Internal “#Samples” columns removed	31
Variables removed for $>30\%$ missingness	3
Retained gait features	159
Completely observed retained features	157
Primary SelfPace participants	133

Appendix

Table 4: Feature preparation in the primary self-paced analysis.

Diagnostic group (site comparison)	Age + sex	Gait	Age + sex + gait
Controls (Bayview vs VA-Seattle)	$0.848\pm 0.012$	$0.883\pm 0.041$	$0.880\pm 0.048$
PD (JHOC vs VA-Seattle)	$0.749\pm 0.041$	$0.956\pm 0.018$	$0.956\pm 0.018$

Appendix

Table 5: Within-diagnosis acquisition-site prediction. Values are mean

\pm

standard deviation from the corresponding participant-level validation analysis.

Analysis	Purpose	Result
Missingness indicators only	Test whether measurement availability identified diagnosis	AUC 0.486
157 complete features	Remove any contribution from feature imputation	AUC 0.895
Average repeated eligible rows	Test deterministic first-record selection	AUC 0.893
Joint site–diagnosis stratification	Preserve the four observed composition groups across folds	AUC 0.897
Ridge logistic regression: gait	Test dependence on L1 regularization	AUC 0.838
Ridge logistic regression: site, age, sex, and gait	Alternative combined estimator	AUC 0.909

Appendix

Table 6: Sensitivity analyses for the primary self-paced (SelfPace) cohort. Values support qualitative interpretation and are not independent confirmatory tests.

Permutation scheme	Mean $\pm$ SD	2.5th–97.5th
		percentiles
Global label permutation	$0.498\pm 0.064$	0.369–0.625
Site-conditioned permutation	$0.867\pm 0.031$	0.805–0.926

Appendix

Table 7: Global and site-conditioned permutation reference summaries.

Figure 4: Global and site-conditioned permutation reference distributions. Markers indicate the mean AUC across 200 permutations per scheme; horizontal intervals show the 2.5th–97.5th percentiles, and the solid vertical line marks the observed fixed-

C

reference AUC of 0.908.

Evaluation	Model input	AUC and uncertainty
Pooled nested cross-validation	159 gait features	$0.835\pm 0.020$
VA-Seattle same-site nested CV	159 gait features	$0.506\pm 0.072$
JHOC + Bayview-to-VA-Seattle stress test	159 gait features	0.487 (0.296–0.684)

Appendix

Table 8: Hurried-paced sensitivity results. Nested-cross-validation values are mean

\pm

standard deviation across repeat-level AUCs. The value in parentheses is a 95% participant-bootstrap confidence interval.

Parkinson's Disease Classification

See all 19 papers

Mar 6, 2026

Topological descriptors of foot clearance gait dynamics improve differential diagnosis of Parkinsonism

Differential diagnosis among parkinsonian syndromes remains a clinical challenge due to overlapping motor symptoms and subtle gait abnormalities. Accurate differentiation is crucial for treatment planning and prognosis. While gait analysis is a well established approach for assessing motor impairments, conventional methods often overlook hidden nonlinear and structural features embedded in foot clearance patterns. We evaluated Topological Data Analysis (TDA) as a complementary tool for Parkinsonism classification using foot clearance time series. Persistent homology produced Betti curves, persistence landscapes, and silhouettes, which were used as features for a Random Forest classifier. The dataset comprised 15 controls (CO), 15 idiopathic Parkinson's disease (IPD), and 14 vascular Parkinsonism (VaP). Models were assessed with leave-one-out cross-validation (LOOCV). Betti-curve descriptors consistently yielded the strongest results. For IPD vs VaP, foot clearance variables minimum toe clearance, maximum toe late swing, and maximum heel clearance achieved 83% accuracy and AUC=0.89 under LOOCV in the medicated (On) state. Performance improved in the On state and further when both Off and On states were considered, indicating sensitivity of the topological features to levodopa related gait changes. These findings support integrating TDA with machine learning to improve clinical gait analysis and aid differential diagnosis across parkinsonian disorders.

Jhonathan Barrios, Wolfram Erlhagen, Miguel F. Gago +2

Centre of Mathematics (CMAT), School of Sciences, University of Minho, Portugal · Neurology Departmet, Hospital da Senhora da Oliveira, Portugal · ICVS, School of Medicine, University of Minho, Portugal +3

Parkinson's Disease Classification Differential Diagnosis Parkinson's Disease Gait Analysis Classification Topological Data Analysis

Ask PDF

Sep 19, 2026

A Compact Stance-Indexed Anterior-Posterior COP Representation for Parkinson's Disease Classification from Plantar VGRF

Parkinson's disease alters gait and bilateral coordination, but machine-learning performance also depends on how continuous gait signals are represented. This study investigates whether preserving anterior-posterior center-of-pressure (AP-COP) information at fixed locations across normalized stance provides a compact and informative representation of plantar-force gait signals. Bilateral vertical ground reaction force recordings from 165 participants in the Gait in Parkinson's Disease Database were evaluated using repeated fully nested participant-level cross-validation. We propose AP-COP10, comprising AP-COP position and bilateral asymmetry across five stance windows. AP-COP10 achieved an AUC of 0.894 and outperformed three harmonized literature-derived COP representations under the same evaluation pipeline. The complementary 25 non-AP-COP descriptors alone achieved an AUC of 0.856, while the complete 35-feature representation achieved 0.908. Removing AP-COP10 from the complete representation produced a statistically supported loss in discrimination, whereas adding the complementary descriptors to AP-COP10 yielded only a small, unsupported improvement. Feature competition indicated that the most informative stance-indexed descriptors were concentrated in early and early-mid stance, while source-study holdout and sensor-perturbation analyses supported the robustness of the representation. These findings indicate that stance-indexed AP-COP retains discriminative information that is not readily recovered by broader engineered gait descriptors, supporting compact and interpretable representations for machine-learning analysis of pathological gait.

Md. Sifat, Sania Akter, Akif Islam +1

Department of Computer Science and Engineering, University of Rajshahi, Rajshahi, Bangladesh

Parkinson's Disease Classification Gait Analysis Medical Diagnosis

Ask PDF

Aug 20, 2026

Aggregate, Don't Adapt: Subject-Level Posterior Aggregation and Transductive Calibration for Cross-Site Parkinsonian Gait Severity

We describe the winning entry to the MoCha 2026 Benchmark and Challenge on Parkinsonian Gait, which predicts MDS-UPDRS gait severity from canonicalized SMPL motion recorded at clinical sites unseen during training. The system reaches 0.6945 macro-F1 on the hidden test and ranked first of 58 entries, ahead of the runner-up at 0.5807 and the organizers' baseline at 0.4289, on a frozen public motion encoder with a single

4\times512

linear layer. Nearly all of the margin comes from three stages usually treated as bookkeeping: reproducing the reference benchmark's exact head recipe, averaging per-walk posteriors within the subject grouping the organizers ship, and a label-free transductive calibration of the feature mean and the decision operating point. Fine-tuning the encoder lost in four distinct forms, and ten alternative encoders were worse. Every ablation number is a paid read on the hidden test, because our own leave-two-cohort-out cross-validation proved anti-correlated with the deciding score over eleven configurations. We give the negative record in full, and identify our largest gain, subject-level aggregation, as the binding ceiling on this benchmark.

Junlong Shen

University of Alberta, Edmonton, Canada

Gait Analysis Parkinson's Disease

Ask PDF

Abstract

Parkinson's Disease Classification

Explore similar work

Topological descriptors of foot clearance gait dynamics improve differential diagnosis of Parkinsonism

A Compact Stance-Indexed Anterior-Posterior COP Representation for Parkinson's Disease Classification from Plantar VGRF

Aggregate, Don't Adapt: Subject-Level Posterior Aggregation and Transductive Calibration for Cross-Site Parkinsonian Gait Severity