Freezing of Gait Prediction using Proactive Agent that Learns from Selected Experience and DDQN Algorithm

Septian Enggar Sukmana; Sang Won Bae; Tomohiro Shibata

cs.LGMar 4, 2026

Freezing of Gait Prediction using Proactive Agent that Learns from Selected Experience and DDQN Algorithm

Authors: Septian Enggar Sukmana, Sang Won Bae, Tomohiro Shibata

Organizations: Kyushu Institute of Technology · Graduate School of Life Science and Systems Engineering, Kyushu Institute of Technology, Kitakyushu, Japan · Stevens Institute of Technology · Charles V. Schaefer, Jr. School of Engineering and Science, Stevens Institute of Technology, New Jersey, United States of America

Abstract

Freezing of Gait (FOG) is a debilitating motor symptom commonly experienced by individuals with Parkinson's Disease (PD) which often leads to falls and reduced mobility. Timely and accurate prediction of FOG episodes is essential for enabling proactive interventions through assistive technologies. This study presents a reinforcement learning-based framework designed to identify optimal pre-FOG onset points, thereby extending the prediction horizon for anticipatory cueing systems. The model implements a Double Deep Q-Network (DDQN) architecture enhanced with Prioritized Experience Replay (PER) allowing the agent to focus learning on high-impact experiences and refine its policy. Trained over 9000 episodes with a reward shaping strategy that promotes cautious decision-making, the agent demonstrated robust performance in both subject-dependent and subject-independent evaluations. The model achieved a prediction horizon of up to 8.72 seconds prior to FOG onset in subject-independent scenarios and 7.89 seconds in subject-dependent settings. These results highlight the model's potential for integration into wearable assistive devices, offering timely and personalized interventions to mitigate FOG in PD patients.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of FOG prediction techniques: (a)fixed-window [ 15 , 9 , 18 , 12 , 17 ] ; (b) threshold by machine learning [ 5 ] ; (c) proactive using RL agent

ID	Gender	Age	Disease duration	H&Y	Tested in
1	M	66	16	3	OFF
2	M	67	7	2	ON
3	M	59	30	2.5	OFF
5	M	75	6	2	OFF
6	F	63	22	2	OFF
7	M	66	2	2.5	OFF

Table 1: Selected Subject on Daphnet in this activity

Figure 2: Time-Based Reward Shaping for FOG Prediction

Parameter	Value
Discount Factor ( $\gamma$ )	0.99
Learning Rate ( $\alpha$ )	0.001
Replay Buffer Size	50000
Cost function	Mean Squared Error (MSE)
Optimizer Function	Adam
Activation Function	ReLU (Rectified Linear Unit)

Table 2: Parameters of the agent model.

Figure 3: Proactive agent action to predict FOG

Panel of Figure 4 — Figure 4: Example FOG prediction point on: (a) Subject 5; (b) Subject 7

Figure 5: Illustration of trained prediction points by replay scheme

Subject Dependent
Variable	Subject ID
	1	2	3	5	6	7	8	9
Mean - reward	120.82	82.80	116.58	119.57	23.65	65.39	43.95	36.00
Mean - prediction point (s)	5.05	3.91	6.03	5.29	2.21	3.55	3.19	2.63
Std Dev - prediction point (s)	1.25	1.93	1.42	1.01	1.87	1.62	0.76	1.31
The longest prediction horizon (s)	6.87	5.08	7.89	7.29	5.74	5.47	7.26	6.70

Table 3: Experimental Results Based on Subject

Study	Mean (s)
Naghavi and Wade [ 15 ]	2.00
Zhang et al. [ 24 ]	0.93
Shalin et al. [ 21 ]	¡ 2.00
El-ziaat et al. [ 2 ]	1.00
Pardoel et al. [ 18 ]	0.80
Pardoel et al. [ 17 ]	0.89

Table 4: Mean Prediction Horizons from Various Studies

Figure 6: Prediction horizon between Subject Independent and Dependent

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Simple Reward Scheme
Variable	Subject ID
	1	2	3	5	6	7	8	9
Mean - reward	74.28	-49.34	0.17	145.05	82.23	96.45	151.50	6.41
Mean - prediction point (s)	5.85	5.76	4.18	5.71	4.71	6.39	4.62	5.91
Std Dev - prediction point (s)	1.47	3.52	3.33	0.57	1.78	1.82	0.00	3.59
The longest prediction horizon (s)	8.49	10.16	11.63	6.24	6.19	11.72	4.62	14.35

Appendix

Table 5: Simple Reward Scheme and Ablation Experiment

Figure 8: Stability Comparison between Simple Reward Scheme vs Without 6 Statistical Parameters

Prioritized Experience Replay

See all 10 papers

Supervised Cross-Modal Feature Alignment for Zero-Wearable Freezing of Gait Detection in Parkinsonism
Sep 8, 2026Aryan Singh, Chandan BiswasCross-Modal Knowledge Distillation Cross-Modal Alignment
NeuroAI Fusion Labs Kolkata, India
Deep Learning-Based Estimation of Ground Reaction Forces in Parkinsonian Gait Using an Optimized Set of IMU Data
Aug 3, 2026Run Lin, Yingtian Tang, Jiawen Xu +6Gait Analysis
Department of Engineering, Faculty of Environment, Science and Economy, University of Exeter, Exeter, EX4 4QF, UK · School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland · Institute for Manufacturing, Department of Engineering, University of Cambridge, Cambridge CB3 0FS, UK +2
Towards Context-Aware Clinical Motion Understanding in Daily Living at Home: Freezing of Gait Detection with Egocentric Vision
Aug 13, 2026Vayalet Stefanova, Diwas Lamsal, Margot Genbrugge +5Gait Analysis Parkinson's Disease
Department of Electrical Engineering (ESAT), KU Leuven, Leuven, Belgium · Neurorehabilitation Research Group (eNRGy), Department of Rehabilitation Sciences, KU Leuven, Leuven, Belgium · Institute of Interdisciplinary Exercise Science and Sports Medicine, Medical School Hamburg, Hamburg, Germany +2

Sep 8, 2026J/K move · Enter open · S save

Supervised Cross-Modal Feature Alignment for Zero-Wearable Freezing of Gait Detection in Parkinsonism

Aryan Singh, Chandan Biswas

NeuroAI Fusion Labs Kolkata, India

Cross-Modal Knowledge Distillation Cross-Modal Alignment Gait Analysis

Open paper AskPDF arXiv

Objective assessment of Freezing of Gait (FoG) in Parkinson's disease (PD) relies predominantly on wearable Inertial Measurement Units (IMUs). While IMUs provide optimal kinematic precision, mandatory sensor attachment restricts continuous clinical deployment. Conversely, unobtrusive vision-based alternatives suffer substantial classification errors during turning-in-place tasks, where geometric self-occlusion degrades deterministic skeletal coordinates and obscures the high-frequency precursors required for FoG detection. To resolve these physical observation limits, we propose a supervised cross-modal subspace distillation framework. During optimisation, pre-trained kinematic data from IMU sensors and contextual clinical metadata act as oracles to guide a deployable visual architecture. By incorporating joint velocity and acceleration derivatives, utilising a confidence-based gating mechanism, the visual model mitigates some of the tracking errors during occlusion events. Empirical evaluations confirm this latent alignment transfers the predictive fidelity of hardware sensors directly into the visual representation, yielding

85.5\%

accuracy, and

82.4\%

balanced accuracy. All the while maintaining a vision only model at inference.

ID	Gender	Age	Disease duration	H&Y	Tested in
1	M	66	16	3	OFF
2	M	67	7	2	ON
3	M	59	30	2.5	OFF
5	M	75	6	2	OFF
6	F	63	22	2	OFF
7	M	66	2	2.5	OFF

ID	Gender	Age	Disease duration	H&Y	Tested in
1	M	66	16	3	OFF
2	M	67	7	2	ON
3	M	59	30	2.5	OFF
5	M	75	6	2	OFF
6	F	63	22	2	OFF
7	M	66	2	2.5	OFF

Abstract

Prioritized Experience Replay

Explore similar work

Supervised Cross-Modal Feature Alignment for Zero-Wearable Freezing of Gait Detection in Parkinsonism

Deep Learning-Based Estimation of Ground Reaction Forces in Parkinsonian Gait Using an Optimized Set of IMU Data

Towards Context-Aware Clinical Motion Understanding in Daily Living at Home: Freezing of Gait Detection with Egocentric Vision

ID	Gender	Age	Disease duration	H&Y	Tested in
1	M	66	16	3	OFF
2	M	67	7	2	ON
3	M	59	30	2.5	OFF
5	M	75	6	2	OFF
6	F	63	22	2	OFF
7	M	66	2	2.5	OFF