See, Learn, Assist: Safe and Self-Paced Robotic Rehabilitation via Video-Based Learning from Demonstration
Authors: Ali Alabbas, Camillo Murgia, Joanne Regan, Philip Long
Organizations: Faculty of Mechanical Engineering, Atlantic Technological University, Galway, Ireland. · Department of Health and Nutritional Science, Atlantic Technological University, Sligo, Ireland.
In this paper, we propose a novel framework that allows therapists to teach robot-assisted rehabilitation exercises remotely via RGB-D video. Our system encodes demonstrations as 6-DoF body-centric trajectories using Cartesian Dynamic Movement Primitives (DMPs), ensuring accurate posture-independent spatial generalisation across diverse patient anatomies. Crucially, we execute these trajectories through a decoupled hybrid control architecture that constructs a spatially compliant virtual tunnel, paired with an effort-based temporal dilation mechanism. This architecture is applied to three distinct rehabilitation modalities: Passive, Active-Assisted, and Active-Resistive, by dynamically linking the exercise's execution phase to the patient's tangential force contribution. To guarantee safety, a Gaussian Mixture Regression (GMR) model is learned on-the-fly from the patient's own limb. This allows the detection of abnormal interaction forces and, if necessary, reverses the trajectory to prevent injury. Experimental validation demonstrates the system's precision, achieving an average trajectory reproduction error of 3.7cm and a range of motion (ROM) error of 5.5 degrees. Furthermore, dynamic interaction trials confirm that the controller successfully enforces effort-based progression while maintaining strict spatial path adherence against human disturbances.
Figures & tables
Fig. 1 : Hardware architecture of the system (left) and body-centric frames constructed from skeleton keypoints (right).
Fig. 2 : A GMR model learned from a subject’s limb during elbow flexion-extension exercise (left). The safety corridor is defined as a ±5 standard deviation around the mean expected force (in red). (Right) Exercise trajectories during shoulder abduction-adduction executed at varying trunk rotation angles. The trajectories are aligned at their starting points to demonstrate the high fidelity of the body-centric encoding.
Exercise
Reproduction Error (cm)
Elbow flexion–extension
1.8
Shoulder internal–external rotation
4.5
Shoulder abduction–adduction
4.9
TABLE I : Reproduction RMSE for the three exercises.
Limb Length (m)
Path Distance (m)
Normalized Ratio (R)
0.61
0.47
0.77
0.55
0.43
0.78
0.53
0.41
0.77
0.50
0.38
0.76
0.50 (Unscaled)
0.47
0.94
TABLE II : Normalized reach ratio ( R ) across different limb lengths: scaled vs. unscaled baseline.
Fig. 3 : Comparison of time-normalized joint angle trajectories for the three upper-limb exercises. Orange lines show the demonstration (subject 1), while green and purple lines show the robot reproductions for subjects 1 and 2.
Fig. 4 : Validation of the virtual tunnel and effort-based temporal dilation. (Left) The subject intentionally applies significant orthogonal forces (in red) while varying tangential effort (in blue). Exercise progress (green) is driven only by ft , slowing when ft<0 . (Right) 3D Cartesian paths confirm that the stiff orthogonal admittance maintains spatial adherence to the reference trajectory despite the significant force applied.
Fig. 5 : Exercise progress vs patient’s delivered force in the active-assisted and active-resistive modes.
Fig. 6 : Force monitoring and adaptive reversibility during the three rehabilitation exercises. The green line indicates exercise progress as a percentage of the demonstration. When the measured interaction force (blue line) exceeds the learned safety limit (dashed red) predicted by the GMR model, the system triggers smooth trajectory reversal to maintain patient comfort.
Human Robotics Laboratory, Department of Mechanical Engineering, The University of Melbourne · Melbourne School of Health Sciences, The University of Melbourne