Real-Time Decoding of Movement Onset and Offset for Brain-Controlled Rehabilitation Exoskeleton
Authors: Kanishka Mitra, Satyam Kumar, Frigyes Samuel Racz, Deland Liu, Ashish D. Deshpande, José del R. Millán
Organizations: Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA. · Chandra Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA. · Department of Neurology, The University of Texas at Austin, Austin, TX, USA. · Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX, USA. · Meta Reality Labs Research, Redmond, WA, USA. · Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, USA.
Robot-assisted therapy can deliver high-dose, task-specific training after neurologic injury, but most systems act primarily at the limb level-engaging the impaired neural circuits only indirectly-which remains a key barrier to truly contingent, neuroplasticity-targeted rehabilitation. We address this gap by implementing online, dual-state motor imagery control of an upper-limb exoskeleton, enabling goal-directed reaches to be both initiated and terminated directly from non-invasive EEG. Eight participants used EEG to initiate assistance and then volitionally halt the robot mid-trajectory. Across two online sessions, group-mean hit rates were 61.5% for onset and 64.5% for offset, demonstrating reliable start-stop command delivery despite instrumental noise and passive arm motion. Methodologically, we reveal a systematic, class-driven bias induced by common task-based recentering using an asymmetric margin diagnostic, and we introduce a class-agnostic fixation-based recentering method that tracks drift without sampling command classes while preserving class geometry. This substantially improves threshold-free separability (AUC gains: onset +56%, p = 0.0117; offset +34%, p = 0.0251) and reduces bias within and across days. Together, these results help bridge offline decoding and practical, intention-driven start-stop control of a rehabilitation exoskeleton, enabling precisely timed, contingent assistance aligned with neuroplasticity goals while supporting future clinical translation.
Figures & tables
Fig. 1 : Experimental setup, task timeline, and decoding pipeline for onset/offset EEG control of the Harmony exoskeleton. (a) Harmony (§ II-B ); (b) EEG cap; (c) stNMES pads under the cuff; (d) arm-mounted LEDs indicating trial phase (white: countdown; green: Start; red: Stop; blue: Rest); (e) fixation-based recentering (§ IV-A ); (f) subject-specific Riemannian decoders (§ II-C ); (g) three spatial targets defining goal-directed trajectories. Blue, green, and orange sliding windows depict EEG samples of shape N×C×T (trials × channels × time) with T=512 points ( 1 s at 512 Hz), as used in § II-C . Onset/offset decisions and robot transitions follow the thresholding logic in § II-C and state machine in § II-C .
Fig. 2 : Grand average spectrogram from the C3 electrode showing changes in spectral power over the course of task trials. Data were obtained by averaging over all trials and all subjects. Vertical lines mark task events (countdown, Start MI, robot moves, Stop MI, robot stops, Rest, robot returns).
Fig. 3 : Online command-delivery outcomes. Stacked bars show group-mean outcome proportions (hit, miss, timeout) for onset and offset in Sessions 2–3; numbers indicate mean hit rate. Offset is computed on attempted trials only (i.e., trials following a successful onset).
Fig. 4 : Online decoding time for hits only. Violin plots show the distribution of decision latencies for onset (cue-locked) and offset (movement-locked to assistance onset) in Sessions 2-3; black bars indicate means.
Fig. 5 : Task-based recentering shifts class separation relative to Si ( Δ median margin). Bars show the group-mean change in median margin, m=dnegative−dpositive , under task-based recentering versus the identity reference Si . Error bars denote 95% CIs across subjects (subject means over runs).
Fig. 6 : AUC by run for onset and offset under task- vs fixation-based recentering. Lines show group-mean AUC; ribbons denote 95% CIs. Inset boxes report paired Wilcoxon p -values for per-subject mean AUC differences (Task − Fix).
Two distinct technologies have gained attention lately due to their prospects for motor rehabilitation: robotics and brain-machine interfaces (BMIs). Harnessing their combined efforts is a largely uncharted and promising direction that has immense clinical potential. However, a significant challenge is whether motor intentions from the user can be accurately detected using non-invasive BMIs in the presence of instrumental noise and passive movements induced by the rehabilitation exoskeleton. As an alternative to the straightforward continuous control approach, this study instead aims to characterize the onset and offset of motor imagery during passive arm movements induced by an upper-body exoskeleton to allow for the natural control (initiation and termination) of functional movements. Ten participants were recruited to perform kinesthetic motor imagery (MI) of the right arm while attached to the robot, simultaneously cued with LEDs indicating the initiation and termination of a goal-oriented reaching task. Using electroencephalogram signals, we built a decoder to detect the transition between i) rest and beginning MI and ii) maintaining and ending MI. Offline decoder evaluation achieved group average onset accuracy of 60.7% and 66.6% for offset accuracy, revealing that the start and stop of MI could be identified while attached to the robot. Furthermore, pseudo-online evaluation could replicate this performance, forecasting reliable online exoskeleton control in the future. Our approach showed that participants could produce quality and reliable sensorimotor rhythms regardless of noise or passive arm movements induced by wearing the exoskeleton, which opens new possibilities for BMI control of assistive devices.
Kanishka Mitra, Frigyes Samuel Racz, Satyam Kumar +2
Chandra Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA. · Department of Neurology, The University of Texas at Austin, Austin, TX, USA. · Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX, USA.
Closed-loop lower-limb exoskeleton control via Electroencephalography (EEG) remains limited by motion artifacts, low signal-to-noise ratio, and binary gait formulations that fail to capture full cortical gait complexity. We propose a 2-block Brain-Computer Interface (BCI) architecture: a trainable session-specific Feature Extraction Block with real-time artifact suppression and multi-domain feature extraction, coupled with a Decoder Block built on a novel Polynomial Time-Varying Layer (PolyTVL)+LSTM for four-state gait classification (Stand, Initiate, Execute, Terminate). Ablation confirmed v01 (PolyTVL+LSTM) outperformed all variants (validation MCC: 0.435, gap: 0.187), with consistent EEG feature discriminability across ROIs and sub-bands (p<0.05). Closed-loop deployment with v01 achieved 55.3% (Rex-assisted) and 52.7% (volitional) gait initiation success, with a mean prediction time of 70.5~ms (+/-41.5), validating real-time feasibility in this pilot study.
Shantanu Sarkar, Saurabh Prasad, Jose L. Contreras-Vidal
IUCRC BRAIN Center, Cullen College of Engineering, University of Houston, Houston, TX, USA
Motor kinematics prediction (MKP) from electroencephalography (EEG) is an important research area for developing movement-related brain-computer interfaces (BCIs). While traditional methods often rely on convolutional neural networks (CNNs) or recurrent neural networks (RNNs), Transformer-based models have shown strong ability in modeling long sequential EEG data. In this study, we propose a CNN-attention hybrid model for decoding hand kinematics from EEG during grasp-and-lift tasks, achieving strong performance in within-subject experiments. We further extend this approach to EEG-EMG multimodal decoding, which yields substantially improved results. Within-subject tests achieve PCC values of 0.9854, 0.9946, and 0.9065 for the X, Y, and Z axes, respectively, computed on the midpoint trajectory between the thumb and index finger, while cross-subject tests result in 0.9643, 0.9795, and 0.5852. The decoded trajectories from both modalities are then used to control a Franka Panda robotic arm in a MuJoCo simulation. To enhance trajectory fidelity, we introduce a copilot framework that filters low-confidence decoded points using a motion-state-aware critic within a finite-state machine. This post-processing step improves the overall within-subject PCC of EEG-only decoding to 0.93 while excluding fewer than 20% of the data points.
Yizhe Li, Shixiao Wang, Jian K. Liu
University of Birmingham, Birmingham, United Kingdom · University of Birmingham