Learning a Speed-adaptive Hip Exoskeleton Control Policy Via Sim-to-real Reinforcement Learning

Designing generalizable control policies for lower-limb exoskeletons remains fundamentally constrained by exhaustive data collection or iterative optimization procedures, which limit accessibility to clinical populations. To address this challenge, we introduce a device-agnostic framework that combines physiologically plausible musculoskeletal simulation with reinforcement learning to enable scalable personalized exoskeleton assistance for both able-bodied and clinical populations. Our control policies not only generate physiologically plausible locomotion dynamics but also capture clinically observed compensatory strategies under targeted muscular deficits, providing a unified computational model of both healthy and pathological gait. Without task-specific tuning, the resulting exoskeleton control policies produce assistive torque profiles at the hip and ankle that align with state-of-the-art profiles validated in human experiments, while consistently reducing metabolic cost across walking speeds. For simulated impaired-gait models, the learned control policies yield asymmetric, deficit-specific exoskeleton assistance that improves both energetic efficiency and bilateral kinematic symmetry without explicit prescription of the target gait pattern. These results demonstrate that physiologically plausible musculoskeletal simulation via reinforcement learning can serve as a scalable foundation for personalized exoskeleton control across both able-bodied and clinical populations, eliminating the need for extensive physical trials.

Developing exoskeleton controllers that generalize across diverse locomotor conditions typically requires extensive motion-capture data and biomechanical labeling, limiting scalability beyond instrumented laboratory settings. Here, we present a physics-based neuromusculoskeletal learning framework that trains a hip-exoskeleton control policy entirely in simulation, without motion-capture demonstrations, and deploys it on hardware via policy distillation. A reinforcement learning teacher policy is trained using a muscle-synergy action prior over a wide range of walking speeds and slopes through a two-stage curriculum, enabling direct comparison between assisted and no-exoskeleton conditions. In simulation, exoskeleton assistance reduces mean muscle activation by up to 3.4% and mean positive joint power by up to 7.0% on level ground and ramp ascent, with benefits increasing systematically with walking speed. On hardware, the assistance profiles learned in simulation are preserved across matched speed-slope conditions (r: 0.82, RMSE: 0.03 Nm/kg), providing quantitative evidence of sim-to-real transfer without additional hardware tuning. These results demonstrate that physics-based neuromusculoskeletal simulation can serve as a practical and scalable foundation for exoskeleton controller development, substantially reducing experimental burden during the design phase.

A significant challenge in exoskeleton robotics is the need to dynamically adapt control profiles to individual motion preferences, thereby ensuring both efficient and comfortable assistance. Currently, since user experience can serve as a comprehensive metric for evaluating the effectiveness of assistance, user preference-based optimization methods have been widely studied for parameter tuning. However, the existing methods rely heavily on extensive human-robot online interactions and suffer from slow optimization speed, which not only induces user fatigue but also compromises optimization effectiveness. Therefore, this paper aims to explore an efficient preference-based optimization framework for personalized exoskeleton assistance that can learn optimal parameters with minimal interaction. We propose a preference-based Bayesian optimization (PbBO) approach that can improve sample efficiency by leveraging knowledge about the sampling distribution of candidate sets. For optimizing six control parameters, PbBO can converge to user-preferred parameters with 90.7% validation accuracy via 20 iterations. Moreover, the hierarchical controller is designed to generate personalized torque for different tasks and achieve interaction torque tracking in real time. The results of treadmill and outdoor experiments demonstrate that the optimized parameters can reduce metabolic rate by 14.5%-15.4%, heart rate by 6.3%-7.6%, and muscle activation by 6.7%-31.5% compared to unassisted walking.