Developing exoskeleton controllers that generalize across diverse locomotor conditions typically requires extensive motion-capture data and biomechanical labeling, limiting scalability beyond instrumented laboratory settings. Here, we present a physics-based neuromusculoskeletal learning framework that trains a hip-exoskeleton control policy entirely in simulation, without motion-capture demonstrations, and deploys it on hardware via policy distillation. A reinforcement learning teacher policy is trained using a muscle-synergy action prior over a wide range of walking speeds and slopes through a two-stage curriculum, enabling direct comparison between assisted and no-exoskeleton conditions. In simulation, exoskeleton assistance reduces mean muscle activation by up to 3.4% and mean positive joint power by up to 7.0% on level ground and ramp ascent, with benefits increasing systematically with walking speed. On hardware, the assistance profiles learned in simulation are preserved across matched speed-slope conditions (r: 0.82, RMSE: 0.03 Nm/kg), providing quantitative evidence of sim-to-real transfer without additional hardware tuning. These results demonstrate that physics-based neuromusculoskeletal simulation can serve as a practical and scalable foundation for exoskeleton controller development, substantially reducing experimental burden during the design phase.
Figures & tables
Fig. 1: Simulation-to-real workflow for learning and deploying hip-exoskeleton control policy. Left (Neuromusculoskeletal simulation): a privileged teacher policy is trained in a predictive human simulation across randomized target speeds ( 0.7 – 1.5 m/s) and slopes ( −5∘ to +5∘ ) using a two-stage curriculum: Curriculum 1 learns stable locomotion without exoskeleton actuation (no-exoskeleton), then Curriculum 2 introduces bilateral hip exoskeleton actuation (exoskeleton-assisted), enabling the policy to learn assistive torques. The teacher outputs reduced-dimensional actions via a muscle-synergy prior, comprising lower-limb muscle excitations ( esyn→elimb ), direct trunk-muscle excitations ( etrunk ), and raw bilateral hip-exoskeleton torques ( τraw ), which are smoothed by a first-order low-pass filter (LPF) to yield applied torques ( τfilt ). Simulation state and reward are fed back to optimize the teacher. Right (Real world): the teacher is distilled into a temporal convolutional network student policy that maps a short history of unilateral thigh inertial measurement unit (IMU) gyroscope signals ( θ˙thigh ) to unilateral hip-torque commands for onboard control; the resulting τraw is passed through the onboard LPF and applied to the physical exoskeleton. The dashed red arrows denote the distillation link from simulated rollouts to the deployable IMU-only policy.
Fig. 2: Human-exoskeleton musculoskeletal model used in simulation. Frontal and lateral view of the H2190 full-body musculoskeletal model, comprising 21 degrees of freedom and 90 Hill-type musculotendon actuators. Bilateral hip exoskeleton actuators apply flexion-extension torques about the left and right hip joints.
Parameter
Value
Algorithm
Soft Actor–Critic (SAC)
Implementation
Stable-Baselines3
Policy
Multilayer perceptron
Actor/Critic sizes
[512, 512, 256]
Parallel envs
30
Stage 1 steps
0 – 5×107
TABLE I: SAC hyperparameters and curriculum schedule.
Fig. 3: Comparison of thigh gyroscope signals between simulation and hardware during level-ground walking at 1.2 m/s. Simulation signals were computed as femur angular velocity expressed in the femur local frame; hardware signals were recorded from an IMU mounted on the distal thigh bar of the exoskeleton. The rotational axes were defined with respect to an upright standing posture: x corresponds to the anteroposterior axis, y to the vertical axis, and z to the mediolateral axis. Solid lines show stride-averaged means and shaded regions indicate ±1 SD across strides. The mediolateral component (gyro z ) exhibited the strongest agreement ( r=0.55 ) and was therefore selected as the sole input modality for student policy training.
Fig. 4: Teacher-student agreement in hip torque prediction. Representative time series of right hip assistance torque produced by the privileged teacher policy (black) and the distilled student controller (teal). The student is a temporal convolutional network that predicts hip torque from a short history of femur mediolateral gyroscope measurements expressed in the local segment frame. The example trial was collected on level ground while commanding a sequence of target speeds ( 0.7 , 1.1 , and 1.5 m/s), each maintained for 5 s. The first 0.95 s of the trial contains no student output, as the input history window has not yet been fully populated. For this trial, the student closely tracked the teacher output ( R2=0.93 ).
Fig. 5: Robotic hip exoskeleton hardware and onboard software architecture. (a) Robotic hip exoskeleton designed to apply bilateral hip flexion-extension torques during locomotion. (b) Onboard software architecture of the machine learning co-processor. Input sensor signals are logged via a dedicated I/O process; a parallel inference process generates real-time hip torque commands using the pre-trained student policy.
Fig. 6: Joint-level biomechanical comparison between simulation and human walking. Sagittal-plane hip, knee, and ankle joint angles (left column of each slope) and net joint moments normalized by body mass (right column of each slope) are shown over the gait cycle for ramp descent ( −5∘ ), level ground ( 0∘ ), and ramp ascent ( +5∘ ). Colored curves denote simulation results across walking speeds (0.7–1.5 m/s), while the gray band indicates the mean ± standard deviation of open-source human experimental data [ 36 ] . All simulation traces correspond to the curriculum stage 2 no-exoskeleton condition, where exoskeleton torques were clamped to zero.
Fig. 7: Learned assistance torque profile compared with conventional controllers. Representative hip assistance torque profiles over the gait cycle for a spline-based controller (Spline Control, red; 1.25 m/s) [ 37 ] , a biological-torque-based controller (Bio torque, green; 1.2 m/s) [ 15 ] , and the learned simulation policy (Simulation, blue; 1.2 m/s). Each profile was normalized by its peak hip flexion torque magnitude for visualization and peak-timing comparison. Markers indicate the timing of the hip extension peak (minimum, triangles) and hip flexion peak (maximum, circles).
Fig. 8: Peak-timing comparison between the biological hip joint moment and the learned hip-exoskeleton assistance profile. (a) Representative gait-cycle waveforms during level-ground walking ( 0∘ , 1.2 m/s), showing the sagittal-plane biological hip joint moment (black) and exoskeleton assistive torque (blue), with markers indicating the extension and flexion peaks. (b) Distribution of extension-peak and flexion-peak timings (percentage of the gait cycle) for the hip joint moment and exoskeleton torque across trials.
Fig. 9: Effect of exoskeleton assistance on muscle activation and positive joint power across conditions. Summary metrics are shown as a function of walking speed for three slope conditions (ramp descent, level ground, and ramp ascent), related to no-exoskeleton conditions. The top row reports mean muscle activation averaged across all muscles over a 15 s walking segment; the bottom row reports mean positive joint power averaged across bilateral hip, knee, and ankle joints in the sagittal plane.
Fig. 10: Comparison of hip assistance torque profiles between simulation and hardware. Gait-cycle-normalized right hip assistance torque is shown for ramp descent (RD), level ground (LG), and ramp ascent (RA). Blue curves denote simulated torque profiles across walking speeds from 0.7 to 1.5 m/s. Red curves show torque profiles generated by the distilled student controller on the physical device, evaluated at discrete speeds from 0.7 to 1.5 m/s in 0.2 m/s increments. Color intensity increases with walking speed.
Designing generalizable control policies for lower-limb exoskeletons remains fundamentally constrained by exhaustive data collection or iterative optimization procedures, which limit accessibility to clinical populations. To address this challenge, we introduce a device-agnostic framework that combines physiologically plausible musculoskeletal simulation with reinforcement learning to enable scalable personalized exoskeleton assistance for both able-bodied and clinical populations. Our control policies not only generate physiologically plausible locomotion dynamics but also capture clinically observed compensatory strategies under targeted muscular deficits, providing a unified computational model of both healthy and pathological gait. Without task-specific tuning, the resulting exoskeleton control policies produce assistive torque profiles at the hip and ankle that align with state-of-the-art profiles validated in human experiments, while consistently reducing metabolic cost across walking speeds. For simulated impaired-gait models, the learned control policies yield asymmetric, deficit-specific exoskeleton assistance that improves both energetic efficiency and bilateral kinematic symmetry without explicit prescription of the target gait pattern. These results demonstrate that physiologically plausible musculoskeletal simulation via reinforcement learning can serve as a scalable foundation for personalized exoskeleton control across both able-bodied and clinical populations, eliminating the need for extensive physical trials.
Itak Choi, Ilseung Park, Eni Halilaj +1
Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213 USA.
Data-driven joint-moment predictors offer a scalable alternative to laboratory-based inverse-dynamics pipelines for biomechanics estimation and exoskeleton control. Meanwhile, physics-based reinforcement learning (RL) enables simulation-trained controllers to learn dynamics-aware assistance strategies without extensive human experimentation. However, quantitative verification of simulation-trained exoskeleton torque predictors, and their impact on human joint power injection, remains limited. This paper presents (1) an RL framework to learn exoskeleton assistance policies that reduce biological joint moments, and (2) a validation pipeline that verifies the trained control networks using an open-source gait dataset through inference and comparison with biological joint moments. Simulation-trained multilayer perceptron (MLP) controllers are developed for level-ground and ramp walking, mapping short-horizon histories of bilateral hip and knee kinematics to normalized assistance torques. Results show that predicted assistance preserves task-intensity trends across speeds and inclines. Agreement is particularly strong at the hip, with cross-correlation coefficients reaching 0.94 at 1.8 m/s and 0.98 during 5° decline walking, demonstrating near-matched temporal structure. Discrepancies increase at higher speeds and steeper inclines, especially at the knee, and are more pronounced in joint power comparisons. Delay tuning biases assistance toward greater positive power injection; modest timing shifts increase positive power and improve agreement in specific gait intervals. Together, these results establish a quantitative validation framework for simulation-trained exoskeleton controllers, demonstrate strong sim-to-data consistency at the torque level, and highlight both the promise and the remaining challenges for sim-to-real transfer.
Zihang You, Xianlian Zhou
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA
Daily locomotion encompasses diverse activities and frequent transitions between them, yet most exoskeleton controllers are designed for a single activity or a narrow set of related movements. Changes in activity therefore typically require explicit mode switching and separately tuned or retrained controllers. Simulation-based learning reduces the need for hardware-based tuning but generally retains this limitation. Here we present UniExo, a framework that first constructs a multi-skill musculoskeletal human policy and then jointly trains an exoskeleton control policy with it. Four single-skill imitation experts for walking, turning, running and backward walking are distilled into a single network structured by a skill latent and subsequently fine-tuned through reinforcement learning on transition sequences. The resultant unified human policy achieves a mean tracking success rate of 94.7% on unseen clips of the four skills and exhibits greater robustness to perturbations than its constituent experts. A single hip exoskeleton controller (UniExo) is initialized from hip moment prediction of the human policy and co-adapted with it through multi-agent reinforcement learning across the four skills. This co-adaptation shifts the timing of the assistance torque and raises the fraction of positive work delivered to the hip. When deployed on a custom hip exoskeleton, the controller generalizes across four treadmill speeds in six participants and assists one participant through a continuous route of all four skills and their transitions, without skill labels or explicit mode switching. UniExo thus provides a step towards replacing activity-specific controllers with unified, user-specific controllers that support diverse locomotor activities and the transitions between them.
Yifei Yuan, Jakob Wolf, Ghaith Androwis +1
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA · Kessler Foundation, West Orange, NJ 07052, USA