Organizations: Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA · Kessler Foundation, West Orange, NJ, 07052, USA
Effective exoskeleton assistance requires co-adaptation: as the device alters joint dynamics, the user reorganizes neuromuscular coordination, creating a non-stationary learning problem. Most learning-based approaches do not explicitly account for the sequential nature of human motor adaptation, leading to training instability and poorly timed assistance. We propose Staged Multi-Agent Training (SMAT), a four-stage curriculum designed to mirror how users naturally acclimate to a wearable device. In SMAT, a musculoskeletal human actor and a bilateral hip exoskeleton actor are trained progressively: the human first learns unassisted gait, then adapts to the added device mass; the exoskeleton subsequently learns a positive assistance pattern against a stabilized human policy, and finally both agents co-adapt with full torque capacity and bidirectional feedback. We implement SMAT in the MyoAssist simulation environment using a 26-muscle lower-limb model and an attached hip exoskeleton. Our musculoskeletal simulations demonstrate that the learned exoskeleton control policy produces an average 10.1% reduction in hip muscle activation relative to the no-assist condition. We validated the learned controller in an offline setting using open-source gait data, then deployed it to a physical hip exoskeleton for treadmill experiments with five subjects. The resulting policy delivers consistent assistance and predominantly positive mechanical power without the need for any explicitly imposed timing shift (mean positive power: 13.6 W at 6 Nm RMS torque to 23.8 W at 9.3 Nm RMS torque, with minimal negative power) consistently across all subjects without subject-specific retraining.
Figures & tables
Fig. 1: Overview of the SMAT framework. Two PPO-based actors, a musculoskeletal human actor πh and an exoskeleton actor πe , interact with a shared physics simulation through a shared critic Vψ . Human-Only Training (left): only πh updates via imitation learning from a reference gait. Human-Exo Joint Training (center): both actors train simultaneously. Hardware Evaluation (right): the trained exoskeleton policy is deployed via sim-to-real transfer, running on a Raspberry Pi 4B to command bilateral MyActuator X8-25 actuators during treadmill walking trials. In SMAT, Stage 1 resides in the human only training mode; Stage 2 adapts the human actor with added exoskeleton mass; Stage 3 freezes the human actor and only trains the exo actor for positive assistance timing; and finally Stage 4 trains both actors for co-adaptation. Policies trained in each stage are subsequently transferred to the next stage.
Reward term
Stage 1
Stage 2
Stage 3
Stage 4
rfwd
0.8
0.8
1.5
1.5
rmuscle
0.01
0.01
0.15
0.15
rΔa
0.005
0.005
0.05
0.05
rhip-act
—
—
2.0
5.0
rexo
—
—
4.0
4.0
rΔτ
—
—
—
1.0
TABLE I: Stage-wise reward configuration and weights used in the multi-stage curriculum.
Parameter
S1
S2–S4
Learning rate
5×10−5
3×10−5
PPO clip range
0.15
0.15
Rollout steps / env
2048
2048
Minibatch size
16384
16384
Epochs per update
20
20
Discount γ
0.99
0.99
TABLE II: PPO training hyperparameters. Values are shared across stages unless noted otherwise.
Item
Specification
Configuration
Custom-designed bilateral hip exoskeleton, Fig. 1 .
Assistance plane
Sagittal plane (hip flex/ext)
Active DOF
1 per side (hip flexion and extension)
Passive DOF
1 per side (hip abd/add)
Actuator
MyActuator X8-25 BLDC (2 total)
Peak torque capacity
25 Nm per hip
TABLE III: Hip exoskeleton hardware specifications.
Fig. 2: Training reward curves and joint kinematics for Stages 1 and 2 (left and right columns, respectively). Top: Episode reward over training steps. Bottom: Mean ± 1 s.d. hip, knee, and ankle angle trajectories over the normalized gait cycle; right limb (solid blue), left limb (dashed red). Vertical lines: mean toe-off.
Fig. 3: Stage 4 co-adapted profiles. (a) Exoskeleton assistance power reward (left axis, green) and hip muscle activation penalty (right axis, red dashed) over training. (b) Hip angular velocity over the gait cycle (positive: flexion; negative: extension). (c) Normalized exoskeleton torque. (d) Exoskeleton mechanical power. Panels (b–d): bilateral average; shaded region: ± 1 SD. Dotted vertical line: mean toe-off.
Fig. 4: Right-side hip muscle activation over the gait cycle comparing Stage 2 (no assistive torque, blue) and Stage 4 (maximum assistive torque 25 Nm, orange). Shaded bands: mean ± 1 s.d. Dashed vertical lines mark mean toe-off for each condition.
Fig. 5: Ablation comparing right-hip normalized exo torque over the gait cycle under four conditions: Stage 3 only without Stage 4 co-adaptation (dashed green); Stage 4 only initialized from Stage 2 without Stage 3 pre-training (dashed red, mean across three runs); Stage 4 only without rexo(4) , initialized from Stage 2 (dash-dot orange); and the full Stage 3 + Stage 4 (SMAT) pipeline (solid blue). Dotted vertical lines mark mean toe-off.
10 Nm
15 Nm
Subject No.
τRMS (Nm)
τMAX (Nm)
MPP (W)
MNP (W)
τRMS (Nm)
τMAX (Nm)
MPP (W)
MNP (W)
S1
6.12
8.12
14.93
− 0.16
9.40
12.25
26.58
− 0.14
S2
6.13
8.14
17.26
− 0.11
9.67
12.13
29.91
− 0.10
S3
5.87
8.07
11.74
− 0.15
9.07
12.39
20.02
− 0.18
S4
5.89
7.95
12.90
− 0.09
9.13
12.03
22.01
− 0.18
S5
5.97
8.02
11.27
− 0.10
9.14
12.30
20.51
− 0.10
TABLE IV: Per-Subject Exoskeleton Joint Torque and Power Metrics Across Assistance Conditions
Fig. 6: Speed generalization on the GT dataset [ 19 ] (10 subjects, 0.6, 1.2, and 1.8 m/s; blue/green/red). Top: Gait-cycle-normalized group mean hip torque; solid: exo torque, dashed: biological hip torque (no-exo condition, amplitude scaled for timing comparison). Bottom: Hip exo power; thin lines: individual subjects, bold: group mean. Dashed vertical line: toe-off ( ≈ 62 %).
Fig. 7: Hip kinematics, exo torque, and mechanical power under assisted conditions (0 % = peak hip flexion; 0 Nm: blue, 10 Nm: green, 15 Nm: red). Top: Group-mean right hip angle. Middle: Exoskeleton torque. Bottom: Right hip assistance power. Shaded bands: ±1 SD; thin lines: subject means.
Learning-based controllers can deliver exoskeleton assistance after training entirely in physics-based simulation, yet few controllers that address human-device co-adaptation have been validated on real users by whole-body metabolic measurement, the standard benchmark for assistive walking. Co-adaptation is challenging: as the device alters joint dynamics, the wearer reorganizes neuromuscular coordination, producing a non-stationary learning problem. Staged Multi-Agent Training (SMAT), a four-stage curriculum that progressively trains a musculoskeletal human actor and a bilateral hip exoskeleton actor, was introduced and shown to reduce simulated hip-muscle activation and provide positive assistance on hardware. This article provides the first physiological validation of SMAT. The policy was deployed on a hip exoskeleton and tested with eight healthy adults, with metabolic cost measured by indirect calorimetry across no-exoskeleton, passive, and active conditions. Active assistance lowered net metabolic rate by 19.7% relative to the passive device (p < 0.001). Biomechanical analysis confirmed predominantly positive hip mechanical power across all subjects (positive-power ratio 0.98), and the policy generalized across walking speeds and terrains. Together, these results show that a single simulation-trained SMAT policy, deployed without subject-specific retraining, delivers a significant metabolic benefit on real users while remaining robust beyond the conditions it was trained on.
Yifei Yuan, Jakob Wolf, Ghaith Androwis +1
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA · Kessler Foundation, West Orange, NJ 07052, USA
Developing exoskeleton controllers that generalize across diverse locomotor conditions typically requires extensive motion-capture data and biomechanical labeling, limiting scalability beyond instrumented laboratory settings. Here, we present a physics-based neuromusculoskeletal learning framework that trains a hip-exoskeleton control policy entirely in simulation, without motion-capture demonstrations, and deploys it on hardware via policy distillation. A reinforcement learning teacher policy is trained using a muscle-synergy action prior over a wide range of walking speeds and slopes through a two-stage curriculum, enabling direct comparison between assisted and no-exoskeleton conditions. In simulation, exoskeleton assistance reduces mean muscle activation by up to 3.4% and mean positive joint power by up to 7.0% on level ground and ramp ascent, with benefits increasing systematically with walking speed. On hardware, the assistance profiles learned in simulation are preserved across matched speed-slope conditions (r: 0.82, RMSE: 0.03 Nm/kg), providing quantitative evidence of sim-to-real transfer without additional hardware tuning. These results demonstrate that physics-based neuromusculoskeletal simulation can serve as a practical and scalable foundation for exoskeleton controller development, substantially reducing experimental burden during the design phase.
Ilseung Park, Changseob Song, Inseung Kang
Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213 USA.
Daily locomotion encompasses diverse activities and frequent transitions between them, yet most exoskeleton controllers are designed for a single activity or a narrow set of related movements. Changes in activity therefore typically require explicit mode switching and separately tuned or retrained controllers. Simulation-based learning reduces the need for hardware-based tuning but generally retains this limitation. Here we present UniExo, a framework that first constructs a multi-skill musculoskeletal human policy and then jointly trains an exoskeleton control policy with it. Four single-skill imitation experts for walking, turning, running and backward walking are distilled into a single network structured by a skill latent and subsequently fine-tuned through reinforcement learning on transition sequences. The resultant unified human policy achieves a mean tracking success rate of 94.7% on unseen clips of the four skills and exhibits greater robustness to perturbations than its constituent experts. A single hip exoskeleton controller (UniExo) is initialized from hip moment prediction of the human policy and co-adapted with it through multi-agent reinforcement learning across the four skills. This co-adaptation shifts the timing of the assistance torque and raises the fraction of positive work delivered to the hip. When deployed on a custom hip exoskeleton, the controller generalizes across four treadmill speeds in six participants and assists one participant through a continuous route of all four skills and their transitions, without skill labels or explicit mode switching. UniExo thus provides a step towards replacing activity-specific controllers with unified, user-specific controllers that support diverse locomotor activities and the transitions between them.
Yifei Yuan, Jakob Wolf, Ghaith Androwis +1
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA · Kessler Foundation, West Orange, NJ 07052, USA