Learning a Speed-adaptive Hip Exoskeleton Control Policy Via Sim-to-real Reinforcement Learning

Bin Li; Zhimin Hou; Jiacheng Hou; Zenian Liang; Tong Wu; Teng Ma; Chenglong Fu

cs.ROSep 23, 2026

Learning a Speed-adaptive Hip Exoskeleton Control Policy Via Sim-to-real Reinforcement Learning

Authors: Bin Li, Zhimin Hou, Jiacheng Hou, Zenian Liang, Tong Wu, Teng Ma, Chenglong Fu

Organizations: Division of Industrial Data Science (DIDS), School of Data Science, Lingnan University, Hong Kong · Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, 518055, China

Abstract

Providing personalized exoskeleton assistance across varying walking speeds remains challenging. Existing online optimization methods are sample-inefficient, requiring extensive human-in-the-loop (HIL) evaluations to optimize the entire assistive torque profile. Sim-to-real reinforcement learning (RL) offers a promising alternative but cannot directly account for individual user preferences. We propose a framework integrating sim-to-real RL with online preference learning for personalized exoskeleton assistance. Specifically, assistance timing is learned in simulation by training RL policies with human musculoskeletal models across varying walking speeds. The learned policies are then distilled and deployed on a physical hip exoskeleton using onboard sensory observations. Gaussian-process-based preference learning further personalizes the assistance magnitude through pairwise user comparisons. By decoupling assistance timing learning in simulation from magnitude optimization in real-world experiments, our framework substantially reduces the online optimization space. Human-subject experiments demonstrate efficient identification of personalized assistive torque profiles across varying walking speeds with fewer real-world evaluations.

Figures & tables

Fig. 1 : Overview of the proposed framework. (a) Learning a human control policy across varying walking speeds. (b) Learning hip exoskeleton teacher policies through interaction with the human musculoskeletal model. (c) Distilling the fully observed teacher policies into a deployable TCN-based student policy using onboard sensory observations. (d) Online preference-based personalization of assistance magnitude after deployment on the physical exoskeleton.

Panel of Figure 2 — Fig. 2 : Simulation results of human motion reproduction and muscle activation. (a) Speed-adaptive behavior of the learned human motion policy. Hip, knee, and ankle joint kinematics and moments across walking speeds from ${0.5}$ to $1.7~\mathrm{m/s}$ , with orange curves denoting the reference motion; (b) Comparison of RMS muscle activation across assistance methods at ${0.9}$ , ${1.25}$ , and $1.6~\mathrm{m/s}$ .

Fig. 3 : Comparison of assistance performance during variable-speed walking (acceleration from

0.9~\mathrm{m/s}

1.6~\mathrm{m/s}

and deceleration from

1.6~\mathrm{m/s}

0.9~\mathrm{m/s}

).(a) Commanded assistive torque over the speed-transition sequence. (b) Corresponding positive mechanical power and gait-level mean positive power (MPP). The MLP-PPO and LSTM-PPO policies are trained only at a fixed walking speed of

1.25~\mathrm{m/s}

and are directly applied to the acceleration and deceleration stages for comparison.

Subject	DOFC				Ours				Online Tuning
Subject	$\bm{\tau}_{\mathrm{RMS}}(Nm)$	$\bm{\tau}_{\mathrm{MAX}}(Nm)$	MPP	MNP	$\bm{\tau}_{\mathrm{RMS}}$	$\bm{\tau}_{\mathrm{MAX}}$	MPP	MNP	$\bm{\tau}_{\mathrm{RMS}}$	$\bm{\tau}_{\mathrm{MAX}}$	MPP	MNP
S1	2.57	4.04	4.30	-0.090	3.85	5.25	5.26	-0.004	6.35	9.96	8.63	-0.192
S2	3.69	5.37	6.60	-0.022	3.09	4.52	5.06	-0.012	3.23	5.36	5.49	-0.093
S3	3.27	4.87	6.40	-0.321	3.06	4.61	5.17	-0.059	5.66	8.08	10.28	-0.094
S4	4.12	5.83	5.79	-0.089	3.54	5.18	3.55	-0.011	4.66	9.00	7.59	-0.123
S5	3.24	4.64	4.83	-0.020	3.90	5.74	5.64	-0.040	5.69	8.94	8.77	-0.005

TABLE I : Commanded Torque and Mechanical Power Across Subjects

Panel of Figure 5 — Fig. 4 : Experimental results of muscle activation and resistive assistance. (a) Comparison of muscle activation across three assistance methods at three walking speeds. (b) Resistance ratio, defined as the fraction of samples exhibiting negative assistive mechanical power within 100 ms after hip angular-velocity zero crossings.

	Human		Exo
Parameter	Phase 1	Phase 2	Phase 1	Phase 2
$r_{v}$	1.0	0.5	0.5	3/2
$r_{\bar{v}}$	0	0.5	0.5	3/2
$r_{\dot{v}}$ and $r_{\overline{\dot{v}}}$	0	0.5	0.5	3.0
$r_{m}$	0.1	0.05	0.15	0.15
$r_{\Delta m}$	0.1	0	0.05	0.05

TABLE II : Reward Weights for All different policy-learning stages.

	Human		Exo
Parameter	Phase 1	Phase 2	Phase 1	Phase 2
Learning rate	$1\times 10^{-4}$	$1\times 10^{-4}$	$3\times 10^{-5}$	$3\times 10^{-5}$
$n_{\mathrm{steps}}$	512	512	2048	2048
Batch size	8192	8192	8192	16384
$n_{\mathrm{epochs}}$	30	30	20	20
$\gamma$	0.99	0.99	0.99	0.99

TABLE III : Hyperparameters of PPO used for all policies learning.

Explore similar work

Musculoskeletal Motion Imitation for Learning Personalized Exoskeleton Control Policy in Impaired Gait
Apr 10, 2026Itak Choi, Ilseung Park, Eni Halilaj +1Assistive Robotics Ankle Exoskeleton Control
Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213 USA.
Learning Hip Exoskeleton Control Policy via Predictive Neuromusculoskeletal Simulation
Mar 4, 2026Ilseung Park, Changseob Song, Inseung KangRobotic Control Sim-to-Real Transfer
Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213 USA.
Personalized Lower-limb Exoskeleton Assistance via Preference-based Bayesian Optimization
Aug 10, 2026Xiao-Yin Liu, Guotao Li, Weiqun Wang +1Bayesian Optimization Ankle Exoskeleton Control
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · CASIA-MUST Joint Laboratory of Intelligence Science and Technology, Institute of Systems Engineering, Macau University of Science and Technology, Macao, China

Apr 10, 2026J/K move · Enter open · S save

Musculoskeletal Motion Imitation for Learning Personalized Exoskeleton Control Policy in Impaired Gait

Itak Choi, Ilseung Park, Eni Halilaj, Inseung Kang

Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213 USA.

Assistive Robotics Ankle Exoskeleton Control RL Control

Open paper AskPDF arXiv

Designing generalizable control policies for lower-limb exoskeletons remains fundamentally constrained by exhaustive data collection or iterative optimization procedures, which limit accessibility to clinical populations. To address this challenge, we introduce a device-agnostic framework that combines physiologically plausible musculoskeletal simulation with reinforcement learning to enable scalable personalized exoskeleton assistance for both able-bodied and clinical populations. Our control policies not only generate physiologically plausible locomotion dynamics but also capture clinically observed compensatory strategies under targeted muscular deficits, providing a unified computational model of both healthy and pathological gait. Without task-specific tuning, the resulting exoskeleton control policies produce assistive torque profiles at the hip and ankle that align with state-of-the-art profiles validated in human experiments, while consistently reducing metabolic cost across walking speeds. For simulated impaired-gait models, the learned control policies yield asymmetric, deficit-specific exoskeleton assistance that improves both energetic efficiency and bilateral kinematic symmetry without explicit prescription of the target gait pattern. These results demonstrate that physiologically plausible musculoskeletal simulation via reinforcement learning can serve as a scalable foundation for personalized exoskeleton control across both able-bodied and clinical populations, eliminating the need for extensive physical trials.

Abstract

Explore similar work

Musculoskeletal Motion Imitation for Learning Personalized Exoskeleton Control Policy in Impaired Gait

Learning Hip Exoskeleton Control Policy via Predictive Neuromusculoskeletal Simulation

Personalized Lower-limb Exoskeleton Assistance via Preference-based Bayesian Optimization