Toward Global Intent Inference for Human Motion by Inverse Reinforcement Learning

Sarmad Mehrdad; Maxime Sabbah; Vincent Bonnet; Ludovic Righetti

cs.ROMar 8, 2026

Toward Global Intent Inference for Human Motion by Inverse Reinforcement Learning

Authors: Sarmad Mehrdad, Maxime Sabbah, Vincent Bonnet, Ludovic Righetti

Organizations: Machines in Motion Laboratory, New York University, USA · LAAS-CNRS, Université Paul Sabatier, CNRS, Toulouse, France · Image and Pervasive Access Laboratory (IPAL), CNRS-UMI, Singapore · Artificial and Natural Intelligence Toulouse Institute (ANITI), Toulouse

Abstract

This paper investigates whether a single, unified cost function can explain and predict human reaching movements, in contrast with existing approaches that rely on subject- or posture-specific optimization criteria. Using the Minimal Observation Inverse Reinforcement Learning (MO-IRL) algorithm, together with a seven-dimensional set of candidate cost terms, we efficiently estimate time-varying cost weights for a standard planar reaching task. MO-IRL provides orders-of-magnitude faster convergence than bilevel formulations, while using only a fraction of the available data, enabling the practical exploration of time-varying cost structures. Three levels of generality are evaluated: Subject-Dependent Posture-Dependent, Subject-Dependent Posture-Independent, and Subject-Independent Posture-Independent. Across all cases, time-varying weights substantially improve trajectory reconstruction, yielding an average 27% reduction in RMSE compared to the baseline. The inferred costs consistently highlight a dominant role for joint-acceleration regulation, complemented by smaller contributions from torque-change smoothness. Overall, a single subject- and posture-agnostic time-varying cost function is shown to predict human reaching trajectories with high accuracy, supporting the existence of a unified optimality principle governing this class of movements.

Figures & tables

Fig. 1: (a) Biomechanical model definition, showing the beginning and the end of the pointing task. (b) Five different initial postures for the pointing task [ 7 ] .

Label	Name	Equation	Reference
$\Phi_{1}$	Cartesian velocity (CV)	$\sum_{t=0}^{T}\boldsymbol{V}_{t}^{T}\boldsymbol{V}_{t}\,\Delta t$	[ 16 ]
$\Phi_{2}$	Energy (Eng)	$\sum_{t=0}^{T}\left\|\boldsymbol{v}_{t}^{T}\mathbf{\boldsymbol{\tau}}_{t}\right\|\Delta t$	[ 30 , 8 ]
$\Phi_{3}$	Geodesic (Geo)	$\sum_{t=0}^{T}\boldsymbol{v}_{t}^{T}M(\boldsymbol{q}_{t})\boldsymbol{v}_{t}\Delta t$	[ 9 ]
$\Phi_{4}$	Joint acceleration (JA)	$\sum_{t=0}^{T}\boldsymbol{a}_{t}^{T}\boldsymbol{a}_{t}\Delta t$	[ 6 ]
$\Phi_{5}$	Joint torque change (JTC)	$\sum_{t=0}^{T}\dot{\boldsymbol{\tau}}_{t}^{T}\dot{\boldsymbol{\tau}}_{t}\Delta t$	[ 38 , 28 ]
$\Phi_{6}$	Joint velocity (JV)	$\sum_{t=0}^{T}\boldsymbol{v}_{t}^{T}\boldsymbol{v}_{t}\Delta t$	[ 3 ]

TABLE I: (Discretized) Biomechanical cost functions

Fig. 2: Averaged estimated weights, and cost contribution, across all subjects for initial postures 1-5, produced through SDPD analysis. Plots show a prominent importance of the Joint Acceleration feature in the cost function ( $\Phi_{4}$ ).

Fig. 3: RMSE values boxplot comparison of all investigated cases against the baseline, averaged across all subjects. Red line in each boxplot indicates the mean value.

Fig. 4: RMSE values (deg) boxplot comparison of all methods against the baseline, averaged across all initial postures. Red line in each boxplot indicates the mean value.

	MO-IRL SDPD (Ours)			Baseline
Initial Posture	$q_{1}$	$q_{2}$	Avg	$q_{1}$	$q_{2}$	Avg
P1	8.56	8.47	8.51	16.92	15.40	16.16
P2	7.77	7.17	7.47	10.06	13.84	11.95
P3	14.46	9.76	12.11	20.60	19.83	20.21
P4	5.52	13.33	9.43	9.28	19.84	14.56
P5	11.99	10.36	11.17	13.54	17.35	15.45

TABLE II: RMSE results of SDPD weights for reaching task generated by MO-IRL, in comparison with the baseline.

	MO-IRL SDPI (Ours)			Baseline
Initial Posture	$q_{1}$	$q_{2}$	Avg	$q_{1}$	$q_{2}$	Avg
P1	8.39	9.80	9.10	16.92	15.40	16.16
P2	12.40	9.94	11.17	10.06	13.84	11.95
P3	19.71	15.17	17.44	20.60	19.83	20.21
P4	8.75	13.76	11.26	9.28	19.84	14.56
P5	12.32	14.83	13.57	13.54	17.35	15.45

TABLE III: RMSE results of SDPI weights for reaching task generated by MO-IRL, in comparison with the baseline.

Fig. 5: Weights and cost contributions of general cost weights proposed for the reaching task in SIPI case. The RMSE comparison boxplots between all existing demonstrations and predictions are presented in the figure at the bottom.

Fig. 6: Comparison of SDPD, SDPI, SIPI, and baseline predictions of the subject’s joint values, against the human demonstration.

	MO-IRL SIPI (Ours)			Baseline
Initial Posture	$q_{1}$	$q_{2}$	Avg	$q_{1}$	$q_{2}$	Avg
P1	6.98	8.69	7.83	16.92	15.40	16.16
P2	13.09	9.26	11.17	10.06	13.84	11.95
P3	20.85	15.23	18.04	20.60	19.83	20.21
P4	7.37	14.36	10.86	9.28	19.84	14.56
P5	13.55	16.00	14.77	13.54	17.35	15.45

TABLE IV: RMSE results of SIPI weights for reaching task generated by MO-IRL, in comparison with the baseline.

Human Motion Prediction

Human motion prediction forecasts how people will move, supporting applications from animation and activity understanding to safe robot interaction.

See all 53 papers

Cost-Matching Model Predictive Control for Efficient Reinforcement Learning in Humanoid Locomotion
Mar 30, 2026Wenqi Cai, Kyriakos G. Vamvoudakis, Sébastien Gros +1Humanoid Robot Locomotion Model Predictive Control
Electrical Engineering Program, New York University Abu Dhabi (NYUAD), 129188, UAE. · School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30332, United States. · Department of Engineering Cybernetics, Norwegian University of Science and Technology (NTNU), Trondheim, Norway.
Trust Region Inverse Reinforcement Learning: Explicit Dual Ascent using Local Policy Updates
May 10, 2026Anish Diwan, Davide Tateo, Christopher E. Mower +3Reinforcement Learning Policy Optimization
Technical University of Darmstadt · Robotics Institute Germany (RIG) · Lund University +3
Gradient based Bilevel for Inverse Optimal Control, a Riemannian approach
Jun 9, 2026Ahmed-Manaf Dahmani, Vincent Bonnet, David Daney +1Bilevel Optimization Optimal Control
INRIA Bordeaux, Universit´e de Bordeaux, Talence, France · INRIA Nancy, Universit´e de Lorraine, Villers-l`es-Nancy, France · LAAS-CNRS, Universit´e de Toulouse, CNRS, Toulouse, France

Mar 30, 2026J/K move · Enter open · S save

Cost-Matching Model Predictive Control for Efficient Reinforcement Learning in Humanoid Locomotion

Wenqi Cai, Kyriakos G. Vamvoudakis, Sébastien Gros, Anthony Tzes

Electrical Engineering Program, New York University Abu Dhabi (NYUAD), 129188, UAE. · School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30332, United States. · Department of Engineering Cybernetics, Norwegian University of Science and Technology (NTNU), Trondheim, Norway.

Humanoid Robot Locomotion Model Predictive Control Model-Based RL

Open paper AskPDF arXiv

In this paper, we propose a cost-matching approach for optimal humanoid locomotion within a Model Predictive Control (MPC)-based Reinforcement Learning (RL) framework. A parameterized MPC formulation with centroidal dynamics is trained to approximate the action-value function obtained from high-fidelity closed-loop data. Specifically, the MPC cost-to-go is evaluated along recorded state-action trajectories, and the parameters are updated to minimize the discrepancy between MPC-predicted values and measured returns. This formulation enables efficient gradient-based learning while avoiding the computational burden of repeatedly solving the MPC problem during training. The proposed method is validated in simulation using a commercial humanoid platform. Results demonstrate improved locomotion performance and robustness to model mismatch and external disturbances compared with manually tuned baselines.

Abstract

Human Motion Prediction

Explore similar work

Cost-Matching Model Predictive Control for Efficient Reinforcement Learning in Humanoid Locomotion

Trust Region Inverse Reinforcement Learning: Explicit Dual Ascent using Local Policy Updates

Gradient based Bilevel for Inverse Optimal Control, a Riemannian approach