A Shared Control Framework for Mobile Robots with Planning-Level Intention Prediction

Jinyu Zhang; Lijun Han; Feng Jian; Lingxi Zhang; Hesheng Wang

cs.RONov 12, 2025

A Shared Control Framework for Mobile Robots with Planning-Level Intention Prediction

Authors: Jinyu Zhang, Lijun Han, Feng Jian, Lingxi Zhang, Hesheng Wang

Abstract

In mobile robot shared control, effectively understanding human motion intention is critical for seamless human-robot collaboration. This paper presents a novel shared control framework featuring planning-level intention prediction. A path replanning algorithm is designed to adjust the robot's desired trajectory according to inferred human intentions. To represent future motion intentions, we introduce the concept of an intention domain, which serves as a constraint for path replanning. The intention-domain prediction and path replanning problems are jointly formulated as a Markov Decision Process and solved through deep reinforcement learning. In addition, a Voronoi-based human trajectory generation algorithm is developed, allowing the model to be trained entirely in simulation without human participation or demonstration data. Extensive simulations and real-world user studies demonstrate that the proposed method significantly reduces operator workload and enhances safety, without compromising task efficiency compared with existing assistive teleoperation approaches.

Figures & tables

Fig. 1: System configuration. The human operator guides the robot to visit all sub-goals $\mathbf{g}_{s}$ before reaching the final goal $\mathbf{g}_{f}$ . The operator observes the environment and robot status through a monitor interface. Based on the visual feedback, the operator provides control commands $\mathbf{u}^{h}$ to the shared controller, which computes the robot control input $\mathbf{u}$ for robot motion control.

Fig. 2: The proposed shared control framework. The locally shared controller computes safe robot control commands $\mathbf{u}$ based on the human control input $\mathbf{u}^{h}$ and the robot’s desired path $\tau^{r}$ . when the human operator intervenes, the reinforcement learning policy jointly encodes the environment, task priors, and the motion intentions of both the robot and the human, and determines whether to trigger path replanning and, if replanning is triggered, predicts the geometric parameters $(h,r)$ of the intention domain $\mathcal{D}_{I}$ . During replanning, the predicted intention domain is constructed and then used as a constraint to generate a new robot desired path $\tau^{r}$ that is consistent with the inferred human intention. The reinforcement learning policy is trained in simulation using H-PPO, where the human desired trajectories are generated by a Voronoi-based human trajectory generation algorithm without requiring human data. Purple denotes environment and task priors, sky blue indicates robot intentions, green represents human motion intentions, and yellow shows inferred intentions in the form of an intention domain. Dark blue components correspond to modules used for policy training.

Fig. 3: Voronoi-based human trajectory generation algorithm: (a) Extract the Discretized Voronoi Diagram (DVD) from the occupancy grid map. The start and goal are denoted by the green and red points. (b) Convert DVD into a Voronoi Graph. The sampled points are indicated by yellow nodes. (c) Generate initial trajectory along the Voronoi Graph edges, construct circular corridors and sample via-points within each corridor. (d) Trajectory planning under both corridor and via-point constraints

Model	$e_{MED}$ /m ( $\downarrow$ )	$f$	$2\phi$ / ∘	$r_{fail}$ /% ( $\downarrow$ )
PG-Baseline	$1.32\pm 0.06$	$1.00\pm 0.00$	-	-
$\beta=0.0$	$0.96\pm 0.11$	$0.29\pm 0.04$	$92.41\pm 7.76$	$1.16\pm 0.70$
$\beta=0.1$	$0.97\pm 0.12$	$0.27\pm 0.03$	$69.18\pm 6.22$	$1.03\pm 0.68$
$\beta=0.5$	$\mathbf{0.88\pm 0.11}$	$0.20\pm 0.03$	$56.22\pm 6.64$	$1.09\pm 0.68$
$\beta=2.0$	$0.93\pm 0.12$	$0.17\pm 0.03$	$33.46\pm 5.98$	$1.42\pm 0.79$
$w_{2}=0.0$	$1.28\pm 0.17$	$0.41\pm 0.06$	$90.74\pm 10.51$	$5.97\pm 1.89$

TABLE I: Evaluation with Generated Human Trajectories. Results are averaged over three runs with different random seeds.

Fig. 4: Heat map visualization of the spatial distribution of replanning frequency, evaluated over 256 human-desired trajectories. The discretized Voronoi diagram is overlaid in green lines.

Fig. 5: Monitor interface in simulation. Left: the robot’s front camera view. Right: the occupancy grid map with the robot’s desired trajectory, final goal and guiding path.

Fig. 6: Visualization of four sub-goal distribution scenarios.

Case	Condition	$T_{c}$ / s ( $\downarrow$ )	$T_{h}$ / s ( $\downarrow$ )	$\rho$ ( $\downarrow$ )	$D_{\mathrm{clear}}$ / m ( $\uparrow$ )
(a)	HL	$\mathbf{31.05\pm 1.31}$	$30.30\pm 0.89$	$97.65\pm 2.19\%$	$0.54\pm 0.03$
	SC	$33.87\pm 0.90$	$\mathbf{8.20\pm 1.69}$	$\mathbf{24.22\pm 5.27}\%$	$\mathbf{0.61\pm 0.00}$
(b)	HL	$\mathbf{34.94\pm 1.69}$	$34.38\pm 1.61$	$98.39\pm 0.13\%$	$0.54\pm 0.02$
	SC	$38.83\pm 0.58$	$\mathbf{15.37\pm 0.36}$	$\mathbf{39.59\pm 0.49}\%$	$0.54\pm 0.01$
(c)	HL	$\mathbf{64.35\pm 4.21}$	$63.65\pm 4.22$	$98.91\pm 0.07\%$	$0.55\pm 0.03$
	SC	$67.97\pm 2.11$	$\mathbf{33.51\pm 3.49}$	$\mathbf{49.35\pm 5.66}\%$	$\mathbf{0.58\pm 0.02}$

TABLE II: Evaluation results with human in the loop under four different sub-goal distributions.

Fig. 7: (a)-(c): Replanned paths and corresponding intention domains from one representative experiment (nine in total). (d): Actual versus guiding trajectory, with human-intervened segments shown in green.

Fig. 8: Visualization of the four intention domain shapes: Cone, Rectangle, Ellipse, and Corridor. All shapes are defined in the human’s local reference frame with identical height $h$ and radius $r$ . The forward boundaries, highlighted in red, indicate candidate regions for computing the predicted sub-goal (blue star). This sub-goal is selected as the collision-free point on the boundary that is closest to the final goal $g_{f}$ (red flag). The blue solid line illustrates the replanned path segment navigating to the predicted sub-goal.

Shape	$T_{c}$ / s ( $\downarrow$ )	$T_{h}$ / s ( $\downarrow$ )	$\rho$ ( $\downarrow$ )	$\Delta T_{c}$ ( $\downarrow$ )	$\Delta T_{h}$ ( $\downarrow$ )	$D_{\mathrm{clear}}$ / m ( $\uparrow$ )	$N_{\mathrm{replan}}$ ( $\downarrow$ )	$T_{\mathrm{infer}}$ / ms ( $\downarrow$ )
Rect	$81.80\pm 6.68$	$52.49\pm 10.49$	$63.85\pm 8.09\%$	$27.12\pm 10.38\%$	$-17.53\pm 16.48\%$	$0.57\pm 0.02$	$12.67\pm 1.53$	$\mathbf{17.54\pm 4.74}$
Elli	$80.91\pm 2.27$	$42.82\pm 7.63$	$52.78\pm 8.01\%$	$25.75\pm 3.52\%$	$-32.72\pm 11.99\%$	$0.57\pm 0.00$	$10.67\pm 0.58$	$20.36\pm 5.55$
Corr	$70.33\pm 3.68$	$37.66\pm 3.40$	$53.76\pm 7.12\%$	$9.29\pm 5.71\%$	$-40.82\pm 5.24\%$	$\mathbf{0.59\pm 0.01}$	$13.00\pm 5.29$	$22.07\pm 8.72$
Cone	$\mathbf{67.97\pm 2.11}$	$\mathbf{33.51\pm 3.49}$	$\mathbf{49.35\pm 5.66}\%$	$\mathbf{5.63\pm 3.29}\%$	$\mathbf{-47.35\pm 5.49}\%$	$0.58\pm 0.02$	$\mathbf{8.67\pm 0.58}$	$18.57\pm 3.46$

TABLE III: Comparative performance of different intention domain shapes in the human-in-the-loop simulation under Case (c).

Case	Method	$T_{c}$ / s ( $\downarrow$ )	$T_{h}$ / s ( $\downarrow$ )	$\rho$ ( $\downarrow$ )	$\Delta T_{c}$ ( $\downarrow$ )	$\Delta T_{h}$ ( $\downarrow$ )	$D_{\mathrm{clear}}$ / m ( $\uparrow$ )	$N_{\mathrm{replan}}$ ( $\downarrow$ )	$T_{\mathrm{infer}}$ / ms ( $\downarrow$ )
(c)	Heuristic	$72.68\pm 2.36$	$34.31\pm 1.27$	$\mathbf{47.07\pm 2.17}\%$	$12.96\pm 3.66\%$	$-46.09\pm 2.00\%$	$0.57\pm 0.01$	$21.67\pm 1.15$	$22.21\pm 2.88$
	RL	$\mathbf{67.97\pm 2.11}$	$\mathbf{33.51\pm 3.49}$	$49.35\pm 5.66\%$	$\mathbf{5.63\pm 3.29}\%$	$\mathbf{-47.35\pm 5.49}\%$	$\mathbf{0.58\pm 0.02}$	$\mathbf{8.67\pm 0.58}$	$\mathbf{18.57\pm 3.46}$
(d)	Heuristic	$69.06\pm 2.45$	$29.20\pm 2.71$	$42.23\pm 2.70\%$	$1.93\pm 3.62\%$	$-56.90\pm 4.00\%$	$0.49\pm 0.01$	$30.00\pm 8.54$	$18.71\pm 3.69$
	RL	$\mathbf{64.88\pm 4.39}$	$\mathbf{20.49\pm 3.59}$	$\mathbf{31.49\pm 4.06}\%$	$\mathbf{-4.24\pm 6.48}\%$	$\mathbf{-69.76\pm 5.30}\%$	$\mathbf{0.50\pm 0.01}$	$\mathbf{5.67\pm 1.53}$	$\mathbf{16.65\pm 4.70}$

TABLE IV: Comparative performance between the heuristic baseline and the RL method.

Fig. 9: Physical experimental setup. Left: Experimental environment with the robot performing a search-and-rescue task. Right: The operator remotely controlled the TurtleBot 4 using an Xbox controller while monitoring the environment.

Fig. 10: Results of the user study. Bar plots show the mean and standard error of the evaluated measures. Asterisks indicate statistical significance (*

p<0.05

, **

p<0.01

, ***

p<0.001

). (a) Task completion time. (b) Interaction time. (c) Trajectory clearance (safety). (d) System Usability Scale (SUS) scores. (e) NASA Task Load Index (NASA-TLX) subscale scores. (f) User ratings for the task-specific statements on a 7-point Likert scale.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Discount factor	0.99
Maximum time steps	$2\times 10^{7}$
Batch size	512
Update epochs	8
Learning rate	$5\times 10^{-6}$
Clipping parameter $\epsilon$	0.3

Appendix

TABLE V: Hyperparameters for H-PPO Training

Mobile Robotics

Mobile robotics studies how ground, aerial, and aquatic robots perceive, navigate, and act in changing real-world environments.

See all 135 papers

Oct 18, 2026

Intent-Informed Model Predictive Control for Shared Autonomy in Remote Manipulation Tasks

Shared autonomy is a control paradigm that combines human cognitive abilities with robot precision to enable effective remote manipulation in complex environments. A key challenge is how to adapt the level of autonomy using information extracted from the human, task, and/or environment. In this paper, we propose a novel arbitration strategy that dynamically adjusts the level of autonomy in the system using the entropy of a belief distribution derived from a multimodal intent inference. Human intent is modeled with a Dynamic Bayesian Network (DBN), which updates beliefs over goals given real-time operator gaze data and robot arm motion during teleoperation. We then formulate the blending of human input and autonomous control as a Model Predictive Control (MPC) problem, where the system's autonomous behavior is guided by the belief distribution. This approach enables continuous, gradual transitions across the full spectrum of autonomy levels, from direct teleoperation to full autonomy. To validate our approach, we conducted experiments to compare the performance of our proposed algorithm against two MPC-based shared autonomy methods. Compared to the baselines, our approach achieved reductions of at least 61% in object retrieval time, 44% in trajectory length, and 60% in human control input. Participants reported high perceived control, predictability, intent recognition, and responsiveness, along with low workload.

Lorena Maria Genua, Camilla Tabasso, Mya Schroder +3

Amazon Fulfillment Technologies & Robotics, Westborough MA, USA · Institute for Experiential Robotics, Northeastern University, Boston MA, USA · School of Computer, Data & Information Sciences, University of Wisconsin - Madison, Madison WI, USA

Robotic Manipulation Robot Teleoperation Model Predictive Control

Ask PDF

Sep 15, 2026

HINT-Plan: Human Intention-Aware Robot Task Planning in Context-Rich Environments using Vision Language Models

Approaches to incorporating human awareness into mobile robot decision-making mainly focus on collision avoidance in low-level motion planning, often overlooking the challenges posed by human presence and high-level behavior. To address this vacancy, we present HINT-Plan, a novel approach to integrate human intention prediction into robot task planning. HINT-Plan employs Vision Language Models (VLMs) to anticipate high-level human intentions from third-person image observations, convert them into goal states, and solve joint task-planning problems. To effectively enable scene awareness in context-rich environments, we use hierarchical Scene Graphs (SGs) as high-level representations of the environment, and translate environmental topology and actionable knowledge into formal planning language to ensure executable plans. Evaluated in a photorealistic simulation, HINT-Plan achieves an overall success rate of 69.71% in joint human-robot task planning, substantially outperforming the baselines by up to 35.29%, while also reducing functional conflicts. The results show the effectiveness of explicitly incorporating inferred human intentions into formal multi-agent task planning for proactive human-aware robot decision-making.

Yuchen Liu, Luigi Palmieri, Lujun Li +3

University of Stuttgart, Germany · University of Luxembourg, Luxembourg · Bosch Center for Artificial Intelligence, Germany

Human-Robot Collaboration Robot Task Planning Multi-Agent Planning Scene Graphs

Ask PDF

Jul 10, 2026

Robot Trajectron V3: A Probabilistic Shared Control Framework for SE(3) Manipulation

We aim to address the challenge of teleoperating robotic arms for high-degree-of-freedom (high-DoF) manipulation tasks, which is cognitively demanding and error-prone, particularly when relying on low-bandwidth interfaces. We propose Robot Trajectron V3 (RT-V3), a probabilistic shared control framework designed for

SE(3)

grasping tasks. RT-V3 formulates shared control as Bayesian inference by learning a prior over user intent and combining it with real-time user commands to estimate the posterior intent distribution. The prior models user intent as a distribution over future trajectories conditioned on past robot dynamics and visual scene context. The intent prior is parameterized by a transformer-based conditional generative model that reasons over point clouds and candidate grasp poses, together with a factorized translation-rotation representation that improves learning efficiency in high-dimensional action spaces. During execution, RT-V3 continuously estimates the posterior distribution over future trajectories by combining the learned intent prior with a user-command likelihood derived from the observed control input, enabling continuous intent refinement and shared assistance. Comprehensive experiments demonstrate that RT-V3 achieves high accuracy in trajectory prediction and competitive performance in reactive planning. Furthermore, real-world user studies indicate that RT-V3 significantly outperforms baseline methods in terms of success rate and efficiency, while substantially reducing the user's physical and mental workload.

Pinhao Song, Zhongxi Li, Ze Fu +2

KU Leuven, Dept. Mechanical Engineering, Research unit Robotics, Automation and Mechatronics, B-3000 Leuven, Belgium. · KU Leuven, Dept. Electrical Engineering, Research unit Processing Speech and Images, B-3000 Leuven, Belgium.

Robotic Control Robotic Grasping Robot Teleoperation Robot Motion Prediction Trajectory Prediction

Ask PDF

Abstract

Mobile Robotics

Explore similar work

Intent-Informed Model Predictive Control for Shared Autonomy in Remote Manipulation Tasks

HINT-Plan: Human Intention-Aware Robot Task Planning in Context-Rich Environments using Vision Language Models

Robot Trajectron V3: A Probabilistic Shared Control Framework for SE(3) Manipulation