In human-robot co-manipulation, the robot's trajectory should accommodate the human's, where human sensorimotor behavior is affected by the task's spatial and temporal constraints, as well as personal factors. Several robot trajectory planning methods have been proposed to embed human sensorimotor models in the robot trajectory planning, but these methods do not consider temporal constraints that adapt to how the human behaves when interacting with the robot. To better align human and robot collaborative motion, this work adapts a stochastic optimal control problem with a final state constraint from the task constraints and a temporal constraint from Fitts' law. A recursive least squares method is used to estimate the Fitts' law parameters, adapting the robot's motion duration to that of the human, thus allowing the robot to adapt to how the human behaves when interacting with the robot. It is hypothesized that a speed increase can result from the robot augmenting the accuracy. The feasibility of the approach is validated on a positioning task where the robot controls degrees of freedom which have repeatable motion, and the human adapts motion in the complementary degrees of freedom. The approach is shown to be computationally suited to online use, it converges as the human and robot mutually adapt, and can increase the movement speed, although thorough human studies are needed in future work.
Figures & tables
Figure 1: The proposed sensorimotor planner generates a human-like trajectory (blue) to a authority handover plane (red), where the robot controls the vertical z-axis motion with a high impedance, and the human controls the horizontal x/y-plane motion. By including the known handover plane as a final state constraint, the robot eliminates the primary source of task risk, which is collisions with the working plane (e.g., assembly plane) (yellow). The exact position within the plane (dark green) is determined by the human to adapt to various goal positions (diverging arrows, light green). The robot’s motion should be synchronized so the human doesn’t need to wait for the robot, even when a new goal location is found. The proposed planner includes a Fitts’-law-based temporal constraint, adapting it online using observed movement durations and distances.
Figure 2: Overview of the proposed sensorimotor planning framework. The planner models aspects of human motor control and task constraints. Dashed lines indicate which aspect the planner components infer or model. The temporal constraint models how humans determine movement duration from task difficulty and adapts to speed increases from robotic augmentation; the scaled stochastic model embodies human motor control principles to generate smooth robot trajectories; the final state constraint exploits the known authority handover plane position to suppress endpoint variability for accuracy augmentation. The solid line represents the physical force interaction between the impedance controller and the human body during co-manipulation.
Figure 3: End-effector velocity profiles for both sets of simulated tasks: Set A (left) and Set B (right).
Set A: fixed width W=0.100 m, varying distance
Distance (m)
Width (m)
Duration (s)
Solve time (s)
0.150
0.100
0.8500
1.6802
0.300
0.100
0.9700
2.0249
0.450
0.100
1.0400
2.6004
0.600
0.100
1.0900
3.3500
–
–
0.9875±0.0901
2.4139±0.6326
Table 1: Task statistics for movements with varying distance and width.
Figure 4: Measured movement data and RLS parameter adaptation for baseline (Condition A, red) and the proposed method (Condition B,green). Each dot represents a single trial, and each line shows the Fitts’ law prediction after an RLS update. The initial Fitts’ law relationship is indicated by the blue dotted line (a = 0.3, b = 0.35).
Figure 5: Final Fitts’ law relationships after 15 RLS updates for both conditions, showing predicted movement duration at constant task difficulty (ID = 2.47).
Figure 6: Endpoint trajectories (left) and velocity profiles (right) from Model A [ 10 ] and Model B [ 16 ] for targets (black circles) at five different distances and a constant target width. The 2D ellipses represent the predicted endpoint covariance from signal-dependent noise, not distributions from multiple trajectory samples. Vertical lines in the velocity profiles indicate durations predicted by Fitts’ law with parameters a=0.65 , b=0.12 .
Effective human-robot teaming is crucial for the practical deployment of robots in human workspaces. However, optimizing joint human-robot plans remains a challenge due to the difficulty of modeling individualized human capabilities and preferences. While prior research has leveraged the multi-cycle structure of domains like manufacturing to learn an individual's tendencies and adapt plans over repeated interactions, these techniques typically consider task-level and motion-level adaptation in isolation. Task-level methods optimize allocation and scheduling but often ignore spatial interference in close-proximity scenarios; conversely, motion-level methods focus on collision avoidance while ignoring the broader task context. This paper introduces RAPIDDS, a framework that unifies these approaches by modeling an individual's spatial behavior (motion paths) and temporal behavior (time required to complete tasks) over multiple cycles. RAPIDDS then jointly adapts task schedules and steers diffusion models of robot motions to maximize efficiency and minimize proximity accounting for these individualized models. We demonstrate the importance of this dual adaptation through an ablation study in simulation and a physical robot scenario using a 7-DOF robot arm. Finally, we present a user study (n=32) showing significant plan improvement compared to non-adaptive systems across both objective metrics, such as efficiency and proximity, and subjective measures, including fluency and user preference. See this paper's companion video at: https://youtu.be/55Q3lq1fINs.
Alex Cuellar, Michael Hagenow, Julie Shah
MIT CSAIL · UW Madison Department of Computer Science
Robot manipulation policies trained through imitation learning inherit not only the demonstrated behavior but also the conservative execution tempo of robot demonstrations. Existing acceleration approaches can execute faster than the original demonstrations, but determine the appropriate acceleration primarily from robot-side information or a predefined set of tempo factors, leaving open how to obtain a task-appropriate reference for how fast each manipulation phase should progress. We introduce EgoSpeedUp, a framework that uses human manipulation as temporal supervision for robot imitation learning. Our key insight is that human demonstrations naturally reveal task-appropriate, phase-wise manipulation tempo. Given slow robot demonstrations and human demonstrations of the same task, EgoSpeedUp aligns corresponding manipulation phases, estimates their relative execution tempos from multiple human demonstrations, and transfers the resulting phase-wise tempo by retiming the robot demonstrations. The retimed demonstrations are then used for standard behavior cloning, allowing the robot to retain its executable manipulation behavior while learning to perform it at a human-informed tempo. Across two real-world manipulation tasks, EgoSpeedUp improves the task success rate by an average of 25 percentage points (pp) while reducing successful execution time by 36.5%. These results demonstrate that human manipulation tempo provides an effective temporal reference for learning faster and more reliable robot policies.
Hanbit Oh, Yukiyasu Domae, Takuma Yagi
Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Japan
This abstract addresses the incorporation of human motion prediction into proactive and dynamic human-aware motion planning, with the goal of enabling safe collaboration between humans and robots. A deep learning, graph-based model is used to forecast human motion and is integrated into a planning framework. This framework employs a static roadmap along with a time-variant A* algorithm to modify the trajectory of a UR5e manipulator. This method greatly improves human-robot interaction and enables proactive collision avoidance by combining precise motion forecasts with adaptive trajectory planning.
Elena Basei, Edoardo Lamon, Matteo Saveriano +2
Department of Information Engineering and Computer Science, Universit`a di Trento, Trento, Italy. · Interdepartmental Robotics Labs (IDRA), University of Trento, Trento, Italy · Department of Industrial Engineering, Universit`a di Trento, Trento, Italy.