cs.RONov 3, 2026
SaveHaptic Shared Control for Multi-Robot Systems
Organizations: Unité de recherche : IRISA UMR6074
Abstract
Multi-agent systems offer significant advantages in terms of flexibility and redundancy, enabling the efficient distribution of complex tasks among multiple robots. However, their deployment requires advanced control strategies to ensure optimal cooperation between autonomous agents and human operators. We propose a modification of the global connectivity maintenance algorithm to incorporate measurements taken in agent's local frame, using sensors with limited fields of view. This algorithm, responsible for the autonomous behavior of the multi-robot system, is combined with different adaptive supervision strategies, allowing the user to exert control at varying scales over the entire group. (i) Single Agent Control (S): The operator directly controls a single robot, (ii) Pinning Control (P): The operator imposes constraints on multiple robots to influence the group, (iii) Centroid Control (C): The operator acts on the system's center of mass to steer the entire fleet. In conjunction with the connectivity maintenance algorithm, these methods ensure compliance with inter-agent constraints while providing haptic feedback to the user regarding the system's connectivity state. The effectiveness of these approaches was evaluated through a user study, comparing performance with and without haptic feedback across multiple scenarios tailored to the different control modes.
Figures & tables
Figure 1.1: MAS application examples. Fig. 1.1a : Simulation of a micro-robot team in a 3D endovascular environment recreated from CT scans. The robots are represented by the red and blue spheres. Fig. 1.1b : KUKA robots working together in the automotive industry.
Figure 1.2: MRS applications examples. Fig. 1.2a : Hercules robots being tested at Amazon’s BOS27 Robotics Innovation Hub ( AFP, 2022a ) . Fig. 1.2b : Stationary robot arm inside Amazon’s facility, designed to grasp and handle packages on conveyors and shelves ( Alan S. Brown, 2021a ) .
Figure 1.3: Examples of decentralization. Fig. 1.3a : a group of ants collaboratively forming a bridge to reach a distant location, studied in Reid et al., 2015a . Fig. 1.3b : Osaka’s Festival of Fire and Water, contingent collectively carrying a Mikoshi (portable shrine).
Figure 1.4: Fig. 1.4a : complete graph, Fig. 1.4b : strongly connected graph, Fig. 1.4c : connected graph, Fig. 1.4d : disconnected graph.
Figure 1.5: Graphs with different connectivity eigenvalues
Figure 1.6: Drone race opposing human pilots and an autonomous drone, presented in Kaufmann et al., 2023a . The autonomous drone was trained using a trajectory dataset captured via global sensing techniques (laser tracker) and dynamically adapts its flight-path mid-race by detecting racing gates with local sensing techniques (onboard camera).
Figure 1.7: UVDAR and UVDAR-COM systems. Fig. 1.7a : Configuration of the UVDAR system, featuring UV LEDs circled in green and cameras circled in red. Fig. 1.7b : UVDAR-COM system workflow, including time tracking of UV markers using 4D Hough Transformation, frame reconstruction, error detection, and data decoding.
Figure 2.1: Fig. 2.1a : Mechanical teleoperation system developed by Goertz in 1954. The system allowed operators to manipulate radioactive materials from a safe location. Fig. 2.1b : PHANToM device, 6 Degreed of Freedom haptic device developed by Sensable Technologies.
Figure 2.2: Fig. 2.2a : dense point cloud reconstructions generated by ORB-SLAM2, utilizing estimated keyframe poses and sensor depth maps, presented in Mur-Artal & Tardós, 2017a . Fig 2.2b : map and pose graph from a Swarm-SLAM experiment involving three robots, detailed in Lajoie & Beltrame, 2023a .
Figure 2.3: Fig. 2.3a : The Revolute-Revolute-Spherical fingertip device presented in Chinello et al., 2017a . Fig 2.3b : An electrode, placed in contact with the finger pad, is controlled by an electrical stimulator attached to the forearm ( Vizcay et al., 2021a )
Figure 2.4: Virtuose 6D haptic device, Haption.
Figure 2.5: Formation control of a group of robots maintaining a desired shape while tracking a moving target and avoiding obstacles, see Dang et al., 2019a .
Figure 2.6: Teleoperation setup. The human operator uses a Virtuose 6D grounded haptic interface (Haption, France) to control a 7 degree of freedom Panda robotic manipulator (Franka Emika, Germany), used as a 6-DoF manipulator. The user receives haptic feedback through the haptic interface and visual feedback from an LCD screen ( Rahal et al., 2023a ) .
Figure 2.7: The shape of with and .
Figure 3.1: Onboard sensors: The green areas in the figures represent the sensing directions of the respective sensors.
Figure 4.1: The shape of the two considered rising/falling activation functions.
Figure 4.2: Relative orientation estimation. Fig. 4.2a Robot senses robot and measures its velocity. Fig. 4.2b The angle between and on the -plane is the relative orientation .
Table 4.1: Algorithm parameters used in our validation.
Figure 4.3: Experimental setup for simulation with Connectivity Maintenance.
Figure 4.4: Drones trajectories during the experiment. Drone 1 is the drone controlled by the human user. Each dot in the trajectory corresponds to the position of the drone at a 5 seconds interval.
Figure 4.5: Validation in simulation. (Top) True vs. the six estimations calculated by the robots over time; the green phases indicate when the human user is providing an external input to the robot leader (robot ). (Bottom) Number of edges in the graph, which changes dynamically as the task evolves.
Figure 4.6: Estimations errors for and . Blue phases indicate when robot is measuring robot ( , see Sect. 4.1.1 ) but robot does not measure robot ( ); yellow phases indicate when robot is measuring robot but robot does not measure robot ; green phases indicate when both robot and are measuring each other. Note how the estimation errors always remain very small apart from the white phase (bottom plot) since, in this case, the two robots are completely disconnected. The error however quickly converges back to zero as soon as one robot starts measuring the other one (vertical red dashed line).
Figure 5.1: The Virtuose device is a high-end 6-DoF haptic interface able to provide 3-DoF translational and 3-DoF rotational feedback.
Figure 5.2: The shape of on the axis with , , , , , and .
Figure 5.3: Pinning control. and are the position of the virtual point and the agent , respectively. is the minimum distance between agent and the virtual point, is the user rotational input and is the velocity control input controlling agent .
Figure 5.4: The projection of the linear elements of the connectivity maintenance force onto the vector and the tangent plan containing .
Figure 5.5: Software implementation. For each simulated drone, one instance of the control and simulation nodes is executed.
Figure 5.6: “Plains” environment. Unity 3D visualization of the robots when using the Single Agent Control method. The leader robot is highlighted.
Figure 5.7: “Desert” environment. Unity 3D visualization of the robots when using the Pinning Control method. The pinned robots are highlighted.
Figure 5.8: Unity 3D visualization of the robots when using the Centroid Control method. The centroid of the group is depicted by the red arrow.
| Control conditions | ||||
| Single Agent | Pinning | Centroid | ||
| without Haptic Feedback | SNH | PNH | CNH | |
| Feedback conditions | with Haptic Feedback | SH | PH | PNH |
Table 5.1: Name of the different control and feedback conditions.
Figure 5.9: Experimental setup for the user study. The environment is displayed on a large screen. The user controls the agents using the Virtuose haptic interface.
Figure 5.10: Experimental results. (a) User perceived effectiveness and intuitiveness, and (b) Measured completion time for the Single Agent Control scenario.
Figure 5.11: User Perceived Effectiveness and Intuitiveness, and Measured Completion Time for Pinning Control Experiment.
Figure 5.12: User Perceived Effectiveness and Intuitiveness, and Proportion of Discovered Environment for Centroid Control Experiment.
Figure 5.13: Agents trajectories throughout the experiment. Each dot in the trajectory corresponds to the position of an agent at a 5 seconds interval.
Figure 5.15: The shape of throughout the experiment and the number of connected agents.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1: ROS 2 nodes running onboard each agent under the same namespace (drone ) and their input/output topics.
Figure A.2: Software implementation for the haptic interface.
Figure A.3: State machine of the centralized_controller_node .
More in this topic
Robotic Control
Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.
See all 606 papers