HECTOR: Human-centric Hierarchical Coordination and Supervision of Robotic Fleets under Continual Temporal Tasks

Shen Wang; Yinhang Luo; Jie Li; Meng Guo

cs.ROApr 13, 2026

HECTOR: Human-centric Hierarchical Coordination and Supervision of Robotic Fleets under Continual Temporal Tasks

Authors: Shen Wang, Yinhang Luo, Jie Li, Meng Guo

Organizations: the School of Advanced Manufacturing and Robotics, Peking University, Beijing 100871, China · National University of Defense Technology, Hunan 410073, China

Abstract

Robotic fleets can be extremely efficient when working concurrently and collaboratively, e.g., for delivery, surveillance, search and rescue. However, it can be demanding or even impractical for an operator to directly control each robot. Thus, autonomy of the fleet and its online interaction with the operator are both essential, particularly in dynamic and partially unknown environments. The operator might need to add new tasks, cancel some tasks, change priorities and modify planning results. How to design the procedure for these interactions and efficient algorithms to fulfill these needs have been mostly neglected in the related literature. Thus, this work proposes a human-centric coordination and supervision scheme (HECTOR) for large-scale robotic fleets under continual and uncertain temporal tasks. It consists of three hierarchical layers: (I) the bidirectional and multimodal protocol of online human-fleet interaction, where the operator interacts with and supervises the whole fleet; (II) the rolling assignment of currently-known tasks to teams within a certain horizon, and (III) the dynamic coordination within a team given the detected subtasks during online execution. The overall mission can be as general as temporal logic formulas over collaborative actions. Such hierarchical structure allows human interaction and supervision at different granularities and triggering conditions, to both improve computational efficiency and reduce human effort. Extensive human-in-the-loop simulations are performed over heterogeneous fleets under various temporal tasks and environmental uncertainties.

Figures & tables

Fig. 1: The considered scenario where a human operator coordinates, interacts, and supervises a large robotic fleet via the proposed framework. Online requests are specified via the designed interface and protocol ( bottom-left ), such as adding or cancelling new temporal tasks, different priorities and deadlines, or direct assignment of certain robots. The fleet reacts autonomously to fulfill different tasks at different regions, both at the global fleet-level ( top ) and the local team-level ( bottom-right ). In the simulated case, $30$ tasks and $450$ subtasks are accomplished by $80$ robots.

Fig. 2: The proposed human-centric framework for hierarchical coordination and supervision of robotic fleets, consisting of three main components: (I) the interaction protocol and interface for four types of online requests ( left ); (II) the automata-guided task assignment and team formation ( middle ); (III) three types of local coordination strategy for different tasks ( right ). Human requests, environment state and execution status are all updated online, for which adaptations for tasks and motions are triggered at different granularities and conditions.

Method	Task Specification	Online Task	Uncertain/ Unknown Subtasks	Human Interaction	Dynamic Constraints	Local Coordination
STyLuS* [ 5 ]	LTL	$\times$	$\times$	$\times$	$\surd$	$\times$
ScRATCHeS [ 21 ]	CaTL	$\times$	$\times$	$\times$	$\surd$	$\times$
DecTree [ 22 ]	LTL	$\surd$	$\surd$	$\times$	$\times$	$\times$
cLTL $+$ [ 26 ]	cLTL	$\times$	$\times$	$\times$	$\surd$	$\times$
HieraTL [ 33 ]	hLTL	$\surd$	$\surd$	$\times$	$\times$	$\times$
Flow-based [ 34 ]	Ordered	$\times$	$\surd$	$\times$	$\times$	$\times$

TABLE I: Comparison with Related Work

Fig. 3: The proposed protocols for human-fleet interaction: the human command and intervention module ( left ); the task decomposition and planning modules ( right ); and the visualization module ( middle ).

Fig. 4: Integrated interface for human-fleet interaction, which includes scenario visualization ( top-left ), temporal ordering of subtasks ( top-middle ), panel to specify operator requests ( top-right ), local plans and execution progress for teams and robots are visualized online ( bottom ).

Fig. 5: Illustration of the automaton-guided search tree, where the node expands over not only the local plans of each team

\{\Gamma_{k},\,k\in\mathcal{K}\}

, but also the progress of each mission

\{\widehat{Q}_{m},\,m\in\mathcal{M}\}

. The leaf nodes of complete assignments may correspond to different number of teams.

Fig. 6: Illustration of the capacity-based and redundancy-aware team formation. Given the task assignment $\nu_{K^{\star}}^{\star}$ , the team formation $\{\mathcal{N}_{1},\cdots,\mathcal{N}_{K^{\star}}\}$ are the determined optimally under the redundancy margin in ( 18 ).

Fig. 7: Local coordination results for the static and known local tasks described in Sec. V-C1 . In total $14$ delivery subtasks (filled circles) are assigned to $5$ robots (filled pentagons), of which the trajectories are shown for fully-actuated ( left ) and non-holonomic ( right ) robots.

Fig. 8: Execution results for the case of static and unknown subtasks in Sec. V-C2 , e.g., the “search and rescue” task. Left : The initial trajectories for coverage by $5$ robots with unknown targets (in circles); Right : The actual trajectories after discovering, assigning and executing $8$ rescue subtasks.

Fig. 9: Execution results of the dynamic and known subtasks in Sec. V-C3 . Top : the trajectories of $5$ robots (filled hexagons) and $5$ targets (filled circles) at different snapshots; Bottom : the evolution of assigned coalitions for each target. Note that different colors reflect different capacity constraints.

Fig. 10: Illustration of the change of execution status between navigation, execution and synchronization ( left ), for a team $\mathcal{C}_{k}$ given its local plan $\Gamma_{k}$ ( right ) in ( 17 ) and ( 19 ). Note that the concurrency constraint between tasks $\omega_{2},\omega_{5}$ are enforced by the synchronization state (red line).

Fig. 11: Illustration of the receding-horizon scheme for coordination and online adaptation in Sec. V-D . Note that the number of teams is increased from $3$ ( left ) to $5$ after the task assignment and team formation ( right ), and the tasks $\omega_{3},\omega_{4}$ are not preempted after replanning.

Fig. 12: Simulated human-in-the-loop scenario of $80$ robots in dynamic environment. Top : in total $32$ tasks and $482$ subtasks are performed during the mission time of $180.4\,\mathrm{s}$ , with the time elapsed from left to right. Clearly $\kappa_{1}$ to $\kappa_{4}$ are issued in the process; Bottom-left : the snapshots of execution of tasks at $t=15\,\mathrm{s}$ , where different markers indicate different types of robots; Bottom-right : gantt charts of teams executing assigned tasks at $t=0\,\mathrm{s}$ , $t=15\,\mathrm{s}$ and $t=130\,\mathrm{s}$ . Note that tasks $\omega_{1}-\omega_{9}$ are released at $t=0\,\mathrm{s}$ , $\omega_{10}-\omega_{14}$ at $t=30\,\mathrm{s}$ , $\omega_{15}-\omega_{19}$ at $t=60\,\mathrm{s}$ , $\omega_{20}-\omega_{26}$ at $t=100\,\mathrm{s}$ , $\omega_{27}-\omega_{30}$ at $t=130\,\mathrm{s}$ , and $\omega_{31},\omega_{32}$ are issued by operator at $t=110\,\mathrm{s}$ .

Fig. 13: Final trajectories of local task executions for robots under the dynamic constraints of maximum curvature $\kappa\leq 3$ . Top : delivery task $\omega_{3}$ at $t=22\,\mathrm{s}$ for a team of $8$ robots and $18$ subtasks; Middle : surveillance task $\omega_{7}$ at $t=25\,\mathrm{s}$ for a team of $20$ robots and $16$ subtasks; Bottom : dynamic capture task $\omega_{5}$ at $t=13\,\mathrm{s}$ for a team of $22$ robots and $18$ subtasks.

Fig. 14: Evolution of enrolled robots and tasks accomplished, as new missions are specified by

\varphi_{0}-\varphi_{10}

. Top : the number of enrolled robots with different status, including navigation, waiting, and task execution; Bottom : number of remaining tasks that are assigned and unassigned.

Fig. 15: Sankey diagram of all robots that participated in the overall mission, along with the number of teams and their compositions. Note that subteams are labeled $N_{0}-N_{49}$ and robots are labeled $A_{0}-A_{79}$ .

Method	Resp. Time [s]	Ave. Resp. [s]	Ave/Max Plan [s]	N/W/E Robots	Succ. Rate [%]
Ours	184.4	34.6	0.46/0.76	19/3/27	100
MILP	327.8	99.1	90.3/144.0	23/7/18	100
SAMP-Task	286.0	72.6	0.45/0.74	24/7/15	100
SAMP-Subtask	230.1	45.6	3.8/8.0	16/5/30	86
ScRATCHeS	257.4	64.9	1.25/7.60	13/3/19	100
Hulk(+poset)	194.4	40.5	4.66/7.59	24/8/24	100

TABLE II: Comparison with Baselines

(N{=}80,M{=}30,J{=450})

Fig. 16: Snapshots of baselines Flow and ScRATCHeS. Left : the subtasks in the red circle are unfinished after $t=32\mathrm{s}$ ; Right : robots marked in red have long navigation distance due to inefficient assignments.

$\mathbf{(N,M,J)}$	Failure $\rho$	Resp. Time [s]	Ave/Max Plan [s]	N/W/E Robots	Succ. Rate [%]
(120, 50, 750)	0.05	378.2	0.46/0.82	26/5/27	100
(120, 50, 750)	0.10	346.6	0.51/0.79	29/5/25	100
(150, 80, 1202)	0.05	699.1	0.53/0.88	29/3/31	100
(150, 80, 1202)	0.10	788.3	0.55/0.90	26/7/36	100
(170, 100, 1509)	0.05	923.1	0.65/1.48	30/6/30	100
(170, 100, 1509)	0.10	1120	0.87/1.65	31/3/29	100

TABLE III: Scalability and Robustness Analysis

Fig. 17: Computational time analysis across the five core modules with varying numbers of robots

N

and tasks

M

Fig. 18: Parameter sensitivity and performance analysis of the HECTOR framework: parameter horizon $H$ in assignment ( top-left ), redundancy margin $\alpha$ in team formation ( top-right ), the number of finished task in current horizon to trigger the replan ( bottom-left ), and the curvature $\kappa$ of robots to plan the trajectories in task coordination ( bottom-right ).

Fig. 19: Illustration of the distributed dynamic coalition formation under different communication delays. Top : the ideal scenario without communication delay; Bottom : scenario with stochastic delays ranging from $10^{1}$ to $10^{2}\,\mathrm{ms}$ .

Fig. 20: High-fidelity ROS simulation of the proposed framework under the scene for disaster relief. Top-left : 2 UAVs assigned to static known tasks (green), 4 robots to dynamic known tasks (purple), and 6 robots to static unknown tasks (yellow); Middle : the task execution trajectories for 10 tasks over different regions and $36$ subtasks. Bottom-right : the execution timeline of each robot completing assigned tasks in 34.6 $\,\mathrm{s}$ on average.

Term	Definition	Reference
$\Phi_{t}$	Set of missions known at time $t>0$ .	Sec. V-B
$\tau_{i}$	Local plan of robot $i\in\mathcal{N}$ .	Sec. IV-A
$\mathcal{K}=\{\kappa_{1,2,3,4}\}$	Operator requests.	Sec. IV-B
$\mathcal{B}_{\varphi_{m}}$	Büchi automaton for mission $\varphi_{m}$ .	Sec. V-B
$\widehat{Q}_{m}$	Set of reachable states in automaton $\mathcal{B}_{\varphi_{m}}$ .	Sec. V-B
$\mathfrak{T}$	Search tree structure defined as $(\mathcal{V},\rightarrow)$ .	Sec. V-B

TABLE IV: Nomenclature of Key Variables and Definitions

Multi-Robot Coordination

Multi-robot coordination studies how teams of robots plan, communicate, allocate tasks, and act together safely and efficiently.

See all 326 papers

HULK: Large-scale Hierarchical Coordination under Continual and Uncertain Temporal Tasks
May 9, 2026Qingyuan Luo, Jie Li, Meng GuoMulti-Agent Coordination Hierarchical Planning
the Department of Mechanics and Engineering Science, College of Engineering, Peking University, Beijing 100871, China · National University of Defense Technology, Hunan 410073, China
Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision
Oct 1, 2026Seabin Lee, Sujeong Park, Nayoung Kim +3Human-Robot Collaboration Multi-Robot Task Allocation
Sogang University, Seoul, Korea. 1Dept. of Electronic Engineering. · Sogang University, Seoul, Korea. 2Dept. of Artificial Intelligence.
Melding LLM and temporal logic for reliable human-swarm collaboration in complex scenarios
May 8, 2026Junfeng Chen, Yuxiao Zhu, An Zhuo +6Swarm Robotics Large Language Model-Based Robot Planning
School of Advanced Manufacturing and Robotics, Peking University, Beĳing & 100871, China. · 2Division of Natural and Applied Sciences, Duke Kunshan University, Suzhou & 215316, China. · School of Automaton, Southeast University, Nanjing & 210096, China. +1

May 9, 2026J/K move · Enter open · S save

HULK: Large-scale Hierarchical Coordination under Continual and Uncertain Temporal Tasks

Qingyuan Luo, Jie Li, Meng Guo

the Department of Mechanics and Engineering Science, College of Engineering, Peking University, Beijing 100871, China · National University of Defense Technology, Hunan 410073, China

Multi-Agent Coordination Hierarchical Planning Multi-Robot Task Allocation

Open paper AskPDF arXiv

Multi-agent systems can be extremely efficient when working concurrently and collaboratively, e.g., for delivery, surveillance, search and rescue. Coordination of such teams often involves two aspects: selecting appropriate subteams for different tasks in various areas, and coordinating agents in the subteams to execute the associated subtasks. Existing work often assumes that the tasks are static and known beforehand, where an integer program can be formulated and solved offline. However, in many applications, the team-wise tasks are generated online continually by external requests, and the amount of subtasks within each task is uncertain, e.g., the number of packages to deliver or victims to rescue. The aforementioned offline solution becomes inadequate as it would require constant re-computation for the whole team and global communication to broadcast the results. Thus, this work tackles the large-scale coordination problem under continual and uncertain temporal tasks, specified as temporal logic formulas over collaborative actions. The proposed hierarchical framework, HULK, consists of two interleaved layers: the rolling assignment of currently known tasks to subteams within a certain horizon, and the dynamic coordination within a subteam given the detected subtasks during online execution. Thus, coordination is performed hierarchically at different granularities and triggering conditions, improving computational efficiency and robustness. The method is validated rigorously over large-scale heterogeneous systems under various temporal tasks and environment uncertainties.

Abstract

Multi-Robot Coordination

Explore similar work

HULK: Large-scale Hierarchical Coordination under Continual and Uncertain Temporal Tasks

Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision

Melding LLM and temporal logic for reliable human-swarm collaboration in complex scenarios