Real-Time Human-Adaptive Task Allocation for Multi-Human Multi-Robot Supervision
Authors: Seabin Lee, Sujeong Park, Nayoung Kim, Sungjin Park, Haechan Jung, Changjoo Nam
Organizations: Sogang University, Seoul, Korea. 1Dept. of Electronic Engineering. · Sogang University, Seoul, Korea. 2Dept. of Artificial Intelligence.
We propose a human-factor-aware method of allocating robot supervision tasks to multiple human operators. In scenarios where multiple operators occasionally teleoperate multiple robots to help the robots overcome difficulties, the allocation of the supervisory control tasks to humans needs to consider the real-time cognitive states of individual operators. However, most existing methods assume fixed supervisory capacity per operator and overlook fluctuations in the human factors such as workload and fatigue. As a result, workload distribution can be unbalanced where some operators become overloaded while the others remain underused. Our method dynamically regulates supervisory capacity and allocates tasks in a way that maintains balanced mental workload, prevents overload, and improves overall team performance. The allocation method uses a greedy strategy that minimizes estimated operator workloads with task prioritization. Robots are assigned to operators by reflecting their current supervisory capacity where the required effort depends on the types of tasks. In the user study, the analysis across predefined time intervals shows that the proposed method consistently achieves higher performance and lower behavioral signs of fatigue compared to a baseline method that does not consider human factors. These results highlight adaptive capacity adjustment as an effective preventive mechanism for sustaining operator performance in long-duration, high-demand settings.
Figures & tables
Figure 1 : Overview of HAMA. Inputs include NASA-TLX workload, subjective fatigue, blink count, success count, and AOI dwell time. The system updates operator capacity by predefined time intervals and assigns robots online based on AOI dwell time. Operators perform teleoperation for robots that are in deadlock or help-signal situations. The diagram shows three operators and the flow from sensing to assignment.
Figure 2 : Overview of the experimental setup. (a) Full view of the simulated warehouse environment. Multiple robots navigate passages to perform tasks such as deadlock resolution and charging station docking. (b) Example of a deadlock situation involving multiple robots in a narrow passage. (c) Docking zone with charging indicators. Robots must request help to align correctly for docking. Keyboard control scheme is shown at the bottom-left, and the operator display at the bottom-center.
Figure 3 : Example of a deadlock cluster in a narrow warehouse passage. Green check marks indicate robots identified as boundary robots, based on convex hull location and directional clearance. Robots marked with the red alert icon are internal to the deadlock cluster and cannot move until boundary robots are cleared.
Figure 4 : Study timeline. After briefing, 2-back tests, and eye-tracker calibration, teams completed two allocation conditions in counterbalanced order. Each condition consisted of 10 repetitions of a 4-minute supervision task followed by a brief survey. A scheduled break separated the two blocks. Total time per team was about 90 min ( ≈ 80 min task time).
Figure 5 : View of the VR control interface as seen by an operator. Multiple robot camera feeds are shown. AOI dwell time and gaze rays were only visible during debugging, not in the actual experiment.
Condition
HAMA
Baseline
Capacity update
✓
×
Attention-based weighting
✓
×
Boundary-based reallocation
✓
✓
Table I : Allocation features in HAMA and baseline conditions
Metric
Method
Session
3
4
5
6
7
8
9
10
Overall
NASA-TLX ↓
HAMA
29.03 ± 7.65
29.73 ± 7.73
30.43 ± 8.85
30.80 ± 9.54
32.30 ± 8.78
31.60 ± 9.03
31.57 ± 9.54
32.00 ± 9.31
30.93 ± 8.77
Baseline
30.03 ± 7.96
30.93 ± 8.10
30.53 ± 8.52
30.77 ± 8.89
31.17 ± 8.96
31.03 ± 9.60
30.87 ± 9.80
31.87 ± 10.16
30.90 ± 8.91
Cognitive Fatigue ↓
HAMA
4.07 ± 2.02
4.57 ± 2.03
4.73 ± 2.16
5.17 ± 2.38
5.67 ± 2.50
5.67 ± 2.47
5.73 ± 2.43
5.57 ± 2.33
5.15 ± 2.34
Baseline
4.53 ± 2.18
4.90 ± 2.20
5.20 ± 2.44
5.53 ± 2.53
5.90 ± 2.59
5.93 ± 2.60
6.10 ± 2.38
6.23 ± 2.51
5.54 ± 2.47
Blink Count ↓
HAMA
44.47 ± 42.33
49.30 ± 46.65
53.90 ± 47.63
58.47 ± 56.60
58.90 ± 58.75
62.47 ± 53.67
58.80 ± 56.87
61.03 ± 53.62
55.92 ± 51.86
Table II : Session-wise mean ± SD (Sessions 3–10). Higher is better for Performance ( ↑ ), and lower is better for the other measures ( ↓ ).
Metric
Effect
F
ηp2
post - hoc
NASA-TLX
Session
3.342 ∗
0.504
3<7,10
Algorithm
0.001
0
-
Cognitive Fatigue
Session
8.182 ∗∗∗
0.713
4,5<6 3<4,5<7,8,9,10
Algorithm
1.546
0.051
-
Blink Count
Session
1.558
0.322
-
Algorithm
6.077 ∗
0.173
-
Table III : Results of two-way RM-ANOVA in experiment.
Figure 6 : Session-wise trends (left) and overall mean with SD (right) for NASA-TLX, Cognitive Fatigue, Blink Count, Performance, and Capacity. Panels are stacked from top to bottom. Asterisks mark a significant main effect of the algorithm for Blink Count and Performance ( p<0.05 ).
Figure 7 : Plot of changes in the average relative difficulty ratio by session.
Smart Manufacturing Thrust, System Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · College of Future Technology, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China