SUBTA: A Framework for Supported User-Guided Bimanual Teleoperation in Structured Assembly
Authors: Xiao Liu, Prakash Baskaran, Songpo Li, Simon Manschitz, Wei Ma, Dirk Ruiken, Soshi Iba
Organizations: Authors are with Honda Research Institute USA, San Jose, CA 95134 USA. · Authors are with Honda Research Institute Europe GmbH, 63073 Offenbach am Main, Germany.
In human-robot collaboration, shared autonomy enhances human performance through precise, intuitive support. Effective robotic assistance requires accurately inferring human intentions and understanding task structures to determine optimal support timing and methods. In this paper, we present SUBTA, a supported teleoperation system for bimanual assembly that couples learned intention estimation, scene-graph task planning, and context-dependent motion assists. We validate our approach through a user study (N=12) comparing standard teleoperation, motion-support only, and SUBTA. Linear mixed-effects analysis revealed that SUBTA significantly outperformed standard teleoperation in position accuracy (p<0.001, d=1.18) and orientation accuracy (p<0.001, d=1.75), while reducing mental demand (p=0.002, d=1.34). Post-experiment ratings indicate clearer, more trustworthy visual feedback and predictable interventions in SUBTA. The results demonstrate that SUBTA greatly improves both effectiveness and user experience in teleoperation.
Figures & tables
Fig. 1: SUBTA : The proposed Supported Teleoperation framework assists users in assembly tasks by (i) estimating the task and generating a plan, and (ii) providing real-time support through visual feedback and motion-level corrections.
Fig. 2: Supported teleoperation system : The user teleoperates the robot to assemble the “horse” structure. The top row shows the scene graph encoding spatial relationships for task monitoring and planning. The middle row depicts the robot successfully executing the block assembly. The bottom row presents the digital twin environment, where task estimation and planning ((highlighted in red) are visualized to guide the user.
Fig. 3: Task Planner : the planner computes the most likely target pose for the next block based on the graph edit distance (GED) and adds the corresponding node to the graph.
TABLE II: Context-dependent behaviours for pick–place motion support.
Fig. 4: Eight Block assembly tasks.
Model
Action (L)
Action (R)
Task
Naive CNN
73.53 (6.40)
72.34 (5.61)
77.58 (4.38)
Hierarchical InEs
43.97 (10.97)
38.93 (8.90)
49.43 (8.62)
HAR-Transformer
81.49 (4.55)
79.95 (4.39)
90.12 (3.71)
Ours
81.73 (4.47)
80.48 (4.84)
89.82 (3.75)
TABLE III: Comparison of F1-scores (% mean ± std. dev.)
Fig. 5: Examples of participants performing three different assembly tasks using the SUBTA system during the user study. The supported workspace is projected on the left with the user included, while the actual robot workspace is shown on the right.
Hypothesis
M3 vs. M2
M2 vs. M1
M3 vs. M1
Supported
H1a : Pos. err.
**
**
Yes
H1b : Ori. err.
**
**
**
Yes
H3 : Mental
NS
**
**
Yes
H4 : SUS
NS
Yes
NS (not significant); ** p<0.01 ; * 0.01<p<0.05 .
TABLE IV: Linear mixed-effects model results across the three modes
Task
Mode
Time (sec)
Success Rate
Progress to Complete
Orientation Error (deg)
Position Error (meter)
“Tuningfork-ly”
M1
74.56 ± –
11.1%
38.9% ± 25.3%
3.710 ± –
0.022 ± –
M2
71.02 ± –
12.5%
43.8% ± 29.1%
2.540 ± –
0.013 ± –
M3
87.05 ± 38.26
22.2%
50.0% ± 33.1%
0.920 ± 0.382
0.011 ± 0.006
“Arch”
M1
64.14 ± 13.51
62.5%
74.9% ± 34.7%
5.288 ± 3.310
0.028 ± 0.018
M2
71.32 ± 15.13
85.7%
95.1% ± 12.9%
2.568 ± 2.032
0.013 ± 0.004
M3
57.86 ± 7.17
100.0%
100.0% ± 0.0%
2.104 ± 2.224
0.007 ± 0.004
TABLE V: Per-task and overall summaries (mean ± std) by mode. Best value per task is bolded.
Fig. 6: NASA-TLX ratings for mental demand, physical demand, and frustration, along with SUS scores across the three modes.
(1) Predicted target block visualization reduced ambiguity (2) Predicted target block visualization increased my confidence (3) Visualization was accurate and trustworthy (4) Visualization was uncluttered
Motion
(3) Hand snapping made grasping easier (4) Object snapping made placement easier (5) Snapping increased my confidence (6) Interventions were predictable and transparent
Same as M2 (items 3–6)
Agency
(7) I always felt in control (8) Easy to override/disengage snapping
Same as M2 (items 7–8)
TABLE VI: Post-experiment questionnaire items for M2 and M3 on a 10-point Likert scale (1 = strongly disagree, 10 = strongly agree).
Fig. 7: Full distributions with mean, median, and 95% confidence intervals for the post-experiment comparison of M2 and M3.
Department of Data and Systems Engineering, The University of Hong Kong, Hong Kong SAR, China. · Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China.