cs.ROSep 11, 2026

MR-GLi: Mixed Reality-Based Gripper-Linked Overlays for Underwater Robot Arm Teleoperation via Bilateral Control

Authors: Masashi Sasago, Masato Kobayashi, Yuki Uranishi

Organizations: Graduate School of Information Science and Technology, The University of Osaka · D3 Center, The University of Osaka · Graduate School of Maritime Sciences, Kobe University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Visual torque feedback supports underwater bilateral teleoperation, but the benefit of mixed reality (MR) over conventional monitor presentation remains unclear. We present MR-GLi, an MR interface that spatially registers a reaction torque indicator and wrist-camera image to the robot gripper. Twenty participants performed lift and pick-and-place tasks with rigid and compliant objects in a counterbalanced within-subject comparison with a 2D monitor, using identical visual-feedback content and four-channel bilateral control. MR-GLi provided gripper-linked access to visual feedback while maintaining a similar level of torque-regulation performance to the 2D monitor. Subjective evaluation further indicated reduced perceived burden associated with shifting attention between the workspace and visual feedback. These results demonstrate the feasibility of gripper-linked MR overlays for underwater bilateral teleoperation and highlight the importance of considering information access in addition to task performance. Additional material: https://mertcookimg.github.io/mr-gli/

Figures & tables

Fig. 1: MR-GLi in operation. Inside the head-mounted display, the reaction torque indicator and wrist-camera image are registered to the gripper and follow the arm as it moves (inset). The monitor condition presents the same visual information on a display beside the tank.
Fig. 2: System overview of MR-GLi and the 2D Monitor configuration.
Fig. 3: Leader and follower arms, sharing three rotational joints (J1–J3) and a two-fingered gripper (J4) driven by waterproof Dynamixel XW540-T260 actuators. An RGB camera sits at the follower wrist; only the follower is submerged.
Fig. 4: The RTI encoding, shared by both configurations. Bar length is the filling ratio of Eq. ( 3 ); hue follows Eq. ( 4 ).
Fig. 5: Tracking and calibration in MR-GLi.
Fig. 6: Experimental setup. Left: the operator sits beside the tank, so the workspace is in front of them in both conditions — seen directly under 2D Monitor and through passthrough under MR-GLi. Centre: the two configurations, labelled 2D and MR in the photographs, i.e. 2D Monitor and MR-GLi. Right: the rigid polyurethane block and compliant cellulose sponge used in trials, and the practice block.
Component2D MonitorMR-GLi
Four-channel bilateral control✓✓
RTI (identical encoding, thresholds)✓✓
Wrist-camera image of the gripper✓✓
Covarying with placement:
RTI and camera registered to the gripper–✓
Direct unmediated view of the tank✓passthrough
TABLE I: Comparison of the two display configurations.
Fig. 7: Experimental protocol under the two display conditions.
Fig. 8: The two tasks. Lift : grasp and hold the object at a marked height for five seconds. Pick&Place : grasp, carry across the tank, release into the basket. Both were performed on both objects.
 Optimal [%]Low [%]High [%]MAE [N ⋅\cdot m]SD [N ⋅\cdot m]Trial time [s] ∗
2D Monitor86.366.726.920.0560.05716.74
MR-GLi86.426.257.340.0590.05314.14
Difference, MR-GLi −- 2D Monitor
pp.546.498.475.409.409.053
rr0.140.160.170.190.190.43
TABLE II: Experimental results
Fig. 9: Subjective measures by configuration ( n=20n=20 ).
Measure2D MonitorMR-GLirrpp
SUS76.2 [21.2]77.5 [21.2]0.15.506
NASA-TLX49.5 [23.4]41.7 [22.3]0.26.261
Mental62.5 [41.2]37.5 [37.5]0.48.034
Physical42.5 [32.5]40.0 [22.5]0.06.825
Temporal22.5 [30.0]27.5 [26.2]0.04.888
Performance32.5 [50.0]30.0 [60.0]0.001.000
TABLE III: Subjective measures at participant level ( n=20n=20 ), median [IQR]. Higher is better for SUS and the two five-point items, lower for NASA-TLX and its sub-scales.
More in this topic

Robot Teleoperation

Robot teleoperation studies how people remotely control robots through interfaces, sensing, feedback, and autonomy, despite distance, delay, and embodiment gaps.

See all 207 papers

Explore similar work

CardsList

  1. A 3DGS-Driven Dynamic Viewpoint and Vibrotactile Framework for Subsea Teleoperation Validated via fNIRS

    Jul 10, 2026Fang Xu, Tianyu Zhou, Ruitong Tian +2Robot TeleoperationHaptic Feedback

    Informatics, Cobots and Intelligent Construction (ICIC) lab, Dept. of Civil and Costal Engineering, Weil Hall 360, University of Florida · RoboPI Lab, Dept. of Electrical and Computer Engineering, University of Florida · Informatics, Cobots and Intelligent Construction (ICIC) Lab, Dept. of Civil and Costal Engineering, 460F Weil Hall, University of Florida, Gainesville, FL 32611

  2. Towards Reliable Underwater Diver-Robot Interaction: Gesture Design, Interaction Logic, and Real-World Evaluation

    Sep 20, 2026Yingqi Liu, Kanzhong Yao, Zimeng Peng +4Robotic ControlGesture Recognition

    University of Science and Technology of China, Hefei, China. · Institute of Artificial Intelligence (TeleAI), China Telecom, Shanghai 200232, China.

  3. DexTwist: Dexterous Hand Retargeting for Twist Motion via Mixed Reality-based Teleoperation

    May 12, 2026Dongmyoung Lee, Chengxi Li, Dongheui LeeDexterous ManipulationContact-Rich Robotic Manipulation

    Autonomous Systems Lab, Institute of Computer Technology, TU Wien, Vienna, Austria. · Institute of Robotics and Mechatronics (DLR), German Aerospace Center, Wessling, Germany.