Wearable robots such as supernumerary robotic limbs (SRLs) can augment human capability, but their effectiveness depends strongly on the operator's sense of embodiment, comprising the sense of agency (SoA) and the sense of ownership (SoO). When an SRL is detached from the body so that the operator can act in two places at once, both the sense of agency and task performance tend to degrade. This paper presents the Multi-modal Embodiment Stimulation Shoulder Pad (MESSPa), a wearable interface that combines one-to-one motion transfer with spatially distributed pneumatic (balloon) haptic feedback on the shoulder, with the goal of preserving the sense of agency across attached and detached states. We describe the system design, the control mapping, and a controlled experiment using a 2×2 mixed design that crosses the control interface between groups (MESSPa, N=13, vs. keyboard control, N=12 from our previous study) and attachment (on-shoulder/detached) within participants, under single- and dual-task loads. Compared with keyboard control in the detached condition, MESSPa significantly reduced task completion time and significantly improved the sense of agency in the detached state, while narrowing the performance gap between the attached and detached conditions. The sense of ownership did not change significantly. These results indicate that combining motion transfer with positional haptic feedback is an effective way to recover agency when a robotic limb is operated remotely.
Figures & tables
Fig. 1: Illustration of MESSPa in use with the arm detached (experimental placement: Fig. 5 a). The HTC VIVE tracker on top of the shoulder pad maps the operator’s shoulder motion to the arm one-to-one, while four shoulder-mounted balloons deliver positional haptic feedback. The remote arm and the task screen are visible through the operator’s view.
Fig. 2: The MESSPa shoulder pad. Motion transfer is provided by an HTC VIVE tracker; haptic feedback is provided by four balloons (25 × 25 mm each) arranged on the shoulder with 80 mm spacing, corresponding to the four quadrants of the arm’s workspace (Section III-C ).
Fig. 3: Pneumatic system. The compressor supply is regulated to 0.05 MPa by a proportional valve; four solenoid valves driven by an Arduino MEGA inflate the balloons according to the arm’s quadrant, as commanded by Unity.
Fig. 4: Balloon test bed used for force validation, consisting of a force gauge and a balloon locking mechanism that holds the balloon against the gauge while the input pressure is varied.
Fig. 5: (a) Experimental setup, showing the on-shoulder (attached) and detached conditions without MESSPa. (b) The Unity task screen, with the timer, the main laser-pointing target (green circle), and the sub-task colour indicator.
Sense of Agency (SoA)
A1
It felt like I could control the robot arm as if it were my own arm
A2
The movements of the robot arm were caused by my actions
A3
I felt as if the movements of the robot arm were influencing my own movements
A4
I felt as if the robot arm was moving by itself
Sense of Ownership (SoO)
O1
I felt as if the robot arm was a part of my body
TABLE I: Sense of Embodiment Questionnaire (adapted from [ 27 ] ).
Fig. 6: Task completion time across the four wearing conditions, for the single and dual task, with the baseline from the previous experiment [ 15 ] . Brackets mark the three planned comparisons: ** p<0.01 , * p<0.05 (Bonferroni-adjusted), n.s. not significant.
Fig. 7: Error distance (mm) across the four wearing conditions, for the single and dual task. Brackets mark the three planned comparisons: ** p<0.01 , * p<0.05 (Bonferroni-adjusted), n.s. not significant.
Fig. 8: Sense of agency and sense of ownership in the detached condition, with and without MESSPa. Agency improves significantly with MESSPa ( p=0.017 ); ownership is unchanged.
In robot teleoperation, haptic feedback can be used to help human operators accomplish dexterous manipulation tasks. However, existing haptic feedback methods try to replicate high-fidelity sensory haptics that are felt in real world interactions, which are constrained by the sensing and feedback hardware capability and may lead to higher workload. To addresses these limitations, this work introduces semantic haptics for teleoperation, which uses abstract haptic patterns to convey critical information about robot states. We categorize robot states into "Confirmations" and "Exceptions", implement a modular haptic rendering pipeline in robot simulation, and deliver semantic haptic feedback to operators through pneumatic and vibrotactile wristbands. This simplifies hardware requirements and enables one-to-many mappings between haptic patterns and robot states. Through three evaluation studies, we identify the most effective semantic haptic design for a common pick and place teleoperation task and compare semantic haptics to other teleoperation feedback approaches including sensory haptics and visual feedback. Results suggest that while semantic haptics performs similarly as other feedback in unimanual tasks, it achieves superior performance in bimanual tasks, with reduced task workload, increased situational awareness, and overall preference.
Multimodal agents (MMAs) sustain interaction through reasoning, memory, tools, and feedback, most visibly as computer-use agents, while robotic systems (RSs) couple sensing and actuation under physical dynamics. We define multimodal embodied agents (MMEAs) as goal-directed systems that couple multimodal task reasoning with physical action and revise decisions from the resulting feedback. This raises a central question: what changes when a multimodal agent moves from computer-use to robot-use? We introduce PAPAV, a unified capability-centric framework of five recurring functions: Perceive the current state, Anticipate action effects, Plan a feasible course, Act through an interface or body, and Verify the outcome. Defined by function rather than architecture, these capabilities let PAPAV identify what all three share and then compare MMEAs with MMAs and with RSs. Five physical constraints, from partial observability to unverifiable outcomes, leave fewer choices fixed in advance and less room to reverse errors. Across 64 benchmarks, explicit evaluation centers on Act; only 5 assess Anticipate and 3 assess Verify, leaving both largely hidden behind task success. These constraints also frame the open challenges we identify. PAPAV therefore provides a common basis for designing reliable physical agents and measuring progress across all five capabilities rather than task success alone.
Assistive robotic systems face a fundamental trade-off: fully autonomous systems lack user agency, while fully user-controlled systems demand continuous cognitive effort. Existing shared autonomy approaches blend human and robot commands but are mostly deployed in separate physical bodies. We introduce co-embodiment with variable autonomy, where human and robot share a single physical body and operate at different autonomy levels across task phases, from mutual autonomy during object search and grasping to human-dominant control during actuation. We present a co-embodied, wearable robotic hand that has its own mind'' and operates with variable autonomy levels. A learning-from-demonstration visuomotor diffusion policy enables autonomous grasping when the user positions the hand near known objects. Once grasped, the system signals completion and the human can actuate the grasped tool (drill, spray bottle, infrared thermometer, lighter, and ice-cream scoop) via hands-free head gestures. The human retains veto authority at all times through a release gesture that returns the system to the initial phase. Unlike blended autonomy, where control is continuously negotiated, our co-embodied approach consists of variable autonomy from full human control to full independent actions while maintaining physical coupling, realizing a one body, two minds paradigm. In a user study with 44 participants performing five bimanual tasks, users rapidly adapted to this two minds'' paradigm: completion times improved by 23.3% across trials (p<0.001, Cohen's d=0.94), the best-performing policy variant reached a 93.6% task success rate, and acceptance ratings were high (5.70/7 overall impression, 5.52/7 daily use willingness). This work establishes co-embodiment with variable autonomy as a viable approach for assistive robotics, enabling human-robot collaboration through co-embodiment.
Piotr Koczy, Yuchong Zhang, Danica Kragic +1
KTH Royal Institute of Technology, Stockholm, Sweden. · INCAR Robotics AB, Stockholm, Sweden.