Temporal Tactile Encoding and Compliance for Intent-Aware Robot-to-Human Bimanual Handover
Authors: Pasquale Marra, Stefano Berti, Gabriele Mario Caddeo, Lorenzo Natale
Organizations: Humanoid Sensing and Perception, Istituto Italiano di Tecnologia, Genoa, Italy. · DIBRIS, Universit`a di Genova, Via All’Opera Pia, 13, Genoa, Italy.
Reliable robot-to-human handover requires the robot to infer when the person is ready to receive the object and release it safely, comfortably, and at the right time. This is challenging because visual observations alone may not disambiguate clear taking intent from accidental contact, weak grasping, wrong-direction forces, or transient interactions. In this work we treat human-robot handover as an intrinsically multimodal problem. Our approach couples a VLA model with a compliance controller that reduces interaction forces during object transfer. We finetune the VLA model with human demonstrations using RGB observation, temporally encoded tactile feedback and proprioception. We evaluate the complete system in a human-subject study against two baselines: one without tactile feedback and one using tactile feedback without compliance control. We hypothesize that combining compliance and temporal tactile encoding yields the most reliable and comfortable handovers, as compliance facilitates physical interaction while tactile history captures sustained taking intent. Performance is measured through objective metrics and an ad-hoc questionnaire. The results show that the two components provide complementary benefits and substantially outperform the baselines. Project page: https://hsp-iit.github.io/temporal-tactile-bimanual-handover/.
Figures & tables
Fig. 2: System overview. The robot’s camera, fingertip forces F (estimated from the raw Xela taxels), and joint states feed either the teleoperation interface (Meta Quest 3, during demonstration collection) or the learned policy (GR00T with the temporal tactile encoder TTE, at evaluation time). Both produce the same references: a desired head orientation Rh , tracked by the head controller ( qhref ), and Cartesian hand poses Pr , Pl . The compliance module converts the hand references into compliant references Pr,c , Pl,c based on the measured fingertip forces (Eq. 4 ); the torso-and-arms controller then computes the torso and arm joint references qtref , qrref , qlref sent to the robot.
Fig. 3: Challenging Test Set, episode 6 (repeated shakes and taps). Top: recorded pull force and hand distance. Bottom: the hand opening predicted by each policy in the final action of the 16-action chunk, with the 0.35m release threshold shown as a dotted line. The shaded region marks the final pull that triggers release. A1 incorrectly predicts release after non-causal pulls, whereas the TTE-based A2 detects it only during the final genuine pull.
Fig. 4: Full-system behavior (P1) in hold-expected trial T5. After grasping the box (first frame), the participant applies two brief pulls peaking at 3.8N , equal to the final genuine pull, but the robot keeps its hands closed (second and third frames). Sustaining the same force for ∼1.5s , rather than ∼0.2s , triggers release (fourth and final frames). The force and hand-distance plots show narrow transient peaks followed by a sustained pull, after which the hand distance increases. Thus, release depends on sustained pulling evidence rather than force magnitude alone. Faces are mosaicked for privacy.
Fig. 5: Success rate — participant got the expected outcome (release on R trials, hold on H trials) — for the three policies, broken down by all 10 trial types.
Metric
P1
P2
P3
compliance+TTE
compliance
TTE
Objective – 300 scored trials
Success [%] ↑
93
54
82
Success, Tr=2.8 s [%] ↑
92
45
80
Release delay, median [s] ↓
1.70
2.35
2.00
Peak pull force [N] ↓
4.07
4.67
4.74
TABLE I: Objective and subjective metrics, 10 participants × 10 trials per policy.
Fig. 6: Left: Post-block questionnaire ratings (all 10 items, mean ± s.d. over 10 participants). The row titles summarize the meaning of the questions. Right: the overall best-to-worst ranking, represented as #(rank position):(number of occurrences).
Robot-to-human object handover is an essential skill for robot assistants, from serving drinks at home to passing surgical tools in the operating room. We expect robots to perform handover robustly -- to release the object only after a firm human grasp while ignoring incidental touches. Existing passive-sensing methods struggle to generalize across diverse objects and human behaviors, as they lack informative perturbations to disambiguate different contact conditions, such as firm grasp versus incidental touch. We propose an active sensing approach for robust handovers: the robot applies information-gathering motions and senses the resulting human-applied forces to infer the contact state. A firm grasp produces forces in multiple directions, while an accidental touch does not. To capture this distinction, we model the contact state with a Bayesian linear model: a distribution over piecewise-linear mappings from robot motions to human-applied forces. This model enables firm grasp detection and active information gathering. In experiments with 12 participants and 30 diverse rigid objects, our method achieved a 97.5% success rate -- over 30% higher than two common baselines.
We present R2HandoverSim, a simulation benchmark for robot-to-human (R2H) object handovers. Although R2H handover methods have advanced rapidly, the lack of standardized evaluation protocols impedes objective comparison. Our benchmark enables reproducible evaluation by systematically comparing four baselines on their predicted shared grasp poses. We conduct a user study with 30 participants, analyze baseline performance, and show that simulation results correlate with real-world evaluation outcomes. Crucially, five complementary metrics (planning feasibility, reachability, grasp stability, grasp affordance, and safety) better reflect user-perceived handover quality than overall success rate alone. Website and code: https://robot-future.github.io/r2handoversim/.
Hanxin Zhang, Abdulqader Dhafer, Hongbiao Dong +1
DANiLab, University of Leicester, Leicester, UK · School of Computing and Mathematical Sciences, University of Leicester, Leicester, UK · School of Metallurgy and Materials, University of Birmingham, Birmingham, UK
Collaborative robots are increasingly sharing workspaces with human operators, making tool handover a frequent and safety-critical micro-interaction. However, traditional static handovers often lead to awkward grasps when handling asymmetric industrial tools. This paper presents a receiver-centered voice-driven adaptive handover system for mechanical tools, built on a Franka cobot. Using an LLM for intention recognition and MediaPipe for real-time 3D hand tracking, the framework dynamically adjusts the end-effector's orientation to present tools in an ergonomically optimal, handle-first pose. A within-subjects study compared this adaptive approach with an object-agnostic static baseline. The results demonstrate that the adaptive system reduces the grasp delay for asymmetric tools, improving the fluency of the interaction. Furthermore, the adaptive strategy improved specific trust-related perceptions, particularly motion predictability and perceived task simplicity.
Federico Biagi, Dario Onfiani, Simone Silenzi +1
Department of Engineering "Enzo Ferrari", University of Modena and Reggio Emilia, Italy.