The Missing Touch: Spatially Distributed Tactile Feedback Brings Teleoperation Closer to Human Dexterity

Rohan Kota; Gregory Reardon; J. Edward Colgate

cs.ROAug 19, 2026

The Missing Touch: Spatially Distributed Tactile Feedback Brings Teleoperation Closer to Human Dexterity

Authors: Rohan Kota, Gregory Reardon, J. Edward Colgate

Organizations: Center for Robotics and Biosystems, Northwestern University, Evanston, IL.

Abstract

A fundamental challenge in robotic teleoperation is enabling an operator to control a remote robot as effortlessly and intuitively as their own hands. Despite the growing use of teleoperation to collect demonstration data for training autonomous robot policies, teleoperated robot performance still falls significantly short of human dexterity, even for basic tasks. Here, we present evidence that a key factor contributing to this performance gap is the absence of spatially distributed tactile feedback. Using a two-degree-of-freedom (DoF) bilateral force-feedback telemanipulator paired with a 32-DoF tactile fingertip display, we show that operator performance improves significantly when localized deformations on the remote manipulator are faithfully reproduced on the operator's fingertip. In a series of teleoperation tasks, reproducing distributed contact information not only accelerated task performance but also brought teleoperated movements closer to natural human behavior by minimizing corrective actions and task completion steps, thereby reducing the deviation between teleoperated and natural trajectories by 29 $\unicode{x2013}$ 79%. Furthermore, we found that increasing the resolution of the tactile feedback $\unicode{x2014}$ by refining how finely the measured displacements were quantized for reproduction $\unicode{x2014}$ compressed the state-space distribution of teleoperated motions, which has been associated with improved training outcomes for autonomous robot policies. Together, these results suggest that spatially distributed tactile feedback is essential for closing the gap between human and teleoperated dexterity and training the next generation of autonomous robots.

Figures & tables

Figure 1: Spatially distributed cutaneous feedback improves naturalness of teleoperated motion. ( A ) Humans performing tasks through a robotic teleoperation system (green) move more slowly, less accurately, and less naturally than during direct manipulation (orange). Although prior works demonstrate that haptic feedback reduces the teleoperation gap in speed and accuracy, closing it fully requires restoring natural movement patterns (dashed arrow). ( B ) Participants used a 2-degree-of-freedom (DoF) leader-follower teleoperation device with bilateral kinesthetic force feedback, driven by brushless DC motors and capstan transmissions. The device allows teleoperators to move in the vertical and horizontal directions. ( C ) A vision-based tactile sensor serving as the robot “finger” records spatially distributed contact during interaction with the environment. ( D ) A 32-DoF cutaneous shape display reproduces this distributed contact information on the operator’s fingertip via controlled inflation of small elastic domes. ( E ) Teleoperated movement patterns converge toward natural human movements as the spatial resolution of cutaneous feedback increases. Renderings of the tactile displays illustrate the four feedback conditions: Off (left panel): no cutaneous feedback; 1D (second panel): uniform inflation across the display upon contact; 2D (third panel): independent inflation of the upper and lower display halves based on contact location; Full (right panel): localized inflation matching the distribution of contact on the robot finger.

Figure 2: Button discrimination task experimental results. Box plots show median (center line), interquartile range (box), 1.5 $\times$ interquartile range (whiskers), and outliers (circles). Unless otherwise noted, pairwise comparisons were performed between the Full condition (orange) and the Off (blue), 1D (purple), and 2D (magenta) conditions (stars indicate statistical significance). ( A ) Participants were asked to press one of two nearby buttons separated by 1 cm without visual feedback. Idealized trajectories illustrate characteristic movement patterns observed under each feedback condition. ( B ) Trajectory of median distance from the participant’s baseline trajectory (red) for each feedback condition (target: Button 2; odd-numbered participants shown). When participants received no cutaneous feedback, they frequently overshot the button and reversed directions (“switchbacks”). Higher resolutions of cutaneous feedback produced trajectories with a smaller DTW distance to the participants’ baseline trajectory (darker lines). Shaded region: button location. ( C ) Box plots of participants’ mean overshoot distance for each condition. ( D ) Box plots of participants’ mean DTW distance for each condition, normalized by their mean distance across all trials. Pairwise comparisons were performed between all six feedback-condition pairs. ( E ) Box plots of within-participant standard deviation of DTW distance. ( F ) Box plots of participants’ mean completion time for each condition, normalized by their mean completion time across all trials. Pairwise comparisons were performed between all six feedback-condition pairs.

Figure 3: Peg rolling task experimental results. Box plots show median (center line), interquartile range (box), 1.5 $\times$ interquartile range (whiskers), and outliers (circles). Unless otherwise noted, pairwise comparisons were performed between the Full condition (orange) and the Off (blue), 1D (purple), and 2D (magenta) conditions (stars indicate statistical significance). ( A ) Participants were asked to roll a peg forward to a hard stop and then backward to its starting position. Task completion consisted of a series of rolling strokes punctuated by “loopbacks” to reset the finger relative to the peg. ( B ) Prototypical trajectories from one participant during the forward-rolling phase, segmented into rolling strokes (green), loopbacks (blue), finger slips past the peg (red), and miscellaneous movements (gray). Example cutaneous-display activation patterns are shown at the beginning (triangle), middle (circle), and end (star) of a stroke. ( C ) Box plots of participants’ mean stroke length for each condition. ( D ) Box plots of participants’ mean number of strokes per trial for each condition. ( E ) Box plots of participants’ mean DTW distance for each condition, normalized by their mean distance across all trials. Pairwise comparisons were performed between all six feedback-condition pairs. ( F ) Box plots of within-participant standard deviation of DTW distance. ( G ) Box plots of participants’ mean completion time for each condition, normalized by their mean completion time across all trials. Pairwise comparisons were performed between all six feedback-condition pairs. ( H ) Box plots of within-participant standard deviation of completion time.

Figure 4: Post-experiment questionnaire results. After completing each task, participants completed a modified NASA-TLX questionnaire, rating perceived performance (Q1), mental demand (Q2), task difficulty (Q3), and frustration (Q4) on a 7-point scale (1 = not at all, 7 = very). Box plots show median (center line), interquartile range (box), 1.5

\times

interquartile range (whiskers), and outliers (circles). ( A ) Responses for the button discrimination task. ( B ) Responses for the peg-rolling task.

Figure 5: Improving dataset consistency for autonomous policy training. Aggregating demonstrations collected with higher-resolution cutaneous feedback produces more concentrated trajectory distributions and reduced inter-trial variability. ( A ) Overlaid trajectories from all participants and trials in the button discrimination task (target: Button 2) for each feedback condition. ( B ) Overlaid trajectories from all participants and trials in the peg-rolling task (forward roll only) for each feedback condition. ( C ) State-space occupancy heatmaps for the peg-rolling task. Higher-resolution feedback yields a more concentrated state space distribution. ( D ) Box plots of mean pairwise DTW distance between trials from the same participant in the button discrimination task. Box plots show median (center line), interquartile range (box), 1.5 $\times$ interquartile range (whiskers), and outliers (circles). ( E ) Box plots of mean pairwise DTW distance between trials from the same participant in the peg-rolling task. Box plot details: see above. ( F ) Violin plots of all pairwise inter-trial DTW distances in the button discrimination task. Solid lines indicate medians and dashed lines indicate upper and lower quartiles. The upper quartile DTW distance is noted next to each of the respective plots. ( G ) Violin plots of all pairwise inter-trial DTW distances in the peg-rolling task. Violin plot details: see above.

Figure S1 : Mapping from vision-based tactile sensor to cutaneous shape display. ( A ) The vision-based tactile sensor makes contact with objects in the environment. ( B ) The vision-based tactile sensor outputs a $320\times 240$ RGB image. ( C ) A mapping algorithm converts the raw RGB sensor image into a pixel inflation pattern (solid blue circles) that mirrors the pixel layout of the cutaneous shape display (blue circles). ( D ) The corresponding pixels on the cutaneous shape display are inflated.

	2D	1D	Off
Full	$W=0.0$ $p<0.001^{***}$	$W=0.0$ $p<0.001^{***}$	$W=0.0$ $p<0.001^{***}$

Table S1 : Button Discrimination Task: Overshoot Distance. Pairwise comparisons of participants’ mean overshoot distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=0.0$ $p<0.001^{***}$	$W=4.5$ $p=0.004^{**}$	$W=9.0$ $p=0.007^{**}$

Table S2 : Button Discrimination Task: Number of Switchbacks. Pairwise comparisons of participants’ mean number of switchbacks between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$F(1,572)=1.361$ $p=0.244$	$F(1,572)=8.125$ $p=0.014^{*}$	$F(1,572)=64.471$ $p<0.001^{***}$
2D		$F(1,572)=2.836$ $p=0.185$	$F(1,572)=47.100$ $p<0.001^{***}$
1D			$F(1,572)=26.821$ $p<0.001^{***}$

Table S3 : Button Discrimination Task: Trajectory Naturalness (DTW Distance). Pairwise comparisons of DTW distance (normalized by each participants’ mean DTW distance across all trials) between all six feedback-condition pairs. Statistics are from two-tailed coefficient tests.

	2D	1D	Off
Full	$W=11.0$ $p=0.013^{*}$	$W=0.0$ $p<0.001^{***}$	$W=0.0$ $p<0.001^{***}$

Table S4 : Button Discrimination Task: Variability in Trajectory Naturalness Pairwise comparisons of within-participant standard deviation of DTW distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$F(1,572)=2.445$ $p=0.237$	$F(1,572)=5.076$ $p=0.074$	$F(1,572)=69.188$ $p<0.001^{***}$
2D		$F(1,572)=0.475$ $p=0.491$	$F(1,572)=45.621$ $p<0.001^{***}$
1D			$F(1,572)=36.784$ $p<0.001^{***}$

Table S5 : Button Discrimination Task: Completion Time. Pairwise comparisons of task completion time (normalized by each participants’ mean completion time across all trials) between all six feedback-condition pairs. Statistics are from two-tailed coefficient tests.

	2D	1D	Off
Full	$W=4.0$ $p=0.003^{**}$	$W=6.0$ $p=0.003^{**}$	$W=0.0$ $p<0.001^{***}$

Table S6 : Button Discrimination Task: Variability in Completion Time. Pairwise comparisons of within-participant standard deviation of completion time between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=8.0$ $p=0.039^{*}$	$W=1.5$ $p=0.018^{*}$	$W=5.0$ $p=0.039^{*}$

Table S7 : Peg Rolling Task: Slip Frequency. Pairwise comparisons of participants’ mean number of slips per trial between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=53.0$ $p=0.004^{**}$	$W=54.0$ $p=0.004^{**}$	$W=55.0$ $p=0.003^{**}$

Table S8 : Peg Rolling Task: Stroke Length. Pairwise comparisons of participants’ mean stroke length between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=6.0$ $p=0.014^{*}$	$W=3.0$ $p=0.010^{*}$	$W=1.0$ $p=0.006^{**}$

Table S9 : Peg Rolling Task: Number of Strokes. Pairwise comparisons of participants’ mean number of strokes between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$F(1,476)=10.833$ $p=0.003^{**}$	$F(1,476)=24.632$ $p<0.001^{***}$	$F(1,476)=51.071$ $p<0.001^{***}$
2D		$F(1,476)=2.794$ $p=0.095$	$F(1,476)=14.861$ $p<0.001^{***}$
1D			$F(1,476)=4.767$ $p=0.059$

Table S10 : Peg Rolling Task: Trajectory Naturalness (DTW Distance). Pairwise comparisons of DTW distance (normalized by each participants’ mean DTW distance across all trials) between all six feedback-condition pairs. Statistics are from two-tailed coefficient tests.

	2D	1D	Off
Full	$W=0.0$ $p=0.003^{**}$	$W=3.0$ $p=0.005^{**}$	$W=0.0$ $p=0.003^{**}$

Table S11 : Peg Rolling Task: Variability in Trajectory Naturalness. Pairwise comparisons of within-participant standard deviation of DTW distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$F(1,476)=27.082$ $p<0.001^{***}$	$F(1,476)=74.124$ $p<0.001^{***}$	$F(1,476)=104.059$ $p<0.001^{***}$
2D		$F(1,476)=11.597$ $p=0.001^{***}$	$F(1,476)=24.969$ $p<0.001^{***}$
1D			$F(1,476)=2.533$ $p=0.112$

Table S12 : Peg Rolling Task: Completion Time. Pairwise comparisons of task completion time (normalized by each participants’ mean completion time across all trials) between all six feedback-condition pairs. Statistics are from two-tailed coefficient tests.

	2D	1D	Off
Full	$W=0.0$ $p=0.003^{**}$	$W=0.0$ $p=0.003^{**}$	$W=0.0$ $p=0.003^{**}$

Table S13 : Peg Rolling Task: Variability in Completion Time. Pairwise comparisons of within-participant standard deviation of completion time between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	Off $\ \xrightarrow{}$ 1D	1D $\ \xrightarrow{}$ 2D	2D $\ \xrightarrow{}$ Full
Q1	$W=0.0$ $p<0.001^{***}$	$W=0.0$ $p=0.016^{*}$	$W=7.5$ $p=0.094$
Q2	$W=59.0$ $p=0.022^{*}$	$W=51.0$ $p=0.022^{*}$	$W=25.0$ $p=0.047^{*}$
Q3	$W=78.0$ $p<0.001^{***}$	$W=41.0$ $p=0.031^{*}$	$W=28.0$ $p=0.117$
Q4	$W=62.5$ $p=0.010^{*}$	$W=28.0$ $p=0.016^{*}$	$W=21.5$ $p=0.141$

Table S14 : Button Discrimination Task: Questionnaire Responses. Pairwise comparisons of post-experiment questionnaire responses between adjacent feedback conditions ( Off vs. 1D , 1D vs. 2D , and 2D vs. Full ). Statistics are from one-tailed Wilcoxon signed-rank tests.

	Off $\ \xrightarrow{}$ 1D	1D $\ \xrightarrow{}$ 2D	2D $\ \xrightarrow{}$ Full
Q1	$W=6.0$ $p=0.266$	$W=2.5$ $p=0.059$	$W=6.0$ $p=0.250$
Q2	$W=21.0$ $p=0.047^{*}$	$W=40.5$ $p=0.047^{*}$	$W=30.5$ $p=0.047^{*}$
Q3	$W=15.0$ $p=0.063$	$W=36.0$ $p=0.012^{*}$	$W=25.0$ $p=0.063$
Q4	$W=10.0$ $p=0.125$	$W=15.0$ $p=0.094$	$W=18.5$ $p=0.125$

Table S15 : Peg Rolling Task: Questionnaire Responses. Pairwise comparisons of post-experiment questionnaire responses between adjacent feedback conditions ( Off vs. 1D , 1D vs. 2D , and 2D vs. Full ). Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=19.0$ $p=0.065$	$W=15.0$ $p=0.065$	$W=0.0$ $p<0.001^{***}$

Table S16 : Button Discrimination Task: Per-Participant Inter-Trial Trajectory Variability. Pairwise comparisons of participants’ mean inter-trial DTW distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=0.0$ $p=0.003^{**}$	$W=0.0$ $p=0.003^{**}$	$W=0.0$ $p=0.003^{**}$

Table S17 : Peg Rolling Task: Per-Participant Inter-Trial Trajectory Variability. Pairwise comparisons of participants’ mean inter-trial DTW distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$U=2548888.0$ $p<0.001^{***}$	$U=2525668.0$ $p<0.001^{***}$	$U=1235360.0$ $p<0.001^{***}$

Table S18 : Button Discrimination Task: Aggregated Inter-Trial Trajectory Variability. Pairwise comparisons of all pairwise inter-trial DTW distances between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Mann-Whitney U tests.

	2D	1D	Off
Full	$U=9220395.0$ $p<0.001^{***}$	$U=4593005.0$ $p<0.001^{***}$	$U=4261698.0$ $p<0.001^{***}$

Table S19 : Peg Rolling Task: Aggregated Inter-Trial Trajectory Variability. Pairwise comparisons of all pairwise inter-trial DTW distances between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Mann-Whitney U tests.

Explore similar work

Aug 3, 2026cs.RO

Semantic Haptic Feedback Enhances Dexterous Robotic Teleoperation

In robot teleoperation, haptic feedback can be used to help human operators accomplish dexterous manipulation tasks. However, existing haptic feedback methods try to replicate high-fidelity sensory haptics that are felt in real world interactions, which are constrained by the sensing and feedback hardware capability and may lead to higher workload. To addresses these limitations, this work introduces semantic haptics for teleoperation, which uses abstract haptic patterns to convey critical information about robot states. We categorize robot states into "Confirmations" and "Exceptions", implement a modular haptic rendering pipeline in robot simulation, and deliver semantic haptic feedback to operators through pneumatic and vibrotactile wristbands. This simplifies hardware requirements and enables one-to-many mappings between haptic patterns and robot states. Through three evaluation studies, we identify the most effective semantic haptic design for a common pick and place teleoperation task and compare semantic haptics to other teleoperation feedback approaches including sensory haptics and visual feedback. Results suggest that while semantic haptics performs similarly as other feedback in unimanual tasks, it achieves superior performance in bimanual tasks, with reduced task workload, increased situational awareness, and overall preference.

Bingjian Huang, Sahar Aseeri, Jonas Schmidtler +9

Robot Teleoperation Haptic Feedback Robotic Manipulation

Ask PDF

Aug 22, 2026cs.RO

Remote Surfaces at Your Fingertips: Electrovibration-Based Tactile Feedback for Robot Teleoperation via Touchscreen Interfaces

Enabling operators to perceive and interact with remote environments naturally is a fundamental challenge in robotic teleoperation. This is especially critical in tasks involving physical interaction, where real-time haptic awareness improves operational safety and effectiveness. Existing kinesthetic haptic feedback methods suffer from instability during rigid surface contacts and remain sensitive to communication delays, while visual cue-based force feedback imposes additional cognitive load and limits sustained situational awareness. This work presents a teleoperation interface that conveys remote surface interactions to the operator through electrovibration-based tactile feedback, enabling naturally mapped force reflection while avoiding the stability issues associated with kinesthetic feedback and the latency limitations of mechanical actuators. A user study (N=21) evaluated interface usability, sense of presence, and operator workload under two force reflection conditions: visual feedback and electrovibration-based tactile feedback. Characterisation experiments further assessed path-following accuracy and response time across both conditions. Results show that tactile feedback significantly reduced response time by 15.35% (p=0.002, d=0.96) and increased the sense of presence by 31% (p<0.001, d=0.90) compared to visual feedback, while imposing comparable workload and usability across both conditions. These findings demonstrate that electrovibration-based tactile feedback is a viable and effective modality for robot teleoperation, improving operator responsiveness and sense of presence in contact-rich manipulation tasks, with direct applicability to safety-critical domains such as nuclear maintenance.

Alperen Kenan, Juan José García Cárdenas, Adriana Tapus +2

Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom · Computer Science and System Engineering Department (U2IS), Autonomous Systems and Robotics Lab, ENSTA, Institut Polytechnique de Paris, Paris, France · Kempten University of Applied Sciences, Kempten, Germany

Robot Teleoperation Haptic Feedback Contact-Rich Robotic Manipulation

Ask PDF

Jun 22, 2026cs.RO

DexTeleop-0: Force-Aware Bimanual Dexterous Teleoperation with Ego-Centric Perception towards Shared Autonomy

Fine-grained, bimanual dexterous manipulation remains a foundational challenge in robotics. Traditional teleoperation systems often fail in contact-rich tasks because embodiment gaps hinder accurate kinematic mapping, while tactile and force feedback remain absent. Consequently, data collection efficiency for high-precision tasks remains prohibitively low. To address these limitations, we propose a tactile-driven adaptation strategy designed to enable fine-grained manipulation on top of teleoperation pipelines. Instantiated within our bimanual dexterous framework, DexTeleop-0, this strategy introduces a real-time optimization loop that bridges the embodiment gap by translating coarse human tracking intents into precise, force-compliant robotic commands with tactile sensing. By estimating accurate contact points and leveraging a tactile-enabled fingertip force-sensing profile, the system dynamically computes localized corrections using the operational space Jacobian with respect to joint angle updates. We rigorously evaluate this tactile-driven adaptation strategy across both simulated environments and real-world hardware. Compared with representative baselines, the proposed method consistently achieves higher task success rates and improved execution efficiency in robust grasping, disturbance-resilient manipulation, and complex dexterous tasks.

Haichao Liu, Yuyao Jiang, Hyunsun Park +2

Nanyang Technological University, Singapore · OOJU, USA

Dexterous Manipulation Bimanual Robotic Manipulation Contact-Rich Robotic Manipulation Robot Teleoperation Tactile Sensing for Robotic Manipulation

Ask PDF