The Missing Touch: Spatially Distributed Tactile Feedback Brings Teleoperation Closer to Human Dexterity

Rohan Kota; Gregory Reardon; J. Edward Colgate

cs.ROAug 19, 2026

The Missing Touch: Spatially Distributed Tactile Feedback Brings Teleoperation Closer to Human Dexterity

Authors: Rohan Kota, Gregory Reardon, J. Edward Colgate

Organizations: Center for Robotics and Biosystems, Northwestern University, Evanston, IL.

Abstract

A fundamental challenge in robotic teleoperation is enabling an operator to control a remote robot as effortlessly and intuitively as their own hands. Despite the growing use of teleoperation to collect demonstration data for training autonomous robot policies, teleoperated robot performance still falls significantly short of human dexterity, even for basic tasks. Here, we present evidence that a key factor contributing to this performance gap is the absence of spatially distributed tactile feedback. Using a two-degree-of-freedom (DoF) bilateral force-feedback telemanipulator paired with a 32-DoF tactile fingertip display, we show that operator performance improves significantly when localized deformations on the remote manipulator are faithfully reproduced on the operator's fingertip. In a series of teleoperation tasks, reproducing distributed contact information not only accelerated task performance but also brought teleoperated movements closer to natural human behavior by minimizing corrective actions and task completion steps, thereby reducing the deviation between teleoperated and natural trajectories by 29 $\unicode{x2013}$ 79%. Furthermore, we found that increasing the resolution of the tactile feedback $\unicode{x2014}$ by refining how finely the measured displacements were quantized for reproduction $\unicode{x2014}$ compressed the state-space distribution of teleoperated motions, which has been associated with improved training outcomes for autonomous robot policies. Together, these results suggest that spatially distributed tactile feedback is essential for closing the gap between human and teleoperated dexterity and training the next generation of autonomous robots.

Figures & tables

Figure 1: Spatially distributed cutaneous feedback improves naturalness of teleoperated motion. ( A ) Humans performing tasks through a robotic teleoperation system (green) move more slowly, less accurately, and less naturally than during direct manipulation (orange). Although prior works demonstrate that haptic feedback reduces the teleoperation gap in speed and accuracy, closing it fully requires restoring natural movement patterns (dashed arrow). ( B ) Participants used a 2-degree-of-freedom (DoF) leader-follower teleoperation device with bilateral kinesthetic force feedback, driven by brushless DC motors and capstan transmissions. The device allows teleoperators to move in the vertical and horizontal directions. ( C ) A vision-based tactile sensor serving as the robot “finger” records spatially distributed contact during interaction with the environment. ( D ) A 32-DoF cutaneous shape display reproduces this distributed contact information on the operator’s fingertip via controlled inflation of small elastic domes. ( E ) Teleoperated movement patterns converge toward natural human movements as the spatial resolution of cutaneous feedback increases. Renderings of the tactile displays illustrate the four feedback conditions: Off (left panel): no cutaneous feedback; 1D (second panel): uniform inflation across the display upon contact; 2D (third panel): independent inflation of the upper and lower display halves based on contact location; Full (right panel): localized inflation matching the distribution of contact on the robot finger.

Figure 2: Button discrimination task experimental results. Box plots show median (center line), interquartile range (box), 1.5 $\times$ interquartile range (whiskers), and outliers (circles). Unless otherwise noted, pairwise comparisons were performed between the Full condition (orange) and the Off (blue), 1D (purple), and 2D (magenta) conditions (stars indicate statistical significance). ( A ) Participants were asked to press one of two nearby buttons separated by 1 cm without visual feedback. Idealized trajectories illustrate characteristic movement patterns observed under each feedback condition. ( B ) Trajectory of median distance from the participant’s baseline trajectory (red) for each feedback condition (target: Button 2; odd-numbered participants shown). When participants received no cutaneous feedback, they frequently overshot the button and reversed directions (“switchbacks”). Higher resolutions of cutaneous feedback produced trajectories with a smaller DTW distance to the participants’ baseline trajectory (darker lines). Shaded region: button location. ( C ) Box plots of participants’ mean overshoot distance for each condition. ( D ) Box plots of participants’ mean DTW distance for each condition, normalized by their mean distance across all trials. Pairwise comparisons were performed between all six feedback-condition pairs. ( E ) Box plots of within-participant standard deviation of DTW distance. ( F ) Box plots of participants’ mean completion time for each condition, normalized by their mean completion time across all trials. Pairwise comparisons were performed between all six feedback-condition pairs.

Figure 3: Peg rolling task experimental results. Box plots show median (center line), interquartile range (box), 1.5 $\times$ interquartile range (whiskers), and outliers (circles). Unless otherwise noted, pairwise comparisons were performed between the Full condition (orange) and the Off (blue), 1D (purple), and 2D (magenta) conditions (stars indicate statistical significance). ( A ) Participants were asked to roll a peg forward to a hard stop and then backward to its starting position. Task completion consisted of a series of rolling strokes punctuated by “loopbacks” to reset the finger relative to the peg. ( B ) Prototypical trajectories from one participant during the forward-rolling phase, segmented into rolling strokes (green), loopbacks (blue), finger slips past the peg (red), and miscellaneous movements (gray). Example cutaneous-display activation patterns are shown at the beginning (triangle), middle (circle), and end (star) of a stroke. ( C ) Box plots of participants’ mean stroke length for each condition. ( D ) Box plots of participants’ mean number of strokes per trial for each condition. ( E ) Box plots of participants’ mean DTW distance for each condition, normalized by their mean distance across all trials. Pairwise comparisons were performed between all six feedback-condition pairs. ( F ) Box plots of within-participant standard deviation of DTW distance. ( G ) Box plots of participants’ mean completion time for each condition, normalized by their mean completion time across all trials. Pairwise comparisons were performed between all six feedback-condition pairs. ( H ) Box plots of within-participant standard deviation of completion time.

Figure 4: Post-experiment questionnaire results. After completing each task, participants completed a modified NASA-TLX questionnaire, rating perceived performance (Q1), mental demand (Q2), task difficulty (Q3), and frustration (Q4) on a 7-point scale (1 = not at all, 7 = very). Box plots show median (center line), interquartile range (box), 1.5

\times

interquartile range (whiskers), and outliers (circles). ( A ) Responses for the button discrimination task. ( B ) Responses for the peg-rolling task.

Figure 5: Improving dataset consistency for autonomous policy training. Aggregating demonstrations collected with higher-resolution cutaneous feedback produces more concentrated trajectory distributions and reduced inter-trial variability. ( A ) Overlaid trajectories from all participants and trials in the button discrimination task (target: Button 2) for each feedback condition. ( B ) Overlaid trajectories from all participants and trials in the peg-rolling task (forward roll only) for each feedback condition. ( C ) State-space occupancy heatmaps for the peg-rolling task. Higher-resolution feedback yields a more concentrated state space distribution. ( D ) Box plots of mean pairwise DTW distance between trials from the same participant in the button discrimination task. Box plots show median (center line), interquartile range (box), 1.5 $\times$ interquartile range (whiskers), and outliers (circles). ( E ) Box plots of mean pairwise DTW distance between trials from the same participant in the peg-rolling task. Box plot details: see above. ( F ) Violin plots of all pairwise inter-trial DTW distances in the button discrimination task. Solid lines indicate medians and dashed lines indicate upper and lower quartiles. The upper quartile DTW distance is noted next to each of the respective plots. ( G ) Violin plots of all pairwise inter-trial DTW distances in the peg-rolling task. Violin plot details: see above.

Figure S1 : Mapping from vision-based tactile sensor to cutaneous shape display. ( A ) The vision-based tactile sensor makes contact with objects in the environment. ( B ) The vision-based tactile sensor outputs a $320\times 240$ RGB image. ( C ) A mapping algorithm converts the raw RGB sensor image into a pixel inflation pattern (solid blue circles) that mirrors the pixel layout of the cutaneous shape display (blue circles). ( D ) The corresponding pixels on the cutaneous shape display are inflated.

	2D	1D	Off
Full	$W=0.0$ $p<0.001^{***}$	$W=0.0$ $p<0.001^{***}$	$W=0.0$ $p<0.001^{***}$

Table S1 : Button Discrimination Task: Overshoot Distance. Pairwise comparisons of participants’ mean overshoot distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=0.0$ $p<0.001^{***}$	$W=4.5$ $p=0.004^{**}$	$W=9.0$ $p=0.007^{**}$

Table S2 : Button Discrimination Task: Number of Switchbacks. Pairwise comparisons of participants’ mean number of switchbacks between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$F(1,572)=1.361$ $p=0.244$	$F(1,572)=8.125$ $p=0.014^{*}$	$F(1,572)=64.471$ $p<0.001^{***}$
2D		$F(1,572)=2.836$ $p=0.185$	$F(1,572)=47.100$ $p<0.001^{***}$
1D			$F(1,572)=26.821$ $p<0.001^{***}$

Table S3 : Button Discrimination Task: Trajectory Naturalness (DTW Distance). Pairwise comparisons of DTW distance (normalized by each participants’ mean DTW distance across all trials) between all six feedback-condition pairs. Statistics are from two-tailed coefficient tests.

	2D	1D	Off
Full	$W=11.0$ $p=0.013^{*}$	$W=0.0$ $p<0.001^{***}$	$W=0.0$ $p<0.001^{***}$

Table S4 : Button Discrimination Task: Variability in Trajectory Naturalness Pairwise comparisons of within-participant standard deviation of DTW distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$F(1,572)=2.445$ $p=0.237$	$F(1,572)=5.076$ $p=0.074$	$F(1,572)=69.188$ $p<0.001^{***}$
2D		$F(1,572)=0.475$ $p=0.491$	$F(1,572)=45.621$ $p<0.001^{***}$
1D			$F(1,572)=36.784$ $p<0.001^{***}$

Table S5 : Button Discrimination Task: Completion Time. Pairwise comparisons of task completion time (normalized by each participants’ mean completion time across all trials) between all six feedback-condition pairs. Statistics are from two-tailed coefficient tests.

	2D	1D	Off
Full	$W=4.0$ $p=0.003^{**}$	$W=6.0$ $p=0.003^{**}$	$W=0.0$ $p<0.001^{***}$

Table S6 : Button Discrimination Task: Variability in Completion Time. Pairwise comparisons of within-participant standard deviation of completion time between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=8.0$ $p=0.039^{*}$	$W=1.5$ $p=0.018^{*}$	$W=5.0$ $p=0.039^{*}$

Table S7 : Peg Rolling Task: Slip Frequency. Pairwise comparisons of participants’ mean number of slips per trial between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=53.0$ $p=0.004^{**}$	$W=54.0$ $p=0.004^{**}$	$W=55.0$ $p=0.003^{**}$

Table S8 : Peg Rolling Task: Stroke Length. Pairwise comparisons of participants’ mean stroke length between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=6.0$ $p=0.014^{*}$	$W=3.0$ $p=0.010^{*}$	$W=1.0$ $p=0.006^{**}$

Table S9 : Peg Rolling Task: Number of Strokes. Pairwise comparisons of participants’ mean number of strokes between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$F(1,476)=10.833$ $p=0.003^{**}$	$F(1,476)=24.632$ $p<0.001^{***}$	$F(1,476)=51.071$ $p<0.001^{***}$
2D		$F(1,476)=2.794$ $p=0.095$	$F(1,476)=14.861$ $p<0.001^{***}$
1D			$F(1,476)=4.767$ $p=0.059$

Table S10 : Peg Rolling Task: Trajectory Naturalness (DTW Distance). Pairwise comparisons of DTW distance (normalized by each participants’ mean DTW distance across all trials) between all six feedback-condition pairs. Statistics are from two-tailed coefficient tests.

	2D	1D	Off
Full	$W=0.0$ $p=0.003^{**}$	$W=3.0$ $p=0.005^{**}$	$W=0.0$ $p=0.003^{**}$

Table S11 : Peg Rolling Task: Variability in Trajectory Naturalness. Pairwise comparisons of within-participant standard deviation of DTW distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$F(1,476)=27.082$ $p<0.001^{***}$	$F(1,476)=74.124$ $p<0.001^{***}$	$F(1,476)=104.059$ $p<0.001^{***}$
2D		$F(1,476)=11.597$ $p=0.001^{***}$	$F(1,476)=24.969$ $p<0.001^{***}$
1D			$F(1,476)=2.533$ $p=0.112$

Table S12 : Peg Rolling Task: Completion Time. Pairwise comparisons of task completion time (normalized by each participants’ mean completion time across all trials) between all six feedback-condition pairs. Statistics are from two-tailed coefficient tests.

	2D	1D	Off
Full	$W=0.0$ $p=0.003^{**}$	$W=0.0$ $p=0.003^{**}$	$W=0.0$ $p=0.003^{**}$

Table S13 : Peg Rolling Task: Variability in Completion Time. Pairwise comparisons of within-participant standard deviation of completion time between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	Off $\ \xrightarrow{}$ 1D	1D $\ \xrightarrow{}$ 2D	2D $\ \xrightarrow{}$ Full
Q1	$W=0.0$ $p<0.001^{***}$	$W=0.0$ $p=0.016^{*}$	$W=7.5$ $p=0.094$
Q2	$W=59.0$ $p=0.022^{*}$	$W=51.0$ $p=0.022^{*}$	$W=25.0$ $p=0.047^{*}$
Q3	$W=78.0$ $p<0.001^{***}$	$W=41.0$ $p=0.031^{*}$	$W=28.0$ $p=0.117$
Q4	$W=62.5$ $p=0.010^{*}$	$W=28.0$ $p=0.016^{*}$	$W=21.5$ $p=0.141$

Table S14 : Button Discrimination Task: Questionnaire Responses. Pairwise comparisons of post-experiment questionnaire responses between adjacent feedback conditions ( Off vs. 1D , 1D vs. 2D , and 2D vs. Full ). Statistics are from one-tailed Wilcoxon signed-rank tests.

	Off $\ \xrightarrow{}$ 1D	1D $\ \xrightarrow{}$ 2D	2D $\ \xrightarrow{}$ Full
Q1	$W=6.0$ $p=0.266$	$W=2.5$ $p=0.059$	$W=6.0$ $p=0.250$
Q2	$W=21.0$ $p=0.047^{*}$	$W=40.5$ $p=0.047^{*}$	$W=30.5$ $p=0.047^{*}$
Q3	$W=15.0$ $p=0.063$	$W=36.0$ $p=0.012^{*}$	$W=25.0$ $p=0.063$
Q4	$W=10.0$ $p=0.125$	$W=15.0$ $p=0.094$	$W=18.5$ $p=0.125$

Table S15 : Peg Rolling Task: Questionnaire Responses. Pairwise comparisons of post-experiment questionnaire responses between adjacent feedback conditions ( Off vs. 1D , 1D vs. 2D , and 2D vs. Full ). Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=19.0$ $p=0.065$	$W=15.0$ $p=0.065$	$W=0.0$ $p<0.001^{***}$

Table S16 : Button Discrimination Task: Per-Participant Inter-Trial Trajectory Variability. Pairwise comparisons of participants’ mean inter-trial DTW distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$W=0.0$ $p=0.003^{**}$	$W=0.0$ $p=0.003^{**}$	$W=0.0$ $p=0.003^{**}$

Table S17 : Peg Rolling Task: Per-Participant Inter-Trial Trajectory Variability. Pairwise comparisons of participants’ mean inter-trial DTW distance between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Wilcoxon signed-rank tests.

	2D	1D	Off
Full	$U=2548888.0$ $p<0.001^{***}$	$U=2525668.0$ $p<0.001^{***}$	$U=1235360.0$ $p<0.001^{***}$

Table S18 : Button Discrimination Task: Aggregated Inter-Trial Trajectory Variability. Pairwise comparisons of all pairwise inter-trial DTW distances between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Mann-Whitney U tests.

	2D	1D	Off
Full	$U=9220395.0$ $p<0.001^{***}$	$U=4593005.0$ $p<0.001^{***}$	$U=4261698.0$ $p<0.001^{***}$

Table S19 : Peg Rolling Task: Aggregated Inter-Trial Trajectory Variability. Pairwise comparisons of all pairwise inter-trial DTW distances between the Full condition and the 2D , 1D , and Off conditions. Statistics are from one-tailed Mann-Whitney U tests.

Explore similar work

Semantic Haptic Feedback Enhances Dexterous Robotic Teleoperation
Aug 3, 2026Bingjian Huang, Sahar Aseeri, Jonas Schmidtler +9Robot Teleoperation Haptic Feedback
Remote Surfaces at Your Fingertips: Electrovibration-Based Tactile Feedback for Robot Teleoperation via Touchscreen Interfaces
Aug 22, 2026Alperen Kenan, Juan José García Cárdenas, Adriana Tapus +2Robot Teleoperation Haptic Feedback
Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom · Computer Science and System Engineering Department (U2IS), Autonomous Systems and Robotics Lab, ENSTA, Institut Polytechnique de Paris, Paris, France · Kempten University of Applied Sciences, Kempten, Germany
DexTeleop-0: Force-Aware Bimanual Dexterous Teleoperation with Ego-Centric Perception towards Shared Autonomy
Jun 22, 2026Haichao Liu, Yuyao Jiang, Hyunsun Park +2Dexterous Manipulation Bimanual Robotic Manipulation
Nanyang Technological University, Singapore · OOJU, USA

Aug 3, 2026, cs.ROJ/K move · Enter open · S save

Semantic Haptic Feedback Enhances Dexterous Robotic Teleoperation

Bingjian Huang, Sahar Aseeri, Jonas Schmidtler, Joseph Zhang+8

Robot Teleoperation Haptic Feedback Robotic Manipulation

Open paper AskPDF arXiv

In robot teleoperation, haptic feedback can be used to help human operators accomplish dexterous manipulation tasks. However, existing haptic feedback methods try to replicate high-fidelity sensory haptics that are felt in real world interactions, which are constrained by the sensing and feedback hardware capability and may lead to higher workload. To addresses these limitations, this work introduces semantic haptics for teleoperation, which uses abstract haptic patterns to convey critical information about robot states. We categorize robot states into "Confirmations" and "Exceptions", implement a modular haptic rendering pipeline in robot simulation, and deliver semantic haptic feedback to operators through pneumatic and vibrotactile wristbands. This simplifies hardware requirements and enables one-to-many mappings between haptic patterns and robot states. Through three evaluation studies, we identify the most effective semantic haptic design for a common pick and place teleoperation task and compare semantic haptics to other teleoperation feedback approaches including sensory haptics and visual feedback. Results suggest that while semantic haptics performs similarly as other feedback in unimanual tasks, it achieves superior performance in bimanual tasks, with reduced task workload, increased situational awareness, and overall preference.

Abstract

Explore similar work

Semantic Haptic Feedback Enhances Dexterous Robotic Teleoperation

Remote Surfaces at Your Fingertips: Electrovibration-Based Tactile Feedback for Robot Teleoperation via Touchscreen Interfaces

DexTeleop-0: Force-Aware Bimanual Dexterous Teleoperation with Ego-Centric Perception towards Shared Autonomy