ZeroTouch: Tactile-Supervised Visual Contact Estimation for Contact-Rich Manipulation

Dmitriy Kosenkov; Daniia Zinniatullina; Miguel Altamirano Cabrera; Iana Zhura; Mikhail Derevianchenko; Dzmitry Tsetserukou

cs.ROSep 18, 2026

ZeroTouch: Tactile-Supervised Visual Contact Estimation for Contact-Rich Manipulation

Authors: Dmitriy Kosenkov, Daniia Zinniatullina, Miguel Altamirano Cabrera, Iana Zhura, Mikhail Derevianchenko, Dzmitry Tsetserukou

Organizations: Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology, Moscow, Russian Federation.

Abstract

Reliable robotic grasping benefits from estimating the evolving physical interaction and selecting a grasp-dependent compression target. Tactile sensors provide direct interaction measurements but require dedicated hardware at deployment. We introduce ZeroTouch, a tactile-supervised framework that predicts dense contact deformation, the instantaneous six-axis wrench, and a grasp-dependent desired compression target from wrist RGB observations, gripper state, and local gravity direction. Tactile measurements are used only as privileged supervision during training and are not required at deployment. On the full validation set, the complete architecture reduces normal-force MAE from 2.017 N for a state-only baseline to 0.531 N. In physical evaluation with 20 trials per condition, ZeroTouch achieves 95% success on an unseen object, 80% in a seen-object/unseen-grasp condition, and 90% under a content/load shift. Under the same evaluation protocol, OpenVLA achieves 25%, 40%, and 55%, while SmolVLA achieves 10%, 25%, and 35%, respectively.

Figures & tables

Fig. 1: Overview of ZeroTouch. From wrist RGB, gripper state, and local gravity direction, the model predicts contact deformation, the instantaneous 6D wrench, and the desired grasp force. The framework operates across different gripper poses and grasp locations on the object, while tactile sensing is used only during training.

Fig. 2: ZeroTouch framework. Wrist RGB observations, gripper state, and local gravity direction are used to predict contact deformation, the instantaneous 6D wrench, and the desired grasp force. Tactile sensing is used only during training as privileged supervision.

Model	Train	Val	$\Delta$ Val
	$F_{z}$ MAE (N)		(%)
State only	0.651	2.017	0.0
Pooled RGB + state	0.185	1.115	44.7
Attention + FiLM	0.173	0.828	58.9
+ tactile-map loss	0.108	0.752	62.7
+ wrench token	0.117	0.531	73.7

TABLE I: Architecture ablation for normal-force prediction. Validation contains held-out objects and grasp configurations.

\Delta

Val denotes error reduction relative to the state-only baseline.

Output	Metric	Train	Val	Test
		187	31	46
$F_{x}$	MAE [N]	0.058	0.117	0.385
$F_{y}$	MAE [N]	0.059	0.150	0.292
$F_{z}$	MAE [N]	0.111	0.531	0.828
$M_{x}$	MAE [N m]	0.120	0.405	0.649
$M_{y}$	MAE [N m]	0.127	0.635	0.849

TABLE II: Quantitative evaluation of the final ZeroTouch model. Errors are aggregated at the episode level. Validation is used for model selection, whereas Test is reserved for final generalization evaluation.

Fig. 3: Six-axis contact estimation for a representative grasp. The example shows the wrist RGB observation, tactile deformation, and recorded and predicted force and moment components. The tactile sensor is used as an offline reference and is not an inference input to ZeroTouch.

Fig. 4: Current-force and desired-force prediction for two representative grasp episodes. Solid curves show the recorded normal force, dashed curves show the visual force prediction, and horizontal lines denote the recorded and predicted desired compression targets. Vertical markers indicate the corresponding threshold crossings. The examples illustrate the relative force-target criterion used by ZeroTouch; closed-loop grasp termination is evaluated separately in the physical robot experiments.

Fig. 5: Generalization of tactile deformation prediction to unseen objects and held-out grasp configurations. Rows show RGB observations, recorded tactile deformation, and predicted deformation. The examples include both objects excluded from training and objects seen during training but evaluated in previously unseen poses and grasp configurations.

Fig. 6: Effect of object loading on predicted contact. Columns compare empty and loaded cups and bottles. Rows show the RGB input, visual attention, recorded tactile deformation, predicted deformation, and normal-force response.

Fig. 7: Normal-force prediction for the same bottle across four grasp configurations. Changing grasp location and orientation changes the recorded interaction even though object identity is fixed.

Fig. 8: UMAP visualization of the learned interaction latent space. Each point represents the median contact latent of one grasp episode. Labels and train/validation membership are used only for post-hoc visualization and are not inputs to UMAP.

Method	Unseen object	Unseen grasp	Content shift
SmolVLA	2/20 (10%)	5/20 (25%)	7/20 (35%)
OpenVLA	5/20 (25%)	8/20 (40%)	11/20 (55%)
ZeroTouch	19/20 (95%)	16/20 (80%)	18/20 (90%)

TABLE III: Physical grasping success over 20 trials per condition.

Explore similar work

PredTac: Learning Contact-Rich Manipulation with Predicted Touch
Sep 14, 2026Weijia Fan, Daqiang GuoTactile Sensing Contact-Rich Robotic Manipulation
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.
SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?
Oct 2, 2026Chen Yang, Linzhe Shi, Changjie Wu +7Contact-Rich Robotic Manipulation Vision-Language-Action Models
Tsinghua University · Amap, Alibaba Group · Hong Kong Polytechnic University
TacZero: Training-Free Peg Insertion Using a General-Purpose Vision-Language Model with Tactile Feedback
Oct 6, 2026Kazutoshi TanakaZero-Shot Robotic Manipulation Contact-Rich Robotic Manipulation
OMRON SINIC X Corporation, Bunkyo-ku, Tokyo 113-0033, Japan

Sep 14, 2026, cs.ROJ/K move · Enter open · S save

PredTac: Learning Contact-Rich Manipulation with Predicted Touch

Weijia Fan, Daqiang Guo

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.

Tactile Sensing Contact-Rich Robotic Manipulation Robotic RL

Open paper AskPDF arXiv

Contact-rich manipulation benefits from tactile feedback, yet physical tactile sensors introduce hardware, calibration, synchronization, and maintenance costs that complicate policy learning and deployment. We formulate predicted touch as an alternative to measured tactile input and present PredTac, a framework that learns to infer tactile states from causal visual observations and robot states and uses the predicted touch as an explicit interface for policy learning and execution. A tactile predictor is first trained with tactile supervision and then used to provide contact information without requiring measured tactile input during downstream policy training or execution. We evaluate PredTac across three contact-rich manipulation tasks in simulation and on a real robot, and further examine how policy performance depends on the predicted contact content. In simulation goal-offset evaluations, predicted-touch policies achieve 27.0%, 52.0%, and 44.7% success on USB, Barbed-spike, and Valve, respectively, improving over the visual baseline by 8.0-13.7 percentage points. On the real robot, predicted-touch ACT achieves 70.0%, 50.0%, and 90.0% success on USB insertion, Barbed extraction, and Valve rotation, respectively, with a three-task mean of 70.0%, approaching measured-touch ACT at 72.2% and substantially outperforming visual ACT at 21.1%. Fixed-policy interventions further show that performance is sensitive to the spatial structure of predicted contact, with spatial rearrangement at fixed value distributions reducing Valve success by 10.7 percentage points. These results demonstrate that predicted touch can provide useful contact information for contact-rich manipulation without requiring tactile sensing as a policy input.

Abstract

Explore similar work

PredTac: Learning Contact-Rich Manipulation with Predicted Touch

SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?

TacZero: Training-Free Peg Insertion Using a General-Purpose Vision-Language Model with Tactile Feedback