Zero-Shot Transfer of Force Map Estimation Across GelSight Mini Sensors

Julio Castaño Amoros; Pablo Gil

cs.CVAug 18, 2026

Zero-Shot Transfer of Force Map Estimation Across GelSight Mini Sensors

Authors: Julio Castaño Amoros, Pablo Gil

Organizations: AUROVA Lab, Computer Science Research Institute, University of Alicante, San Vicente del Raspeig, 03690, Spain · Department of Physics, Systems Engineering and Signal Theory, University of Alicante, San Vicente del Raspeig, 03690, Spain

Abstract

Despite the rapid industrialization of the touch sensor manufacturing process, most of these sensors are still handmade in research laboratories. This complicates standardizing their performance, requiring the repetition of data collection and training models for each unit produced. To address this problem, this paper presents a method that can generalize the estimation of 3D force maps across different GelSight Mini sensor units, regardless of the sensor version. Specifically, the method consists of two stages: a domain adaptation stage, in which the input tactile image is reconstructed as a general tactile image using a UniT-based model; and a stage for estimating 3D force maps employing a U-Net network. Our proposal achieves promising results in both steps, such as an SSIM of 0.9338 +- 0.0358 in the image reconstruction phase and an MAE_F of 1.1294 +- 1.5934(N) in the force estimation phase.

Explore similar work

TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception
Jun 24, 2026Lan Wei, Gurmeher Khurana, Sirine Bhouri +6Vision-Based Tactile Sensing Few-Shot Learning
Imperial College London, UK. · Queen Mary University of London, UK · King’s College London, UK
Dense Force Estimation with an Event-based Optical Tactile Sensor
Jun 8, 2026Agis Politis, René Zurbrügg, Valentina CavinatoTactile Sensing Robotic Tactile Sensing
Sony Advanced Visual Sensing, Zurich, Switzerland · Computer Vision and Geometry Group at ETH Zürich
BIDETA: Brain-Inspired Data-Efficient Tactile Adaptation for Unseen Sensors
Sep 8, 2026Boheng Liu, Lan Wei, Ziyu Li +4Vision-Based Tactile Sensing Few-Shot Domain Adaptation
School of Computer Science and Technology Beijing Institute of Technology Beijing, China

Jun 24, 2026, cs.ROJ/K move · Enter open · S save

TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception

Lan Wei, Gurmeher Khurana, Sirine Bhouri, Wenhao Hong+5

Imperial College London, UK. · Queen Mary University of London, UK · King’s College London, UK

Vision-Based Tactile Sensing Few-Shot Learning Robotic Tactile Sensing Self-Supervised Visual Representation Learning

Open paper AskPDF arXiv

Vision-based tactile sensors (VBTSs) enable robots to infer contact geometry and force-related cues by imaging deformation through an internal camera, yet generalisation across sensor designs remains poorly understood. We present TacVerse, a multi-sensor dataset and benchmark for cross-sensor vision-based tactile perception. The dataset contains 106,800 tactile images from seven VBTSs and supports three downstream tasks: shape classification, grating classification, and force regression. Experiments are conducted under three settings: within-sensor training, zero-shot cross-sensor transfer, and few-shot adaptation. Strong within-sensor performance across all tasks indicates that the collected tactile observations are informative for the target objectives. Direct cross-sensor transfer, however, leads to substantial degradation. Shape classification is comparatively robust, whereas grating classification and force regression are more sensitive to sensor shift. Few-shot adaptation for force regression consistently improves performance on unseen target sensors but does not fully close the gap to within-sensor upper bounds. A representation study further shows that MAE (Masked Autoencoder) pretraining provides the most consistent gains across tasks and sensors. TacVerse provides a controlled testbed for studying sensor shift, data-efficient adaptation, and self-supervised learning in tactile perception.

Abstract

Explore similar work

TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception

Dense Force Estimation with an Event-based Optical Tactile Sensor

BIDETA: Brain-Inspired Data-Efficient Tactile Adaptation for Unseen Sensors