TacBridge: Learning Unified Tactile Representations through Cross-Sensor Correspondence
Organizations: Department of Engineering Mathematics, University of Bristol, Bristol, BS16 1QY, United Kingdom. · OpenDIM Lab, School of Aerospace Engineering, Beijing Institute of Technology, Beijing, China. · Department of Computer Science, Cardiff University, Cardiff, CF10 3AT, United Kingdom. · Department of Computer Science, University of Leeds, Leeds, LS2 9JT, United Kingdom. ·
Abstract
Optical tactile sensors record tactile videos that capture local deformation and its temporal evolution during contact, supporting robot manipulation, force estimation, and tactile representation learning. Differences in imaging geometry, background appearance, illumination, resolution, and deformation response lead to sensor-dependent observations of the same contact across GelSight, DIGIT, GelSlim, TacTip, and other optical tactile sensors. This variability limits representation transfer and the joint use of multi-sensor data. We introduce TacBridge, a framework for learning unified representations from multi-sensor tactile videos through cross-sensor correspondence. Specifically, multi-sensor masked tactile-video pre-training learns spatiotemporal representations across sensors. We then train the same masked model jointly on same-sensor and cross-sensor reconstruction, using paired source and target videos of matched contact events. Experiments on cross-sensor translation and downstream perception demonstrate the cross-sensor generalization and task transferability of the learned tactile representations.
Figures & tables
| Model | TAG GS | Cloth GS | Pose DG | Slip F1 DG | Force Change RMSE DG | Slip F1 Mini | Force Change RMSE Mini |
|---|---|---|---|---|---|---|---|
| CLIP | 51.65 | 26.76 | 54.54 | 33.13 | 174.39 | 85.47 | 177.67 |
| VJEPA (S) | 54.67 | 18.66 | 55.09 | 83.33 | 105.63 | 97.00 | 121.31 |
| MAE (S) | 59.47 | 19.40 | 55.92 | 83.30 | 98.33 | 97.50 | 102.64 |
| AnyTouch | 80.82 | 24.34 | 56.22 | 40.60 | 169.42 | 88.92 | 162.41 |
| TacBridge Encoder (ours) | 73.13 | 35.65 | 63.79 | 85.77 | 91.26 | 97.90 | 104.56 |
| Cross-sensor translation | Same-sensor recon. | ||||||
|---|---|---|---|---|---|---|---|
| Encoder | PSNR | cPSNR | cPSNR | rFID-r | PSNR | cPSNR | rFID-r |
| CLIP | 29.23 | 24.78 | 8.70 | 85.2 | 29.30 | 24.86 | 87.3 |
| VJEPA (S) | 29.28 | 24.99 | 8.78 | 84.8 | 29.38 | 25.14 | 87.4 |
| MAE (S) | 30.16 | 25.99 | 10.32 | 71.9 | 30.26 | 26.14 | 73.9 |
| AnyTouch | 30.52 | 26.30 | 10.30 | 63.5 | 30.69 | 26.46 | 65.8 |
| TacBridge (ours) | 33.23 | 28.83 | 11.93 | 58.3 | 33.38 | 28.98 | 60.1 |
| Encoder | OOD (unseen fabrics) | ID (seen fabrics) |
|---|---|---|
| CLIP [ 35 ] | 67.1 | 91.3 |
| VJEPA (S) | 69.1 | 92.2 |
| MAE (S) | 67.1 | 94.5 |
| AnyTouch [ 8 ] | 70.2 | 92.7 |
| TacBridge (linear) | 73.0 | 95.5 |
| TacBridge (attentive pooling) | 74.2 | 99.0 |
Tactile Sensing
Tactile sensing measures contact forces, deformation, texture, and motion to help robots perceive and act through physical interaction.
See all 155 papers