Visuo-Tactile Representation Learning

Visuo-tactile representation learning studies how to combine images with tactile signals so robots can perceive contact, estimate physical properties, and act when vision alone is unreliable. Common approaches include contrastive alignment, spatial matching of contact points to visual geometry, cross-attention, and masked reconstruction of tactile or point-cloud features. Temporal and predictive models also learn how touch changes during interaction, often conditioning imitation-learning or vision-language-action policies; large datasets and benchmarks test transfer across tasks, objects, and sensors. A central challenge is making representations generalize across different sensors and unseen materials while determining when touch, predicted touch, or vision-touch fusion actually improves task performance.