Visuo-Tactile Representation Learning

Overview

Visuo-tactile representation learning studies how to combine images with tactile signals so robots can perceive contact, estimate physical properties, and act when vision alone is unreliable. Common approaches include contrastive alignment, spatial matching of contact points to visual geometry, cross-attention, and masked reconstruction of tactile or point-cloud features. Temporal and predictive models also learn how touch changes during interaction, often conditioning imitation-learning or vision-language-action policies; large datasets and benchmarks test transfer across tasks, objects, and sensors. A central challenge is making representations generalize across different sensors and unseen materials while determining when touch, predicted touch, or vision-touch fusion actually improves task performance.

Latest papers

CardsList

71 papers6 papers in the last four weeks, down 60% on the four weeks before. 0.0% of all new papers.

  1. TaF-Foundation: Wrench-Grounded Cross-Sensor Tactile Representations

    Nov 16, 2026Jiaming Jiang, Yuzhe Huang, Baokun Yang +8Robotic ManipulationRobot Manipulation Datasets

  2. BAT-WAM: Bridging Action and Touch in World-Action Models for Dexterous Hands

    Nov 7, 2026Dong Fang, Feixuan Huang, Yuanxin Zhong +5Robotic ManipulationTactile World Models

  3. MDF++: Visuo-Tactile World Action Model with End-to-end Representation Learning

    Nov 6, 2026Zixuan Huang, Jinhao Yu, Zachary Charlick +1World Models for RoboticsRobotic Manipulation

  4. TouchAnnotator: Multi-Source Tactile Supervision with Privileged-View Distillation

    Nov 6, 2026Ao Li, Zhihao Yuan, Chubin Zhang +6Visual-to-Tactile PredictionVisuo-Tactile Representation Learning

  5. Which Physical Tactile Cues Matter for Contact-Rich Imitation Learning? A Controlled Real-Robot Study

    Nov 4, 2026Xuhao Qin, Xiyan Huang, Haoyu Dong +2Vision-Based Tactile SensingContact-Rich Robotic Manipulation

  6. Touch-JEPA: Tactile Representation Learning from Hand-Object Video with Limited Tactile Supervision

    Oct 20, 2026Minseong Kweon, Qiao Feng, Yiming Huang +1Visuo-Tactile Representation LearningHand-Object Interaction

  7. Ego2Contact: Predicting Touch from Egocentric Human Video for Contact-Rich Robot Manipulation

    Oct 16, 2026Qianqian Yang, Dayou Li, Bangya Liu +11Visual-to-Tactile PredictionVision-Based Tactile Sensing

  8. OpenViTac: Learning and Benchmarking Visuo-Tactile Policies in a Unified Sim-and-Real Framework

    Oct 7, 2026Yifan Wu, Qin Li, Nan Min +14Robot Manipulation BenchmarksTactile Sensing for Robotic Manipulation

  9. TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning

    Oct 7, 2026Dayou Li, Hao Wang, Qianqian Yang +23Visual-to-Tactile PredictionVisuo-Tactile Representation Learning

  10. Equivariant Visual-Tactile Diffusion Policy for Contact-Rich Manipulation

    Oct 2, 2026Lik Hang Kenny Wong, Yiyao Ma, Xiu-Shen Wei +5Robot Imitation LearningRobot Learning

  11. TacDyn-WAM: Learning Implicit Tactile Dynamics in a Heterogeneous Visuo-Tactile World Action Model

    Sep 30, 2026Enyi Wang, Mingxin Wang, Quan Shi +12World Models for RoboticsVisuo-Tactile Representation Learning

  12. OccluDex: Hierarchical 3D Visuo-Tactile Representation Learning for Egocentric Dexterous Manipulation under Self-Occlusion

    Sep 30, 2026Ziheng Xu, Yueyuan Chen, Xinyuan He +6Visuo-Tactile Representation LearningGeneralization in Robotic Manipulation

  13. What to Attend, What to Keep: Skill-Conditioned Visuotactile Representation with Progress-Guided Event Memory

    Sep 29, 2026Amir-Hossein Shahidzadeh, Seungjae Lee, Eadom Dessalene +5Robot Task Progress EstimationContact-Rich Robotic Manipulation

  14. Uni-VLaT: Whole-Body Tactile Adaptation of VLA Policies for Humanoid Loco-Manipulation

    Sep 28, 2026Zihao Wang, Shutong Liu, Siqi Zheng +5Humanoid Loco-ManipulationTactile Sensing

  15. Unified Visual-Tactile-Action Modeling from Human Demonstrations for Dexterous Manipulation

    Sep 28, 2026Wenqiao Li, Qianyou Zhao, Jiawen Hao +5Contact-Rich Robotic ManipulationVisuo-Tactile Representation Learning

  16. TACTIC: Understanding Tactile Encoders and Conditioning for Contact-rich Robot Manipulation Policies

    Sep 25, 2026Seongjin Bien, Débora Oliveira Makowski, Carlo Kneissl +6Visuo-Tactile Representation LearningVision-Based Tactile Sensing

  17. VisTacAlign: Co-Training Dexterous Policies on Tactile Human and Robot Demonstrations using Visual-Tactile Alignment

    Sep 25, 2026Julien Poffet, Matthew Strong, Ankush Dhawan +5Robotic ManipulationRobot Imitation Learning

  18. CableVLA: Simulation-Privileged Global-Local Representation Learning for Cable Routing

    Sep 22, 2026Zhifei Teng, Bo Feng, Xiang Zou +4Visuo-Tactile Representation LearningSim-to-Real Transfer

  19. DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation

    Sep 21, 2026Haoran Yuan, Zekai Wang, Boning Shao +4World Models for RoboticsContact-Rich Robotic Manipulation

  20. HapticWAM: Distilling Imagined Touch into a World-Action Model without Inference-Time Tactile Sensing

    Sep 20, 2026Mikhail Sannikov, Ilya Mikhalchuk, Konstantin Gubernatorov +3Visual-to-Tactile PredictionWorld Models for Robotics

  21. Agile-WAM: An Agile Tactile World Action Model for Contact-Rich Robot Control

    Sep 17, 2026Hanchu Zhou, Brendan Lynch, Raman Goyal +4Robotic ControlWorld Models for Robotics

  22. DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination

    Sep 8, 2026Yankai Fu, Ning Chen, Junkai Zhao +5Contact-Rich Robotic ManipulationVisuo-Tactile Representation Learning