Visuo-Tactile Representation Learning

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 43

All topics

TactX: Learning Shared Tactile Representations Across Diverse Sensors
Jun 30, 2026Junsung Park, Sachin Bhadang, Carmelo Sferrazza +2Cross-Modal Representation Learning Contact-Rich Robotic Manipulation
UC San Diego · Seoul National University · Amazon FAR
Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation
Jun 29, 2026Shengqi Xu, Guojin Zhong, Yang Liu +7Robot Policy Learning Contact-Rich Robotic Manipulation
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · NeoteAI +1
TacGen: Touch Is a Necessary Dimension of Physical-World Representation -- Addressing Tactile Data Scarcity with Scalable Vision-to-Touch Alignment and Generation
Jun 28, 2026Wanghao Ye, Aarosh Das, Sihan Chen +19Visuo-Tactile Representation Learning Multimodal Contrastive Learning
University of Maryland, College Park · Carnegie Mellon University · University of Alabama +2
HT-Bench: Benchmarking and Learning Dexterous Full-Hand Tactile Representations with Egocentric Vision
Jun 17, 2026Yuzhe Huang, Jiaping Wu, Jiaming Jiang +6Representation Learning Visuo-Tactile Representation Learning
Beihang University · Rimbot · BUPT +3
T-Rex: Tactile-Reactive Dexterous Manipulation
Jun 15, 2026Dantong Niu, Zhuoyang Liu, Zekai Wang +31Representation Learning Visuo-Tactile Representation Learning
UC Berkeley · NVIDIA · La Sapienza University +3
ContactWorld: What Representations Matter for Vision-Tactile Latent World Models in Contact-Rich Manipulation
Jun 11, 2026Zhiyuan Zhang, Pokuang Zhou, Kaidi Zhang +7Representation Learning World Models for Robotics
Department of Industrial Engineering, Purdue University, West Lafayette, IN 47907, USA · Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA
Tac-DINO: Learning Vision-Tactile Features with Patch Alignment
Jun 10, 2026Hong Li, Yankang Dong, Yue Xu +8Representation Learning Tactile Sensing
Shanghai Jiao Tong University · Ant Group · Shanghai Innovation Institute
Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning
Jun 10, 2026Shengcheng Luo, Xiyan Huang, Zhe Xu +3Robotic Grasping Representation Learning
1ShanghaiTech University · 2Beijing Institute for General Artificial Intelligence
TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representation
Jun 10, 2026Kailin Lyu, Di Wu, Pengwei Zhang +12Representation Learning Visuo-Tactile Representation Learning
Institute of Automation, Chinese Academy of Sciences · Zhongguancun Academy · National University of Singapore +4
AetheRock: An Arm-Worn Robot Teaching System for Force-Guided Vision-Tactile Learning
Jun 8, 2026Hong Li, Yue Xu, Yihan Tang +8Representation Learning Contact-Rich Robotic Manipulation
Shanghai Jiao Tong University · Beijing Institute for General Artificial Intelligence (BIGAI) · Ant Group +1
RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation
Jun 7, 2026Shengcheng Luo, Kefei Wu, Xiaoying Zhou +3Contact-Rich Robotic Manipulation Visuo-Tactile Representation Learning
ShanghaiTech University · Beijing Institute for General Artificial Intelligence
Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation
Jun 4, 2026Rickmer Krohn, Erik Helmut, Niklas Funk +3Tactile Sensing Contact-Rich Robotic Manipulation
Interactive Robot Perception & Learning, TU Darmstadt · Hessian AI · Robotics Institute Germany +1
Cross-Modal Visuo-Tactile Representation Learning with Action Chunking Transformers for Contact-Rich Manipulation
Jan 31, 2026Yaohua Liu, Rong Fu, Amir H. Gandomi +2Representation Learning Cross-Modal Representation Learning
Chinese Medicine Guangdong Laboratory, Hengqin, Zhuhai 519031, China · Institute of Collaborative Innovation, University of Macau, Macau 999078, China · Faculty of Engineering & IT, University of Technology Sydney, Ultimo, NSW 2007, Australia +3

Jun 30, 2026, cs.ROJ/K move · Enter open · S save

TactX: Learning Shared Tactile Representations Across Diverse Sensors

Junsung Park, Sachin Bhadang, Carmelo Sferrazza, Sha Yi+1

UC San Diego · Seoul National University · Amazon FAR

Cross-Modal Representation Learning Contact-Rich Robotic Manipulation Visuo-Tactile Representation Learning Generalization in Robotic Manipulation Cross-Embodiment Robot Learning Tactile Sensing for Robotic Manipulation

Open paper AskPDF arXiv

Tactile sensors provide critical information for contact-rich manipulation, yet tactile representations and policies remain tightly coupled to each specific sensor, limiting transferability across robots and hardware platforms. We propose TactX, a framework for learning a transferable tactile representation across sensors spanning three fundamentally different transduction modalities: resistive, magnetic, and vision-based. TactX maps heterogeneous tactile observations into a shared latent space through modality-specific encoders trained on paired contact data. Such paired interactions provide a natural alignment signal across modalities, and the encoders are jointly trained across all sensor pairs, inducing a consistent latent space for all sensor types. Our experiments show that TactX aligns tactile representations across sensors while preserving object-level contact information, as evidenced by sensor-identity prediction and object classification in the learned latent space. We evaluate TactX on four contact-rich manipulation tasks: pick-and-place, plug insertion, board wiping, and object reorientation, and show that policies trained with one sensor transfer zero-shot to physically distinct sensors through the shared latent. This improves the average success rate from 27.5% for vision-only policy to 45.9%, providing a step toward sensor-agnostic tactile manipulation.

Momentum

Related topics

Latest papers 43