Tactile-Guided Robot Learning: From Representation Learning to Foundation Models
Organizations: School of Computer Science, University College London · MIT CSAIL, Massachusetts Institute of Technology · Department of Robotics, University of Michigan · School of Engineering, University of Tokyo
Abstract
Tactile sensing is becoming central to robust, adaptive, and dexterous robotic manipulation. Beyond its traditional role in contact detection and reactive control, touch is increasingly used to learn transferable representations, condition manipulation policies, and support embodied physical reasoning. Yet existing surveys largely emphasize tactile hardware and perception, leaving the rapid shift toward learning-based tactile manipulation insufficiently synthesized. This paper reviews tactile-guided skill learning from a learning-centric perspective, organized by the functional role of touch across the learning and control pipeline. We examine tactile representation and policy learning, tactile foundation models, predictive and adaptive control, and generalization across objects, tasks, sensors, embodiments, and simulation-to-real transfer. Across these directions, we identify common design principles and remaining challenges in learning transferable physical representations, integrating semantic reasoning with contact-level control, and scaling tactile learning across heterogeneous data and hardware. Finally, we outline emerging directions toward tactile foundation and world models that can predict, reason about, and adapt physical interactions. Together, this review characterizes the transition of touch from a task-specific feedback signal toward a core modality for general-purpose embodied intelligence. An accompanying database of the reviewed literature and associated resources is available at https://sites.google.com/view/touch-in-robot-learning-review.
Figures & tables
| Learning Paradigm | Optimization Objective | Tactile Integration | Typical Architectures | Typical Strengths | Main Limitations |
|---|---|---|---|---|---|
| Imitation Learning ( Guzey et al., 2023 ) ( Xu et al., 2025a ) ( Huang et al., 2024a ) ( Zhao et al., 2026a ) ( Wi et al., 2025 ) ( Zhao et al., 2025a ) | Behavior cloning from expert demonstrations | - Feature fusion - Pretrained tactile representations - Latent conditioning | - Behavior Cloning - ACT - Transformer - Diffusion Policy - Nearest-Neighbor Retrieval | Data-efficient learning, stable training, precise contact behaviors | Limited to demonstrated behaviors, weak recovery from unseen failures |
| Reinforcement Learning ( Yin et al., 2025a ) ( Tang et al., 2025 ) ( Ye et al., 2026 ) ( Field et al., 2025 ) ( Church et al., 2022 ) | Reward maximization through interaction | - State representation - Reward shaping - Teacher-student learning - Latent representations | - SAC - PPO - Actor-Critic - Teacher-Student Policies | Adaptive online interaction, robust recovery, force regulation | Sample inefficiency, reward engineering, sim-to-real challenges |
| Diffusion Policies ( Xue et al., 2025 ) ( Yin et al., 2025b ) ( Wu et al., 2025b ) ( Lin et al., 2025 ) ( Zhu et al., 2026b ) | Conditional generative action modeling | - Observation conditioning - Latent conditioning - Reactive refinement | - DDPM - Latent Diffusion - Diffusion Policy - U-Net - Transformer | Multi-modal action generation, smooth trajectories, uncertainty modeling | Large demonstration datasets, iterative sampling, high computational cost |
| Visuo-Tactile Fusion | Joint multimodal policy learning | - Early/late fusion - cross-attention - token fusion - geometric fusion - adaptive gating | - PointNet - Cross-Attention - Token Fusion - Gated Fusion - Multimodal Transformers | Robust perception under occlusion and contact uncertainty | Fusion strategy remains task-dependent and architecture-specific |
| Model | Foundation Backbone | Tactile Encoder | Fusion | Action Generation | Primary Role | Generalization |
|---|---|---|---|---|---|---|
| TLA ( Hao et al., 2026 ) | Qwen2-VL | ViT | Language-conditioned tactile token fusion | Autoregressive | Foundation policy | Zero-shot objects |
| Tactile-VLA ( Huang et al., 2025b ) | + Gemma | MLP | Token-level multimodal fusion + force prediction | Flow Matching | Foundation policy + CoT | Objects, force-sensitive tasks |
| TaF-VLA ( Huang et al., 2026b ) | ViT + causal Transformer | Force-aligned latent representation (InfoNCE) | Flow Matching | Force-aware policy | Cross-sensor, objects | |
| Octopi-1.5 ( Yu et al., 2025 ) | Qwen2-VL | CLIP | Retrieval-Augmented Generation (RAG) | – | Tactile reasoning | Object retrieval |
| VTLA ( Zhang et al., 2026a ) | Qwen2-VL | ViT | Vision-guided temporally enhanced tactile tokens | Autoregressive + DPO | Foundation policy | Zero-shot Sim2Real |
| TacVLA ( Zhang et al., 2026b ) | MLP | Contact-aware tactile gating | Flow Matching | Adaptive foundation policy | Visual disturbances |
| Generalization Type | Transfer Scenario | Representative Strategies | Representative Methods | Current Progress | Remaining Challenges |
|---|---|---|---|---|---|
| Object | Novel object geometry, size, material, friction, compliance | Large-scale pretraining, multimodal representations, tactile foundation models | TLA ( Hao et al., 2026 ) , Tactile-VLA ( Huang et al., 2025b ) , TacDiffusion ( Huang et al., 2026b ) , Touch in the Wild ( Zhu et al., 2026b ) | High | Large appearance and contact distribution shifts |
| Manipulation Task | Different manipulation skills | Multitask learning, foundation models, transferable tactile representations | T3 ( Zhao et al., 2024 ) , TaF-VLA ( Huang et al., 2026b ) , Touch in the Wild ( Zhu et al., 2026b ) | Moderate | Transfer across different manipulation behaviors |
| Tactile Sensor | Different tactile sensing technologies | Sensor-invariant representations, self-supervised learning, cross-modal alignment | Sparsh ( Higuera et al., 2024 ) , FTP-1 ( Yuan et al., 2026 ) , TaF-VLA ( Huang et al., 2026b ) | Moderate | Different sensing principles and morphologies |
| Robot Embodiment | Different robot hands, grippers and manipulators | Cross-embodiment representation learning, latent alignment | TactAlign ( Wi et al., 2025 ) , FTP-1 ( Yuan et al., 2026 ) | Limited | Different kinematics and control spaces |
| Sim-to-Real | Simulation real robot | Domain randomization, teacher-student learning, latent adaptation, fine-tuning | Reactive Diffusion Policy ( Xue et al., 2025 ) , PPO-based methods ( Yin et al., 2025a ) , Touch in the Wild ( Zhu et al., 2026b ) | Moderate | Realistic contact mechanics remain difficult to simulate |
| Environment | Lighting, clutter, disturbances, object placement, contact uncertainty | Robust multimodal fusion, online adaptation, large-scale pretraining | VITAL ( George et al., 2024 ) | Moderate | Real-world deployment under changing environments |
Robot Skill Learning
Robot skill learning studies how robots acquire, adapt, and combine reusable behaviors from demonstrations, interaction, and other data.
See all 320 papers