cs.ROOct 14, 2026

Tactile-Guided Robot Learning: From Representation Learning to Foundation Models

Authors: Kiyanoush Nazari, Luchen Li, Haoshu Fang, Nima Fazeli, Zhibin Li, Fumiya Iida, Thomas George Thuruthel

Organizations: School of Computer Science, University College London · MIT CSAIL, Massachusetts Institute of Technology · Department of Robotics, University of Michigan · School of Engineering, University of Tokyo

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Tactile sensing is becoming central to robust, adaptive, and dexterous robotic manipulation. Beyond its traditional role in contact detection and reactive control, touch is increasingly used to learn transferable representations, condition manipulation policies, and support embodied physical reasoning. Yet existing surveys largely emphasize tactile hardware and perception, leaving the rapid shift toward learning-based tactile manipulation insufficiently synthesized. This paper reviews tactile-guided skill learning from a learning-centric perspective, organized by the functional role of touch across the learning and control pipeline. We examine tactile representation and policy learning, tactile foundation models, predictive and adaptive control, and generalization across objects, tasks, sensors, embodiments, and simulation-to-real transfer. Across these directions, we identify common design principles and remaining challenges in learning transferable physical representations, integrating semantic reasoning with contact-level control, and scaling tactile learning across heterogeneous data and hardware. Finally, we outline emerging directions toward tactile foundation and world models that can predict, reason about, and adapt physical interactions. Together, this review characterizes the transition of touch from a task-specific feedback signal toward a core modality for general-purpose embodied intelligence. An accompanying database of the reviewed literature and associated resources is available at https://sites.google.com/view/touch-in-robot-learning-review.

Figures & tables

Fig. 1: Scope of tactile-guided skill learning for robotic manipulation. We organize the literature according to the functional role of touch, spanning tactile representation learning, tactile-conditioned policies, tactile-informed foundation and world models, predictive and adaptive control, and generalization across objects, tasks, sensors, embodiments, environments, and the simulation-to-real gap.
Fig. 2: Self-supervised representation learning paradigms for tactile sensing. Reconstruction-based methods (a-d) recover a masked or transformed observation; contrastive methods (e-h) pull positive pairs together in a shared latent space while separating negatives; non-contrastive and predictive methods (i-k) match or predict latent targets without explicit negatives.
Learning ParadigmOptimization ObjectiveTactile IntegrationTypical ArchitecturesTypical StrengthsMain Limitations
Imitation Learning ( Guzey et al., 2023 ) ( Xu et al., 2025a ) ( Huang et al., 2024a ) ( Zhao et al., 2026a ) ( Wi et al., 2025 ) ( Zhao et al., 2025a )Behavior cloning from expert demonstrations- Feature fusion - Pretrained tactile representations - Latent conditioning- Behavior Cloning - ACT - Transformer - Diffusion Policy - Nearest-Neighbor RetrievalData-efficient learning, stable training, precise contact behaviorsLimited to demonstrated behaviors, weak recovery from unseen failures
Reinforcement Learning ( Yin et al., 2025a ) ( Tang et al., 2025 ) ( Ye et al., 2026 ) ( Field et al., 2025 ) ( Church et al., 2022 )Reward maximization through interaction- State representation - Reward shaping - Teacher-student learning - Latent representations- SAC - PPO - Actor-Critic - Teacher-Student PoliciesAdaptive online interaction, robust recovery, force regulationSample inefficiency, reward engineering, sim-to-real challenges
Diffusion Policies ( Xue et al., 2025 ) ( Yin et al., 2025b ) ( Wu et al., 2025b ) ( Lin et al., 2025 ) ( Zhu et al., 2026b )Conditional generative action modeling- Observation conditioning - Latent conditioning - Reactive refinement- DDPM - Latent Diffusion - Diffusion Policy - U-Net - TransformerMulti-modal action generation, smooth trajectories, uncertainty modelingLarge demonstration datasets, iterative sampling, high computational cost
Visuo-Tactile FusionJoint multimodal policy learning- Early/late fusion - cross-attention - token fusion - geometric fusion - adaptive gating- PointNet - Cross-Attention - Token Fusion - Gated Fusion - Multimodal TransformersRobust perception under occlusion and contact uncertaintyFusion strategy remains task-dependent and architecture-specific
TABLE I: Comparison of the major learning paradigms for tactile-conditioned policy learning.
ModelFoundation BackboneTactile EncoderFusionAction GenerationPrimary RoleGeneralization
TLA ( Hao et al., 2026 )Qwen2-VLViTLanguage-conditioned tactile token fusionAutoregressiveFoundation policyZero-shot objects
Tactile-VLA ( Huang et al., 2025b )π0\pi_{0} + GemmaMLPToken-level multimodal fusion + force predictionFlow MatchingFoundation policy + CoTObjects, force-sensitive tasks
TaF-VLA ( Huang et al., 2026b )π0.5\pi_{0.5}ViT + causal TransformerForce-aligned latent representation (InfoNCE)Flow MatchingForce-aware policyCross-sensor, objects
Octopi-1.5 ( Yu et al., 2025 )Qwen2-VLCLIPRetrieval-Augmented Generation (RAG)–Tactile reasoningObject retrieval
VTLA ( Zhang et al., 2026a )Qwen2-VLViTVision-guided temporally enhanced tactile tokensAutoregressive + DPOFoundation policyZero-shot Sim2Real
TacVLA ( Zhang et al., 2026b )π0.5\pi_{0.5}MLPContact-aware tactile gatingFlow MatchingAdaptive foundation policyVisual disturbances
TABLE II: Comparison of recent tactile foundation and world models for embodied robotic manipulation.
Fig. 3: Representative architectures for integrating tactile information into foundation models for robotic manipulation. The paradigms include (a) direct multimodal conditioning, where tactile tokens are fused with vision, language, and proprioception within a VLM/VLA backbone; (b) semantic alignment of tactile representations with vision and language prior to policy conditioning; (c) physical grounding of tactile features through force-aligned representations; (d) predictive architectures that use tactile world models to generate future visuo-tactile interaction states for action generation; and (e) tactile-augmented reasoning, where tactile observations and retrieved interaction memories condition a VLM/VLA for semantic understanding and decision making.
Generalization TypeTransfer ScenarioRepresentative StrategiesRepresentative MethodsCurrent ProgressRemaining Challenges
ObjectNovel object geometry, size, material, friction, complianceLarge-scale pretraining, multimodal representations, tactile foundation modelsTLA ( Hao et al., 2026 ) , Tactile-VLA ( Huang et al., 2025b ) , TacDiffusion ( Huang et al., 2026b ) , Touch in the Wild ( Zhu et al., 2026b )HighLarge appearance and contact distribution shifts
Manipulation TaskDifferent manipulation skillsMultitask learning, foundation models, transferable tactile representationsT3 ( Zhao et al., 2024 ) , TaF-VLA ( Huang et al., 2026b ) , Touch in the Wild ( Zhu et al., 2026b )ModerateTransfer across different manipulation behaviors
Tactile SensorDifferent tactile sensing technologiesSensor-invariant representations, self-supervised learning, cross-modal alignmentSparsh ( Higuera et al., 2024 ) , FTP-1 ( Yuan et al., 2026 ) , TaF-VLA ( Huang et al., 2026b )ModerateDifferent sensing principles and morphologies
Robot EmbodimentDifferent robot hands, grippers and manipulatorsCross-embodiment representation learning, latent alignmentTactAlign ( Wi et al., 2025 ) , FTP-1 ( Yuan et al., 2026 )LimitedDifferent kinematics and control spaces
Sim-to-RealSimulation →\rightarrow real robotDomain randomization, teacher-student learning, latent adaptation, fine-tuningReactive Diffusion Policy ( Xue et al., 2025 ) , PPO-based methods ( Yin et al., 2025a ) , Touch in the Wild ( Zhu et al., 2026b )ModerateRealistic contact mechanics remain difficult to simulate
EnvironmentLighting, clutter, disturbances, object placement, contact uncertaintyRobust multimodal fusion, online adaptation, large-scale pretrainingVITAL ( George et al., 2024 )ModerateReal-world deployment under changing environments
TABLE III: Taxonomy of generalization in tactile-conditioned manipulation policies.
More in this topic

Robot Skill Learning

Robot skill learning studies how robots acquire, adapt, and combine reusable behaviors from demonstrations, interaction, and other data.

See all 319 papers

Explore similar work

CardsList
Oct 21, 2026

Tactile-Based Robotic Manipulation: From Mechanics-Driven to Data-Driven Perspectives

Tactile sensing provides robots with direct access to physical interactions at the contact interface, making it a critical modality for robust and dexterous manipulation. Over the past several decades, tactile-based manipulation research has evolved from mechanics-driven approaches, which explicitly interpret tactile signals through contact mechanics and task-specific reasoning, to data-driven approaches that learn tactile representations and manipulation policies directly from data. This paper reviews the development of tactile-based manipulation from these two complementary perspectives. We first introduce the fundamentals of tactile sensing, and then review, from a mechanics-driven perspective, how those signals form manipulation-relevant state representations and subsequently influence the robot action. We then examine the recent advances in data-driven tactile manipulation, covering imitation learning, reinforcement learning, vision-language-action models, and predictive models. Finally, we discuss open challenges and future opportunities toward scalable, generalizable, and robust tactile-enabled robotic systems.

Jingyi Xiang, Yixuan Wang, Wenzhen Yuan

Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, Illinois, USA, 61801

Tactile Sensing for Robotic ManipulationRobotic Tactile Sensing
AskPDF
Jun 30, 2026

TactX: Learning Shared Tactile Representations Across Diverse Sensors

Tactile sensors provide critical information for contact-rich manipulation, yet tactile representations and policies remain tightly coupled to each specific sensor, limiting transferability across robots and hardware platforms. We propose TactX, a framework for learning a transferable tactile representation across sensors spanning three fundamentally different transduction modalities: resistive, magnetic, and vision-based. TactX maps heterogeneous tactile observations into a shared latent space through modality-specific encoders trained on paired contact data. Such paired interactions provide a natural alignment signal across modalities, and the encoders are jointly trained across all sensor pairs, inducing a consistent latent space for all sensor types. Our experiments show that TactX aligns tactile representations across sensors while preserving object-level contact information, as evidenced by sensor-identity prediction and object classification in the learned latent space. We evaluate TactX on four contact-rich manipulation tasks: pick-and-place, plug insertion, board wiping, and object reorientation, and show that policies trained with one sensor transfer zero-shot to physically distinct sensors through the shared latent. This improves the average success rate from 27.5% for vision-only policy to 45.9%, providing a step toward sensor-agnostic tactile manipulation.

Junsung Park, Sachin Bhadang, Carmelo Sferrazza +2

UC San Diego · Seoul National University · Amazon FAR

Cross-Modal Representation LearningContact-Rich Robotic ManipulationVisuo-Tactile Representation LearningGeneralization in Robotic ManipulationCross-Embodiment Robot LearningTactile Sensing for Robotic Manipulation
AskPDF
Jun 11, 2026

FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation

Despite the success of vision-based generalist robotic policies, existing tactile-based policies remain tied to fixed embodiments and sensor setups. This is because tactile signals are highly heterogeneous across hardware, making cross-sensor generalization difficult. We present FTP-1,the first generalist foundation tactile policy pretrained to acquire transferable tactile manipulation abilities across diverse sensors and embodiments. FTP-1 supports varied tactile inputs, including image-, array-, and state-based signals, by using heterogeneous encoders to project them into unified morphology-aware latent tokens that are jointly modeled by a shared tactile Transformer expert. Pretrained on around 3,000 hours of tactile manipulation data aggregated from 26 data sources, spanning human and robot demonstrations across 21 sensors, FTP-1 learns tactile skills that transfer beyond the sensors seen during pretraining. Across downstream finetuning experiments spanning 5 hardware configurations, FTP-1 improves contact-rich manipulation on seen sensor setups by +17.2% and, surprisingly, transfers to two previously unseen tactile-sensor setups, achieving a +31% gain in success rate. FTP-1 establishes the first unified foundation baseline for tactile manipulation, providing future tactile policies with a shared model-level starting point. Pretrained models, datasets, training code and more visualization at https://ftp1-policy.github.io.

Chengbo Yuan, Zicheng Zhang, Mingjie Zhou +14

Tsinghua University · Shanghai Qi Zhi Institute · Sharpa +5

Robot Skill LearningContact-Rich Robotic ManipulationRobot Foundation ModelsGeneralization in Robotic ManipulationCross-Embodiment Robot Learning
AskPDF