cs.RONov 18, 2026

TacPilot: Does Touch Need to Be Everywhere?

Authors: Yunzhen Zhang, Yuxiao Zhu, Jie Yang, Yufeng Miao, Bing Luo

Organizations: Duke Kunshan University · Helix Robotics

Abstract

Tactile feedback helps visual manipulation policies respond to local contact, but changes in tactile observations do not always require corrective motion, and sustained corrections do not always require new tactile features. We introduce TacPilot, a tactile residual framework that separates representation updates from action intervention around a frozen visual policy. TacPilot uses a perception gate to determine when tactile features need refreshing and updates only affected regions while reusing unchanged features. A separate intervention gate determines whether bounded Cartesian and gripper residuals modify the nominal visual action. This separation allows tactile perception and action correction to follow different spatial and temporal demands during manipulation. In offline expert-trajectory replay, local feature reuse reduces encoded regions by 56.4% relative to periodic full encoding, with essentially unchanged action predictions. In an exploratory simulation evaluation spanning two cylinder sizes, TacPilot succeeds in 26 of 30 trials (86.7%). We further validate TacPilot on OpenArm v2 equipped with Paxini tactile sensors across four contact-rich manipulation tasks.

Figures & tables

Explore similar work

CardsList
  1. PredTac: Learning Contact-Rich Manipulation with Predicted Touch

    Sep 14, 2026Weijia Fan, Daqiang GuoVisual-to-Tactile PredictionTactile Sensing

  2. TacPAC: Tactile Prediction and Real-Time Action Correction in World-Action Models for Contact-Rich Manipulation

    Sep 4, 2026Zipei Ma, Xiaofei Wei, Junzhe Jiang +2Contact-Rich Robotic ManipulationHaptic Feedback

  3. OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

    Jul 4, 2026Kelin Yu, Haode Zhang, Harish Ravichandar +2Robot Policy AdaptationRobot Skill Learning