cs.ROOct 17, 2026

Touch Where It Matters: Learning Local Tactile Policies for Fine-Grained Gripper Control

Authors: Mingyuan Ba, Yanan Zhou, James Zhao, Weiming Zhi

Organizations: School of Computer Science, The University of Sydney, Australia

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Visual imitation policies can organise task motion, yet contact-rich object use demands gripper adjustments beyond coarse opening and closing. We study fine-grained gripper control as a local feedback problem within ongoing task execution. Our approach assigns tactile feedback a dedicated control role during contact-rich task stages. We introduce ContactPolicy, a learned local controller that takes over the gripper while a visual base policy continues to control the arm. ContactPolicy combines tactile and force histories with current robot state to jointly predict gripper increments and their modulation, updating the gripper target from each new observation. An Intent policy manages control handover between the visual base policy and ContactPolicy. Real-world experiments on container opening, object handling, and tool operation show improved task success over both the Visuomotor Policy and Tactile-Force Policy baselines. Execution analysis illustrates how the gripper adjusts contact within an ongoing task, including periods of target retention and fine-grained changes as interaction conditions evolve. Arm-state ablations and failure analysis further examine how execution context supports local regulation. These results support task-local tactile policies for fine-grained gripper control within visual manipulation.

Figures & tables

Fig. 1: Touch where it matters. Visual arm control with local tactile gripper regulation. Photos and tactile examples are not time-synchronised.
Fig. 2: System overview. The visual base policy advances the arm; Intent assigns gripper control to the visual base policy or ContactPolicy.
Fig. 3: Gripper control handover. The visual base policy continues arm motion while Intent CLOSE activates local gripper regulation. Intent OPEN returns gripper control to the visual base policy after the operation, linking the local feedback stage to the surrounding task execution.
Fig. 4: Local tactile policy. Contact histories and current robot state condition an increment Δg^t\Delta\hat{g}_{t} and modulation decision mtm_{t} . Their product enters the gripper-command stage, which updates the target at 60 Hz.
Fig. 5: Action and modulation supervision. Demonstrated increments train predictions, while update/hold labels supervise the modulation head.
Fig. 6: Six contact-rich manipulation tasks. (a) Tube Opening, (b) Cup Separation, (c) Bag Clipping, (d) Lens Blowing, (e) Beaker Pouring, and (f) Petri Dispensing test fine-grained gripper control during container opening, object handling, and tool operation, with contact feedback during task motion.
Fig. 7: Beaker Pouring phases and contact signals. Gripper width (0–1) and force magnitude (N, 0.1 s means) accompany representative task frames. Red markers indicate phases; the first 5 s are compressed. Frames provide approximate phase correspondence; the inset enlarges the visible water stream.
Fig. 8: Bag Clipping contact stages. Frames show acquiring the clip, applying it to the bag, and releasing it. Paired tactile difference images from the gripper on the left reveal changing fingertip contact across these stages.
Fig. 9: Arm-state ablation. Success over 25 trials per task improves with arm-state inputs on Lens Blowing, Beaker Pouring, and Petri Dispensing.
Fig. 10: Premature actuation during task motion. Without arm-state input, the blower is squeezed flat before its nozzle reaches the lens; the beaker is squeezed before alignment with the receiving vessel is complete, causing premature water discharge; and liquid leaks from the dropper while it is being moved towards the dish.
More in this topic

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,685 papers

Explore similar work

CardsList

  1. What is the Better Curriculum: Controller-Shaped Grasping Behavior for Contact Force-Sensitive Manipulation

    Sep 22, 2026Ziyan Feng, Zizhao Yuan, Yulong Fu +6Robotic GraspingForce-Controlled Robotic Manipulation

    The Hong Kong University of Science and Technology (Guangzhou)

  2. Contact-Grounded Policy: Dexterous Visuotactile Policy with Generative Contact Grounding

    Mar 5, 2026Zhengtong Xu, Yeping Wang, Ben Abbatematteo +4Robotic ManipulationContact-Rich Robotic Manipulation

    Purdue University · University of Wisconsin–Madison · Meta Reality Labs Research

  3. Learning Force-Regulated Robotic Manipulation with a Low-Cost Tactile-Force-Controlled Gripper

    Feb 10, 2026Xuhui Kang, Tongxuan Tian, Sung-Wook Lee +3Robot Policy LearningRobotic Grasping

    Department of Computer Science, University of Virginia, Charlottesville, VA, USA. · Department of Computer Science, Columbia University, New York, NY, USA.