Master Micro Residual Correction with Adaptive Tactile Fusion and Force-Mixed Control for Contact-Rich Manipulation

Xingting Li; Yifan Xie; Han Liu; Wei Hou; Guangyu Chen; Shoujie Li; Wenbo Ding

cs.ROMar 16, 2026

Master Micro Residual Correction with Adaptive Tactile Fusion and Force-Mixed Control for Contact-Rich Manipulation

Authors: Xingting Li, Yifan Xie, Han Liu, Wei Hou, Guangyu Chen, Shoujie Li, Wenbo Ding

Organizations: Shenzhen Ubiquitous Data Enabling Key Lab, Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China. · School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China. · School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639956, Singapore.

Abstract

Robotic contact-rich and fine-grained manipulation remains a significant challenge due to complex interaction dynamics and the competing requirements of multi-timescale control. While current visual imitation learning methods excel at long-horizon planning, they often fail to perceive critical interaction cues like friction variations or incipient slip, and struggle to balance global task coherence with local reactive feedback. To address these challenges, we propose M2-ResiPolicy, a novel Master-Micro residual control architecture that synergizes high-level action guidance with low-level correction. The framework consists of a Master-Guidance Policy (MGP) operating at 10 Hz, which generates temporally consistent action chunks via a diffusion-based backbone and employs a tactile-intensity-driven adaptive fusion mechanism to dynamically modulate perceptual weights between vision and touch. Simultaneously, a high-frequency (60 Hz) Micro-Residual Corrector (MRC) utilizes a lightweight GRU to provide real-time action compensation based on TCP wrench feedback. This policy is further integrated with a force-mixed PBIC execution layer, effectively regulating contact forces to ensure interaction safety. Experiments across several demanding tasks including fragile object grasping and precision insertion, demonstrate that M2-ResiPolicy significantly outperforms standard Diffusion Policy (DP) and state-of-the-art Reactive Diffusion Policy (RDP), achieving a 93% damage-free success rate in chip grasping and superior force regulation stability.

Figures & tables

Fig. 1 : M 2 -ResiPolicy Framework. Our dual-stream architecture decouples global planning (MGP) from high-frequency local correction (MRC), unified by a force-mixed PBIC execution layer.

Fig. 2 : Overview of the proposed M 2 -ResiPolicy framework. Left: The 10 Hz Master-Guidance Policy encodes multimodal observations and generates temporally coherent action chunks via confidence-gated cross-attention fusion, while the 60 Hz Micro-Residual Corrector predicts residual compensation from aligned TCP wrench feedback. Right: The reference pose $x_{\mathrm{ref}}$ is executed through the force-mixed PBIC layer at 125 Hz to produce a compliant pose command $x_{\mathrm{cmd}}$ for stable contact interaction.

Fig. 3 : System setup and data streams. A teleoperated UR7e platform collects visuotactile, RGB, pose, and TCP wrench data. Visuotactile signals, RGB, and end-effector pose are sampled at 10 Hz, while TCP wrench is sampled at 60 Hz.

Fig. 4 : Evaluation tasks. Key stages of four benchmarks. From top to bottom: Chip Transfer , Plug Insertion , Whiteboard Wiping , and Block Assembly . Each row illustrates a typical execution sequence, with fingertip visuotactile observations visualized in the corner to reflect local deformations and interaction-state changes during contact.

Fig. 5 : Baseline Architectures. We compare our hierarchical MGP+MRC framework against the open-loop Diffusion Policy (DP) and the end-to-end Reactive Diffusion Policy (RDP).

Method	Chip Transfer			Surface Wiping			Plug Insertion			Block Assembly			Score
Method	Pick	Move	Place	Pick	Contact	Wipe	Grasp	Align	Insert	Grasp	Align	Stack	Avg.
DP (Baseline)	43%	40%	37%	80%	67%	63%	70%	63%	43%	57%	53%	43%	0.55
DP w. tactile	63%	57%	53%	83%	73%	70%	77%	70%	53%	67%	60%	50%	0.65
RDP (Tactile)	90%	87%	83%	97%	93%	90%	90%	87%	73%	93%	87%	77%	0.87
RDP (Force)	70%	63%	60%	97%	93%	93%	87%	87%	80%	87%	83%	80%	0.82
M 2 -ResiPolicy	100%	97%	93%	97%	97%	97%	93%	93%	80%	97%	93%	87%	0.94

TABLE I : Policy Performance Across Task-Specific Execution Stages (Success Rate)

Fig. 6 : Failure Analysis. (Top) Baseline failures: Perception errors cause crushing/slippage; Control rigidity leads to force stops/jamming. (Bottom) Force profiles contrast unsafe baseline spikes (Red) with our stable regulation (Blue).

Variants	Chip Transfer	Surface Wiping	Plug Insertion	Block Assembly
Full Model (Ours)	93%	97%	80%	87%
w/o Adaptive Fusion	80% ( $\downarrow$ 13)	93% ( $\downarrow$ 04)	73% ( $\downarrow$ 07)	77% ( $\downarrow$ 10)
w/o MRC (Residual)	90% ( $\downarrow$ 03)	93% ( $\downarrow$ 04)	67% ( $\downarrow$ 13)	70% ( $\downarrow$ 17)
w/o Impedance (Rigid)	87% ( $\downarrow$ 06)	90% ( $\downarrow$ 07)	70% ( $\downarrow$ 10)	77% ( $\downarrow$ 10)

TABLE II : Ablation Analysis: Success Rate Drops

Fig. 7 : Zero-Shot Generalization Scenarios.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,685 papers

Tube Diffusion Policy: Reactive Visual-Tactile Policy Learning for Contact-rich Manipulation
Apr 26, 2026Teng Xue, Alberto Rigo, Bingjian Huang +4Robot Policy Learning Contact-Rich Robotic Manipulation
Meta Reality Labs Research · Idiap Research Institute · École Polytechnique Fédérale de Lausanne (EPFL) +2
Force Policy: Learning Hybrid Force-Position Control Policy under Interaction Frame for Contact-Rich Manipulation
Feb 25, 2026Hongjie Fang, Shirun Tang, Mingyu Mei +9Contact-Rich Robotic Manipulation Force-Controlled Robotic Manipulation
Noematrix · Shanghai Jiao Tong University · Zhejiang University +2
Touch Where It Matters: Learning Local Tactile Policies for Fine-Grained Gripper Control
Oct 17, 2026Mingyuan Ba, Yanan Zhou, James Zhao +1Robotic Manipulation Contact-Rich Robotic Manipulation
School of Computer Science, The University of Sydney, Australia

Apr 26, 2026J/K move · Enter open · S save

Tube Diffusion Policy: Reactive Visual-Tactile Policy Learning for Contact-rich Manipulation

Teng Xue, Alberto Rigo, Bingjian Huang, Jiayi Shen+3

Meta Reality Labs Research · Idiap Research Institute · École Polytechnique Fédérale de Lausanne (EPFL)+2

Robot Policy Learning Contact-Rich Robotic Manipulation Robust Control Diffusion Policy Imitation Learning

Open paper AskPDF arXiv

Contact-rich manipulation is central to many everyday human activities, requiring continuous adaptation to contact uncertainty and external disturbances through multi-modal perception, particularly vision and tactile feedback. While imitation learning has shown strong potential for learning complex manipulation behaviors, most existing approaches rely on action chunking, which fundamentally limits their ability to react to unforeseen observations during execution. This limitation becomes especially critical in contact-rich scenarios, where physical uncertainty and high-frequency tactile feedback demand rapid, reactive control. To address this challenge, we propose Tube Diffusion Policy (TDP), a novel reactive visual-tactile policy learning framework that bridges diffusion-based imitation learning with tube-based feedback control. By leveraging the expressive power of generative models, TDP learns an observation-conditioned feedback flow around nominal action chunks, forming an action tube that enables fast and adaptive reactions during execution. We evaluate TDP on the widely used Push-T benchmark and three additional challenging visual-tactile dexterous manipulation tasks. Across all benchmarks, TDP consistently outperforms state-of-the-art imitation learning baselines. Two real-world experiments further validate its robust reactivity under contact uncertainty and external disturbances. Moreover, the step-wise correction mechanism enabled by action tube significantly reduces the required denoising steps, making TDP well suited for real-time, high-frequency feedback control in contact-rich manipulation.

Abstract

Robotic Manipulation

Explore similar work

Tube Diffusion Policy: Reactive Visual-Tactile Policy Learning for Contact-rich Manipulation

Force Policy: Learning Hybrid Force-Position Control Policy under Interaction Frame for Contact-Rich Manipulation

Touch Where It Matters: Learning Local Tactile Policies for Fine-Grained Gripper Control