Contact-rich insertion requires robots to combine visual alignment with tactile adjustment. However, existing visuotactile chunk-based policies collapse tactile feedback into a single generation-time fusion pathway, leaving persistent contact states, abrupt contact events, and future contact consequences implicitly modeled in a single representation. We present PACE (Phase-Aware Contact-Event Conditioning), a temporal-role tactile conditioning framework for chunk-based insertion policies. PACE organizes a shared tactile stream into three temporal roles: persistent contact states condition action-chunk generation, abrupt contact events correct the generated chunk, and contact-consequence prediction shapes action representations during training. This design preserves chunk-level action continuity while assigning distinct pathways to different temporal roles of contact feedback. Across five real-robot insertion tasks, PACE achieves an average success rate of 90.7%, outperforming vision-only and generation-time tactile-fusion baselines by over 44 percentage points; on the ManipulationNet benchmark, PACE maintains 88.7% success across six geometry-clearance configurations, including an industrial-grade 0.1-mm clearance. Ablations confirm the complementary contributions of the three temporal roles, mechanistic analyses show that PACE separates normal insertion progress from contact events requiring immediate correction, and cross-signal and cross-architecture experiments indicate that this temporal-role principle extends across different physical feedback signals and policy backbones. Videos and materials are available at https://pace-insertion.github.io/.
Figures & tables
Figure 1: PACE. A temporal-role tactile conditioning framework that organizes one tactile stream into three roles: contact state conditions generation via affine modulation, contact events correct the chunk via gated residual, and contact-consequence prediction shapes representations during training.
Figure 2: Overview of PACE. Multimodal observations enter specialized encoders; a shared tactile bus feeds three temporal-role pathways. Phase: smoothed contact state modulates Transformer encoder features via layer-wise affine conditioning (generation-time). Pred.: decoder representations are supervised to predict future contact changes (training only). Event: contact-event cues produce a gated residual that corrects the generated action chunk (post-generation). The backbone is an ACT-style CVAE-Transformer.
Figure 3: Experimental setup. Left: force-feedback teleoperation system (Force Dimension sigma.7 leader, robot follower). Right: five real insertion tasks and the ManipulationNet benchmark.
Method
Key
2-pin
3-pin
USB
Ethernet
Avg.
Vision-only ACT
4/30
4/30
2/30
1/30
4/30
10.0%
Vision-only DP
5/30
6/30
4/30
3/30
5/30
15.3%
π0
6/30
8/30
4/30
4/30
6/30
18.7%
ACT + Tactile Cross-Attn
10/30
15/30
11/30
9/30
14/30
39.3%
ACT + Tactile Concat
13/30
17/30
12/30
13/30
14/30
46.0%
PACE
27/30
29/30
28/30
27/30
25/30
90.7%
Table 1(a): Quantitative comparison on real insertion tasks.
Peg
3 mm
1 mm
0.1 mm
Cyl.
25/25
22/25
21/25
Hex.
24/25
22/25
19/25
Table 1(b): ManipulationNet peg-in-hole results.
Method
Key
2-pin
3-pin
USB
Eth.
Avg.
PACE w/o Pred.
25/30
27/30
26/30
26/30
24/30
85.3%
PACE w/o Phase
24/30
26/30
21/30
23/30
24/30
78.7%
PACE w/o Event
20/30
22/30
18/30
19/30
21/30
66.7%
Phase-only
17/30
20/30
16/30
18/30
19/30
60.0%
PACE
27/30
29/30
28/30
27/30
25/30
90.7%
Table 2: Ablation study on PACE design choices.
Figure 4: Mechanistic visualization of PACE’s temporal-role separation during insertion. Top: calibrated end-effector contact force (N). Middle: Phase pathway output, visualized as the second principal component of the feature-wise affine modulation parameters ( γ,β ) (a.u.). Bottom: Event pathway output, shown as the gate-weighted residual correction level relative to its pre-contact resting level (a.u.). Phase exhibits sustained shifts across different contact phases, whereas Event shows sharp excursions around collision events and does not simply increase with the sustained high force of stable insertion.
Table A3: Contact-consequence prediction head parameterization.
Component
Additional Parameters
ϕphase (condition encoder + modulator)
0.546 M
ϕgate
0.005 M
ϕres
0.024 M
ϕpred (training only)
0.266 M
PACE total
0.841 M
Appendix
Table A4: Additional parameter count of PACE modules.
Parameter
Value
Optimizer
AdamW
Learning rate
1×10−5
Batch size
32
Training epochs
10000
Chunk size H
20
KL weight λkl
10
Appendix
Table C1: Training hyperparameters.
Task
Best epoch
Key
7456
2-pin
7102
3-pin
8126
USB
8003
Ethernet
7914
Cyl-3mm
7571
Appendix
Table C2: Selected checkpoints by task.
Item
Value
Demonstrations per task
~160 episodes
Episode length
~300 steps
RGB image resolution
200 × 266
Tactile sensor resolution (Vitai GF225)
240 × 240
Action space
7-DoF joint position
Appendix
Table C3: Dataset and observation statistics.
Temporal Ensemble
Event correction
Operates on
Predictions from multiple overlapping action chunks at the same time step
Generated action chunk
Mechanism
Fixed exponential weighted averaging
Learned gated residual from latest tactile history
Signal driven
No
Yes, tactile differential
Contact-aware
No
Yes
Appendix
Table D1: Comparison between temporal ensemble and Event correction.
Method
Chunk
Key
2-pin
3-pin
USB
Eth.
Avg.
Phase-only
20
17/30
20/30
16/30
18/30
19/30
60.0%
Phase-only (chunk=10)
10
14/30
17/30
13/30
15/30
15/30
49.3%
PACE
20
27/30
29/30
28/30
27/30
25/30
90.7%
Appendix
Table D2: Short-chunk control on real insertion tasks.
Method
2-pin
Key
RDP
19/30
12/30
PACE
29/30
27/30
Appendix
Table D3: Comparison with RDP on two real insertion tasks (successes/30).
Task
PACE
ACT + Tactile Concat
Key
27/30 [0.74, 0.97]
13/30 [0.27, 0.61]
2-pin
29/30 [0.83, 0.99]
17/30 [0.39, 0.73]
3-pin
28/30 [0.79, 0.98]
12/30 [0.25, 0.58]
USB
27/30 [0.74, 0.97]
13/30 [0.27, 0.61]
Ethernet
25/30 [0.66, 0.93]
14/30 [0.30, 0.64]
Appendix
Table D4: Real-robot successes and Wilson 95% confidence intervals.
Method
3 mm
1 mm
0.1 mm
Vision-only ACT
30%
10%
0%
Vision-only DP
10%
0%
0%
PACE
98%
88%
80%
Appendix
Table D5: Baseline success rates on ManipulationNet, averaged over cylindrical and hexagonal pegs. Each geometry-clearance configuration uses 25 rollouts.
Policy
Feedback on
Feedback off
PACE
29/30
21/30
ACT + Tactile Concat
17/30
7/30
Appendix
Table D6: Policy success under force-feedback collection conditions (2-pin; 30 rollouts per method).
Metric
FF on
FF off
Rel. change
p -value / d
Force variance (N 2 )
4.8
12.7
−62%
p<10−13 , d=0.80
Inter-trajectory deviation (N)
1.07
1.92
−44%
p<10−4 , d=0.55
Pairwise force-curve correlation
0.67
0.54
+24%
—
Peak contact force (N)
10.0±2.2
16.7±7.1
−40%
p<10−24 , d=1.28
Episode length CV
0.108
0.132
−18%
—
Appendix
Table D7: Demonstration quality metrics (2-pin, 162 episodes per condition). Relative change is reported as FF-on vs. FF-off; where tested, differences are statistically significant (Mann-Whitney U test).
Figure D1: Full demonstration quality comparison (2-pin, 162 episodes). (a–b) Force curve overlays. (c) Per-episode variance distribution. (d) Inter-trajectory standard deviation over normalized time. (e) Peak force distribution. Force-feedback demonstrations exhibit consistently lower variance, tighter trajectory bundles, and more predictable peak forces.
Figure E1: Representative failure case on USB insertion. The policy produces multiple correction attempts that move toward the target, but coupled lift–translation and insufficient retreat shift the contact point, accumulate object tilt, and lead to terminal jamming.
High-precision connector insertion remains challenging for robotic systems due to tight mechanical tolerances, partial observability during contact, and multimodal uncertainty arising from occlusion and contact ambiguity. Successful insertion requires closed-loop contact guidance that continuously integrates global alignment cues with local contact feedback to produce stable corrective actions under interaction. In this work, we present ContactDP (Contact-Guided Diffusion Policy for Tight Insertion Tasks), a multimodal diffusion-policy framework for contact-rich insertion. ContactDP jointly integrates wrist RGB observations, fingertip tactile sensing, and wrist-mounted force-torque measurements to infer contact state and generate temporally consistent corrective motions during insertion. To ensure stable execution under contact, the learned policy operates together with a hybrid position-force controller that provides compliant low-level interaction. We evaluate our approach on a suite of industrial-grade connector insertion tasks with varying connector geometries, grasp conditions, and initial misalignment. Across all tasks, ContactDP significantly outperforms vision-only diffusion policies for performance, reliability and generalization.
Chengyi Xing, Shaoxiong Yao, Diego Romeres +1
Mitsubishi Electric Research Labs, Cambridge, MA, US
Contact-rich precision insertion is a key manipulation skill in robotic assembly. Tight clearances make insertion more sensitive to alignment errors and prone to collisions and jamming, while variations in geometry and clearance across parts further complicate policy reuse. We present a reinforcement learning framework that trains insertion policies entirely in simulation for direct deployment without real-world demonstrations or policy fine-tuning. By combining target poses with compact three-dimensional fingertip force feedback, the policy learns to search for alignment and correct its motion despite errors in the estimated hole position. A decoupled gated reward coordinates alignment and insertion. Force-signal smoothing and state-independent standard deviations stabilize the learning process. The resulting policies perform real-world insertion across multiple hole geometries with a minimum nominal clearance of 0.02 mm and improve success while reducing peak contact forces under hole-position errors. Cross-clearance and cross-geometry evaluations further confirm policy generalization. The system achieved the first perfect score of 20/20 on ManipulationNet's peg-in-hole benchmark under its Human-in-the-Loop protocol, with fully autonomous insertion motions. A single policy trained only on a simulated hexagonal insertion task achieved an overall success rate of 95.0% across eight unseen real-world insertion tasks. These results show that learning entirely in simulation can yield precision insertion skills that can be deployed directly and reused across real-world tasks. The project website (https://mzhsoul.github.io/InsertAnything/) provides open-source simulation and real-robot experiment scripts, assets, and trained checkpoints.
Zhenghua Ma, Xinpan Meng, Zeyu Liu +4
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China, and the State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · PaXini AI Technology (Beijing) Co., Ltd., Beijing 100080, China
Robots that autonomously determine their actions from language instructions and sensory observations could perform new contact-rich manipulation tasks without task-specific training or hand-designed rules. To perform these tasks, robots must infer how objects contact one another and move as a result, then select actions. For contact inference and action selection, prior approaches involve designing estimation models and tactile feedback control laws, or learning models for object-motion estimation, action-outcome prediction, and action selection from tactile data. Instead, we propose TacZero, which uses a pretrained general-purpose vision-language model (VLM) to interpret visual and tactile observations and select robot actions without additional tactile or manipulation training or task-specific rules for contact interpretation or action selection. TacZero provides the VLM with camera images, robot state, and three-axis tactile responses represented as numerical values or vectors overlaid on the images. From these observations and interaction history, the VLM generates commands specifying target end-effector positions and gripper opening or closing, which a low-level controller executes. In real-world cylindrical-peg insertion experiments, TacZero succeeded in 15 of 20 trials with numerical tactile input, compared with 10 of 20 without tactile input. This study provides a concrete starting point for further research on contact-rich manipulation using general-purpose VLMs and highlights challenges in pursuing this direction.
Kazutoshi Tanaka
OMRON SINIC X Corporation, Bunkyo-ku, Tokyo 113-0033, Japan