PHASE: Compliance-Enabled Tactile Phase Retrieval for Few-Shot Insertion Learning
Authors: Jeremy Siburian, Cristian C. Beltran-Hernandez, Tatsuya Matsushima, Yusuke Iwasawa, Masashi Hamaya, Mai Nishimura
Organizations: OMRON SINIC X Corporation, 5-24-5 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan · The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8654, Japan
Contact-rich assembly tasks such as peg-in-hole insertion remain difficult to learn from limited demonstrations. While retrieval-augmented imitation learning, which augments target demonstrations with relevant prior data, offers a promising direction, its applicability to contact-rich manipulation remains largely unexplored. Contact-rich insertion unfolds over multiple phases from search to insert, and retrieving phase-specific experience from prior data in principled ways remains an open question. Our key insight is that a compliant wrist enables the robot to sustain contact throughout execution, producing rich tactile and force signals that naturally reveal the phase structure of insertion and inform what should be retrieved. Based on this insight, we present PHASE (PHase-Aware Segmentation and REtrieval), a framework for compliance-enabled tactile phase retrieval that integrates multimodal contact-aware representation learning, variable-length phase segmentation from tactile signals, and phase-consistent retrieval for policy learning. We evaluate PHASE on real-world peg-in-hole insertion across five peg geometries, comparing against retrieval strategies drawn from state-of-the-art methods under a shared policy architecture. PHASE improves the overall success rate by 13 percentage points over the strongest non-phase-aware baseline, and improves performance under unseen initial positions by 30 percentage points. These results demonstrate that aligning retrieval with interaction-defined contact phases substantially improves robustness in few-shot insertion learning.
Figures & tables
Fig. 1: PHASE: Phase-Aware Segmentation & Retrieval. Given a few target task demonstrations, PHASE leverages compliance-enabled tactile and force signals to segment prior demonstrations into variable-length contact phases, retrieves phase-consistent segments, and trains robust policies that generalize across geometries and distribution shifts.
Fig. 2: Overview of PHASE. The framework consists of three stages: (1) Tactile Representation Learning , where a masked tactile-proprioceptive encoder [ 17 ] learns embeddings that capture force-driven interaction dynamics; (2) Phase-Aware Segmentation & Retrieval , where demonstrations are segmented using tactile-estimated torque signals to identify interaction-defined contact phases and variable-length segments are retrieved accordingly; and (3) Policy Training , where an ACT policy [ 18 ] is trained on the retrieved, phase-consistent segments for robust peg-in-hole insertion.
Fig. 3: Tactile-Based Phase Segmentation. The top panel shows the tactile-estimated torque magnitude ∥τ^∥ (dark curve) and the signed components of τ^ (light curves), while the bottom panel shows the coefficient of variation (CV) of the torque magnitude. The boundary between search and insert is detected at the first frame where CV<η after the contact peak, indicating the onset of stable engagement. Robot snapshots (inset) show the physical interaction during each phase.
Fig. 4: Compliant Robot for Tactile Phase Retrieval. The UR5e, which includes an integrated F/T sensor, is equipped with a soft wrist providing passive 6D compliance, and a parallel gripper with a 3×3 distributed tactile sensor. Gripper pose is tracked via a motion capture system. Demonstrations are collected through teleoperation for peg-in-hole insertion tasks with multiple peg geometries and varied initial poses.
Seen Shapes
Unseen Shapes
Method
Circle
Square
Rectangle
Oval
Hexagon
Total
Target-Only
13/20
6/20
9/20
15/20
7/20
50/100 (50%)
BC-Prior
19/20
10/20
4/20
17/20
14/20
64/100 (64%)
Retrieval-Single [ 8 ]
13/20
7/20
7/20
14/20
15/20
56/100 (56%)
Retrieval-Full [ 38 ]
16/20
9/20
7/20
19/20
13/20
64/100 (64%)
TABLE I: Real-world success rate over 20 trials per shape. Totals are reported over 100 trials per method. Best and second-best results are highlighted.
Metric
Single
Pairs
Fixed
Windows
Full
Episodes
PHASE
Training frames
4,479
8,820
10,596
15,548
Phase balance (0–1)
0.99
0.98
0.87
0.98
Temporal coherence
0.72
1.00
1.00
1.00
TABLE II: Offline retrieval characteristics across retrieval strategies ( k=20 ). ↓ : lower is better.
Method
Circle
Square
Rectangle
Oval
Hexagon
Total
Target-Only
0/20
0/20
0/20
0/20
0/20
0/100 (0%)
BC-Prior
5/20
2/20
3/20
5/20
2/20
17/100 (17%)
Retrieval-Single [ 8 ]
0/20
1/20
1/20
0/20
0/20
2/100 (2%)
Retrieval-Full [ 38 ]
4/20
4/20
4/20
2/20
0/20
14/100 (14%)
Retrieval-Window [ 4 ]
6/20
5/20
5/20
0/20
0/20
16/100 (16%)
PHASE (Ours)
12/20
8/20
12/20
9/20
6/20
47/100 (47%)
TABLE III: Success rate under unseen starting positions (20 trials per shape; 100 total). Best and second-best results per column are highlighted.
Contact-rich precision insertion is a key manipulation skill in robotic assembly. Tight clearances make insertion more sensitive to alignment errors and prone to collisions and jamming, while variations in geometry and clearance across parts further complicate policy reuse. We present a reinforcement learning framework that trains insertion policies entirely in simulation for direct deployment without real-world demonstrations or policy fine-tuning. By combining target poses with compact three-dimensional fingertip force feedback, the policy learns to search for alignment and correct its motion despite errors in the estimated hole position. A decoupled gated reward coordinates alignment and insertion. Force-signal smoothing and state-independent standard deviations stabilize the learning process. The resulting policies perform real-world insertion across multiple hole geometries with a minimum nominal clearance of 0.02 mm and improve success while reducing peak contact forces under hole-position errors. Cross-clearance and cross-geometry evaluations further confirm policy generalization. The system achieved the first perfect score of 20/20 on ManipulationNet's peg-in-hole benchmark under its Human-in-the-Loop protocol, with fully autonomous insertion motions. A single policy trained only on a simulated hexagonal insertion task achieved an overall success rate of 95.0% across eight unseen real-world insertion tasks. These results show that learning entirely in simulation can yield precision insertion skills that can be deployed directly and reused across real-world tasks. The project website (https://mzhsoul.github.io/InsertAnything/) provides open-source simulation and real-robot experiment scripts, assets, and trained checkpoints.
Zhenghua Ma, Xinpan Meng, Zeyu Liu +4
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China, and the State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · PaXini AI Technology (Beijing) Co., Ltd., Beijing 100080, China
Contact-rich insertion requires robots to combine visual alignment with tactile adjustment. However, existing visuotactile chunk-based policies collapse tactile feedback into a single generation-time fusion pathway, leaving persistent contact states, abrupt contact events, and future contact consequences implicitly modeled in a single representation. We present PACE (Phase-Aware Contact-Event Conditioning), a temporal-role tactile conditioning framework for chunk-based insertion policies. PACE organizes a shared tactile stream into three temporal roles: persistent contact states condition action-chunk generation, abrupt contact events correct the generated chunk, and contact-consequence prediction shapes action representations during training. This design preserves chunk-level action continuity while assigning distinct pathways to different temporal roles of contact feedback. Across five real-robot insertion tasks, PACE achieves an average success rate of 90.7%, outperforming vision-only and generation-time tactile-fusion baselines by over 44 percentage points; on the ManipulationNet benchmark, PACE maintains 88.7% success across six geometry-clearance configurations, including an industrial-grade 0.1-mm clearance. Ablations confirm the complementary contributions of the three temporal roles, mechanistic analyses show that PACE separates normal insertion progress from contact events requiring immediate correction, and cross-signal and cross-architecture experiments indicate that this temporal-role principle extends across different physical feedback signals and policy backbones. Videos and materials are available at https://pace-insertion.github.io/.
Reliable insertion of industrial connectors remains a central challenge in robotics, requiring sub-millimeter precision under uncertainty and often without full visual access. Vision-based approaches struggle with occlusion and limited generalization, while learning-based policies frequently fail to transfer to unseen geometries. To address these limitations, we leverage tactile sensing, which captures local surface geometry at the point of contact and thus provides reliable information even under occlusion and across novel connector shapes. Building on this capability, we present Touch2Insert, a tactile-based framework for arbitrary peg insertion. Our method reconstructs cross-sectional geometry from high-resolution tactile images and estimates the relative pose of the hole with respect to the peg in a zero-shot manner. By aligning reconstructed shapes through registration, the framework enables insertion from a single contact without task-specific training. To evaluate its performance, we conducted experiments with three diverse connectors in both simulation and real-robot settings. The results indicate that Touch2Insert achieved sub-millimeter pose estimation accuracy for all connectors in simulation, and attained an average success rate of 86.7% on the real robot, thereby confirming the robustness and generalizability of tactile sensing for real-world robotic connector insertion.
Masaru Yajima, Yuma Shin, Rei Kawakami +2
Institute of Science Tokyo, Tokyo, Japan. · Mitsubishi Electric, Kanagawa, Japan.