EquiContact: A Hierarchical SE(3) Vision-to-Force Equivariant Framework for Spatially Generalizable Contact-rich Tasks
Organizations: University of California, Berkeley · Yonsei University
Abstract
This paper presents a framework for learning vision-based robotic policies for contact-rich manipulation tasks, which generalize spatially across task configurations. We propose EquiContact, a novel hierarchical framework composed of a high-level vision planner, such as a Diffusion Equivariant Descriptor Field (Diff-EDF) and a low-level compliant visuomotor policy, such as a Geometric Compliant Action Chunking Transformer (G-CompACT). G-CompACT, operating using localized observations from wrist-mounted RGB images, end-effector pose feedback in the form of a geometrically consistent error vector (GCEV) and force-torque readings, produces actions defined on the end-effector frame. We show that the entire EquiContact pipeline is -equivariant under equivariant reference-frame estimation and left-invariant visual features, from perception to force control. Once the robot learns to execute a contact-rich task, such as inserting a peg-in-hole (PiH), it can execute similar tasks at different locations and orientations without retraining of the low-level policy. We also describe three key components for attaining spatially generalizable contact-rich policies: geometrically consistent compliance, the use of localized policies, and induced equivariance. Real-world peg-in-hole (PiH) experiments demonstrate near-perfect success rates under spatially out-of-distribution configurations, validating the proposed framework and principles, while additional screwing and surface-wiping experiments evaluate the transfer of the localized policy design beyond insertion. The experimental videos and more details can be found on the project website: https://equicontact.github.io/EquiContact-website/
Figures & tables
| RQ | Methods | Observation | Action | Test Scenario | SR |
| RQ2 | ACT w/o GAC | World Pose | World Pose | Flat (In-Dist.) | 3 / 10 |
| RQ2 | ACT w/ GAC | World Pose | World Pose | Flat (In-Dist.) | 18 / 20 |
| RQ3 | CompACT | World Pose, FT | World Pose, Gains | Flat (In-Dist.) | 19 / 20 |
| Flat (OOD) | 0 / 10 | ||||
| RQ3 | EquiContact | GCEV, FT | Relative Pose, Gains | Flat (OOD) | 20 / 20 |
| (Place, Ours) | Tilted ( , OOD) | 19 / 20 |
| Test Scenario | SR | Failure |
|---|---|---|
| Flat OOD | 20 / 20 | – |
| Tilted OOD | 19 / 20 | 1 Insertion |
| Test Scenario | Screwing | Wiping |
|---|---|---|
| Flat OOD | 10 / 10 | 10 / 10 |
| Tilted , OOD | 9 / 10 | 10 / 10 |
| Names | Values |
|---|---|
| Image Size | |
| Learning Rate (policy) | |
| Learning Rate (Vision Encoder) | |
| Epochs | |
| Batch Size |
| Task | # Episodes | Mean Length | Max Length |
|---|---|---|---|
| Peg-grasping | 58 | 574.41 | 763 |
| Peg-insertion | 138 | 442.99 | 649 |
| Screwing | 91 | 1075.6 | 1478 |
| Wiping | 76 | 734.45 | 1087 |
| Testing Scenarios | Screwing | Wiping |
|---|---|---|
| in | 4 / 10 | 10 / 10 |
| in | 0 / 10 | 0 / 5 |
| in | 2 / 10 | 0 / 5 |
| pick | 7.173 | 6.933 | 6.199 | 7.650 | 15.90 | 15.67 |
| place | 13.75 | 8.241 | 5.999 | 3.806 | 5.560 | 5.660 |
| Backbone | Learning Rate | Learning Rate | Success Rate | |
|---|---|---|---|---|
| ( ) | ( ) | In-dist | OOD | |
| RN18 | 10 / 10 | 6 / 10 | ||
| CLIP-RN50-frozen | 3 / 10 | 3 / 10 | ||
| CLIP-RN50-SBT | 10 / 10 | 0 / 10 | ||
| CLIP-RN50 ( proposed ) | 10 / 10 | 10 / 10 | ||
Contact-Rich Robotic Manipulation
Contact-rich robotic manipulation studies how robots plan and control physical interactions such as grasping, insertion, pushing, and dexterous object handling.
See all 445 papers