Equivariant Volumetric Grasping
Organizations: KU Leuven, Dept. Mechanical Engineering, Research unit Robotics, Automation and Mechatronics. · AI Thrust, HKUST(GZ) · KU Leuven, Dept. Electrical Engineering, Research unit Processing Speech and Images.
Abstract
We propose a new volumetric grasp model that is equivariant to rotations around the vertical axis, leading to a significant improvement in sampling efficiency. Our model employs a tri-plane volumetric feature representation -- i.e., the projection of 3D features onto three canonical planes. We introduce a novel tri-plane feature design in which features on the horizontal plane are equivariant to rotations, while the sum of features from the other two planes remains invariant to reflections induced by the same transformations. We further develop equivariant adaptations of two state-of-the-art volumetric grasp planners, GIGA and IGD. Specifically, we derive a new equivariant formulation of IGD's deformable attention mechanism and propose an equivariant generative model of grasp orientations based on flow matching. We provide a detailed analytical justification of the proposed equivariance properties and validate our approach through extensive simulated and real-world experiments. Our results demonstrate that the proposed projection-based design reduces both computational and memory costs. Moreover, the equivariant grasp models built on top of our tri-plane features consistently outperform their non-equivariant counterparts, achieving higher performance within a real-time cost constraint. Video and code can be viewed in: https://mousecpn.github.io/evg-page/
Figures & tables
| Method | Packed | Pile | Latency (ms) | ||
| GSR (%) | DR (%) | GSR (%) | DR (%) | ||
| VGN* ( Breyer et al., 2021 ) | 72.5 2.6 | 76.7 1.7 | 59.3 2.9 | 43.5 2.9 | 9 |
| GIGA* ( Jiang et al., 2021 ) | 84.8 2.2 | 85.1 2.5 | 69.5 1.3 | 49.0 3.4 | 24 |
| GraspNet-1B Baselines* ( Fang et al., 2020 ) | 49.9 2.3 | 40.1 2.2 | 50.2 4.2 | 30.0 2.3 | 77 |
| GSNet* ( Wang et al., 2021 ) | 67.8 2.5 | 60.1 3.2 | 58.3 3.8 | 51.3 4.6 | 156 |
| GPD* ( Ten Pas et al., 2017 ) | 41.8 2.9 | 34.1 3.4 | 22.7 1.1 | 9.0 0.7 | 2138 |
| Method | Packed | Pile | Latency (ms) | ||
| GSR (%) | DR (%) | GSR (%) | DR (%) | ||
| PointNetGPD ( Liang et al., 2019 ) | 79.3 1.8 | 82.5 2.9 | 75.6 2.3 | 77.0 2.8 | 149 |
| VGN ( Breyer et al., 2021 ) | 80.2 1.6 | 86.2 2.0 | 64.9 2.2 | 69.1 3.2 | 9 |
| GIGA ( Jiang et al., 2021 ) | 89.9 1.7 | 87.6 2.0 | 76.3 2.4 | 80.9 4.1 | 24 |
| GIGA (HR) ( Jiang et al., 2021 ) | 91.4 1.5 | 88.5 1.4 | 86.5 1.2 | 80.8 1.9 | 53 |
| EdgeGraspNet ( Huang et al., 2023 ) | 92.5 0.9 | 94.3 1.1 | 91.5 1.3 | 92.5 1.3 | 843/685 |
| Method | Packed | Pile | Adv | |||
|---|---|---|---|---|---|---|
| GSR (%, 95% CI) | DR (%, 95% CI) | GSR (%, 95% CI) | DR (%, 95% CI) | GSR (%, 95% CI) | DR (%, 95% CI) | |
| GIGA ( Jiang et al., 2021 ) | 76.7 (66/86) [66.8, 84.4] | 88.0 (66/75) [78.7, 93.6] | 61.1 (44/72) [49.6, 71.5] | 58.7 (44/75) [47.4, 69.1] | 72.5 (66/91) [62.6, 80.6] | 88.0 (66/75) [78.7, 93.6] |
| EdgeGraspNet ( Huang et al., 2023 ) | 73.4 (58/79) [62.8, 81.9] | 77.3 (58/75) [66.7, 85.3] | 62.1 (41/66) [50.1, 72.9] | 54.7 (41/75) [43.4, 65.4] | 72.2 (57/79) [61.4, 80.8] | 76.0 (57/75) [65.2, 84.2] |
| VN-EdgeGraspNet ( Huang et al., 2023 ) | 71.3 (57/80) [60.5, 80.0] | 76.0 (57/75) [65.2, 84.2] | 67.7 (44/65) [55.6, 77.8] | 58.7 (44/75) [47.4, 69.1] | 79.5 (58/73) [68.8, 87.1] | 77.3 (58/75) [66.7, 85.3] |
| IGD ( =1) ( Song et al., 2024 ) | 78.0 (64/82) [67.9, 85.6] | 85.3 (64/75) [75.6, 91.6] | 58.0 (51/88) [47.5, 67.7] | 68.0 (51/75) [56.8, 77.5] | 78.2 (61/78) [67.8, 85.9] | 81.3 (61/75) [71.1, 88.5] |
| ICGNet ( Zurbrügg et al., 2024 ) | 72.2 (57/79) [61.4, 80.8] | 76.0 (57/75) [65.2, 84.2] | 71.1 (54/76) [60.0, 80.0] | 72.0 (54/75) [61.0, 80.9] | 69.9 (51/73) [58.6, 79.2] | 68.0 (51/75) [56.8, 77.5] |
| No. | Ablated models | GSR (%) | DR (%) | GPU Memory (MB) | GFLOPs | Params (M) | Latency (ms) |
|---|---|---|---|---|---|---|---|
| 1 | 3D UNet | 86.8 1.1 | 85.5 1.5 | 17226 | 44.30 | 2.24 | 22 |
| 2 | Eq. 3D UNet | 89.9 1.5 | 88.8 2.0 | 17208 | 44.30 | 0.22 | 22 |
| 3 | Tri-UNet | 84.6 1.1 | 85.3 2.4 | 3485 | 1.90 | 0.84 | 24 |
| 4 | Tri-UNet ( -separated) | 87.0 1.2 | 84.8 1.6 | 3292 | 1.90 | 1.61 | 24 |
| 5 | Eq. Tri-UNet (no V2TP) | 90.3 0.6 | 87.4 1.7 | 3860 | 1.90 | 0.40 | 24 |
| 6 | Tri-UNet (with V2TP) | 90.9 1.4 | 86.6 1.3 | 3405 | 1.95 | 1.69 | 25 |
| Orientation generation method | Graspness | EDA | GC | GSR (%) | DR (%) |
|---|---|---|---|---|---|
| FM (10-step) | ✓ | 91.1 2.5 | 89.1 2.1 | ||
| FM (10-step) | ✓ | ✓ | 93.5 1.0 | 91.1 0.5 | |
| FM (10-step) | ✓ | ✓ | ✓ | 97.3 1.1 | 92.1 1.2 |
| DDPM (10-step) | ✓ | ✓ | ✓ | 92.6 0.9 | 88.1 1.1 |
| DDPM (100-step) | ✓ | ✓ | ✓ | 92.6 1.4 | 85.0 2.2 |