Organizations: The Department of Electronic Engineering, Tsinghua University, Beijing 100084, China. · The Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong, China.
Single-view RGB-D grasp detection remains a common choice in 6-DoF robotic grasping systems, which typically requires a depth sensor. While RGB-only 6-DoF grasp methods has been studied recently, their inaccurate geometric representation is not directly suitable for physically reliable robotic manipulation, thereby hindering reliable grasp generation. To address these limitations, we propose MG-Grasp, a novel depth-free 6-DoF grasping framework that achieves high-quality object grasping. Leveraging two-view 3D foundation model with camera intrinsic/extrinsic, our method reconstructs metric-scale and multi-view consistent dense point clouds from sparse RGB images and generates stable 6-DoF grasp. Experiments on GraspNet-1Billion dataset and real world demonstrate that MG-Grasp achieves state-of-the-art (SOTA) grasp performance among RGB-based 6-DoF grasping methods.
Figures & tables
Fig. 1: Pipeline of MG-Grasp: given sparse posed RGB images, we first perform Depth Aggregation and Dense Correspondence to obtain up-to-scale pointmaps, and then recover metric scale via Triangulation-based Scale Recovery. The metric pointmaps are further refined with multi-view consistency optimization to produce consistent geometry, which is finally used for grasp guidance and 6-DoF grasp generation.
Fig. 2: Left: metrically scaled pointmaps before refinement; right: refined pointmaps. The highlighted regions (red boxes) illustrate misaligned surface layers before refinement, which are largely removed after refinement, resulting in multi-view consistent geometry for grasping.
Method
Data
Seen
Similar
Novel
Average
PointnetGPD [ 29 ]
RGB-D
25.96/27.59
22.68/24.38
9.23/10.66
19.29/20.88
GraspNet [ 2 ]
RGB-D
27.59/29.88
26.11/27.84
10.55/11.51
21.41/23.08
TransGrasp [ 31 ]
RGB-D
39.81/35.97
29.32/29.71
13.83/11.41
27.65/25.70
HGGD [ 4 ]
RGB-D
59.36/60.26
51.20/48.59
22.17/18.43
44.24/42.43
FlexLog [ 37 ]
RGB-D
72.81 / 69.44
65.21 / 59.01
30.04 / 23.67
56.02 / 50.67
GraspNeRF [ 13 ]
RGB
22.49/24.61
14.15/17.67
11.08/12.86
15.91/18.38
TABLE I: Results on GraspNet Dataset, showing AP on the RealSense/Kinect splits respectively
Method
Seen
Similar
Novel
Average
FeedForward-based (VGGT [ 18 ] )
52.29/51.54
47.98/43.18
20.73/ 22.90
40.33/39.21
SLAM-based (MASt3R-SLAM [ 33 ] )
50.39/43.42
38.54/37.50
18.19/14.10
35.71/31.67
MVS-based (MVSAnywhere [ 25 ] )
42.64/39.12
33.92/32.41
14.83/12.78
30.46/28.10
Ours
63.70 / 66.80
56.03 / 57.35
23.22 /20.47
47.65 / 48.21
TABLE II: Results of different geometry pipelines on GraspNet
Fig. 3: Comparison of unprojected point clouds produced by different geometry pipelines. The highlighted regions illustrate misaligned surface.
Fig. 4: Effect of the number of input views on GraspNet grasping performance (AP).
(A)
(B)
Seen
Similar
Novel
Average
✗
✗
29.27/23.99
25.48/19.51
14.32/10.64
23.02/18.05
✗
✓
40.73/35.06
37.10/23.57
20.90/11.85
32.91/23.49
✓
✗
55.29/39.95
43.67/38.66
20.22/16.25
39.73/31.62
✓
✓
63.70 / 66.80
56.03 / 57.35
23.22 / 20.47
47.65 / 48.21
TABLE III: Ablation study of different modules on GraspNet. (A) Triangulation-based scale recovery; (B) multi-view refinement.
Fig. 5: Real world setting.
Scene
Success
Attempt
1
5
6
2
5
5
3
5
5
4
5
6
5
5
6
6
5
6
TABLE IV: REAL ROBOTIC GRASP EXPERIMENT RESULTS
Module
Time (s)
Ratio (%)
MobileSAM masking
0.06
2.1
Depth Aggregation and Correspondence
0.87
30.5
Triangulation-based Scale Recovery
0.11
3.8
Multi-View Refinement
1.34
46.8
Grasp-aware Guidance
0.05
1.6
Grasp Generation
0.30
10.6
TABLE V: Runtime breakdown of MVR-Grasp per inference
Method
Time (s)
VG-Grasp [ 32 ]
11.20
FeedForward-based (VGGT [ 18 ] )
0.61
Ours
2.86
TABLE VI: End-to-end runtime comparison (seconds per inference).
Robotic grasping from single-view observations remains a critical challenge in manipulation. However, existing methods still struggle to generate reliable grasp candidates and stably evaluate grasp feasibility under incomplete geometric information. To address these limitations, we present SuperGrasp, a new two-stage framework for single-view parallel-jaw grasping. In the first stage, we introduce a Similarity Matching Module that efficiently retrieves valid and diverse grasp candidates by matching the input single-view point cloud with a precomputed primitive dataset based on superquadric coefficients. In the second stage, we propose E-RNet, an end-to-end network that expands the grasp-aware region and takes the initial grasp closure region as a local anchor region, capturing the contextual relationship between the local region and its surrounding spatial neighborhood, thereby enabling more accurate and reliable grasp evaluation and introducing small-range local refinement to improve grasp adaptability. To enhance generalization, we construct a primitive dataset containing 1.2k standard geometric primitives for similarity matching and collect a point cloud dataset of 100k samples from 124 objects, annotated with stable grasp labels for network training. Extensive experiments in both simulation and real-world environments demonstrate that our method achieves stable grasping performance and good generalization across novel objects and clutter scenes.
Task-Oriented Grasping (TOG) has emerged as an important skill for purposeful robotic manipulation. Given the task of using a knife to "slice bread," a robot must choose to grasp from the handle, whereas for "hand[ing] the knife over," it is preferable for the handle to remain unobstructed for safe handoff. Current approaches for TOG either focus on 2D planar grasping, remain limited by their reliance on task-agnostic grasp generators, or are constrained by reliance on finite, manually-annotated, task-specific grasp datasets that cannot cover all possible grasp-task pairs. To that end, we present TGrasp6D, a vision-language based approach for generating task-oriented 6DoF grasps in a zero-shot manner. Given a task and a single RGB-D image of an associated object, TGrasp6D reasons about the appropriate semantic affordance for the given task and object geometry to produce an initial task- informed prior. This generated prior is further refined via 6DoF pose optimization for collision avoidance while preserving task consistency. Our approach achieves state-of-the-art success rate of 94% on the TaskGrasp dataset, with the best baseline at 74%. We also deploy our approach on a Franka Research 3 robot to demonstrate zero-shot generalization to real world objects
Manav Kulshrestha, S. Apoorva Vashisth, and Aniket Bera are with the IDEAS Lab, Department of Computer Science, Purdue University, West Lafayette, IN, USA. · Damon Conover is with the DEVCOM Army Research Laboratory, Adelphi, MD, USA.
Humans can grasp objects effortlessly, whereas multi-fingered robots are far from this level of generality. We argue that the most natural source of robot grasping data is from humans, who pick up thousands of objects every day. We present HUG, a flow-matching model that generates diverse human grasps for any user-specified object in a single RGB-D image captured from a stereo camera. Using smart glasses, we first collect 1M-HUGs, an egocentric dataset of human grasps spanning 1M frames (27.8 hrs) and 6,707 object instances across 41 buildings. Next, to model the distribution of natural human grasps, our novel flow-matching model fuses RGB and depth observations to output a grasp parameterized by wrist translation, wrist rotation, and MANO hand pose. Predicted grasps can be retargeted to various robot hands, enabling zero-shot grasping in everyday scenes. To standardize evaluation, we build a new simulated benchmark, HUG-Bench, of 90 unseen objects from five geometric categories and various sizes, with metric-scale 3D meshes. We evaluate HUG in the real world on the 30-object test set of HUG-Bench across multiple stereo cameras, robot embodiments, and household environments. HUG outperforms the state-of-the-art grasping baselines by +23% and +34% on our challenging object set. Code, data, benchmark, checkpoints, and an interactive demo are released on our website: https://grasping.io/
Kevin Yuanbo Wu, Tianxing Zhou, Isaac Tu +5
New York University · Tsinghua University · University of Michigan