GOAG: Generative and Object-Agnostic Grasp Planner for Dexterous Robotic Manipulation
Authors: Julien Merand, Boris Meden, Mathieu Grossard, Liming Chen
Organizations: Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France · Ecole Centrale de Lyon, CNRS, LIRIS, UMR5205, Institut Universitaire de France (IUF), F-69130 Ecully, France
Multifingered grasping is a crucial robotic skill, but current deep-learning grasp planners often struggle to generalize to new objects because they are trained on limited, object-specific datasets. We introduce a fundamentally different approach, grounded in the observation that the gripper and the object share identical surface geometry at their mutual contact points. We propose GOAG: Generative and Object-Agnostic Grasp Planner for Dexterous Robotic Manipulation, a novel deep generative model that learns a compact latent representation of a specific gripper's contact surface distribution, enabling the efficient sampling of valid grasp configurations without relying on object-specific training data. We show that by introducing object features only at inference time, our model can effectively retrieve admissible contact areas that are compatible with the gripper's capabilities. We validate our approach through extensive experiments on established grasp protocols in both simulated and real-world scenarios, demonstrating its effectiveness with different grippers from the literature. Our method delivers state-of-the-art results on the objects from the MultiDex dataset, achieving an average success rate of 86.93%. It offers significantly faster processing when generating numerous grasps, while matching the performance of leading approaches specifically trained on this dataset. Unlike these methods, our approach does not rely on object-specific training data, highlighting the advantages of object-agnostic learning. It effectively addresses the generalization challenges faced by traditional data-driven grasp planners. Code and videos are available on our project website https://cea-list.github.io/goagweb/ .
Figures & tables
Fig. 1 : GOAG Paradigm. A successful grasp on an object induces dual contact zones on both object and gripper, at the intersection of the two geometries. Our method is built on this key observation: these contact zones ( C(.) ) are closely the same from either perspective. GOAG capitalizes on this by training exclusively on gripper geometry, allowing it to learn a robust and generalizable grasping strategy without ever being exposed to a grasp database with specific objects geometries.
Grasp Representation
Gripper Pose
Gripper Joint Values
Force Closure
Non- Penetration
Training Set
Working Reference Frame
Optional Grasp Preference Interface
DFC [ 5 ]
Direct
Optimized
Optimized
✓
✓
No
Object
✗
UniGrasp [ 6 ]
Intermediate
IK Solved
IK Solved
✓
✗
Objects + Grippers
Object
✗
GeoMatch [ 7 ]
Intermediate
IK Solved
IK Solved
✓
✗
Objects + Grippers
Object
✗
GenDexGrasp [ 8 ]
Intermediate
Optimized
Optimized
✓
✓
Objects + Grippers
Object
✗
ManiFM [ 9 ]
Intermediate
Optimized
Optimized
✓
✓
Objects + Grippers
Object
Contact Region
DRO-Grasp [ 10 ]
Intermediate
Optimized
Optimized
✓
✓
Objects + Grippers
Object
Palm Orientation
TABLE I : Comparison of Dexterous Grasp Planning Methods.
Fig. 2 : Grasp Taxonomy Adaptation and Contact Sampling. (Top) We adapt the human grasp taxonomy from [ 13 ] to the Allegro Hand geometry. For each grasp type (e.g., C6, F27), we define a corresponding admissible contact region (black points), distinguishing it from the non-contact surface (blue points). (Bottom) Data generation mechanism: We randomly sample specific contact points (red) strictly within the admissible black regions. This allows the model to learn structured, feasible contact distributions based solely on gripper kinematics, independent of any object.
Fig. 3 : Overview of GOAG. Geometrical graspability is learned in an object-agnostic manner by focusing on the gripper’s capabilities. Training: We sample gripper configurations Q to generate H(Q) and corresponding contact points ( C(H(Q)) ). To ensure transferability, we use a Basis Point Set (BPS) encoding tied to the gripper’s workspace. A Conditional Variational Autoencoder (CVAE) is trained to reconstruct these contact distributions, while a Links Mapper (PointNet++) learns to associate contact points with specific gripper links. Inference: A novel object O , positioned at the inverse gripper pose [R,T]−1 , is BPS-encoded. By sampling a latent variable z∈Rψ , the CVAE Decoder generatively predicts diverse, plausible contact points C(O) . The Links Mapper then labels which gripper link should reach each point. Generation: Finally, a Force Closure check ensures the predicted contacts yield a stable grasp, and a Grasp Optimization step outputs the final, refined gripper configuration Q∗ .
Method
Data Driven
Object-Agnostic Training
Success Rate (%) ↑
Efficiency (sec. / grasps) ↓
Diversity (avg.) ↑
Barrett
Allegro
ShadowHand
Avg.
Barrett
Allegro
ShadowHand
T (m)
R (rad)
Q (rad)
DFC [ 5 ]
✗
✓
83.10
82.71
72.15
79.32
>1800
>1800
>1800
0.0607
1.424
0.3579
GenDexGrasp [ 8 ] (full)
✓
✗
70.26
71.48
71.15
70.96
9.78
16.45
14.65
0.0519
1.416
0.2567
DRO-Grasp [ 10 ] (pretrain, w/o controller)
✓
✗
78.30
75.80
63.30
72.47
0.88
0.42
1.72
0.0546
1.515
0.2892
GOAG (w/o FC)
✓
✓
86.30
91.20
74.70
84.07
0.09
0.13
0.15
0.0480
1.396
0.3162
GOAG
✓
✓
87.40
93.20
77.90
86.93
0.18
0.19
0.20
0.0479
1.401
0.3170
TABLE II : In-depth grasp performance analysis on the Multidex [ 8 ] test set. We report our grasp results, on three classic dexterous grippers in terms of success rate, efficiency and diversity.
Fig. 4 : GOAG grasp results on Multidex [ 8 ] objects. Grasps are shown for the Barrett (green), Allegro (pink), and Shadow Hand (purple) grippers.
Method
Per-Dataset Training
DexGraspNet ↑
UniDexGrasp ↑
MultiDex ↑
RealDex ↑
DexGRAB ↑
Avg. (%)
UniDexGrasp [ 25 ]
✓
33.9
23.7
21.6
27.1
20.8
25.42
GraspTTA [ 29 ]
✓
18.6
21.0
30.3
13.3
14.4
19.52
SceneDiffuser [ 31 ]
✓
26.6
28.3
69.8
21.7
39.1
37.10
UGG [ 32 ]
✓
46.9
46.0
55.3
32.7
42.7
44.72
DGA [ 12 ]
✓
57.5
53.1
79.1
44.8
57.9
58.48
GOAG
✗
43.07
49.51
77.90
37.37
62.13
53.97
TABLE III : Grasp generalization assessment across multiple grasp datasets. Following [ 12 ] we evaluate our method performances with the Shadow hand on multiple grasp test sets. We report the success rates as defined in IV-C . It is worth noting that state-of-the-art methods are retrained for each dataset. GOAG has only been trained once on Shadow hand kinematics.
Fig. 5 : Real-world setup and results with Allegro hand on YCB [ 39 ] objects. First row presents the real robot grasps. Second row presents corresponding virtual grasps. Objects have been rotated around the z-axis for a better understanding of the grasp poses.
Current dexterous grasp planners primarily optimize for physical stability, focusing on whether an object can be grasped rather than how it should be grasped to support downstream functional tasks. However, conditioning grasp synthesis on specific human grasp taxonomies typically requires prohibitively expensive, object-annotated datasets. To address these limitations, we propose CoToGrasp, a novel generative framework that synthesizes diverse, stable grasps strictly conditioned on specific contact topologies. To bypass the data collection bottleneck, CoToGrasp is trained entirely in an object-agnostic manner. We introduce a feature-based canonical workspace that projects local object features into a unified gripper-centric domain, effectively decoupling the semantic functional intent from the arbitrary object geometry. By learning the intrinsic contact manifold of the gripper within this workspace, our model achieves zero-shot generalization to unseen objects at inference. Extensive evaluations on the large-scale DexGraspNet dataset demonstrate that CoToGrasp achieves state-of-the-art performance, outperforming existing taxonomy-guided planners. Finally, we demonstrate the physical viability and kinematic feasibility of our synthesized contact topologies on a physical robot platform. Code is available on our project website at https://cea-list.github.io/cotograspweb/ .
Julien Merand, Boris Meden, Liming Chen +1
Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France · Ecole Centrale Lyon, CNRS, LIRIS, UMR5205, Institut Universitaire de France (IUF), F-69130 Ecully, France
Dexterous grasping in multi-object scene constitutes a fundamental challenge in robotic manipulation. Current mainstream grasping datasets predominantly focus on single-object scenarios and predefined grasp configurations, often neglecting environmental interference and the modeling of dexterous pre-grasp gesture, thereby limiting their generalizability in real-world applications. To address this, we propose DGS-Net, an end-to-end grasp prediction network capable of learning dense grasp configurations from single-view point clouds in multi-object scene. Furthermore, we propose a two-stage grasp data generation strategy that progresses from dense single-object grasp synthesis to dense scene-level grasp generation. Our dataset comprises 307 objects, 240 multi-object scenes, and over 350k validated grasps. By explicitly modeling grasp offsets and pre-grasp configurations, the dataset provides more robust and accurate supervision for dexterous grasp learning. Experimental results show that DGS-Net achieves grasp success rates of 88.63% in simulation and 78.98% on a real robotic platform, while exhibiting lower penetration with a mean penetration depth of 0.375 mm and penetration volume of 559.45 mm^3, outperforming existing methods and demonstrating strong effectiveness and generalization capability. Our dataset is available at https://github.com/4taotao8/DGS-Net.
Dexterous grasping is the foundational primitive in embodied AI, demanding massive data to train robust models. As real-world data collection is expensive, simulation has become the mainstream paradigm. Yet, while cluttered scenes best reflect real-world applications, learning to grasp within them is bottlenecked by a critical scarcity of large-scale data. To resolve this, we curate high-quality 3D objects and supporting bases, proposing a scalable seed-and-filter strategy that bypasses sluggish scene-level optimization. This yields an unprecedented benchmark comprising over 2.6 million scenes and 0.4B scene-specific grasp ground truths, featuring diverse realistic layouts paired with rich semantic and geometric observations. Furthermore, we introduce the OmniDex model to overcome the grasp multimodality and last-millimeter precision errors plaguing current generative models. By coupling Soft Winner-Takes-All learning with human-inspired physical constraints during training, and utilizing physics-driven ranking, our approach achieves robust dexterous grasping without the latency of post-optimization. Experimental results show that OmniDex model achieves state-of-the-art performance and strong generalization across diverse scenes, views, and unseen objects.