cs.ROOct 28, 2026

Skeleton-Guided Geometry-Aware Auto-Labeling for Robotic Grasping: From 4-DoF Learning to 6-DoF Execution

Authors: Ali Sabzejou, Farbod Azimmohseni, Mehdi Tale Masouleh, Ahmad Kalhor

Organizations: Human and Robot Interaction Laboratory, School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Robotic grasping relies on large manually annotated datasets, which are costly to create. We introduce a straight-skeleton-based algorithm for generating two-finger grasp keypoints and identifying regions suitable for stable, feasible grasps. Our automated pipeline produces consistent grasp annotations without human intervention. We also propose the Skeleton-based Generative Grasp Convolutional Neural Network (SkelGG-CNN), a 4-DoF architecture that learns geometric grasp features from skeleton-based guidance. To extend auto-labeling to 6-DoF grasping, we develop a generalized quasi-3D skeletonization algorithm that operates directly on point clouds. Using automatically generated 4-DoF labels, SkelGG-CNN achieves accuracies of 95.42%, 96.55%, and 97.1% on Jacquard V1, Jacquard V2, and Cornell, respectively. On the GraspNet novel split, 6-DoF auto-labeling achieves an average precision of 18.33, comparable to state-of-the-art model-based approaches. In simulation, the 6-DoF method achieves success rates of 98.4% on Dex-Net and 89.1% on EGAD!. Real-world experiments with a Delta parallel robot show success rates of 97.4% on single household objects and 95.0% in cluttered scenes, outperforming state-of-the-art methods without domain adaptation. The 4-DoF pipeline adapts across environments and produces annotations approaching human-level precision, enabling efficient training of lightweight models on small datasets. The generalized 6-DoF method provides scalable, geometry-aware annotations for cluttered 3D scenes.

Figures & tables

Figure 1: Experimental setup featuring the Delta parallel robot and an illustration of the SkelGG-CNN model’s output for a real-world scene.
Figure 2: Overview of the proposed method. The pipeline begins with the simulation environment, where RGB-D images are captured from a set of 3D meshes sourced from the Dex-Net dataset. The subsequent stages show the details process of the straight skeleton-based grasp pose estimation algorithm. After this process, the generated grasp poses are used to create the SkelGG-CNN model labels. The skeleton heatmap is generated during the polygon offsetting phase of the algorithm, adding geometric features to the model.
AuthorAutomatic Dataset GenerationCornell ResultsJacquard ResultsGraspNet ResultsEGAD! ResultsCluttered ObjectsCode Available
Morrison et al. (2018)×\times✓×\times×\times×\times✓✓
Xu et al. (2022)×\times✓✓×\times×\times✓×\times
Kumra et al. (2022)×\times✓✓×\times×\times✓✓
Zhang et al. (2023)×\times✓✓×\times×\times✓×\times
Li and Zheng (2024)×\times✓✓×\times×\times✓×\times
Duan et al. (2025)×\times✓✓×\times×\times×\times×\times
Table 1: Comparison of various recent grasping approaches across different datasets and approaches.
Figure 3: Comparison of SkelGG-CNN output heatmaps with the pretrained Gr-ConvNetV2 Kumra et al. (2022) model. The first three columns display the quality and angle heatmaps for both models. The fourth column represents the width heatmap for Gr-ConvNet, whereas SkelGG-CNN uses a skeleton heatmap instead, enabling both grasp width estimation and grasp refinement.
Figure 4: Illustration of polygon offsets with the skeleton specified by red lines.
Figure 5: Skeleton heatmap construction steps.
Figure 6: Example of grasp repositioning, the initial grasp center PciP_{c_{i}} from the quality heatmap is shifted along the orientation θi\theta_{i} to the point PcP_{c} with the maximum skeleton heatmap value. This refinement produces more robust grasps compared to the approaches that only use quality maps. In this figure, blue dots represent initial grasp centers and red dots indicate the final grasp centers after the refinement process.
Figure 7: Architecture of the SkelGG-CNN model.
Figure 8: Overview of the generalized 6-DoF skeleton-based auto-labeling pipeline. (Left) The input object point cloud is partitioned into overlapping horizontal cross-sections (80% overlap) orthogonal to the zz -axis. (Center — upper path: 3D skeletonization) Each slice is processed with a Laplacian contraction skeletonization; skeleton points within a slice are connected using a Minimum Spanning Tree (MST) and then linked to nearest neighbors in adjacent slices to produce a compact 3D skeleton graph that captures the object’s volumetric structure. (Center — lower path: 2D polygon approximation and initial centers) Each cross-section is projected to 2D and approximated with an alpha-shape polygon; a straight skeleton computed on this polygon yields initial central grasp centers. (Right) Candidate skeleton-node pairs are sampled and refined by traversing the 3D skeleton (depth-first search) from these initial centers; at each candidate pair a local plane is fitted to neighboring skeleton edges and the antipodality of the planes is evaluated as the grasp quality measure. (Far right) Top-scoring pairs are selected and visualized as final 6-DoF gripper poses superimposed on the original point cloud.
Input typeJacquard accuracy(IoU)
RGB94.46
Depth95.32
RGB-D95.42
Table 2: Performance comparison of algorithms on Accuracy with IoU (%) on Jacquard V1 dataset.
Model parametersAccuracy (IoU)
951,31692.10
3,801,25295.11
15,196,99695.42
Table 3: Comparison of model performance with different parameter count on the Jacquard V1 dataset.
AuthorsAlgorithmAccuracyLabels
   TrainTest
Zhou et al. (2018)FCGN, ResNet10192.80GTGT
Zhang et al. (2019)ROI-GD, ResNet10193.60GTGT
Morrison et al. (2020b)GG-CNN 284.00GTGT
Wang et al. (2021b)DD-Net, Hourglass97.00GTGT
Yu et al. (2022)SE-ResUNet95.70GTGT
Table 4: Performance comparison of algorithms on Accuracy with IoU (%) on the Jacquard V1 dataset.
AuthorsAlgorithmAccuracyLabels
   TrainTest
He et al. (2016)ResNet5091.37GTGT
He et al. (2016)ResNet10191.35GTGT
He et al. (2016)ResNet15290.30GTGT
Iandola et al. (2016)Squeeze94.76GTGT
Zhang et al. (2018)ShuffleNet94.85GTGT
Table 5: Performance comparison of algorithms on Accuracy with IoU (%) on the Jacquard V2 dataset.
Figure 9: Comparison of test accuracies of the trained models on Dex-Net dataset.
AuthorsAlgorithmAccuracyLabels
   TrainTest
Morrison et al. (2020b)GG-CNN 284.0GTGT
Zhang et al. (2019)ROI-GD, ResNet-10193.6GTGT
Wang et al. (2021b)DD-Net, Hourglass97.2GTGT
Yu et al. (2022)SE-ResUNet98.2GTGT
Xu et al. (2022)GKNet96.9GTGT
Table 6: Performance comparison of algorithms on Accuracy with IoU (%) on the Cornell dataset.
MethodsSeenUnseenNovel
 AP (%)AP 0.8 (%)AP 0.4 (%)AP (%)AP 0.8 (%)AP 0.4 (%)AP (%)AP 0.8 (%)AP 0.4 (%)
GPD Ten Pas et al. (2017)24.3830.1613.4623.1828.6411.329.5010.143.16
GG-CNN Morrison et al. (2018)16.8922.4711.2315.0519.766.197.388.781.32
PointNetGPD Liang et al. (2019)27.5934.2117.8324.3830.8412.8310.6611.243.21
GraspNet Fang et al. (2020)29.8836.1919.3127.8433.1916.6211.5112.923.56
RGBMatter Gou et al. (2021)32.0839.4620.8530.4037.8718.7213.0813.796.01
Table 7: Detailed results on the GraspNet dataset (Kinect), showing average precision (AP) scores.
Figure 10: Results of the quasi-3D auto-labeling method on a real-world scene from the GraspNet dataset, visualized with a blue-to-red gradient indicating grasp quality (red: higher quality, blue: lower quality).
Figure 11: Simulation experiments on the Dex-Net dataset. The first row shows the predicted grasp rectangles generated by the SkelGG-CNN model, along with the corresponding skeleton and quality heatmaps. The second row illustrates the robotic arm’s approach phase. Finally, the third row presents the successfully grasped objects from two different grasps.
CategoryMethodEGAD!Dex-Net TestDex-Net Unseen
LearningGG-CNN54.089.688.0
GR-ConvNetV2-94.492.2
SkelGG-CNN ∗62.497.695.2
Geometric2D Auto-Labeling60.892.888.0
Quasi-3D Auto-Labeling89.198.496.0
Table 8: Success rate comparison of algorithms on Dex-Net and EGAD! datasets.
Figure 12: Illustration of the objects used for real robotic experiments.
Figure 13: Grasp predictions produced by the SkelGG-CNN model for single objects in a real-world environment. The figure displays both quality and skeleton maps, highlighting the identified grasp configurations.
Figure 14: Demonstration of a Delta parallel robot grasping objects in a cluttered scene. The first column presents the skeleton and quality output of the SkelGG-CNN model for the initial scene. For each grasp attempt, grasps are generated based on the captured image of the scene in its current state. The second row highlights the bounding box of the selected grasp at each stage.
AlgorithmTraining DatasetHousehold IsolatedHousehold Cluttered
SAE, struct. reg. Lenz et al. (2015)Cornell89.0 (89/100)-
AlexNet-based CNN Pinto and Gupta (2016)Custom66.0 (99/150)38.4 (50/130)
Robust Best Grasp Johns et al. (2016)ModelNet in Sim80.0 (80/100)-
Multi-grasp Res-50 Chu et al. (2018)Cornell89.0 (89/100)-
DexNet 2.0 Mahler et al. (2017)DexNet in Sim80.0 (40/50)-
GPD Gualtieri et al. (2016)CAD models-77.3 (116/138)
Table 9: Performance comparison on household isolated and cluttered scenes.
Figure 15: A failure case where the object slipped from the gripper after being grasped.
More in this topic

Robotic Grasping

Robotic grasping studies how robots perceive, plan, and execute stable, task-appropriate contacts with objects across varied shapes and conditions.

See all 316 papers

Explore similar work

CardsList
Oct 26, 2026

MMGNet: Multi-Modal 6-DoF Grasp Synthesis for Cluttered Scenes

Grasping is a foundational capability in robotic manipulation. Scene-level 6-DoF grasp synthesis relies on point clouds for metric geometry, yet real-world point observations are often noisy, incomplete, and over-smoothed. RGB images provide object-region, boundary, and appearance cues, yet two bottlenecks limit their effective use in scene representation and local grasp reasoning. We present MMGNet, a Multi-Modal Grasp Synthesis Network that addresses these bottlenecks through hierarchical image-to-3D feature injection and rotational graspness modeling. Hierarchical injection integrates multi-scale image features throughout 3D representation formation, while the rotational graspness jointly predicts the grasp approach direction and in-plane rotation before organizing local image-informed features through grasp-shaded grouping. Ablation studies show that these two modules provide complementary gains. MMGNet achieves a mean AP of 60.34 on GraspNet-1Billion with RealSense, surpassing all compared methods. In real-world experiments with adversarial grasp objects, MMGNet improves the grasp success rate over AnyGrasp by 8.46 percentage points. Code, models, and data will be released upon publication.

Rui Cao, Jia-Hui Pan, Yichuan Li +4

Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, China · Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · Sino-German College of Intelligent Manufacturing, Shenzhen Technology University, Shenzhen, China

Robotic GraspingMultimodal Sensor Fusion
AskPDF
May 31, 2026

GraspGen-X: Cross-Embodiment 6-DOF Diffusion-based Grasping

We study cross-embodiment 6-DOF robot grasping. Unlike prior works, we require the model not only to generalize to novel objects / scenes but also to novel gripper morphologies and physical grasping processes. Our method extends diffusion model based generative 6-DOF grasping models to condition on the additional gripper's representation. We propose a swept-volume heuristic for encoding the gripper. We train our cross-embodiment model with procedural grippers and a large-scale dataset of 2 Billion grasps. In simulation experiments, our model has the best zero-shot generalization to novel real-world grippers and objects over baseline methods. Our model also serves as a good initialization for fine-tuning to adapt to novel grippers. In ablations, we demonstrate the efficiency of our sweep-volume gripper representation and our procedural gripper training dataset. Last, we show zero-shot generalization to real-world novel grippers for 6-DOF grasping, surpassing baselines in cross-embodiment generalization.

Beining Han, Yu-Wei Chao, Erwin Coumans +5

1NVIDIA · 2Princeton University

Robotic GraspingGeneralization in Robotic ManipulationCross-Embodiment Robot LearningDiffusion Models
AskPDF
Mar 17, 2026

MG-Grasp: Metric-Scale Geometric 6-DoF Grasping Framework with Sparse RGB Observations

Single-view RGB-D grasp detection remains a common choice in 6-DoF robotic grasping systems, which typically requires a depth sensor. While RGB-only 6-DoF grasp methods has been studied recently, their inaccurate geometric representation is not directly suitable for physically reliable robotic manipulation, thereby hindering reliable grasp generation. To address these limitations, we propose MG-Grasp, a novel depth-free 6-DoF grasping framework that achieves high-quality object grasping. Leveraging two-view 3D foundation model with camera intrinsic/extrinsic, our method reconstructs metric-scale and multi-view consistent dense point clouds from sparse RGB images and generates stable 6-DoF grasp. Experiments on GraspNet-1Billion dataset and real world demonstrate that MG-Grasp achieves state-of-the-art (SOTA) grasp performance among RGB-based 6-DoF grasping methods.

Kangxu Wang, Siang Chen, Chenxing Jiang +3

The Department of Electronic Engineering, Tsinghua University, Beijing 100084, China. · The Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong, China.

Robotic Grasping
AskPDF