Robotic grasping is typically formulated as the problem of identifying successful actions from a space of candidate grasp poses. However, the organization of successful actions within this space has received less attention. We study the multiscale structure of viable robotic grasps in SE(3) and investigate whether this structure can be exploited for more efficient exploration. Using a large-scale grasp dataset, we show that successful grasp sets exhibit heterogeneous and reproducible connectivity structure across objects. We then introduce a connectivity-aware sampling strategy that incrementally explores the currently observed grasp space by prioritizing potential bridges between components, structural frontiers, boundary extensions, and geometric novelty. In controlled reconstruction experiments, the method recovers the connectivity structure of successful grasp sets substantially more efficiently than random sampling and farthest-point sampling. We further evaluate whether connectivity acquired under hidden grasp viability can improve subsequent grasp discovery, and whether structural experience from previously explored objects can be retrieved and transferred to unseen objects. These results suggest that the spatial organization of viable actions provides information relevant to grasp-space exploration beyond the viability of individual candidate actions. More broadly, they motivate structure-aware exploration as a means of exploiting the geometry of viable action spaces in robotic manipulation.
Figures & tables
Fig. 1 : Convergence of the grasp-space connectivity signature with sample size. The curve shows the mean within-object structural distance E(N) across 100 objects, with 95% bootstrap confidence intervals across objects. Independent samples become increasingly consistent as the number of successful grasps increases.
Fig. 2 : Within-object and between-object structural distances at N=50 , 100 , and 300 successful grasps. Within-object distances decrease strongly with sample size, whereas between-object distances remain substantially larger and comparatively stable. Sampling variability therefore decreases with N while object-specific differences in multiscale connectivity persist. Vertical lines show the interquartile range and circles denote medians.
Fig. 3 : Examples of viable grasp-space structure across six objects. Successful grasps are colored according to connected components computed in the original normalized SE(3) grasp space, while failed grasps are shown in gray. The two-dimensional MDS embedding is used only for visualization. Each panel reports the connectivity radius ϵ , number of connected components β0 , and relative size L of the largest component.
Random observations
Objects matched by CA
Median CA observations
Median speedup
50
200/200
12
4.17×
100
200/200
21
4.76×
200
192/200
93
2.15×
TABLE I : Sample efficiency of connectivity-aware (CA) exploration in E2. For each Random sampling budget, we report the median number of CA observations required to achieve the same connectivity-reconstruction error. Speedup is computed per object as the Random budget divided by the required CA observations and then summarized by the median.
N
Exploration
Method
Phase 2 successes
Normalized AUC
Mean gain [95% CI] (%)
200
Random
Random
25.13±1.14
0.5020±0.0295
–
NS
30.97±4.45
0.6638±0.1054
+32.66[+29.62,+35.80]
LV
31.20±4.16
0.6699±0.1007
+33.94[+30.93,+37.04]
CA
31.02±4.18
0.6528±0.0983
+30.56[+27.61,+33.57]
CA
Random
25.13±1.14
0.5020±0.0295
–
NS
29.77±4.47
0.6526±0.1050
+30.44[+27.39,+33.51]
TABLE II : Hidden-viability exploitation under Random and connectivity-aware (CA) Phase 1 exploration across candidate-set sizes. Phase 2 success counts and normalized AUC are reported as mean ± SD across objects after averaging over five seeds. Gains are computed relative to Random exploitation under the same Phase 1 exploration policy and reported as mean [95% CI]. NS denotes nearest-success and LV local viability.
Fig. 4 : Phase 2 exploitation performance following connectivity-aware (CA) Phase 1 exploration. Left: mean Phase 2 AUC gain relative to Random exploitation as a function of total candidate-set size N ; error bars show 95% bootstrap confidence intervals across objects. Right: distributions of object-level Phase 2 AUC gains relative to Random exploitation at N=1600 ; points denote individual objects, circles indicate medians, and vertical bars indicate interquartile ranges.
Fig. 5 : E4 cross-object viability-history transfer as a function of memory-library size M and transfer weight α at 200 attempts. Cells show mean normalized AUC ± 95% confidence intervals. Random, FPS, and CA are shown as non-transfer baselines. Moderate transfer weights improve CA, whereas complete reliance on transferred information ( α=1 ) reduces performance.
Fig. 6 : Performance across interaction budgets at fixed memory size M=200 . Cells report mean normalized AUC ± 95% confidence intervals across target objects for Random, FPS, CA, and memory-assisted CA at different transfer weights α .
Fig. 7 : Ranked per-object memory gain at M=200 and α=0.5 . Target objects are independently ranked by ΔnAUC=nAUCCA+Memory−nAUCCA for each budget. Positive values indicate improvement from cross-object memory.
Attempts
CA
CA+Memory
Mean Δ
Median Δ
Win rate
100
0.720
0.737
+0.017
+0.012
68.5%
200
0.747
0.756
+0.009
+0.004
61.0%
400
0.750
0.749
-0.001
≈0.000
48.0%
TABLE III : Effect of cross-object memory across interaction budgets at M=200 and α=0.5 . Δ denotes the paired per-object difference nAUCCA+Memory−nAUCCA .
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
HA=(SA+∪Sj∗+,SA−∪Sj∗−)
Appendix
Algorithm 1 Cross-Object Viability-History Transfer
M
Method
Retrieval (s)
Pool- B (ms)
Total (s)
–
Random
–
1.59±1.58
(1.61±1.58)×10−3
–
FPS
–
0.74±0.09
(2.87±0.31)×10−3
–
CA
–
27.45±1.57
3.73±0.23
200
E4
37.98±0.67
1067±133
42.73±0.81
400
E4
75.90±1.37
1073±103
80.64±1.42
600
E4
113.91±2.19
1077±95
118.66±2.27
Appendix
TABLE IV : Runtime per target at Q=400 (mean ± SD; 25 runs). Retrieval and Pool- B costs are one-time operations.
Dexterous manipulation poses a challenging high-dimensional optimization problem, as useful behaviors require coordinated motion across many hand joints. In reinforcement learning (RL) and sampling-based trajectory optimization, exploration commonly relies on independent robot joint perturbations, making coordinated behaviors difficult to discover. Prior work reduces this search space for grasp learning using low-dimensional spaces of coordinated joint motions learned from human hand data, but this restricts the expressivity required for general manipulation. Some combine learned and joint-space actions to restore expressivity, but this increases dimensionality and introduces redundancy. We study these effects across diverse manipulation settings, varying action dimensionality, exploration strategy, and the source of human data. Our experiments suggest that human-motion priors are most effective when used to structure exploration rather than change the action representation. Motivated by this finding, we propose EigenDEXplore, which induces correlated exploration by adding perturbations along human-derived eigenvectors to independent joint-space noise, leaving the action space unchanged. Across multiple dexterous hands, EigenDEXplore consistently outperforms joint-space and learned action-space baselines in grasping, in-hand reorientation, and contact-rich manipulation. These gains span unstructured and reference-guided RL, trajectory optimization, and sim-to-real deployment, and are largest in settings with less reward shaping and curriculum design.
Grasping is a fundamental robotic capability that bridges perception and physical task execution. This paper studies grasp pose recovery under a perception-to-execution mismatch, where a grasp generated from visual perception may become spatially stale if the object moves before execution, using only sparse tactile interactions and no further visual observations. We propose DA-GRD, Decision-Aware Grasp-Relevant Disambiguation, which maintains a weighted planar belief over possible object configurations and selects tactile probes according to their ability to eliminate hypotheses and improve agreement among candidate task grasps. Rather than fully relocalizing the object, DA-GRD stops when the remaining hypotheses support a common executable grasp. In MuJoCo experiments on ten rigid objects with translations up to 5~cm and yaw perturbations up to ±45∘, DA-GRD achieves an 84.7% physical lift success rate, compared with 9.1% for stale AnyGrasp, 21.2% for the original fix-scan baseline, and 63.7% for fix-scan method adapted with an SE(2) belief. DA-GRD also achieves a 57.3% Task conditioned Success rate. Across objects, it uses a success-average of 4.13 tactile probes over the ten per-object means, corresponding to a 72.5% reduction relative to the fixed 15-probe baselines. Real-world experiments on six objects achieve 71.7% physical lift success and 38.3% task-conditioned success with 4.20 probes on average. These results show that tactile sensing can recover task-relevant grasps under vision-off conditions with limited physical interaction, without requiring complete object localization.
Haoran Wang, Yuteng Sun, Yuanjie Li +6
Michael · Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore
Regrasp planning is often required when one pick-and-place cannot transfer an object from an initial pose to a goal pose while maintaining grasp feasibility. The main challenge is to reason about shared-grasp connectivity across intermediate poses, where discrete search becomes brittle. We propose an implicit multi-step regrasp planning framework based on differentiable pose sequence connectivity metrics. We model grasp feasibility under an object pose using an Energy-Based Model (EBM) and leverage energy additivity to construct a continuous energy landscape that measures pose-pair connectivity, enabling gradient-based optimization of intermediate object poses. An adaptive iterative deepening strategy is introduced to determine the minimum number of intermediate steps automatically. Experiments show that the proposed cost formulation provides smooth and informative gradients, improving planning robustness over other alternatives. They also demonstrate generalization to unseen grasp poses and cross-end-effector transfer, where a model trained with suction constraints can guide parallel gripper grasp manipulation. The multi-step planning results further highlight the effectiveness of adaptive deepening and minimum-step search.
Liang Qin, Weiwei Wan, Kensuke Harada
All authors are with the Graduate School of Engineering Science, The University of Osaka, Japan.