Arm-Aware Guided Dexterous Grasp Generation with Arm-Agnostic Grasp Models

Yongyi Jia; Yongpeng Jiang; Kangchen Lv; Yi Ren; Mingrui Yu; Xiang Li

cs.ROAug 17, 2026

Arm-Aware Guided Dexterous Grasp Generation with Arm-Agnostic Grasp Models

Authors: Yongyi Jia, Yongpeng Jiang, Kangchen Lv, Yi Ren, Mingrui Yu, Xiang Li

Abstract

Dexterous grasp generation that considers arm-related constraints is crucial in real-world scenarios involving arm environment collision avoidance, workspace boundary grasps, and consecutive grasping. Existing hand-centric grasp models, which primarily focus on the floating hand's pose, are insufficient for such cases. Conventional arm-aware methods either rely on rejection sampling to discard infeasible samples or require retraining on arm-specific data, leading to low sample efficiency under adverse conditions or limited generalization across different robots and environments. To overcome these limitations, this letter presents an arm-aware dexterous grasp generation framework that leverages pretrained arm-agnostic grasp models while integrating arm and environmental information only at inference time. Specifically, we formulate arm-aware constrained grasp generation as a joint optimization of hand pose and arm configuration, and derive closed-form gradients for arm-related constraints. Assuming the hand pose distribution is represented by a diffusion model, we prove that gradient-based optimization is equivalent to guided diffusion sampling, steering near-feasible samples toward the feasible region. Through comprehensive evaluation involving 10k objects across 6 scenarios, we demonstrate that the proposed framework generates feasible grasps in highly constrained settings with significantly higher probability, highlighting its advantages in real-world applications. Supplementary materials and appendix are available at https://arm-aware-dexgrasp.github.io/.

Figures & tables

Fig. 1 : Motivation of arm-aware dexterous grasp generation. Real-world grasp execution requires considering arm-related constraints, yet conventional methods rely on rejection sampling to discard even near-feasible hand poses, leading to low efficiency. Our approach guides pretrained arm-agnostic models with these constraints at inference, greatly improving sampling efficiency.

Fig. 2 : Overview of the proposed arm-aware dexterous grasp generation method. Initially, we pretrain an arm-agnostic diffusion model to capture the distribution of wrist poses for floating hands. During sampling, arm kinematics and environment SDF are integrated as constraints, with their gradients guiding the denoising process. This approach significantly enhances the proportion of feasible grasps, adaptable to various arm-hand configurations and constrained environments.

		C. Satisfaction(%)↑
Scene	Method	CFR	RFR	FR	GSR (%)↑	SR (%)↑	OSR (%)↑
	Baseline (RS)	15.31	99.96	15.31	58.17	8.91	55.15
S1	Ours	88.87	99.99	88.87	53.08	47.17	91.77
	Baseline (RS)	31.08	99.24	31.08	60.08	18.68	74.32
S2	Ours	87.91	99.95	87.91	49.11	43.17	91.54
	Baseline (RS)	25.75	72.24	25.75	58.46	15.05	70.31

Table I : Performance of grasp generation in constrained environments (UR5)

		C. Satisfaction(%)↑
Scene	Method	CFR	RFR	FR	GSR (%)↑	SR (%)↑	OSR (%)↑
	Baseline (RS)	10.92	85.22	10.58	54.18	5.73	39.07
S1	Ours	61.03	85.95	59.18	48.50	28.70	84.76
	Baseline (RS)	32.04	85.35	30.31	57.82	17.53	73.60
S2	Ours	85.87	91.86	83.28	47.32	39.41	90.96
	Baseline (RS)	32.56	55.69	32.37	57.09	18.48	75.47

Table II : Performance of grasp generation in constrained environments (Franka)

		C. Satisfaction(%)↑
Arm	Method	CFR	RFR	FR	GSR (%)↑	SR (%)↑	OSR (%)↑
	Baseline (RS)	90.06	26.07	25.74	59.54	15.32	66.33
	Ours	99.25	65.58	65.57	53.04	34.79	89.42
UR5	w/o Reachability *	99.83	25.89	22.49	59.48	15.39	66.92
	w/o Collision *	95.77	67.30	66.13	52.61	34.79	89.41
	Baseline (RS)	78.20	21.44	12.26	58.55	7.18	37.97

Table III : Performance of reachable grasp generation near workspace boundaries

Fig. 3 : Illustration of representative cases highlighting the effects of various constraints. The top row illustrates the arm configurations $\bm{q}^{*}$ during the denoising process of a certain noised sample both with our method and Baseline (RS), where $\bm{q}^{*}$ at $t=T,T/2,0$ are visualized from transparent to opaque. The denoising process starts from the same initial sample shown with the highest transparency. The fully denoised hand and arm configuration are depicted with the lowest transparency. For clarity of illustration, we only display the evolution of hand configuration in (b). The green arm in (c) represents the reference joint configuration $\bm{q}_{\text{cur}}$ based on a hypothetical previous grasp attempt. The bottom row presents the evolution of normalized constraint losses over time.

Arm	Method	C. Satisfaction			GSR (%)↑	SR (%)↑	OSR (%)↑
Arm	Method	CFR (%)↑	FR (%)↑	AJP ( $\text{rad}^{2}$ )↓	GSR (%)↑	SR (%)↑	OSR (%)↑
	Baseline (RS)	35.55	34.28	0.0686	58.51	20.06	71.63
	Ours	76.45	74.76	0.0398	49.79	37.72	90.26
UR5	w/o Collision *	45.11	48.83	0.0372	52.07	23.34	77.25
	w/o Joint *	49.34	46.49	0.0614	56.95	26.48	83.92
	Baseline (RS)	45.52	44.41	0.0778	55.08	24.46	65.38

Table IV : Performance of grasp generation with reference joint configuration

		C. Satisfaction(%)↑
Scene	Method	CFR	RFR	FR	GSR (%)↑	SR (%)↑	OSR (%)↑
	Baseline (PD)	77.80	99.99	77.80	40.98	31.88	87.43
	Baseline (PSR)	82.38	99.99	82.38	24.59	20.26	78.03
S1	Ours	88.87	99.99	88.87	53.08	47.17	91.77
	Baseline (PD)	95.03	99.97	95.01	19.37	18.40	76.74
	Baseline (PSR)	99.98	62.29	62.29	38.01	23.68	81.92

Table V : Comparison with other representative diffusion-based constraint-aware grasp generation methods (UR5)

Fig. 4 : Snapshots of the real-world grasp execution with UR5 and LEAP Hand, under two challenging scenarios. (a) Corridor Scene, (b) Shelf Scene.

Corridor Scene	Object	Banana	Blue Box	Cheezit	Mustard	Toy	Apple	Tea Can	White Tape
Corridor Scene	Top-10 Success Num	8/10	6/10	8/10	8/10	9/10	10/10	7/10	10/10
Shelf Scene	Object	Banana	Blue Box	Cheezit	Mustard	Toy	Apple	Tea Can	White Tape
Shelf Scene	Top-10 Success Num	10/10	9/10	8/10	9/10	9/10	10/10	9/10	10/10

Table VI : Per-object grasp execution success rate with the grasps generated for eight daily objects in two real-world constrained scenarios (Fig. 4 )

Fig. 5 : Snapshots of real-world constrained grasping demonstrating the practical effectiveness of the proposed method. Top: the initial configurations. Bottom: the final grasping configurations. (a) Grasping behind obstacles. (b) Grasping in deep containers. (c) Bi-manual handover grasping.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 12 — Fig. 6 : Visualization of four evaluation scenarios (S1-S4) with one successful grasp involving the robotic arm and dexterous hand, the obstacles painted in green, and the grasped object highlighted in orange. Scene S4 is specifically designed to showcase Franka’s collision avoidance using its kinematic redundancy.

Fig. 7 : Effect of different constraint weights $\lambda_{c}$ and learning rates $\eta_{\lambda}$ .

Fig. 8 : Effect of different sigmoid shape.

Fig. 9 : Effect of different weights coefficient $\alpha$ .

Fig. 10 : Effect of different joint weights $\lambda_{s}$ .

Fig. 11 : Effect of different denoising step $\text{T}_{\text{N}}$ .

Scene	Method	Changes of Metrics(%)↑
Scene	Method	FR	GSR	SR	OSR
S1	w/ proj.	- 4.80	+ 2.54	- 0.41	+ 0.34
S1	w/ noised grad.	- 0.75	- 3.69	- 3.64	- 0.12
S2	w/ proj.	- 18.39	+ 5.52	- 5.19	- 1.41
S2	w/ noised grad.	- 0.12	- 1.96	- 2.29	- 0.11
S3	w/ proj.	+ 0.83	+ 0.81	+ 0.94	+ 0.33

Appendix

Table VII : Performance of the ablation groups compared to ’ Ours ’ in Tab. I

Fig. 12 : The average inference time per successful grasp under different batch size on UR5 and Franka in the Narrow Corridor scenario.

Module	Component	Time Ratio (%)
SDF	Query SDF	56.8
	Link Jacobian	17.8
	Sphere Jacobian	12.2
	Nullspace Projection	6.3
	Others	6.9
IK	Call Solver	41.6

Appendix

Table VIII : Inference-Time Breakdown of Major Components

Arm	IK Num.	Metrics(%)↑
Arm	IK Num.	FR	GSR	SR	OSR
UR5	2	2.74	44.01	1.21	10.06
	4	45.36	56.64	25.69	83.64
	8	73.16	57.80	42.29	90.32
Franka	5	35.47	48.79	17.31	71.58
	10	46.18	51.78	23.91	80.46

Appendix

Table IX : Performance of grasp generation with varying number of IK solutions

Fig. 13 : Comparison of grasp generation near the arm’s workspace boundary. The top right figure shows a planned grasp that the robotic arm cannot reach.

Fig. 14 : Illustration of guided grasp generation for continuously regrasping with the Joint Proximity constraint. The reference initial grasp and the four final grasps are displayed from left to right, with additional grasps omitted for brevity. Please refer to the attached video for the complete grasp sequence.

Fig. 15 : Failure cases in real-world experiments. (a–b) The quality of a small subset of generated grasps is degraded by the guidance, leading to unstable grasps holding only the object’s edge. (c) The dexterous hand unintentionally contacts the object due to open-loop execution. (d) The object shifts in hand because of inaccurate mass parameter estimation or insufficient friction.

Fig. 16 : Architecture of the grasp generation and evaluation networks.

Explore similar work

Mar 17, 2026

EFF-Grasp: Energy-Field Flow Matching for Physics-Aware Dexterous Grasp Generation

Denoising generative models have recently become the dominant paradigm for dexterous grasp generation, owing to their ability to model complex grasp distributions from large-scale data. However, existing diffusion-based methods typically formulate generation as a stochastic differential equation (SDE), which often requires many sequential denoising steps and introduces trajectory instability that can lead to physically infeasible grasps. In this paper, we propose EFF-Grasp, a novel Flow-Matching-based framework for physics-aware dexterous grasp generation. Specifically, we reformulate grasp synthesis as a deterministic ordinary differential equation (ODE) process, which enables efficient and stable generation through smooth probability flows. To further enforce physical feasibility, we introduce a training-free physics-aware energy guidance strategy. Our method defines an energy-guided target distribution using adapted explicit physical energy functions that capture key grasp constraints, and estimates the corresponding guidance term via a local Monte Carlo approximation during inference. In this way, EFF-Grasp dynamically steers the generation trajectory toward physically feasible regions without requiring additional physics-based training or simulation feedback. Extensive experiments on five benchmark datasets show that EFF-Grasp achieves superior performance in grasp quality and physical feasibility, while requiring substantially fewer sampling steps than diffusion-based baselines.

Yukun Zhao, Zichen Zhong, Yongshun Gong +2

Shandong University, Jinan, China

Robotic Grasping Physics-Informed Generative Modeling Flow Matching

Ask PDF

Sep 27, 2026

Steer2Grasp: Inference-Time Embodiment-Aware Steering for Diverse Physically Feasible Grasp Diffusion

Current grasp diffusion models provide rich priors for generation, yet their object-centric approach can violate the kinematic and collision constraints imposed by the embodiment and the environment. Existing embodiment-aware methods primarily perform local corrections around generated grasps through gradient guidance or optimization, making it difficult to recover from fundamentally infeasible modes. We present Steer2Grasp, a training-free, embodiment-agnostic framework for inference-time grasp steering that adapts a frozen Cartesian grasp diffusion model using deployment-specific rewards. Through Feynman-Kac (FK) inspired particle reweighting and resampling, the method reallocates population mass from infeasible to high-reward grasp modes, enabling population-level mode transitions without modifying the pretrained diffusion model or requiring differentiable constraints. The framework enables a unified treatment for single and dual arm grasping through reachability and collision aware rewards, followed by gradient free gripper level local refinement. Across diverse objects, robot embodiments, and constrained environments, our method substantially improves feasible grasp generation while maintaining proximity to the underlying grasp prior.

Vignesh Vembar, Ayush Kaura, A Padmaprabhan +5

Robotics Research Center, IIIT Hyderabad · Johns Hopkins University

Robotic Grasping Inference-Time Guidance

Ask PDF

Jan 31, 2026

DiffMorphGrasp: A Diffusion Framework Conditioned on Hand Morphology for Cross-Embodiment Grasp Generation

As robots are deployed across diverse environments, manipulation tasks require robot hands with different morphologies to satisfy varying task requirements, hardware capabilities, and cost constraints. However, transferring grasping policies learned on one robot hand to another with a different morphology remains a fundamental challenge due to variations in kinematic structures and degrees of freedom across embodiments. To address these limitations, we propose DiffMorphGrasp, the first diffusion-based framework that incorporates hand morphology information into the generative process for cross-embodiment dexterous grasping. The proposed approach maps grasps from diverse robotic hands into a reference hand pose, providing a common space for learning. Grasp generation is then continuously conditioned on structured representations of hand kinematics, encoded as graphs derived from hand configurations, together with object geometry. In addition, we introduce a loss function that computes joint-level errors over the hand kinematic structure to constrain the generated grasps during training. To evaluate DiffMorphGrasp, we construct a new benchmark for assessing grasp generation performance across robot hands with different morphologies. Extensive experiments demonstrate that DiffMorphGrasp achieves strong zero-shot generalization to unseen hand morphologies, enabling scalable cross-embodiment grasp deployment. Project page: https://georgewuzy.github.io/DiffMorphGrasp-website

Zhiyuan Wu, Xiangyu Zhang, Zhuo Chen +3

Department of Engineering, King’s College London, Strand, London, WC2R 2LS, United Kingdom · Imperial College London, London, SW7 2AZ, United Kingdom

Robotic Grasping Cross-Embodiment Robot Learning Zero-Shot Robotic Manipulation Diffusion Models

Ask PDF