Synthesizing realistic articulated hand-object interactions is a fundamental problem in virtual reality, embodied intelligence, and digital human applications. Existing methods for dexterous grasp synthesis typically regress or denoise poses in a joint space that couples global rigid motion with local articulation, which often yields unstable samples and physically implausible contacts. We introduce DEAL-Grasp, built upon the Decoupled Alignment (DEAL) representation, which reformulates grasp synthesis as alignment-space generation: the interaction state comprises task-space geometric anchors and articulation parameters, from which the rigid transform is recovered via closed-form Procrustes alignment while preserving local articulation. On this mixed state, we model grasp generation using heterogeneous-state flow matching with component-wise vector fields, incorporating time-adaptive physical regularization during training. At inference, grasps are synthesized solely by integrating the learned vector field, without test-time optimization or auxiliary physical guidance. Across MultiDex and zero-shot RealDex benchmarks, DEAL-Grasp attains high force-perturbation success rates alongside minimal penetration and high diversity of generated grasps, while substantially reducing native inference latency compared to optimization-heavy baselines. The project page is available at https://wmtlab.github.io/DEAL-Grasp/.
Figures & tables
Figure 1. Overview and core intuition of DEAL-Grasp. (a) Prior methods: Direct joint-space regression of (R,t,θ) couples root and articulation errors under FK , causing kinematic drift at the fingertips (top); discrete SO(3) sampling incurs discretization error (bottom). (b) DEAL design: State H={θ,X} evolves via heterogeneous flow matching over Rnj×Rnx×3 ( 0≤t≤1 ), followed by Kabsch–SVD alignment for global SE(3) recovery. (c) Zero-shot generation: Dual views (top) show precise contact on a hand-shaped target object; the array (bottom) shows diverse grasps on unseen geometries. Three-panel figure. Left: schematic of joint-space regression with fingertip drift and discrete rotation sampling error. Middle: DEAL state of joints and geometric anchors evolving over time, then closed-form rigid alignment. Right: rendered grasps on a hand-shaped object and a grid of grasps on varied unseen objects.
Figure 2. Overview of the DEAL-Grasp framework. The framework generates dexterous grasps by modeling a conditional neural vector field vϕ in a continuous-time domain. The Object Encoder first integrates the object points O with its PCA-based spatial scales s to extract geometric features ( EO,glo,EO,loc ). Simultaneously, the DEAL Encoder maps the stochastic state Ht to the initial query position PH0 via the embedding operator Φ(⋅) , while EH0 denotes the predefined identity tokens used to distinguish different kinematic components. Subsequently, the Iterative Velocity Refinement Decoder (IVRD) evolves these tokens across L transformer layers. In each layer, an AdaLN-based fusion mechanism integrates the context vector C (comprising EO,glo,t,s ) to guide the velocity prediction. Training combines CFM supervision against the linear conditional vector field ut with time-adaptively weighted physical losses (TAW); inference integrates vϕ purely via ODE steps without test-time physical guidance. Pipeline diagram from left to right: object point-cloud encoder; DEAL encoder that embeds joint and anchor state into query tokens; multi-layer IVRD blocks with context fusion; and separate training path with flow matching plus physical losses versus inference path that integrates the learned field alone.
Figure 3. Dense hand surface point clouds evaluated via forward kinematics for physical consistency losses. Blue+red: Pfull (128 points × 17 links) for penetration penalty Lpen . Red only: Ppalmar , the palmar-surface subset used for contact guidance Lpull . Three-dimensional articulated hand with dense surface sample points on every link. Blue and red together mark the full surface set; red alone highlights only the palm-facing samples on the fingers and palm.
Methods
MultiDex
RealDex
Latency (s) ↓
S.R.1 (%) ↑
S.R.6 (%) ↑
Pen. (cm) ↓
Hmean↑
Hstd↓
S.R.1 (%) ↑
S.R.6 (%) ↑
Pen. (cm) ↓
Hmean↑
Hstd↓
UniDexGrasp ( Xu et al., 2023 )
63.08
46.25
1.20
6.67
0.43
45.08
26.83
1.89
6.41
0.49
30.31
GenDexGrasp ( Li et al., 2023 )
76.34
49.35
1.06
6.71
0.60
45.57
26.17
1.54
6.12
0.57
36.57
GrainGrasp ( Zhao et al., 2024 )
67.36
42.45
0.98
6.42
0.58
48.07
29.26
1.43
6.03
0.61
29.91
DRO ( Wei et al., 2024b )
86.31
59.50
1.31
5.87
1.22
59.32
33.73
2.81
4.85
1.23
2.16
UGG (w. opt.) ( Lu et al., 2024 )
83.69
59.29
0.43
6.98
0.15
78.89
61.32
1.21
6.66
0.19
14.10
Table 1. Quantitative comparison on MultiDex and RealDex. Dynamic success rates (S.R.) are evaluated post-controller; other metrics reflect raw generated poses. Best results are in bold .
Figure 4. Qualitative comparison of synthesized grasps. Red circles highlight characteristic physical failure modes in baseline methods, including pronounced mesh interpenetration and detached, hovering fingers. DEAL-Grasp establishes physically consistent contact configurations with natural surface compliance and minimal penetration across both MultiDex and RealDex objects. Grid of rendered robot-hand grasps on everyday objects from MultiDex and RealDex. Baseline rows show fingers sinking into objects or hovering without contact, marked by red circles; the DEAL-Grasp row shows tighter, more natural contacts on the same objects.
Variant
S.R.1 (%) Δ↑
S.R.6 (%) Δ↑
Pen. (cm) Δ↓
HmeanΔ↑
HstdΔ↓
DEAL (Direct Reg.)
−8.45
−13.40
+0.39
−0.33
+0.09
DEAL (Tips-only)
+0.81
−0.23
+0.03
+0.00
+0.02
IVRD (Fixed Query)
−1.72
−3.26
+0.16
−0.23
+0.05
TAW (Constant G(t) )
−3.00
−5.53
+0.27
−0.22
+0.06
TAW (Exponential G(t) )
−1.38
−3.32
+0.01
−0.09
+0.02
CFM → DDPM (100)
−2.75
−9.78
+0.15
+0.08
+0.02
Table 2. Ablation of DEAL-Grasp architectural components. Metrics are reported as relative changes ( Δ ) with respect to the full model.
Figure 5. Attention heatmap analysis of the IVRD module. Max-pooled fingertip–object cross-attention maps across decoder layers and flow steps (warm colors indicate high saliency). Black skeletons denote input spatial queries; intermediate configurations follow a light-to-dark grayscale gradient across increasing layer depths, contrasted with the IVRD (Fixed Query) baseline. Multi-panel attention maps over object point clouds. Black hand skeletons mark fingertip query locations; warm colors show high attention near contact surfaces. Across columns, hand poses darken from light to dark as decoder depth increases, with a separate Fixed Query baseline panel for comparison.
Variant
S.R.1 (%) Δ↑
S.R.6 (%) Δ↑
Pen. (cm) Δ↓
HmeanΔ↑
HstdΔ↓
w/o Phys. (All)
−4.72
−9.33
+0.21
−0.11
+0.02
w/o Pen. ( Lpen )
−0.15
−1.24
+0.35
+0.02
−0.01
w/o Pull ( Lpull )
−1.92
−5.86
−0.04
−0.06
+0.03
Table 3. Ablation of physical supervision terms. Metrics are reported as relative changes ( Δ ) with respect to the full training objective.
Setting
S.R.1 (%) Δ↑
S.R.6 (%) Δ↑
Pen. (cm) Δ↓
HmeanΔ↑
HstdΔ↓
Lat. (s) Δ↓
L=1
−18.70
−22.50
+0.56
−0.55
−0.05
−0.06
L=3
−10.40
−13.30
+0.31
−0.32
−0.01
−0.02
L=8
+0.41
+1.21
−0.04
+0.02
+0.01
+0.02
Table 4. Sensitivity analysis of decoder layers ( L ). Metrics are reported as relative changes ( Δ ) with respect to the default L=6 .
Figure 6. Qualitative results across diverse object scales. Objects are uniformly scaled with maximum extents spanning 5 cm to 30 cm. Across both small and large objects, synthesized hand configurations exhibit conformal finger wrapping and minimal interpenetration, demonstrating consistent geometric conditioning. Rows of rendered hand–object grasps where the same objects appear at increasing physical sizes from about 5 cm to 30 cm, with fingers wrapping small items and spanning larger items without obvious collisions.
Figure 7. Diverse grasp synthesis on MultiDex and RealDex. For each object, multiple configurations (color-coded) exhibit distinct finger articulations and contact distributions, demonstrating multimodal coverage of viable grasps under the DEAL representation. Per-object galleries of several overlaid or side-by-side hand poses in different colors, contacting different faces and finger configurations on each MultiDex and RealDex object.
Figure 8. Physical robot experiments on a UR10e manipulator and Shadow Dexterous Hand. (Left) Experimental platform and 3D-printed test objects. (Right) Successful executions of DEAL-Grasp configurations across diverse geometries, verifying physical feasibility under real-world visual sensing and actuation noise. Composite photograph. Left green-framed panel: laboratory setup with a UR10e arm, ShadowHand, and tabletop test objects. Right blue-framed panel: photo sequences of successful physical grasps on several object shapes.
Figure 9. Effect of object-feature resolution on representative failure cases. Each column compares the same object under two encoder settings: a wristwatch (left) and a toy airplane (right). For each object, both settings share identical initial states H0 . The top row (128 FPS-sampled object points) exhibits local penetration at thin edges or slender protruding parts, whereas the bottom row (192 points) preserves similar terminal grasp topologies with reduced penetration depth. Two-column comparison for a wristwatch and a toy airplane. Top row grasps with 128 object points show local finger penetration at thin edges or protrusions; bottom row grasps with 192 points keep a similar overall pose with visibly less penetration.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10. Heuristic grasp controller for evaluation. Examples show the change from initial geometric contacts (red) to force-closure states (green). Global pose is held fixed while joint angles are adjusted by minimizing contact objectives. Lpull drives the fingers toward the object; Lpen and Lspen limit interpenetration and self-collision. The small residual penetration (green) reflects the contact closure needed for stable simulation. Side-by-side rendered hands on objects with contact regions highlighted. Initial geometric contacts appear in red; after joint-only closing, contacts shift to green force-closure states with slight residual surface pressure.
Methods (Pre-controller)
S.R.1 (%) ↑
S.R.6 (%) ↑
Pen. (cm) ↓
DRO ( Wei et al., 2024b )
83.94
55.16
1.26
UGG (w. opt.) ( Lu et al., 2024 )
83.36
53.40
0.43
DGA (wo. opt.) ( Zhong et al., 2025 )
81.37
56.30
0.61
DGA (w. opt.) ( Zhong et al., 2025 )
88.61
68.02
1.03
DEAL-Grasp (Ours)
89.59
66.41
0.42
Appendix
Table 5. Pre-controller evaluation on MultiDex. Success rates and penetration are measured on raw model outputs without the shared heuristic grasp controller used in the main tables.
Dexterous grasp synthesis has advanced rapidly in generating stable and physically plausible hand poses, but real-world manipulation requires grasps that preserve the function implied by the task. We study open-vocabulary task-oriented dexterous grasp generation, where a robot must infer functional intent from free-form language, ground it in multi-view visual observations and object geometry, and generate an executable high-degree-of-freedom grasp. We present OpenDexGrasp, a unified data and generative modeling framework for this setting. OpenDexVerse provides dual-source supervision organized by the Coverage-to-Alignment (C2A) Recipe: OpenDex-Scale offers large-scale semantic and geometric coverage through automatic grasp synthesis and vision-language annotation, while OpenDex-Align supplies high-quality embodied alignment through human teleoperation and category-level transfer. OpenDexGrasp learns a shared perception-action latent representation that couples open-vocabulary vision-language context with dexterous action generation. Affordance grounding and grasp generation provide complementary supervision over this latent space, enabling direct generation of task-consistent dexterous grasps without a separate affordance-to-pose inference stage. Extensive simulation and real-robot experiments demonstrate improved functional alignment, physical feasibility, generalization to unseen categories, and real-world execution success. Additional details and videos are available at https://opendexgrasp.github.io/.
Jiyao Zhang, Junhan Wang, Tianyu Wang +4
CFCS, School of CS, PKU, China · National Key Laboratory for Multimedia Information Processing, School of CS, PKU, China · PrimeBot +1
Differences in hand geometry, joint parameterization, and kinematic topology pose challenges for cross-embodiment dexterous grasp generation. However, spatial relations among links alone are insufficient to fully capture the hand's kinematic topology. We propose DexGraspFlow, a conditional link-level SE(3) flow-matching framework integrating geometric relations and kinematic topology. The framework encodes local object geometry, link geometry, and current poses. A dual-stream DiT models link--link coordination and link--object interaction, while attention biases derived from kinematic-tree distances provide explicit structural information for modeling evolving geometric interactions. The model jointly learns translational and rotational velocity fields, integrates them to generate link poses, and recovers joint configurations through inverse kinematics. Additionally, inference-time palm-orientation guidance enables controllable grasp generation without retraining. Across three robotic hands and 30 object--hand pairs, DexGraspFlow achieves a 95.33% average simulation success rate, with an average generation latency, including inverse kinematics, approximately 49% lower than that of a state-of-the-art baseline. Further experiments demonstrate zero-shot adaptability to seven finger-removal variants and palm-orientation control. A model trained on demonstrations for the deployed hand achieves a 91% success rate over 100 real-robot grasping trials, supporting the framework's applicability to physical grasp execution.
Bin Zhou, Boyu Ma, Xinyu Zhou +5
MARS Lab, Nanyang Technological University, Singapore · ROKAE Robotics
Denoising generative models have recently become the dominant paradigm for dexterous grasp generation, owing to their ability to model complex grasp distributions from large-scale data. However, existing diffusion-based methods typically formulate generation as a stochastic differential equation (SDE), which often requires many sequential denoising steps and introduces trajectory instability that can lead to physically infeasible grasps. In this paper, we propose EFF-Grasp, a novel Flow-Matching-based framework for physics-aware dexterous grasp generation. Specifically, we reformulate grasp synthesis as a deterministic ordinary differential equation (ODE) process, which enables efficient and stable generation through smooth probability flows. To further enforce physical feasibility, we introduce a training-free physics-aware energy guidance strategy. Our method defines an energy-guided target distribution using adapted explicit physical energy functions that capture key grasp constraints, and estimates the corresponding guidance term via a local Monte Carlo approximation during inference. In this way, EFF-Grasp dynamically steers the generation trajectory toward physically feasible regions without requiring additional physics-based training or simulation feedback. Extensive experiments on five benchmark datasets show that EFF-Grasp achieves superior performance in grasp quality and physical feasibility, while requiring substantially fewer sampling steps than diffusion-based baselines.