Dexterous grasping is fundamental to robotics, yet data-driven grasp prediction heavily relies on large, diverse datasets that are costly to generate and typically limited to a narrow set of gripper morphologies. Analytical grasp synthesis can be used to scale data collection, but necessary simplifying assumptions often yield physically infeasible grasps that need to be filtered in high-fidelity simulators, significantly reducing the total number of grasps and their diversity. We propose a scalable generate-and-refine pipeline for synthesizing large-scale, diverse, and physically feasible grasps. Instead of using high-fidelity simulators solely for verification and filtering, we leverage them as an optimization stage that continuously improves grasp quality without discarding precomputed candidates. More specifically, we initialize an evolutionary search with a seed set of analytically generated, potentially suboptimal grasps. We then refine these proposals directly in a high-fidelity simulator (Isaac Sim) using an asynchronous, gradient-free evolutionary algorithm, improving stability while maintaining diversity. In addition, this refinement stage can be guided toward human preferences and/or domain-specific quality metrics without requiring a differentiable objective. We further distill the refined grasp distribution into a diffusion model for robust real-world deployment, and highlight the role of diversity for both effective training and during deployment. Experiments on a newly introduced Handles dataset and a DexGraspNet subset demonstrate that our approach achieves over 120 distinct stable grasps per object (a 1.7-6x improvement over unrefined analytical methods) while outperforming diffusion-based alternatives by 46-60% in unique grasp coverage.
Figures & tables
Fig. 1 : Visualization of Generated Grasps using our Evolutionary Refinement visualized for two Handles and one object asset. Our evolutionary approach generates diverse, physically stable grasps by refining candidates directly within high-fidelity simulation.
Fig. 2 : Method Overview. Given object meshes, we first generate an initial set of analytical grasp proposals. We then refine these candidates in a high-fidelity simulator using an asynchronous gradient-free evolutionary loop (selection, crossover, mutation, and physics-based evaluation) to obtain a diverse set of physically feasible grasps. Finally, we distill the refined grasp distribution into a diffusion model conditioned on noisy pointcloud observations, enabling efficient sampling at inference time under partial observability.
Fig. 3 : Contact Point Selection for Dexterous Grasping. Two views of a dexterous hand (Xhand) grasping a cup from the Objects dataset. Light blue points indicate candidate contact points on the object surface that lie within the contact distance threshold, while dark blue points denote points outside this threshold. Green points represent the final active contact points selected via farthest point sampling (FPS), which are used to compute grasping commands through the contact Jacobian.
Fig. 4 : Selection of our IKEA Handles Dataset. We provide 90 geometrically distinct handle assets with multiple texture variations, totaling 190 unique simulation-ready assets. Each handle includes high-resolution collision geometry, realistic textures, and articulated joints with compliant degrees of freedom to support stable grasp synthesis. All assets are based on commercially available products from ikea.com and can be directly used in Isaac Sim.
Fig. 5 : Grasp Quality and Diversity Comparison. Radar plots comparing synthesized grasp distributions across different generation strategies. We report grasp stability (success rate), uniqueness of successful grasps at different thresholds (DSG@2cm/DSG@20cm), and diversity via marginal entropies of hand position, orientation, and joint angles. Overall, evolutionary refinement improves stability while maintaining high diversity across both datasets.
Fig. 6 : Convergence of Evolutionary Refinement. Mean distinct stable grasps (blue, left axis) and mean entropy (red, right axis) as a function of simulator environment steps, initialized with 32 analytical seeds. Both metrics exhibit rapid improvement during the first 2.5k steps, then converge to a stable plateau by approximately 10k steps. The simultaneous saturation of both stability and diversity metrics shows that evolutionary refinement efficiently balances grasp quality and coverage within a limited interaction budget.
Fig. 7 : Effect of Diffusion Training vs Evolutionary Refinement. Mean distinct stable grasps for GraspQP, a diffusion model trained on GraspQP samples, and evolutionary refinement, showing that diffusion improves over the raw initialization but remains below evolutionary sampling.
Fig. 8 : Effect of Preference Alignment. Comparison of grasps without (left) and with (right) preference alignment from human feedback. The un-guided evolutionary distribution produces diverse but unconventional grasps, while the preference-aligned version generates grasps that better match human intent and natural grasping patterns.
Fig. 9 : Real-world Cabinet-Handle Rollout. We sample handle grasps from a point-cloud-conditioned diffusion model trained on refined grasp distributions, then plan collision-aware trajectories using cuRobo. These are executed on a Franka Panda arm equipped with a dexterous hand. Three executions across different grasp modes and approach directions demonstrate successful sim-to-real transfer in contact-rich manipulation.
Fig. 10 : Reachability analysis comparing 7-DoF and 6-DoF robot arms. We compare the number of distinct stable grasps achievable across different handle positions. Grasp candidates are generated by GraspQP (solid colors) and our evolutionary approach (hatched) using 32 and 128 seed grasps on 50 handle assets. For 12 sampled handle positions, we report: (a) Minimal Reachability: mean minimum reachable grasps across all object poses; (b) Average Reachability: mean reachable grasps across all poses; (c) Maximum Reachability: mean maximum reachable grasps for the most favorable pose. Error bars indicate standard deviation.
Fig. 11 : Diffusion-based grasp generation. Visualization of the denoising process for grasp pose prediction on a cup (top row) and bar-style handle (bottom row). The sequence shows the evolution from the initial noisy distribution (T=100) to the final predicted grasp pose (T=0) at 20-step intervals for three different grasp samples.
Fig. 12 : Real-world point cloud reconstructions. We capture depth data for objects used in hardware experiments and reconstruct 3D point clouds through our multi-view fusion pipeline. Top row shows cabinet handles with their reconstructed point clouds. Bottom row shows real-world objects paired with their corresponding point cloud reconstructions. These point clouds serve as input to the diffusion model for grasp prediction.
Fig. 13 : Cabinet opening with knob-style handle. Four successful grasp strategies for opening a cabinet with a knob-style handle . Each row shows a different approach, with frames capturing the approach, grasp, and opening phases.
Fig. 14 : Cabinet opening with bar-style handle. Four successful grasp strategies for opening a cabinet with a bar-style handle . Each row shows a different approach, with frames capturing the approach, grasp, and opening phases.
Fig. 15 : Cabinet opening with knob-style handle (variant 2). Four successful grasp strategies for opening a cabinet with a knob-style handle . Each row shows a different approach, with frames capturing the approach, grasp, and opening phases.
Fig. 16 : Cabinet opening with shell-style handle. Four successful grasp strategies for opening a cabinet with a shell-style handle . Each row shows a different approach, with frames capturing the approach, grasp, and opening phases.
Fig. 17 : Grasping household objects. Successful grasp execution on a mug (top row) and bottle (bottom row). Frames show the approach, pre-grasp configuration, and final stable grasp for each object.
Fig. 18 : Lifted object poses. Successfully grasped and lifted household objects showing final stable configurations: tea box, toy figure, cup, and bottle (clockwise from top left).
Fig. 19 : Motion planning with cuRobo. Collision-aware trajectory planning and execution for cabinet manipulation. The environment is approximated using simplified collision geometries: cubic representations for the floor and cabinet, and voxelized models for the handle point cloud. The sphere representation used for the robot is shown on the left. The sequence (left to right) shows successful approach and grasp execution on a bar-style handle.
Fig. 20 : Preference annotation interface. The user interface used for collecting pairwise preference labels over grasp pairs. Annotators compare two grasps and select which appears more natural and human-intent-aligned, or mark them as similar.
Synthesizing high-quality dexterous grasps is a fundamental challenge in robot manipulation, requiring adherence to diversity, kinematic feasibility (valid hand-object contact without penetration), and dynamic stability (secure multi-contact forces). The recent framework Dexonomy successfully ensures broad grasp diversity through dense sampling and improves kinematic feasibility via a simulator-based refinement method that excels at resolving exact collisions. However, its reliance on fixed contact points restricts the hand's reachability and prevents the optimization of grasp metrics for dynamic stability. Conversely, purely gradient-based optimizers can maximize dynamic stability but rely on simplified contact approximations that inevitably cause physical penetrations. To bridge this gap, we propose GraspADMM, a novel grasp synthesis framework that preserves sampling-based diversity while improving kinematic feasibility and dynamic stability. By formulating the refinement stage using the Alternating Direction Method of Multipliers (ADMM), we decouple the target contact points on the object from the actual contact locations on the hand. This decomposition allows the pipeline to alternate between updating the target object points to directly maximize dynamic grasp metrics, and adjusting the hand pose to physically reach these targets while strictly respecting collision boundaries. Extensive experiments demonstrate that GraspADMM significantly outperforms state-of-the-art baselines, achieving a nearly 15% absolute improvement in grasp success rate for type-unaware synthesis and roughly a 100% relative improvement in type-aware synthesis. Furthermore, our approach maintains robust, physically plausible grasp generation even under extreme low-friction conditions.
Liangwang Ruan, Jiayi Chen, He Wang +1
School of Computer Science, Peking University. · Galbot. · School of Intelligence Science and Technology, Peking University.
Denoising generative models have recently become the dominant paradigm for dexterous grasp generation, owing to their ability to model complex grasp distributions from large-scale data. However, existing diffusion-based methods typically formulate generation as a stochastic differential equation (SDE), which often requires many sequential denoising steps and introduces trajectory instability that can lead to physically infeasible grasps. In this paper, we propose EFF-Grasp, a novel Flow-Matching-based framework for physics-aware dexterous grasp generation. Specifically, we reformulate grasp synthesis as a deterministic ordinary differential equation (ODE) process, which enables efficient and stable generation through smooth probability flows. To further enforce physical feasibility, we introduce a training-free physics-aware energy guidance strategy. Our method defines an energy-guided target distribution using adapted explicit physical energy functions that capture key grasp constraints, and estimates the corresponding guidance term via a local Monte Carlo approximation during inference. In this way, EFF-Grasp dynamically steers the generation trajectory toward physically feasible regions without requiring additional physics-based training or simulation feedback. Extensive experiments on five benchmark datasets show that EFF-Grasp achieves superior performance in grasp quality and physical feasibility, while requiring substantially fewer sampling steps than diffusion-based baselines.
Dexterous grasp synthesis has advanced rapidly in generating stable and physically plausible hand poses, but real-world manipulation requires grasps that preserve the function implied by the task. We study open-vocabulary task-oriented dexterous grasp generation, where a robot must infer functional intent from free-form language, ground it in multi-view visual observations and object geometry, and generate an executable high-degree-of-freedom grasp. We present OpenDexGrasp, a unified data and generative modeling framework for this setting. OpenDexVerse provides dual-source supervision organized by the Coverage-to-Alignment (C2A) Recipe: OpenDex-Scale offers large-scale semantic and geometric coverage through automatic grasp synthesis and vision-language annotation, while OpenDex-Align supplies high-quality embodied alignment through human teleoperation and category-level transfer. OpenDexGrasp learns a shared perception-action latent representation that couples open-vocabulary vision-language context with dexterous action generation. Affordance grounding and grasp generation provide complementary supervision over this latent space, enabling direct generation of task-consistent dexterous grasps without a separate affordance-to-pose inference stage. Extensive simulation and real-robot experiments demonstrate improved functional alignment, physical feasibility, generalization to unseen categories, and real-world execution success. Additional details and videos are available at https://opendexgrasp.github.io/.
Jiyao Zhang, Junhan Wang, Tianyu Wang +4
CFCS, School of CS, PKU, China · National Key Laboratory for Multimedia Information Processing, School of CS, PKU, China · PrimeBot +1