FastGrasp: Learning-based Whole-Body Control Method for Fast Dexterous Grasping with Mobile Manipulators

Heng Tao; Yiming Zhong; Zemin Yang; Yuexin Ma

cs.ROApr 14, 2026

FastGrasp: Learning-based Whole-Body Control Method for Fast Dexterous Grasping with Mobile Manipulators

Authors: Heng Tao, Yiming Zhong, Zemin Yang, Yuexin Ma

Organizations: ShanghaiTech University

Abstract

Fast dexterous grasping while a mobile base remains in motion requires coordinated whole-body control and rapid adaptation to physical contact. We propose FastGrasp, a two-stage learning framework that integrates grasp guidance, whole-body control, and tactile feedback. First, a pretrained conditional variational autoencoder generates diverse grasp candidates from object point clouds. Candidates are filtered by approach direction and supporting-surface constraints, then ranked using an envelopment-based criterion combining grasp width and depth coverage measures. Second, a reinforcement learning policy jointly controls the mobile base, arm, and dexterous hand, using the selected grasp as guidance and tactile observations for online grasp adjustment. The policy is trained with domain randomization and deployed with command filtering and tactile-triggered grasp tightening. Simulation experiments show higher grasp success rates than the evaluated baselines under full and partial point-cloud observations, while real-world experiments demonstrate sim-to-real transfer across diverse object geometries.

Figures & tables

Fig. 1: FastGrasp demonstrations in simulation and the real world.

Fig. 2: Real-world robot platform and simulation setup.

Fig. 3: Overview of the FastGrasp framework.

Method	Full Point Cloud			Partial Point Cloud
Method	Easy (60)	Hard (36)	Average	Easy (60)	Hard (36)	Average
Reactive Mobile Manipulation [ 4 ]	10.11(7.58)	5.33(8.11)	8.31(7.78)	10.11(7.58)	5.33(8.11)	8.31(7.78)
One-Stage (OS)	3.13(7.38)	2.87(7.72)	3.03(7.51)	0(-)	0(-)	0(-)
TS + FS [ 35 ]	52.43(1.95)	24.60(3.11)	41.99(2.39)	0(-)	0(-)	0(-)
Ours (TS + ES)	59.50(1.62)	34.42(1.73)	50.09(1.67)	47.17(2.67)	24.09(3.97)	38.51(3.21)

TABLE I: Simulation results on unseen objects. S.R. (%) and HO.D. (cm) are reported as S.R. (HO.D.). Each object is evaluated over

10\times 500

grasp trials. A dash indicates that HO.D. is unavailable because no successful grasp is obtained. OS: one-stage; TS: two-stage; FS: force-direction-based grasp selection adapted from [ 35 ] ; ES: hand-envelopment-based grasp selection.

Reward / penalty
Variant	Easy (60)	Hard (36)	Avg.
w/o $R_{\text{radius}}$	0(-)	0(-)	0(-)
w/o $R_{\text{move}}$	0(-)	0(-)	0(-)
w/o $R_{\text{ori}}$	0(-)	0(-)	0(-)
w/o $R_{\text{height}}$	32.11(2.86)	22.67(2.49)	28.57(2.72)
w/o $R_{\text{pre}}$	0(-)	0(-)	0(-)

TABLE II: Ablation study. Each entry reports S.R. (%) with HO.D. (cm) in parentheses. A dash denotes unavailable HO.D. due to zero successful grasps.

Method	$v_{f}^{\max}=1.3\,\mathrm{m/s}$ , $v_{y}^{\max}=1.0\,\mathrm{rad/s}$			$v_{f}^{\max}=0.65\,\mathrm{m/s}$ , $v_{y}^{\max}=0.5\,\mathrm{rad/s}$
Method	Easy (10)	Hard (6)	Avg.	Easy (10)	Hard (6)	Avg.
Ours w/o DR	0	0	0	0	0	0
Ours w/o LPF	0	0	0	0	0	0
Ours w/o TA	31	20	26.9	33	24	29.6
Ours	35	27	32.0	38	29	34.6

TABLE III: Real-world S.R. (%). Each object is evaluated in 10 consecutive trials.

Fig. 6: Training reward curves over episodes. The red curve denotes the total reward, while the blue curves show reward components associated with different phases of grasp execution.

Explore similar work

Learning Dexterous Grasping from Sparse Taxonomy Guidance
Apr 5, 2026Juhan Park, Taerim Yoon, Seungmin Kim +10Robotic Grasping Taxonomy
Department of Artificial Intelligence, Korea University, Seoul, Korea · Korea University of Technology and Education, Cheonan, Korea · Naver AI Lab, Korea +4
DynaMOMA: Instantaneous Prediction of Grasp Poses for Mobile Manipulation of Dynamic Objects
Jun 24, 2026Zhinan Yu, Junyan Xu, Jiazhao Zhang +9Whole-Body Control Dynawm
National University of Defense Technology · CFCS, Peking University · Defense Innovation Institute, Academy of Military Sciences +2
Steerable and Reactive Grasping Through Modular Design with a Three-Point Interface
Sep 21, 2026Andrew Nguyen, Yonghyeon Lee, Sangbae KimRobotic Grasping Proprioception
University of Michigan, Ann Arbor, MI, USA. · Department of Mechanical Engineering, MIT, Cambridge, MA, USA. · Department of AI, Yonsei University, Seoul, South Korea.

Apr 5, 2026, cs.ROJ/K move · Enter open · S save

Learning Dexterous Grasping from Sparse Taxonomy Guidance

Juhan Park, Taerim Yoon, Seungmin Kim, Joong-Gil Kim+9

Department of Artificial Intelligence, Korea University, Seoul, Korea · Korea University of Technology and Education, Cheonan, Korea · Naver AI Lab, Korea+4

Robotic Grasping Taxonomy

Open paper AskPDF arXiv

Dexterous manipulation requires planning a grasp configuration suited to the object and task, which is then executed through coordinated multi-finger control. However, specifying grasp plans with dense pose or contact targets for every object and task is impractical. Meanwhile, end-to-end reinforcement learning from task rewards alone lacks controllability, making it difficult for users to intervene when failures occur. To this end, we present GRIT, a two-stage framework that learns dexterous control from sparse taxonomy guidance. GRIT first predicts a taxonomy-based grasp specification from the scene and task context. Conditioned on this sparse command, a policy generates continuous finger motions that accomplish the task while preserving the intended grasp structure. Our result shows that certain grasp taxonomies are more effective for specific object geometries. By leveraging this relationship, GRIT improves generalization to novel objects over baselines and achieves an overall success rate of 87.9%. Moreover, real-world experiments demonstrate controllability, enabling grasp strategies to be adjusted through high-level taxonomy selection based on object geometry and task intent.

Abstract

Explore similar work

Learning Dexterous Grasping from Sparse Taxonomy Guidance

DynaMOMA: Instantaneous Prediction of Grasp Poses for Mobile Manipulation of Dynamic Objects

Steerable and Reactive Grasping Through Modular Design with a Three-Point Interface