GraspSense: Physically Grounded Grasp and Grip Planning for a Dexterous Robotic Hand via Language-Guided Perception and Force Maps

Elizaveta Semenyakina; Ivan Snegirev; Mariya Lezina; Miguel Altamirano Cabrera; Safina Gulyamova; Dzmitry Tsetserukou

cs.ROApr 7, 2026

GraspSense: Physically Grounded Grasp and Grip Planning for a Dexterous Robotic Hand via Language-Guided Perception and Force Maps

Authors: Elizaveta Semenyakina, Ivan Snegirev, Mariya Lezina, Miguel Altamirano Cabrera, Safina Gulyamova, Dzmitry Tsetserukou

Organizations: Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology

Abstract

Dexterous robotic manipulation requires more than geometrically valid grasps: it demands physically grounded contact strategies that account for the spatially non-uniform mechanical properties of the object. However, existing grasp planners typically treat the surface as structurally homogeneous, even though contact in a weak region can damage the object despite a geometrically perfect grasp. We present a pipeline for grasp selection and force regulation in a five-fingered robotic hand, based on a map of locally admissible contact loads. From an operator command, the system identifies the target object, reconstructs its 3D geometry using SAM3D, and imports the model into Isaac Sim. A physics-informed geometric analysis then computes a force map that encodes the maximum lateral contact force admissible at each surface location without deformation. Grasp candidates are filtered by geometric validity and task-goal consistency. When multiple candidates are comparable under classical metrics, they are re-ranked using a force-map-aware criterion that favors grasps with contacts in mechanically admissible regions. An impedance controller scales the stiffness of each finger according to the locally admissible force at the contact point, enabling safe and reliable grasp execution. Validation on paper, plastic, and glass cups shows that the proposed approach consistently selects structurally stronger contact regions and keeps grip forces within safe bounds. In this way, the work reframes dexterous manipulation from a purely geometric problem into a physically grounded joint planning problem of grasp selection and grip execution for future humanoid systems.

Figures & tables

Fig. 1: Illustration of the force-map-aware grasp planning approach on a paper cup. (a) The reconstructed 3D object model with the computed force map: colour encodes the locally admissible lateral contact force at each surface point (blue – high admissible load; red – low admissible load). (b) Grasp executed using geometric feasibility criteria alone, without force-map information. (c) Grasp executed with force-map-aware re-ranking: contacts are directed toward the mechanically admissible mid-body and rim zones, and per-finger stiffness is scaled to the locally admissible force, ensuring reliable retention without structural overloading.

Fig. 2: Overview of the proposed pipeline, comprising five stages: task understanding and semantic parsing, object perception and 3D reconstruction, force map construction through physics-informed geometric approximation, grasp candidate generation with functional-goal filtering and force-map-aware re-ranking, and pre-grasp motion generation and grip execution.

Object	Zone 1 [N]	Zone 2 [N]	Base [N]
Paper cup	rim: 41.2	wall: 4.8	88.8
Plastic cup	rim: 3.7	wall: 0.3	8.0
Glass goblet	bowl: 69	stem: 565	989

TABLE I: Admissible Lateral Force per Structural Zone.

Object	Method	Zone	$\bm{F^{*}_{\min}}$	Margin	Viol.
Paper cup	Baseline	wall	4.8	1.4	1/5
	Ours	rim	41.2	7.8	0/5
Plastic cup	Baseline	wall	0.3	1.2	2/5
	Ours	rim	3.7	5.9	0/5
Goblet	Baseline	bowl	69.0	2.1	1/5
	Ours	stem	565	12.4	0/5

TABLE II: Grasp Selection Results. Margin

=F^{*}_{\min}/f_{\max}

; Viol.

=

fingers with

f_{k}{>}F^{*}_{k}

Panel of Figure 5 — Fig. 3: Two geometrically equivalent grasps on a glass goblet (blue $\rightarrow$ low; red $\rightarrow$ high admissible force). Left (rejected): bowl contacts; $F^{*}_{\min}{=}69$ N. Right (selected): stem contacts; $F^{*}_{\min}{=}565$ N.

Panel of Figure 6 — Fig. 4: Contact force vectors for the three controller conditions. Left : under-stiffness, object slips. Centre : over-stiffness, red vectors indicate violations. Right : proposed controller, all vectors green.

Condition	$\bm{f_{\max}}$ [N]	Margin	Viol.	Success
Under-stiffness	0.08	-	0/5	$\times$
Over-stiffness	0.74	0.41	4/5	$\times$
Ours ( $\lambda{=}0.7$ )	0.21	1.43	0/5	✓

TABLE III: Grip Execution Results, Plastic Cup (

F^{*}_{\text{wall}}{=}0.3

N).

Mode	$\bm{\lambda}$	$\bm{f_{\max}}$ [N]	Margin	Success
Gently	0.3	0.09	3.33	✓
Default	0.7	0.21	1.43	✓
Firmly	1.0	0.29	1.03	✓

TABLE IV: Effect of Interaction Mode

\lambda

(proposed controller).

Explore similar work

SECOND-Grasp: Semantic Contact-guided Dexterous Grasping
May 13, 2026Han Yi Shin, Heeju Ko, Jaewon Mun +6Robotic Grasping Language-Conditioned Robot Manipulation
1Korea University · University of Texas at Austin · University of Michigan +1
Real-Time Force Regulation for Whole-Hand Dexterous Grasping
Sep 24, 2026Sang Min Kim, Alexander Alexiev, Tzu-Yuan Lin +3Robotic Grasping Dexterous Manipulation
Department of Electrical and Computer Engineering, Seoul National University, Seoul, Republic of Korea. · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA. · Department of Artificial Intelligence, Yonsei University, Seoul, Republic of Korea.
Exp-Force: Experience-Conditioned Pre-Grasp Force Selection with Vision-Language Models
Mar 9, 2026Siqi Shang, Minchao Huang, Bill Fan +1Robotic Grasping VLMs for Robotics
The University of Texas at Austin, TX, USA.

May 13, 2026J/K move · Enter open · S save

SECOND-Grasp: Semantic Contact-guided Dexterous Grasping

Han Yi Shin, Heeju Ko, Jaewon Mun, Qixing Huang+5

1Korea University · University of Texas at Austin · University of Michigan+1

Robotic Grasping Language-Conditioned Robot Manipulation

Open paper AskPDF arXiv

Achieving reliable robotic manipulation, such as dexterous grasping, requires a synergy between physically stable interactions and semantic task guidance, yet these objectives are often treated as separate, disjoint goals. In this paper, we investigate how to integrate dexterous grasping techniques, i.e., physically stable grasps for object lifting and language-guided grasp generation, to achieve both physical stability and semantic understanding. To this end, we propose SECOND-Grasp (SEmantic CONtact-guided Dexterous Grasping), a unified framework that enables robotic hands to dynamically adjust grasping strategies based on semantic reasoning while ensuring physical feasibility. We begin by obtaining coarse contact proposals through vision-language reasoning to infer where contacts should occur based on object properties, followed by segmentation to localize these regions across views. To further ensure consistency across multiple viewpoints, we introduce Semantic-Geometric Consistency Refinement (SGCR), which refines initial contact predictions by enforcing semantic consistency across views and removing geometrically invalid regions, yielding reliable 3D contact maps. Then, we derive a feasible hand pose for each contact map via inverse kinematics, generating a supervision signal for policy learning. Our approach, trained on DexGraspNet, consistently outperforms baselines in lifting success rate on both seen and unseen categories, achieving 98.2% and 97.7%, respectively, while also improving intent-aware grasping by 12.8% and 26.2%. We further show promising results on additional datasets and robotic hands, including Shadow Hand and Allegro Hand.

Abstract

Explore similar work

SECOND-Grasp: Semantic Contact-guided Dexterous Grasping

Real-Time Force Regulation for Whole-Hand Dexterous Grasping

Exp-Force: Experience-Conditioned Pre-Grasp Force Selection with Vision-Language Models