DeViGrasp: Robust Visual Mobile Grasping for Quadruped Manipulators under Degraded Perception

Liang Zhou; Jiaming Su; Yancong Wei; Kangkang Dong; Houde Liu

cs.ROSep 12, 2026

DeViGrasp: Robust Visual Mobile Grasping for Quadruped Manipulators under Degraded Perception

Authors: Liang Zhou, Jiaming Su, Yancong Wei, Kangkang Dong, Houde Liu

Organizations: Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Shanghai Jiao Tong University Global College, Shanghai Jiao Tong University, Shanghai, China

Abstract

Quadruped manipulators enable mobile grasping in complex environments, yet their whole-body control policies remain vulnerable to unreliable onboard visual perception. Existing methods are typically developed under relatively reliable observations and have not systematically examined how occlusion, segmentation-mask dropout, depth noise, and target-localization jitter affect grasp reasoning and target tracking. To address this gap, we introduce DeViGrasp-Bench, a benchmark for mobile grasping under degraded vision that incorporates controlled visual degradations, seen and unseen objects, multiple difficulty levels, and complex terrains, and evaluates task success, execution efficiency, and action smoothness. We further propose DeViGrasp-Net, a teacher--student framework that combines state-conditioned grasp reasoning with reliability-aware temporal target estimation. The privileged teacher attends to offline grasp candidates conditioned on object, robot, end-effector, and task states, while the deployable student fuses dual-view segmented-depth observations with current, memory, and recovery target hypotheses through Target Hold Memory and Temporal Memory Attention. DeViGrasp-Net outperforms VBC across degradation levels, unseen objects, and complex terrains, and surpasses an adapted DQ-Net across all evaluated degradation levels. Under the Difficult setting, it achieves a success rate of 62.3%, improving upon VBC and DQ-Net by 16.1 and 4.3 percentage points, respectively; under the Hard setting, its margin over DQ-Net increases to 10.5 percentage points. Ablation studies confirm the complementary benefits of grasp-aware supervision and reliability-aware temporal memory.

Figures & tables

Figure 1: Overview of DeViGrasp-Bench . It evaluates quadruped mobile grasping across diverse objects, terrain conditions, and visual degradation levels, including depth noise, mask dropout, occlusion, and target localization jitter.

Level	Depth	Dropout	Occ. P.	Occ. S.	Jitter Std	Jitter Clip
Easy	0.05	0.10	0.10	0.25	0.01	0.03
Medium	0.15	0.20	0.20	0.40	0.02	0.05
Difficult	0.30	0.35	0.30	0.50	0.04	0.07
Hard	0.45	0.50	0.40	0.60	0.06	0.09

Table 1: Visual degradation levels in DeViGrasp-Bench.

Figure 2: Overview of DeViGrasp-Net . A privileged teacher learns grasp-aware whole-body actions from object, robot, task, and grasp-candidate information, while a deployable student uses forward/wrist segmented depth observations and temporal memory attention for robust target estimation under degraded perception.

Figure 3: Visualization of Target Hold Memory under mask dropout and partial occlusion. Memory-retained observations preserve usable target evidence when current observations are corrupted.

Figure 4: Qualitative comparison on complex terrains. DeViGrasp-Net achieves stable grasps across rough flat, slope, and stairs, while VBC often fails from target-tracking and end-effector misalignment.

Model	Easy				Medium				Difficult				Hard
Model	AES $\downarrow$	ASTS $\downarrow$	AS-L1 $\downarrow$	SR $\uparrow$	AES $\downarrow$	ASTS $\downarrow$	AS-L1 $\downarrow$	SR $\uparrow$	AES $\downarrow$	ASTS $\downarrow$	AS-L1 $\downarrow$	SR $\uparrow$	AES $\downarrow$	ASTS $\downarrow$	AS-L1 $\downarrow$	SR $\uparrow$
VBC	100.9	61.2	0.372	43.0	93.6	67.3	0.370	47.2	94.9	66.2	0.370	46.2	108.3	79.7	0.372	29.3
VBC+EMA	91.4	62.8	0.366	47.8	98.5	64.8	0.365	38.5	104.2	70.3	0.361	37.9	107.6	74.7	0.362	28.6
DQ-Net	74.4	51.8	0.339	64.5	74.8	52.7	0.339	66.2	79.5	55.1	0.336	58.0	93.8	61.1	0.337	44.3
DeViGrasp-Net	75.3	54.8	0.291	70.2	76.4	55.0	0.290	68.3	82.6	58.3	0.289	62.3	88.1	61.6	0.287	54.8

Table 2: Main results under different visual degradation levels on flat terrain. Results are averaged over three random seeds; full mean

\pm

std statistics are provided in the supplementary material.

Setting	AES $\downarrow$	ASTS $\downarrow$	AS-L1 $\downarrow$	SR (%) $\uparrow$
VBC	94.9	66.2	0.370	46.2
w/ GA	89.1	64.8	0.278	53.5
w/ GA + THM	85.5	66.2	0.282	57.1
w/ GA + THM + TMA	82.6	58.3	0.289	62.3

Table 3: Ablation study under the Difficult visual degradation level. Results are averaged over three random seeds.

Model	Terrain	AES $\downarrow$	ASTS $\downarrow$	AS-L1 $\downarrow$	SR (%) $\uparrow$
VBC	R. Flat	130.6	81.6	0.311	7.9
	R. Slope	131.5	82.9	0.311	6.4
	R. Stairs	130.3	66.5	0.328	10.1
DeViGrasp-Net	R. Flat	92.1	64.1	0.290	58.4
	R. Slope	89.1	65.7	0.291	61.5
	R. Stairs	111.8	65.7	0.293	34.2

Table 4: Complex-terrain evaluation under Difficult visual degradation. Results are averaged over three random seeds.

Method	GSR-T	GSR-S	OSSR
DQ-Net	80.8	55.8	53.2
DeViGrasp-Net	85.2	60.7	57.6

Table 5: Generalization to low-speed dynamic targets on DQ-Bench Level 1. GSR-T and GSR-S denote teacher and student grasp success rates, respectively, while OSSR denotes the one-shot success rate. Results are averaged over three random seeds.

Figure 5: Generalization to unseen object instances under the Difficult degradation setting.

Robotic Grasping

Robotic grasping studies how robots perceive, plan, and execute stable, task-appropriate contacts with objects across varied shapes and conditions.

See all 316 papers

May 4, 2026

Visibility-Aware Mobile Grasping in Dynamic Environments

We consider mobile grasping in unknown and dynamic environments, where a robot must approach and grasp a target object using a camera with a limited field of view. This setting introduces two visibility constraints. Incomplete environmental observations can create collision risks along the whole-body trajectory, while occlusions can hinder object pose refinement during grasping. Previous systems often assume known or static environments and decouple perception from motion, leaving safety unresolved around unobserved obstacles. We present a unified system that enforces both visibility constraints at runtime through three core components. A velocity-aware gaze policy directs perception toward fast-moving regions of the planned swept volume, a hierarchical subgoal policy samples grasp and intermediate configurations that maintain target visibility, and a real-time whole-body planner replans over a receding horizon as the map updates. Across 400 randomized simulation scenarios and real-world trials on a Fetch mobile manipulator, our system achieves success rates of 70.4% in unknown static environments and 57.9% in dynamic environments, outperforming the decoupled navigation- and-manipulation strategy by 16.3 and 9.3 percentage points, respectively, while significantly reducing collisions.

Tianrun Hu, Anxing Xiao, David Hsu +1

School of Computing, National University of Singapore, Singapore. · Smart Systems Institute, National University of Singapore, Singapore. · Shanghai Innovation Institute, Shanghai, China.

Active Perception Constrained Motion Planning Robotic Grasping Motion Planning Dynamic Obstacle Avoidance

Ask PDF

Apr 14, 2026

FastGrasp: Learning-based Whole-Body Control Method for Fast Dexterous Grasping with Mobile Manipulators

Fast dexterous grasping while a mobile base remains in motion requires coordinated whole-body control and rapid adaptation to physical contact. We propose FastGrasp, a two-stage learning framework that integrates grasp guidance, whole-body control, and tactile feedback. First, a pretrained conditional variational autoencoder generates diverse grasp candidates from object point clouds. Candidates are filtered by approach direction and supporting-surface constraints, then ranked using an envelopment-based criterion combining grasp width and depth coverage measures. Second, a reinforcement learning policy jointly controls the mobile base, arm, and dexterous hand, using the selected grasp as guidance and tactile observations for online grasp adjustment. The policy is trained with domain randomization and deployed with command filtering and tactile-triggered grasp tightening. Simulation experiments show higher grasp success rates than the evaluated baselines under full and partial point-cloud observations, while real-world experiments demonstrate sim-to-real transfer across diverse object geometries.

Heng Tao, Yiming Zhong, Zemin Yang +1

ShanghaiTech University

RL for Robotics Robotic Grasping Mobile Manipulation Whole-Body Robotic Manipulation Whole-Body Control

Ask PDF

Jul 15, 2026

Beyond Visual Grasping: Benchmarking Complex Grasping from Detection to Execution

Robust robotic grasping remains a fundamental challenge for complex real-world applications. Recent advances in large-scale models demonstrate promising capabilities for reasoning in robotic tasks. However, existing benchmarks for grasping primarily focus on isolated, visual-based grasp pose detection, failing to capture the complexity of grasping tasks that require multi-step reasoning and semantic understanding during execution. To address this gap, we propose GCA-Bench, a benchmark featuring challenging grasping with complex action scenarios that involve both scene-level reasoning and semantic constraints. GCA-Bench enables the evaluation of recent large foundation models under the same settings. To demonstrate the effectiveness of our new benchmark, we implement a diverse set of baselines, ranging from traditional grasp detection pipelines to end-to-end learning methods. Empirical studies achieve success rates below 70% on complex grasping scenarios, underscoring critical limitations. In addition, we propose new evaluation metrics, analyze critical failure models, and provide insights to guide the development of more robust and generalizable grasping strategies.

Hanyi Zhang, Khang Nguyen, Charith Munasinghe +10

University of Liverpool, UK · Mohamed bin Zayed University of Artificial Intelligence, UAE · Zurich University of Applied Science, Switzerland +3

Robotic Grasping Robot Foundation Models Robot Manipulation Benchmarks

Ask PDF

Abstract

Robotic Grasping

Explore similar work

Visibility-Aware Mobile Grasping in Dynamic Environments

FastGrasp: Learning-based Whole-Body Control Method for Fast Dexterous Grasping with Mobile Manipulators

Beyond Visual Grasping: Benchmarking Complex Grasping from Detection to Execution