Affordance-Guided Enveloping Grasp Demonstration Toward Non-destructive Disassembly of Pinch-Infeasible Mating Parts
Authors: Masaki Tsutsumi, Takuya Kiyokawa, Gen Sako, Kensuke Harada
Organizations: Department of Systems Innovation, Graduate School of Engineering Science, The University of Osaka, 1-3 Machikaneyama, Toyonaka, Osaka, Japan. · Industrial Cyber-physical Systems Research Center, The National Institute of Advanced Industrial Science and Technology (AIST), 2-3-26 Aomi, Koto-ku, Tokyo, Japan.
Robotic disassembly of complex mating components often renders pinch grasping infeasible, necessitating multi-fingered enveloping grasps. However, visual occlusions and geometric constraints complicate teaching appropriate grasp motions when relying solely on 2D camera feeds. To address this, we propose an affordance-guided teleoperation method that pre-generates enveloping grasp candidates via physics simulation. These Affordance Templates (ATs) are visualized with a color gradient reflecting grasp quality to augment operator perception. Simulations demonstrate the method's generality across various components. Real-robot experiments validate that AT-based visual augmentation enables operators to effectively select and teach enveloping grasp strategies for real-world disassembly, even under severe visual and geometric constraints.
Figures & tables
Fig. 1 : Overview of the proposed affordance-guided teleoperation framework. (a) illustrates the system mapping human hand motions to the underactuated robot hand. (b) outlines the procedure: physics-based AT generation, visual presentation, mapping of tracked human motions, and physical execution.
Fig. 2 : Pinch-infeasible components to be disassembled. (a) Object A requires extraction from a narrow lateral slot. (b) Object B has a wide, flat profile with fragile edges. (c) Object C is a heavy, box-like metallic component located deep within the housing.
Fig. 3 : Sequential process of AT generation in the physics simulator. The left shows the valid sampling region and point cloud evaluation, the center displays the rigid-body interaction during grasp execution, and the right presents the successfully extracted affordance template.
Fig. 4 : Generated affordance templates for target objects A, B, and C. High quality ( dh : Small) grasps are rendered in red.
Fig. 5 : The humanoid multi-fingered hand capable of enveloping grasps and comparison with a human hand.
Fig. 6 : Sequential snapshots of the real-world disassembly process for Object A. The sequence illustrates three key stages: (1) approaching the target via spatial wrist tracking, (2) executing the enveloping grasp guided by the visual AT overlay, and (3) successfully extracting the mating component.
AT Quality Level
Success Rate [%]
High ( dh : Small)
90
Medium ( dh : Mid)
100
Low ( dh : Large)
80
TABLE I : Success rates of real-world disassembly trials for Object A (10 independent trials per configuration).
Fig. 7 : Executed enveloping grasps guided by ATs. The operator successfully modulated finger apertures to match varying geometric demands: (a) a lateral grasp for Object A, (b) a wide thumb spread for the broad profile of Object B, and (c) a deep, hook-like enveloping strategy to securely extract the heavy Object C.
Grasping and holding tools while using them presents a considerable challenge not only for robots but also for humans. Such a challenge is particularly noticeable in processes involving assembly and disassembly, where efficiency and consistency depend on performing rapidly adaptive tasks. Nonetheless, contemporary robotic grasping technologies that can securely manipulate tools during operation frequently have significant constraints. In this paper, introduce ARTiS (Adaptive Robotic Tool Gripper in Disassembly Systems), a novel gripper that combines the adaptability of soft grippers, the dexterity of anthropomorphic hands, and the robustness of rigid mechanisms with a soft palm and fingertips. This unique combination makes it possible to hold tools securely in a variety of situations through using active jamming in the palm and fin-ray adaptation in fingertips. Furthermore, high finger dexterity is achieved through the seven degrees of freedom design, which enables the fingertips to orient to any surface, both for automated solutions and collaborative tasks. A comprehensive evaluation was conducted using a range of conventional disassembly tools to assess the gripper's compliance, durability, and functional versatility. More information, hardware instructions, and videos at https://romanmykhailyshyn.github.io/artis/
Roman Mykhailyshyn, Yukiyasu Domae, Kensuke Harada
Embodied AI Research Team, National Institute of Advanced Industrial Science and Technology (AIST), Tokyo 135-0064, Japan · Graduate School of Engineering Science, The University of Osaka, Osaka 560-8531, Japan
This paper proposes a refinement to existing vision-based portable pipelines for the recognition of graspable object parts. Such pipelines can effectively support automatic grasp selection and preshaping in upper-limb prosthetic devices. However, realistic scenarios in which multiple objects are present still represent a challenge. The proposed method exploits a minor modification to the conventional hardware setup and a custom training approach to extend the applicability of vision-based pipelines. A laser pointer is introduced in the setup as an effective mechanism for object disambiguation. Accordingly, a suitable retraining of the deep convolutional neural networks supporting recognition of graspable parts is implemented. A mixed reality strategy allows the reuse of available high-quality segmentation masks to generate realistic images procedurally, extending the range of scenarios supported by the final device. Experimental results confirm that the proposed pipeline improves target disambiguation accuracy by up to 9% compared with existing solutions, while preserving the computational cost.
Task-oriented grasping (TOG) requires robots to grasp functional parts of objects (e.g., the handle of a mug for pouring), yet these affordance regions are frequently occluded in cluttered scenes. Active perception via next-best-view (NBV) planning can resolve such occlusions by moving the camera for more informative observations. However, existing NBV methods typically optimize viewpoints for grasping the target object as a whole without distinguishing which part is task-relevant. A naive adaptation, fully scanning the target object before predicting the affordance, wastes most of the viewpoint budget on task-irrelevant surfaces (e.g., the mug body for pouring). To address this, we propose ATAP, an Affordance-Targeted Active Perception framework that shifts viewpoint planning from exhaustive target scanning to targeted affordance verification. ATAP hypothesizes the occluded target geometry via a generative shape prior and predicts the affordance distribution over the imagined complete surface. In cluttered scenes, severe occlusion can make the location of the hidden affordance ambiguous, leaving multiple locations plausible given the partial observation. ATAP therefore introduces an uncertainty-aware viewpoint planner that jointly optimizes expected entropy reduction over these competing hypotheses and expected affordance verification gain from real observations. This process iterates until the affordance is sufficiently verified for grasp execution. Experiments in simulation and real-world cluttered scenes show that ATAP substantially improves the functional grasp success rate over fixed-view TOG baselines, and outperforms reconstruction-based active perception with over 57% fewer NBV steps.
Jingzhi Cui, Xuefeng Liu, Feng Han +3
School of Computer Science and Engineering, Beihang University, Beijing, China · School of Artificial Intelligence, Beihang University, Beijing, China