Cross-Embodiment Affordance Transfer for Robotic Manipulation

Sawera Yaseen; Serge Thill; Rekha Raja

cs.ROOct 13, 2026

Cross-Embodiment Affordance Transfer for Robotic Manipulation

Authors: Sawera Yaseen, Serge Thill, Rekha Raja

Organizations: School of Artificial Intelligence, Donders Centre for Cognition, Department of Human-Centred Intelligent Systems, Radboud University, Nijmegen, The Netherlands.

Abstract

Robotic manipulation systems commonly learn object affordances from human-annotated datasets, which reflect interaction possibilities for a human hand rather than the physical constraints of a specific robot. For example, a surface region of an object identified as graspable by a human annotator may be geometrically unreachable or mechanically infeasible for a robot, yet this mismatch is rarely addressed explicitly. To tackle this challenge, we proposed a framework that translates human affordance annotations into robot-executable affordance predictions conditioned on the physical properties of the robot. The network predicts which surface regions of an object are physically executable by the robot, given a 3D point cloud of an object and a descriptor encoding the robot's geometry and constraints. The model was trained using human-annotated affordances together with robotic execution outcomes from multiple sampled approach trials in simulation, where execution outcomes determined which regions are executable for the robot. The evaluation was performed on a household objects dataset using two parallel-jaw gripper configurations on a UR5e robot arm. The framework outperformed a model that does not receive information about the robot. The experimental results demonstrated that affordance prediction must be conditioned on the robot and that human annotations alone are insufficient for learning robot-executable affordance representations.

Figures & tables

Fig. 1 : Human-to-robot affordance gap. Human-interpreted affordance regions do not necessarily correspond to robot-executable regions, as executability depends on the physical capabilities of the acting embodiment.

Fig. 2 : FiLM-conditioned PointNet++ for affordance segmentation. The model takes a 2048-point cloud and a 21-dimensional embodiment descriptor $\mathbf{e}$ . A per-point segmentation head produces logits over three classes: grasp, wrap-grasp, and background.

Fig. 3 : Experimental setup in the PyBullet simulation. It consists of a UR5e manipulator with interchangeable Robotiq 2F-85 and 2F-140 grippers on a tabletop workspace.

Method	Affordance	Prec.	Rec.	F1	IoU
Embodiment-conditioned	Grasp	44.31	89.66	59.31	42.15
	Wrap-Grasp	99.92	65.16	78.88	65.12
	Macro Avg.	72.11	77.41	69.09	53.64
No-embodiment baseline	Grasp	43.32	86.61	57.76	40.60
	Wrap-Grasp	99.83	64.39	78.29	64.32
	Macro Avg.	71.58	75.50	68.02	52.46

TABLE I : Affordance segmentation performance and embodiment-conditioning ablation.

\Delta

denotes the mIoU improvement of the embodiment-conditioned model over the no-embodiment baseline.

Affordance	Embodiment	Prec.	Rec.	F1	IoU
Grasp	2F-85	44.72	90.53	59.87	42.72
	2F-140	43.71	87.26	58.25	41.09
	$\Delta$	-1.01	-3.27	-1.62	-1.63
Wrap-Grasp	2F-85	99.45	26.67	42.06	26.63
	2F-140	99.80	63.34	77.49	63.26
	$\Delta$	+0.35	+36.67	+35.43	+36.63

TABLE II : Cross gripper transfer performance. The grippers used are Robotiq 2F-85 and 2F-140. Each model is trained on one gripper and evaluated on another.

\Delta

is computed as 2F-140

-

2F-85 .

Panel of Figure 6 — Fig. 4 : Qualitative results across seen and unseen objects. Each column corresponds to an object, while rows show the RGB observation, depth representation, reconstructed point cloud, and predicted affordance map.

Object	Dimensions (mm)	Robotiq 2F-85		Robotiq 2F-140
Object	Dimensions (mm)	Grasp	Wrap-Grasp	Grasp	Wrap-Grasp
Spoon	186 $\times$ 83 $\times$ 21	✓	—	✓	—
Fork	198 $\times$ 27 $\times$ 16	✓	—	✓	—
Knife	215 $\times$ 23 $\times$ 21	✓	—	✓	—
Spatula	306 $\times$ 125 $\times$ 33	✓	—	✓	—
Hammer	333 $\times$ 182 $\times$ 33	✓	—	✓	—

TABLE III : Physical affordance feasibility across two gripper configurations and object geometries. Grasp: feasible if the graspable cross-section is less than the gripper maximum span. Wrap-grasp: feasible if the gripper span exceeds the object body diameter and the object is cylindrical or rounded. Flat objects are not amenable to wrap-grasp.

Panel of Figure 8 — Fig. 5 : Representative YCB objects used for evaluating predicted affordances across both gripper configurations. Corresponding physical feasibility analysis is given in Tab. III .

Explore similar work

RoboPCA: Pose-centered Affordance Learning from Human Demonstrations for Robot Manipulation
Mar 8, 2026Zhanqi Xiao, Ruiping Wang, Xilin ChenRobotic Manipulation
The authors are with the Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS), Beijing, 100190, China, and also with the University of Chinese Academy of Sciences, Beijing 100049, China.
RAAP: Retrieval-Augmented Affordance Prediction with Cross-Image Action Alignment
Mar 31, 2026Qiyuan Zhuang, He-Yang Xu, Yijun Wang +3Generalization in Robotic Manipulation Zero-Shot Robotic Manipulation
School of Computer Science and Engineering, and Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Southeast University, Nanjing 211189, China · Southeast University-Monash University Joint Graduate School, Southeast University, Suzhou 215123, China · School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulation
Apr 13, 2026Mingyang Li, Haofan Xu, Haowen Sun +10Synthetic Data Generation for Robotics Robotic Manipulation
School of Artificial Intelligence, Xi’an Jiaotong University

Mar 8, 2026J/K move · Enter open · S save

RoboPCA: Pose-centered Affordance Learning from Human Demonstrations for Robot Manipulation

Zhanqi Xiao, Ruiping Wang, Xilin Chen

The authors are with the Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS), Beijing, 100190, China, and also with the University of Chinese Academy of Sciences, Beijing 100049, China.

Robotic Manipulation

Open paper AskPDF arXiv

Understanding spatial affordances -- comprising the contact regions of object interaction and the corresponding contact poses -- is essential for robots to effectively manipulate objects and accomplish diverse tasks. However, existing spatial affordance prediction methods mainly focus on locating the contact regions while delegating the pose to independent pose estimation approaches, which can lead to task failures due to inconsistencies between predicted contact regions and candidate poses. In this work, we propose RoboPCA, a pose-centered affordance prediction framework that jointly predicts task-appropriate contact regions and poses conditioned on instructions. To enable scalable data collection for pose-centered affordance learning, we devise Human2Afford, a data curation pipeline that automatically recovers scene-level 3D information and infers pose-centered affordance annotations from human demonstrations. With Human2Afford, scene depth and the interaction object's mask are extracted to provide 3D context and object localization, while pose-centered affordance annotations are obtained by tracking object points within the contact region and analyzing hand-object interaction patterns to establish a mapping from the 3D hand mesh to the robot end-effector orientation. By integrating geometry-appearance cues through an RGB-D encoder and incorporating mask-enhanced features to emphasize task-relevant object regions into the diffusion-based framework, RoboPCA outperforms baseline methods on image datasets, simulation, and real robots, and exhibits strong generalization across tasks and categories.

Abstract

Explore similar work

RoboPCA: Pose-centered Affordance Learning from Human Demonstrations for Robot Manipulation

RAAP: Retrieval-Augmented Affordance Prediction with Cross-Image Action Alignment

AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulation