Demonstration-Based Robotic Feeding Using Dynamic Movement Primitives
Authors: Kenzhi Iskandar Wong, Kitti Thamrongaphichartkul, Sri Harsha Turlapati, Yanshuo Lu, Kanichi Doi, Boyu Ma, Bin Zhou, Lin Yang, Ananda Sidarta, Lek Syn Lim, Arun-Kumar Kaliya-Perumal, Jianfei Yang, Domenico Campolo
Organizations: Robotics Research Centre, School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore. · School of Science and Technology, Singapore University of Social Sciences, Singapore. · Department of Mechano-Informatics, University of Tokyo, Japan. · R3 Institute Singapore, Nanyang Technological University, Singapore.
This paper presents a demonstration-based robotic feeding framework that integrates human demonstration, Dynamic Movement Primitive-based motion planning, vision-based scene perception, and robot control. A human operator first demonstrates the scooping and feeding motions while the position and Euler-angle orientation of the spoon are recorded using a motion-capture system. The recorded trajectory is segmented into scooping and feeding motions, which are encoded using separate Dynamic Movement Primitives. Before task execution, an RGB-D camera estimates the participant's mouth position, while two AprilTags are used for robot-frame registration and tray localization. The detected tray pose is used to position the scooping motion, whereas the detected mouth position defines the positional goal of the feeding motion. The approach, scooping, and feeding trajectories are then executed sequentially by a Kinova Gen3 manipulator through a ROS~2-based framework. A real-world experiment demonstrates the feasibility of the integrated pipeline and shows that the learned spoon motions can be reproduced within the detected experimental scene.
Figures & tables
Fig. 1: Overview of the demonstration-based robotic feeding framework. The scooping and feeding motions are recorded using a motion-capture system and encoded as DMPs. Before task execution, the mouth position is detected once using an RGB-D camera, while two AprilTags are used for camera-to-robot frame registration and tray localization. The learned DMP weights, task-relevant configurations, and initial mouth position are used to generate the desired end-effector motion for the robotic manipulator.
Fig. 2: Experimental setup for the robotic feeding task. A Kinova Gen3 manipulator equipped with a spoon performs the scooping and feeding motions, while an RGB-D camera is used to estimate the participant’s mouth position. Two AprilTags are used for camera-to-robot frame registration and tray localization.
Fig. 3: Mouth-perception results under different head poses and mouth configurations.
Fig. 4: Sequential snapshots of the robotic feeding task. From left to right, the manipulator approaches the bowl, executes the learned scooping motion, transports the spoon toward the participant, and reaches the initially detected mouth position.
Fig. 5: Planned end-effector trajectories visualized within the RGB-D point cloud of the experimental scene. The approach trajectory yapproach(t) moves the robot toward the pre-scooping pose, while yscoop(t) and yfeed(t) denote the DMP-generated scooping and feeding trajectories, respectively. The coordinate frames indicate the planned spoon position and orientation along each trajectory.
Robotics and AI Lab, Technological University of Uruguay, Rivera, Uruguay · Center of Technology, Federal University of Santa Maria, Santa Maria, Brazil · Computer Science Center, Federal University of Rio Grande, Rio Grande, Brazil
Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom · Faculty of Electrical Engineering, Kempten University of Applied Sciences, Kempten, Germany