Organizations: Chair of Machine Tools and Control System, RPTU University Kaiserslautern-Landau, 67663 Kaiserslautern, Germany · Innovative Factory Systems, German Institute of Artificial Intelligence, 67663 Kaiserslautern, Germany · Chair of Machine Tools and Control Systems, RPTU University Kaiserslautern-Landau, 67663 Kaiserslautern, Germany
Learning robot skills from scratch is often time-consuming, while reusing data promotes sustainability and improves sample efficiency. This study investigates policy transfer across different robotic platforms, focusing on peg-in-hole task using reinforcement learning (RL). Policy training is carried out on two different robots. Their policies are transferred and evaluated for zero-shot, fine-tuning, and training from scratch. Results indicate that zero-shot transfer leads to lower success rates and relatively longer task execution times, while fine-tuning significantly improves performance with fewer training time-steps. These findings highlight that policy transfer with adaptation techniques improves sample efficiency and generalization, reducing the need for extensive retraining and supporting sustainable robotic learning.
Figures & tables
Figure 1: Ilustration of the peg in hole insertion task.
Figure 2: Diagram of the RL agent using SAC method for robot control.
Figure 3: Diagram illustrating adaptive hybrid motion-force control. FK and IK denote forward and inverse kinematics, respectively, while R0n defines the transformation from the respective frame into task frame.
Figure 4: Simulated environments of UR5e in (a) and Panda (b) models via MuJoCo.
Scenario
Use Case
Remarks
1
Testing UR5e model
-
2
Testing Panda model
-
3
Testing Panda using UR5e model
zero-shot
4
Testing UR5e using Panda model
zero-shot
5
Testing Panda using UR5e model
Fine-tuning
Table 1: Proposed scenarios for evaluating policy transfer.
Figure 5: Results of the use-cases in Table 1 .
Figure 6: Comparison of the training performance for UR5e and Panda robots in MuJoCo.
Department of Aerospace Engineering, Automation and Systems Research Institute (ASRI), Seoul National University, Seoul, Korea · Georgia Institute of Technology, Atlanta, GA, USA