Data-Efficient Robot Learning

Overview

Data-efficient robot learning studies how robots can learn useful skills without costly collections of demonstrations or extensive trial and error on physical hardware. Common approaches include imitation learning, sample-efficient reinforcement learning, world models that support planning or simulated practice, and methods that select, augment, or reuse demonstrations. Human videos, pretrained vision-language-action models, and transfer across robot embodiments can provide useful starting points, while benchmarks such as LIBERO and RoboCasa test manipulation performance in simulation and on real robots. A central challenge is that gains on offline prediction or simulated tasks do not always translate to reliable real-world behavior, and adaptation must improve new skills without losing old ones or requiring excessive supervision.

Latest papers

211 papers27 papers in the last four weeks, down 52% on the four weeks before. 0.2% of all new papers.

Apr 2, 2024

Active Exploration in Bayesian Model-based Reinforcement Learning for Robot Manipulation

Efficiently tackling multiple tasks within complex environments, such as those found in robot manipulation, remains an ongoing challenge in robotics and an opportunity for data-driven solutions such as reinforcement learning (RL). The high cost of data gathering in robotics encourages data-efficient approaches such as model-based RL, which builds a dynamic model of the robot to conduct most policy learning on cheaper simulations, while enabling data reuse and transfer between tasks. In this work, we employ Bayesian deep learning (BDL) with a double purpose. First, we increase the model quality by improving its calibration. Second, we improve data efficiency by performing active learning of the dynamic model during a preliminary exploration phase, actively estimating the information gain of each transition and using it as the exploration reward. We compare several BDL methods in a realistic robot manipulation setup. Our experiments show that our approach matches the quality of relevant alternatives with much lower robot execution requirements and, unlike previous studies validated only on standard RL environments, we tackle robotic arm end-tasks on a complete manipulator simulator such as CoppeliaSim. Code is available at https://github.com/ropertunizar/BayesianActiveExploration.