Demonstration-Free Skill Composition for Navigation and Manipulation
Organizations: School of Computer Science and Engineering, University of New South Wales, Sydney, Australia · Escuela de Ingeniería, Universidad Central de Chile, Santiago, Chile
Abstract
Hierarchical reinforcement learning enables sequential tasks to be represented as compositions of separately trained skills. This paper evaluates demonstration-free hierarchical pipelines for MiniGrid navigation and robotic manipulation, comparing their training requirements and task performance with flat reinforcement learning baselines. The evaluation concerns complete pipelines with task-specific observation, reward and control structures. In navigation, two parameterised skills are reused across unseen FourRooms layouts under manually specified high-level plans. In manipulation, reaching, grasp alignment and lifting are trained separately, with downstream skills initialised from states collected during predecessor rollouts. Across unseen navigation layouts, the hierarchical pipeline substantially outperforms the evaluated flat DQN baselines while requiring fewer training interactions. In manipulation, the hierarchical chain requires fewer training steps than flat SAC and exhibits lower observed variation across seeds. The frozen Pick and Lift skills are further reused for stacking by training an additional Place skill.
Figures & tables
| Hyperparameter | Value |
|---|---|
| Training budget, flat baselines | |
| Training budget, GoTo | |
| Training budget, CrossDoor | |
| Learning rate | |
| Replay buffer size | |
| Learning starts |
| Method | Steps | Success (mean std) | 95% CI |
|---|---|---|---|
| Flat SAC | 292k | ||
| HRL Reach | 132k | ||
| HRL Grasp | 47k | ||
| HRL Lift (10 cm) | 16k | ||
| Full chain | 195k |
| Metric | Result |
|---|---|
| Pick and Lift completion | (85.6%) |
| Place success given handoff | (83.6%) |
| Full stacking success |
Reinforcement Learning
Reinforcement learning studies how agents learn sequential decisions from rewards and interaction, balancing exploration, performance, safety, and adaptation.
See all 1,862 papers