Lazy-MOSAIC: Accelerating Skill-Centric Manipulation Planning with Learned Skill-Effect World Models
Organizations: Robotics Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, United States. · Department of Computer Science, Technion, Israel Institute of Technology, Haifa, Israel.
Abstract
Long-horizon manipulation tasks require a robot to perform a sequence of physical interactions with its environment, typically combining several different skills. Planning such behaviors efficiently is challenging: given a library of parameterized skills, solutions lie in a vast hybrid space of skill sequences and their parameters, and evaluating each candidate skill's feasibility and effect with a high-fidelity world model such as a physics simulator is computationally expensive. MOSAIC, a recent skill-centric approach, lets the skills guide a non-sequential search toward the regions of this space where they are likely to succeed; however, it pays the full cost of simulation for every parameterized skill it considers. In this work, we introduce Lazy-MOSAIC, a lazy planning framework that instead searches over hypothesized skill outcomes. A cheap learned skill-effect world model predicts, in a fraction of a millisecond, a skill's feasibility and the world state it produces. Lazy-MOSAIC then uses these predictions to focus expensive simulation on promising candidate skills and, when a hypothesis is invalidated, repairs only the affected part of the search, while ensuring that every skill in a returned plan is validated in simulation. We evaluate Lazy-MOSAIC on cluttered-shelf retrieval tasks of increasing difficulty and demonstrate it on a physical robot, showing that it plans substantially faster than baselines with fewer simulator queries, while matching or exceeding their success rate in every setting. Visit skill-mosaic.github.io.
Figures & tables
| Task 1 | Task 2 | Task 3 | ||||||||||||||||||||||
| Easy | Medium | Hard | Easy | Medium | Hard | Medium | Hard | |||||||||||||||||
| Method | Time | SR | Len | Time | SR | Len | Time | SR | Len | Time | SR | Len | Time | SR | Len | Time | SR | Len | Time | SR | Len | Time | SR | Len |
| Mosaic | 104.4 | 20/20 | 2.0 | 273.2 | 16/20 | 3.69 | 418.0 | 7/20 | 5.86 | 144.2 | 20/20 | 3.75 | 323.6 | 8/20 | 7.63 | 540.5 | 4/20 | 10.25 | 397.2 | 6/20 | 5.67 | N/A | 0/20 | – |
| Lazy-Mosaic | 44.1 | 20/20 | 2.0 | 67.3 | 19/20 | 3.58 | 175.1 | 13/20 | 6.23 | 41.0 | 20/20 | 3.45 | 121.7 | 10/20 | 7.8 | 256.8 | 7/20 | 10.57 | 60.3 | 11/20 | 6.18 | 584.8 | 4/20 | 10.5 |
| Ranked- Mosaic | 91.6 | 20/20 | 2.0 | 238.9 | 17/20 | 3.71 | 359.4 | 9/20 | 6.11 | 126.5 | 20/20 | 3.95 | 294.8 | 8/20 | 7.38 | 501.0 | 5/20 | 11.0 | 326.7 | 7/20 | 5.86 | N/A | 0/20 | – |
| Lazy-Mosaic w/o uncertainty | 59.0 | 20/20 | 2.0 | 97.2 | 18/20 | 3.83 | 232.8 | 11/20 | 6.45 | 53.0 | 20/20 | 4.0 | 178.9 | 9/20 | 8.11 | 287.8 | 4/20 | 10.75 | 136.5 | 7/20 | 6.29 | 597.1 | 1/20 | 10.0 |
Robot Task Planning
Robot task planning turns goals and observations into executable actions while coordinating skills, motion, uncertainty, and safety.
See all 255 papers