cs.ROFeb 23, 2026

NovaPlan: Zero-Shot Long-Horizon Manipulation via Closed-Loop Video Language Planning

Authors: Jiahui Fu, Junyu Nan, Lingfeng Sun, Hongyu Li, Jianing Qian, Benjamin Yang, Yilun Du, Jennifer L. Barry, +2 more

Organizations: Robotics and AI Institute · Carnegie Mellon University · Brown University · University of Pennsylvania

Abstract

Solving complex long-horizon robotic tasks requires joint reasoning over abstract task structure and low-level physical interaction. While combining Vision-Language Models (VLMs) and video generation models offers a promising path for zero-shot planning, their individual tendencies to hallucinate physics or violate geometric consistency often compound over time, preventing reliable real-world execution. We introduce NovaPlan, a hierarchical framework that enables robust, zero-shot long-horizon manipulation by systematically proposing, verifying, and repairing visual plans. At the high level, a VLM planner decomposes tasks and filters out dynamically inconsistent futures by verifying multiple candidate video rollouts. To translate these imagined futures into reliable physical actions, NovaPlan utilizes a hybrid geometric representation that adaptively switches between object-centric flow and human hand flow. Finally, NovaPlan closes the loop by continuously monitoring execution to verify outcomes and synthesize local, non-prehensile corrective behaviors, such as fingertip poking, when failures occur. Across diverse multi-stage tasks, NovaPlan substantially outperforms prior zero-shot systems, achieving complex assembly and dexterous error recovery entirely without task-specific training or demonstrations. Please visit our project website for additional results: https://nova-plan.github.io/

Explore similar work

CardsList
  1. APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughts

    Jul 9, 2026Emily Jin, Joy Hsu, Yiqing Xu +3Long-Horizon PlanningPlanning

  2. ProAct-VLM: Pre-Failure Vision-Language Task Replanning with Continuous Perception Feedback

    Sep 29, 2026Ahmed Nader Ahmed, Omar Moured, Mughni Irfan Mohammed Abdul +2Long-Horizon ManipulationPerception

  3. Long-Horizon Manipulation via Trace-Conditioned VLA Planning

    Apr 23, 2026Isabella Liu, An-Chieh Cheng, Rui Yan +7Long-Horizon ManipulationRobot State