Robotic manipulation requires sophisticated commonsense reasoning, a capability naturally possessed by large-scale Vision-Language Models (VLMs). While VLMs show promise as zero-shot planners, their lack of grounded physical understanding often leads to compounding errors and low success rates when deployed in complex real-world environments, particularly for challenging tasks like deformable object manipulation. Although Reinforcement Learning (RL) can adapt these planners to specific task dynamics, directly fine-tuning VLMs via real-world interaction is prohibitively expensive, unsafe, and sample-inefficient. To overcome this bottleneck, we introduce DreamPlan, a novel framework for the reinforcement fine-tuning of VLM planners via video world models. Instead of relying on costly physical rollouts, DreamPlan first leverages the zero-shot VLM to collect exploratory interaction data. We demonstrate that this sub-optimal data is sufficient to train an action-conditioned video generation model, which implicitly captures complex real-world physics. Subsequently, the VLM planner is fine-tuned entirely within the "imagination" of this video world model using Odds Ratio Policy Optimization (ORPO). By utilizing these virtual rollouts, physical and task-specific knowledge is efficiently injected into the VLM. Our results indicate that DreamPlan bridges the gap between semantic reasoning and physical grounding, significantly improving manipulation success rates without the need for large-scale real-world data collection. Our project page is https://psi-lab.ai/DreamPlan/.
Figures & tables
Fig. 2: Overview of DreamPlan . Our framework consists of three stages. (1) Zero-shot proposal: given the current observation and goal image, a pretrained VLM planner generates multiple candidate keypoint-based manipulation actions. (2) World model learning: these zero-shot actions are executed to collect diverse action–observation trajectories, which are used to fine-tune an action-conditioned diffusion world model that predicts object deformation outcomes from rendered robot-motion videos. (3) World-model-guided alignment: the trained world model acts as a verifier to evaluate sampled VLM actions by predicting their future outcomes; comparing predicted outcomes yields pairwise preferences (more vs. less goal-consistent actions), which are used to fine-tune the VLM planner via Odds Ratio Policy Optimization (ORPO), aligning it toward physics-consistent behaviors without additional real-world interaction.
Fig. 3: Qualitative comparison of video generation results. Baselines fail to follow the specified actions or produce unrealistic deformations, while our method generates deformations that is both action-consistent and physically plausible, demonstrating its reliability as a verifier for VLM fine-tuning.
Fig. 4: Hardware setup. Our real-world platform consists of two Franka FR3 arms positioned opposite each other to enable bimanual manipulation within a shared workspace. A top-mounted RealSense D435i camera captures the interaction area, where deformable objects are placed on the work surface for automated data collection and further manipulation.
Method
Rope Straightening ↑
Cloth Folding ↑
Toy Arm Repositioning ↑
Avg score ↑
Qwen3-VL-4B
0.10
0.15
0.40
0.22
Qwen3-VL-8B
0.20
0.10
0.70
0.33
Qwen3-VL-32B
0.30
0.05
0.70
0.35
GPT-4o
0.30
0.00
0.55
0.28
Qwen3-VL-8B + DreamPlan RL finetune
0.60
0.35
0.85
0.60
TABLE I: Quantitative results on real-world tasks. Through DreamPlan RL fine-tuning, Qwen3-VL-8B achieves state-of-the-art (SOTA) performance in complex object manipulation, consistently outperforming leading pure zero-shot baselines. See Section IV-A3 for our evaluation protocol.
Fig. 5: Success rate improvements via RL fine-tuning. Our RL pipeline improves success rates by 15%–40% over zero-shot baselines across all evaluated real-world tasks.
Fig. 6: Real-world task setups and qualitative comparison. We evaluate our RL-finetuned Qwen3-VL ( DreamPlan ) on toy arm repositioning, rope straightening, and cloth folding. Leftmost columns show the initial and target states, with green and red arrows indicating predicted action vectors. The time-lapse sequences contrast the successful executions of our RL-finetuned model (top rows, green borders) against the typical failures of the pure zero-shot Qwen3-VL baseline (bottom rows, pink borders).
Full-Scene Prediction
Object-Only Prediction
PSNR ↑
24.70
26.25
TABLE II: Ablation on Object-Only Video Prediction. We compare the PSNR of synthetic rollouts generated by two action-conditioned video world models, trained for full-scene and object-only generation, respectively.
Method
Avg. Score ↑
Time (s) ↓
TFLOPs ↓
Explicit Verification
+ Sample N=4
0.48
926.32
7.82×104
+ Sample N=8
0.50
2605.56
1.56×105
DreamPlan (Ours)
0.60
1.12
15.14
TABLE III: Inference Efficiency and Computational Cost Analysis. We compare the task performance (Avg. Score), inference speed (Time), and computational overhead (TFLOPs) of DreamPlan against a sampling-based explicit verification baseline.
Solving complex long-horizon robotic tasks requires joint reasoning over abstract task structure and low-level physical interaction. While combining Vision-Language Models (VLMs) and video generation models offers a promising path for zero-shot planning, their individual tendencies to hallucinate physics or violate geometric consistency often compound over time, preventing reliable real-world execution. We introduce NovaPlan, a hierarchical framework that enables robust, zero-shot long-horizon manipulation by systematically proposing, verifying, and repairing visual plans. At the high level, a VLM planner decomposes tasks and filters out dynamically inconsistent futures by verifying multiple candidate video rollouts. To translate these imagined futures into reliable physical actions, NovaPlan utilizes a hybrid geometric representation that adaptively switches between object-centric flow and human hand flow. Finally, NovaPlan closes the loop by continuously monitoring execution to verify outcomes and synthesize local, non-prehensile corrective behaviors, such as fingertip poking, when failures occur. Across diverse multi-stage tasks, NovaPlan substantially outperforms prior zero-shot systems, achieving complex assembly and dexterous error recovery entirely without task-specific training or demonstrations. Please visit our project website for additional results: https://nova-plan.github.io/
Jiahui Fu, Junyu Nan, Lingfeng Sun +7
Robotics and AI Institute · Carnegie Mellon University · Brown University +1
Vision-Language Models (VLMs) have achieved strong performance in visual perception and spatial reasoning, leading to growing interest in using them as high-level planners for embodied AI. However, physical tasks require models to integrate spatial information, reason over temporal states, and generate multi-step plans under various constraints, raising the question of whether current VLMs can reliably support such reasoning and planning independently of execution. We introduce Plan-Bench, a benchmark for evaluating the high-level reasoning and multi-step planning capabilities of VLMs relevant to embodied worlds, while isolating the influence of action execution. It covers three complementary dimensions, spanning 16 subclasses and 699 curated samples. Extensive evaluation of 28 VLMs reveals room for improvement, with even the strongest model achieving only 75.0% accuracy. We further find that model scaling generally improves these capabilities. Ablation studies also indicate larger models show increasing potential as high-level planners. Overall, Plan-Bench provides a lightweight probe for evaluating VLMs' reasoning and planning capabilities relevant to embodied environments and assessing their potential as decision makers for physical tasks. Our benchmark and code are available at https://github.com/MinneFeng/Plan-Bench.
Juntong Feng, Qingan Liu, Jingru Dong +8
Dalian University of Technology · Nanyang Technological University
Building generalizable agents for diverse applications remains a fundamental challenge. While imitation learning-based policies succeed in specific training environments, they often fail to generalize to novel scenes and tasks. In this work, we propose World Action Planner, a robot planning system that leverages the reasoning capabilities of Vision-Language Models (VLMs) and the physical grounding of a multi-task pose-image conditioned world model. Our system enables an agent to propose initial action plans and iteratively refine them via optimization and search, reasoning over imagined world model rollouts. We demonstrate that our approach achieves superior performance across compositional tasks, new layouts, and zero-shot generalization scenarios, significantly outperforming state-of-the-art end-to-end policy models such as VLAs and WAMs. Project website at worldactionplanner.github.io