cs.CVNov 17, 2026

Plausible Videos Can Lead to Poor Decisions: Diagnosing and CURE-ing World Models for Planning

Authors: Raphael C. Kim, Michele Santacatterina, Jingsen Zhu, Ramin Zabih

Organizations: Cornell University · NYU Grossman School of Medicine

Abstract

Action-conditioned world models are increasingly used as world models for planning, yet a world model that makes plausible, realistic-looking videos that are consistent with the training data can produce poor decisions. We address two key questions: (1) when can we trust a world model for planning, and (2) what should we do when we cannot? We introduce CURE (Certify, Upweight, Rollout, Explore), a certificate-guided technique that assesses whether a world model can be trusted for planning and determines what to do next: whether to upweight the training data to debias the model; roll out more samples at test time; or explore by collecting new data where support (how well the training data cover the planner's decisions) is missing. We deploy CURE across multiple domains, spanning simulated physics based on PhyWorld, MuJoCo arm control, and Language-Table pushing in simulation and on real videos. The decisions our certificate trusts have lower regret than those it does not (e.g., 0.06 vs. 0.38 on Language-Table sim and 0.07 vs. 0.38 on MuJoCo contact), whereas video perceptual quality does not identify good decisions. By focusing on decision quality rather than perceptual quality, CURE lowers regret over the best pessimistic decision rule by 10-25% across support levels, and over world-action models, search and policy learning by 41-64%, given the same data.

Figures & tables

Explore similar work

CardsList
  1. How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position

    Jun 13, 2026Yang Yu, Shiyuan Zhang, Yifei Sheng +2Model-Based PlanningWorld Model Evaluation

  2. How Much Planning Is Enough? Reducing Search and Computation in World-Model Planning

    Oct 6, 2026Changbai Li, Sirui Li, Yichen Yang +4World Model-Based PlanningEfficient Inference for World Action Models