Back on Track: Deployment-time Policy Improvement via World Model Planning

Colton Crosby; Hao Wang; Mac Schwager; Somil Bansal

cs.ROOct 14, 2026

Back on Track: Deployment-time Policy Improvement via World Model Planning

Authors: Colton Crosby, Hao Wang, Mac Schwager, Somil Bansal

Organizations: The authors are associated with the Aeronautics and Astronautics Department, Stanford University. · The author is associated with the Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California.

Abstract

State-of-the-art visuomotor policies are increasingly capable of executing complex tasks, yet can fail during deployment in ways they cannot predict or recover from on their own. Improving their performance and robustness typically requires collecting more data, retraining, or modifying the policy architecture, all of which are time- and resource-intensive in practice. We instead propose an offline-trained monitoring-and-intervention framework that improves a policy during deployment using only black-box query access. From a limited set of labeled on-policy rollouts, we learn a policy-conditioned liveness value that measures the nominal policy's progress toward task completion; we additionally train a compact latent world model using these rollouts together with play data. At deployment, the liveness value continuously monitors execution. When it detects a loss of progress, a short-horizon planner searches the world model for corrective actions using the same value as its planning objective, after which control returns to the nominal policy. We evaluate our method on the LIBERO-Goal task suite and on a Franka Panda manipulator, using the vision-language-action model $π_{0.5}$ and a diffusion policy as nominal policies, and demonstrate significant improvements in task success rates, while supporting real-time control. Additional results and videos are available on the project website (https://policy-improvement-wm.pages.dev/).

Figures & tables

	t0	t1	t2	t3	t4	t5	t6	t7	t8	t9	Mean
$\chi$	$1.15$	$2.00$	$1.40$	$1.15$	$2.00$	$1.40$	$1.40$	$2.00$	$1.85$	$1.10$
$\Upsilon$	$.02$	$.10$	$.04$	$.00$	$.10$	$.04$	$.04$	$.10$	$.10$	$.02$
$\pi_{b}$	$66.7$	$81.0$	$\mathbf{80.3}$	$57.0$	$84.3$	$58.0$	$76.3$	$98.0$	$71.7$	$53.0$	$72.6$
Ours	$\mathbf{93.7}$	$\mathbf{84.7}$	$79.3$	$\mathbf{67.0}$	$\mathbf{89.3}$	$\mathbf{70.0}$	$\mathbf{80.7}$	$\mathbf{98.3}$	$\mathbf{78.7}$	$\mathbf{73.7}$	$\mathbf{81.5}$
Rescued	$86$	$13$	$9$	$37$	$17$	$58$	$23$	$1$	$37$	$88$	$\mathbf{369}$
Broken	$5$	$2$	$12$	$7$	$2$	$22$	$10$	$0$	$16$	$26$	$\mathbf{102}$

TABLE I : Full LIBERO-Goal suite. Success rate (%),

n\!=\!300

paired trials per task. Rescued and Broken count interventions that flip a trial’s outcome;

\Delta

is their difference.

Fig. 2 : A sample visualization of the wrist camera in Task 0 ( open the middle drawer ). The proposed method (top) guides the robot towards the correct middle drawer, while the nominal policy (bottom) erroneously targets the top drawer.

		Black-box access				White-box reference
Task	$\pi_{b}$	TD3+BC	IQL	LatentSafe	Ours	Filtered-BC
t0	$66.7$	$56.3$	$42.7$	$66.3$	$\mathbf{93.7}$	$81.0$
t3	$57.0$	$7.3$	$34.3$	$57.0$	$67.0$	$\mathbf{75.3}$
t4	$84.3$	$60.0$	$54.3$	$84.7$	$\mathbf{89.3}$	$\mathbf{89.3}$
t9	$53.0$	$35.0$	$37.3$	$28.7$	$\mathbf{73.7}$	$67.0$
Mean	$65.2$	$39.6$	$42.2$	$59.2$	$\mathbf{80.9}$	$78.2$

TABLE II : Comparison against direct-improvement and filtering methods. Success rate (%),

n\!=\!300

per task. TD3+BC and IQL replace

\pi_{b}

; Filtered-BC retrains it with white-box weight access; the remaining methods filter it.

	$+50\%$ Config. perturbation $\chi$				$+50\%$ Object perturbation $\Upsilon$
Task	Set	$\pi_{b}$	Ours	$\Delta$	Set	$\pi_{b}$	Ours	$\Delta$
t0	$1.725$	$43.0$	$\mathbf{71.0}$	$+28.0$	$.03$	$67.0$	$\mathbf{89.0}$	$+22.0$
t3	$1.725$	$41.0$	$\mathbf{48.0}$	$+7.0$	${}^{*}.02$	$54.0$	$\mathbf{68.0}$	$+14.0$
t4	$3.000$	$57.0$	$\mathbf{70.0}$	$+13.0$	$.15$	$\mathbf{78.0}$	$77.0$	$-1.0$
t9	$1.650$	$37.0$	$\mathbf{61.0}$	$+24.0$	$.03$	$51.0$	$\mathbf{62.0}$	$+11.0$
Mean		$44.5$	$\mathbf{62.5}$	$+18.0$		$62.5$	$\mathbf{74.0}$	$+11.5$

TABLE III : Perturbation-severity stress test. Success rate (%) when one perturbation magnitude is increased by

50\%

relative to the main experimental setting and the other is held fixed. “Set” denotes the perturbed value. ∗ Task 3 has no nominal object perturbation, so its object condition introduces

\pm.02

rather than scaling.

Monitor design ablations ( $n\!=\!100$ )
Configuration	t0	t3	t4	t9	Mean	$\Delta$
$\pi_{b}$ (unfiltered)	$73.0$	$56.0$	$85.0$	$57.0$	$67.8$	$-14.5$
Ours (full)	$\mathbf{96.0}$	$\mathbf{66.0}$	$89.0$	$\mathbf{78.0}$	$\mathbf{82.3}$	—
no trigger schedule	$91.0$	$50.0$	$\mathbf{93.0}$	$61.0$	$73.8$	$-8.5$
1-member VF	$91.0$	$58.0$	$90.0$	$57.0$	$74.0$	$-8.3$
2-member VF	$\mathbf{96.0}$	$62.0$	$91.0$	$62.0$	$77.8$	$-4.5$

TABLE IV : Component ablations. Success rate (%) with one component varied from the full configuration; all others pinned.

\Delta

is the change in mean success.

Perturb.	$\pi_{b}$	$+$ Filt.	$\Delta$
all	$40.0$	$\mathbf{60.0}$	$\mathbf{+20.0}$
$0^{\circ}$	$80.0$	$100.0$	$+20.0$
$0$ – $4^{\circ}$	$54.2$	$71.4$	$+17.3$
$4$ – $6.5^{\circ}$	$34.8$	$43.5$	$+8.7$
$6.5$ – $9^{\circ}$	$22.7$	$\mathbf{70.6}$	$\mathbf{+47.9}$

TABLE V : Hardware lunchbox opening task. Success rate (%) for initial-configuration perturbations in

[0,9]^{\circ}

Robot Policy Adaptation

Robot policy adaptation changes or steers learned robot behaviors to handle new tasks, environments, hardware, and user preferences.

See all 173 papers

DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning
Jul 3, 2026Hanchen Cui, Sergio Arnaud, Arjun Majumdar +5World Model-Based Planning Robotic Manipulation
1Fundamental AI Research (FAIR), Meta · University of Minnesota Twin Cities
Enhancing Policy Learning with World-Action Model
Mar 30, 2026Yuci Han, Alper YilmazRobot Policy Learning World Action Models
Photogrammetry and Computer Vision Lab, The Ohio State University, Columbus, OH 43210, USA.
$ω$ -EVA: Envision, Verify, and Act with Latent Interactive World Models
Jun 8, 2026Zhenguo Sun, Yu Sun, Hande Huang +1Robot Policy Learning World Models for Robotics

Jul 3, 2026J/K move · Enter open · S save

DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning

Hanchen Cui, Sergio Arnaud, Arjun Majumdar, Daniel Dugas+4

1Fundamental AI Research (FAIR), Meta · University of Minnesota Twin Cities

World Model-Based Planning Robotic Manipulation Vision-Language-Action Models Test-Time Adaptation of VLA Models Robustness of VLA Models

Open paper AskPDF arXiv

Pretrained vision-language-action (VLA) policies show promising zero-shot generalization, but often fail under deployment-time distribution shift, leading to decreased robustness and inconsistent instruction following. While prior work commonly tackles this by finetuning on in-distribution data, it assumes demonstrations collected on tasks in the target environment. In this work, we propose DREAMSTEER, a deployment-time steering framework for pretrained VLAs without any finetuning or parameter modifications. The key insight in DREAMSTEER is to leverage a latent world model and a value model to steer pretrained VLA policies. During deployment, DREAMSTEER samples candidate action chunks from a VLA policy and predefined motion primitives, imagines their outcomes using an action-conditioned latent world model, and ranks the imagined trajectories with a language-conditioned value model. Across four real-world manipulation benchmarks with unseen objects, DREAMSTEER improves task success rate from 23.75% to 66.25% and instruction-following accuracy from 38.75% to 56.25% over the base VLA policy.

Abstract

Robot Policy Adaptation

Explore similar work

DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning

Enhancing Policy Learning with World-Action Model

ωωω-EVA: Envision, Verify, and Act with Latent Interactive World Models

$ω$ -EVA: Envision, Verify, and Act with Latent Interactive World Models