Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift

Danqi Zhao; Keyu Chen; Wenchao Sun; Minglei Huang; Jianhe Zhang; Yan Zhu; Hao Cheng; Zifan Cao

cs.ROOct 15, 2026

Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift

Authors: Danqi Zhao, Keyu Chen, Wenchao Sun, Minglei Huang, Jianhe Zhang, Yan Zhu, Hao Cheng, Zifan Cao,

Organizations: School of Vehicle and Mobility, Tsinghua University, Beijing, China. · Shanghai Technical Center, DENSO (China) Investment Co., Ltd., Shanghai, China. · DENSO (China) Investment Co., Ltd., China.

Abstract

World-model planners evaluate candidate trajectories by predicting their future consequences, implicitly relying on the plan-to-motion relationship learned during training. At deployment, a different downstream controller may execute the same planned trajectory differently, creating a planning-execution gap between the frozen model's imagined future and actual execution. A natural remedy is to replace the planned trajectory with its predicted or observed execution. However, we find that better execution fidelity does not necessarily yield better decisions: directly substituting predicted execution trajectories yields only a 0.048 PDMS improvement across 10 unseen controller configurations, far smaller than the gain from our proposed adaptation. This suggests a semantic mismatch between a planned trajectory, which serves as a reference for downstream control, and an executed trajectory, which already describes the resulting post-controller motion. We introduce ExecutionBridge, which infers controller-specific execution characteristics from planned-executed trajectory pairs and uses them to adapt the native ego latent rollout while keeping the original planned trajectory unchanged. The planner, world model, and reward head remain frozen. On NAVSIM, ExecutionBridge consistently improves planning across unseen controller configurations and multiple frozen world-model planners, with substantially larger gains in challenging driving scenes where execution mismatch more strongly affects candidate evaluation.

Figures & tables

Fig. 1: Evolution of end-to-end planning paradigms. Model-free planners predict or score trajectories directly from the current scene, without explicitly modeling their future consequences. Model-based planners additionally use a learned world model to imagine candidate-conditioned futures for trajectory evaluation. Conventional model-based planning typically assumes that candidate trajectories are executed according to the plan-to-motion relationship learned during training. Execution-adaptive model-based planning further incorporates controller-specific information, allowing candidate futures to be evaluated under deployment-specific execution.

Fig. 2: Planning–execution gap and motivation for ExecutionBridge. (a) The same planned trajectory can lead to different executions at deployment from those implicitly assumed by the frozen world model, causing its imagined future to deviate from the future actually encountered. (b) Directly substituting the planned trajectory with its executed counterpart improves execution fidelity, but introduces a plan–execution semantic mismatch : the executed trajectory can still be interpreted through the model’s implicit plan-to-motion mapping, and therefore does not necessarily improve candidate ranking. (c) ExecutionBridge instead retains the original planned trajectory as the conditioning input and uses controller-specific execution evidence to adapt the imagined future used for candidate evaluation.

Fig. 3: Architecture of ExecutionBridge. (a) Controller Encoder. Given a support set of planned–executed trajectory pairs $\mathcal{S}_{c}$ , the Controller Encoder summarizes these observations into a compact controller representation $e_{c}$ that captures persistent controller-specific execution characteristics. An auxiliary execution decoder predicts the execution of a held-out query trajectory for training supervision and is discarded at deployment. (b) Controller-Conditioned Latent Bridge. For each candidate, the frozen world model first produces its native rollout from the original planned trajectory. Conditioned on $e_{c}$ , the Latent Bridge predicts a gated correction to the ego latent, which is bounded by a trust-region projection before being injected into the rollout. The resulting controller-conditioned rollout is evaluated by the unchanged reward model for candidate scoring.

Fig. 4: Representative planning results under the standard NAVSIM setting. Compared with frozen WoTE, ExecutionBridge yields trajectory selections that are better aligned with controller-dependent execution in representative collision and drivable-area failure cases.

Method	Input	#Traj.	Traj. Eval.	NC $\uparrow$	DAC $\uparrow$	EP $\uparrow$	TTC $\uparrow$	PDMS $\uparrow$
Human	–	–	–	100.0	100.0	87.5	100.0	94.8
Constant Velocity	–	1	$\times$	69.9	58.8	49.3	49.3	21.6
Ego Status MLP [ 29 ]	–	1	$\times$	93.0	77.3	62.8	83.6	65.6
VADv2 [ 6 ]	C	8192	Rule-based	97.9	91.7	77.6	92.9	83.0
UniAD [ 4 ]	C	1	Rule-based	97.8	91.9	78.8	92.9	83.4
LTF [ 30 ]	C	1	$\times$	97.4	92.8	79.0	92.4	83.8

TABLE I: Planning performance across frozen world-model planners under the standard NAVSIM setting. For each backbone, ExecutionBridge is applied while keeping its native planning components frozen.

	Full set		Low-score subset
Method	PDMS $\uparrow$	$\Delta$	PDMS $\uparrow$	$\Delta$
Frozen WoTE	87.976	–	32.038	–
Predicted execution	88.024	+0.048	36.528	+4.490
ExecutionBridge	88.589	+0.613	46.128	+14.090

TABLE II: Planning under controller-induced execution shift. Results are averaged over ten controller configurations.

Fig. 5: Planning under unseen controller shifts. Performance of the frozen planner and ExecutionBridge across ten unseen deployment controller configurations. (a) ExecutionBridge consistently improves PDMS across controller styles (mean: $87.98\rightarrow 88.59$ , $+0.61$ ). (b) It also improves TTC (mean: $94.53\rightarrow 94.86$ , $+0.33$ ). (c) The improvement is substantially larger on the low-score subset, where PDMS increases from $32.04$ to $46.13$ ( $+14.09$ ). These results indicate that controller-conditioned adaptation is particularly beneficial in challenging driving scenes where candidate evaluation is more sensitive to execution differences.

Fig. 6: Adaptation under representative controller styles. We show four representative controller configurations, including aggressive, conservative, steering-lag, and steering-gain behaviors. Controller-conditioned candidate evaluation consistently improves both PDM and Time to Collision (TTC) across these diverse controller styles. Examples on the right illustrate safety-critical cases in which accounting for controller-specific execution changes the selected trajectory and avoids failures under frozen evaluation.

Fig. 7: Execution fidelity does not imply decision fidelity. (a) Predicted and exact controller execution substantially reduce geometric execution error. (b–c) Nevertheless, candidate-ranking fidelity changes only marginally on the frozen-defined difficult-scene subset. (d) Direct execution substitution improves selected-candidate quality, but ExecutionBridge performs best by preserving the original plan and adapting its imagined future. Pairwise-oracle PDMS in (d) is a candidate-level diagnostic distinct from final navtest PDMS.

Method	Adapt.	$e_{c}$	$\hat{r}^{c,k}$	PDMS $\uparrow$	Low-score $\uparrow$
Frozen WoTE	–	–	–	87.976	32.038
Decision-only	$\xi$	–	–	88.071	35.214
Score calibration	$s$	✓	–	88.184	40.327
Residual-conditioned	$\xi$	✓	✓	88.312	45.550
ExecutionBridge	$\xi$	✓	–	88.589	46.128

TABLE III: Ablation of adaptation target and controller information. “Adapt.” denotes adaptation of the ego latent

\xi

or candidate score

s

;

e_{c}

is the controller representation and

\hat{r}^{c,k}

the candidate-specific execution residual.

$N$	Exec. ADE $\downarrow$	Full PDMS $\uparrow$	Low-score PDMS $\uparrow$
64	1.521	88.445	45.901
128	1.522	88.441	45.955
256	1.521	88.441	45.954
1024	1.521	88.443	45.954

TABLE IV: Sensitivity to support-set size. Results over five held-out controller styles show stable execution and planning performance across

N=64

–

1024

Autonomous Driving Planning

Autonomous driving planning selects safe, feasible vehicle actions and trajectories amid traffic, uncertainty, road rules, and changing goals.

See all 384 papers

Mind the Execution Gap: Action-Semantic Mismatch in World-Model Control
Oct 5, 2026Shengtao Wen, Xiang Chen, Yu Tian +3World Model-Based Planning Action-Conditioned World Models
Nanjing University of Aeronautics and Astronautics, Nanjing, China · Tsinghua University, Beijing, China · Nanjing University, Nanjing, China
Adapt for Planning, Not just Prediction: Optimistic Test-Time World Model Adaptation
Oct 26, 2026Akshay Mete, Dileep Kalathil, P. R. KumarTest-Time Adaptation World Model-Based Planning
ECE Department, Texas A&M University, College Station, Texas, USA.
AdaReP:Adaptive Re-Planning under Model Mismatch for Neural World-Model Predictive Control
Jun 22, 2026Yutian Cheng, Xiaojian Ma, Xianhao Wang +6World Model-Based Planning World Models for Robotics
Shanghai Jiao Tong University · Beijing Institute for General Artificial Intelligence (BIGAI) · University of Science and Technology of China

Oct 5, 2026J/K move · Enter open · S save

Mind the Execution Gap: Action-Semantic Mismatch in World-Model Control

Shengtao Wen, Xiang Chen, Yu Tian, Lingbing Guo+2

Nanjing University of Aeronautics and Astronautics, Nanjing, China · Tsinghua University, Beijing, China · Nanjing University, Nanjing, China

World Model-Based Planning Action-Conditioned World Models Model-Based RL

Open paper AskPDF arXiv

World-model controllers rely on action-conditioned dynamics for prediction and planning, yet real control systems often execute commands asynchronously due to communication delay, packet loss, reordering, and actuator buffering. We study how asynchronous execution changes the action semantics assumed within world-model controllers, rather than treating it only as an external control disturbance. Through controlled interventions, we identify two architecture-dependent failure modes: planning-based controllers such as TD-MPC2 suffer from a future-action timeline mismatch between imagined and executed action sequences, while recurrent world models such as DreamerV3 can attribute observed transitions to commands that were not actually applied. Our analysis shows that TD-MPC2 requires the correct future action sequence during latent dynamics rollout, whereas DreamerV3 requires timely attribution of each transition to the action that generated it. Based on these findings, we introduce two lightweight execution-consistent interfaces, Future-Sequence for TD-MPC2 and Applied-Action Feedback for DreamerV3, that correct these mismatches without modifying the pretrained world models. Experiments across delays, packet loss, reordering, multiple control domains, measured network traces, and a process-separated asynchronous stack consistently support both diagnoses and the corresponding architecture-specific corrections.

Abstract

Autonomous Driving Planning

Explore similar work

Mind the Execution Gap: Action-Semantic Mismatch in World-Model Control

Adapt for Planning, Not just Prediction: Optimistic Test-Time World Model Adaptation

AdaReP:Adaptive Re-Planning under Model Mismatch for Neural World-Model Predictive Control