World Action Critic (WAC) Model: Joint Foresight and Criticism for Policy Learning
Authors: Xiangteng Zhang, Haoyuan Liu, Yang Guan, Jingliang Duan, Yao Lyu, Tao Zhang, Shuai Zhang, Kehua Sheng, Bo Zhang, Shengbo Eben Li
Organizations: School of Vehicle and Mobility, Tsinghua University · SunRisingAI Lab · Didi Voyager Labs, Didi Autonomous Driving · College of AI, Tsinghua University
World models have shown considerable promise in sequential decision-making by anticipating the consequences of actions and providing predictive foresight for policy learning. However, learning to reason about counterfactual actions remains challenging, since predictive objectives may underemphasize information critical for action evaluation, while unexecuted actions lack observed futures for direct supervision. To address these challenges, we propose the World Action Critic (WAC) Model, which proposes multimodal action candidates and jointly learns candidate-conditioned future prediction and consequence evaluation. In our proposed framework, critic supervision directly shapes intermediate predictive representations during rollout, encouraging them to retain information needed to distinguish action quality. Observed futures supervise prediction for executed actions, while external task-quality targets provide evaluation signals for counterfactual candidates. Through joint optimization, critic signals from these candidates propagate into predictive representations without requiring their observed futures. Our theoretical analysis characterizes critic-guided feature retention and bounds candidate-selection regret. Applied to autonomous driving, WAC achieves 93.4 PDMS on NAVSIM-v1, demonstrating the effectiveness of jointly learning predictive foresight and consequence evaluation.
Figures & tables
Figure 1: Overview of world models for decision making. (a) Future prediction provides an auxiliary learning signal for policy features. (b) An action-conditioned simulator supplies imagined states to a planner for reward-based evaluation and search. (c) WAM jointly models future states and actions for action generation. (d) WAC jointly learns candidate-conditioned future prediction and consequence evaluation. Factual future targets ground prediction, while candidate-quality supervision shapes predictive representations for counterfactual action comparison.
Figure 2: Joint prediction and criticism in WAC. Given latent state z and candidate action a , WAC jointly predicts future latent state z′ and quality score q . Asymmetric attention lets the critic read predictive features while keeping critic tokens hidden from latent prediction. Conditions are injected through adaptive layer normalization (AdaLN).
Method
NC ↑
DAC ↑
TTC ↑
Comf. ↑
EP ↑
PDMS ↑
End-to-End Planning
DiffusionDriveV2 [ 66 ]
98.3
97.9
94.8
99.9
87.5
91.2
iPad (ViT-L) [ 16 ]
99.2
97.4
96.3
99.7
87.8
91.7
SparseDriveV2 [ 39 ]
98.5
98.4
95.0
99.9
88.6
92.0
NaviHydra (V4096) [ 46 ]
98.7
98.6
96.2
100.0
88.7
92.7
DrivoR (train) [ 24 ]
98.9
98.3
96.2
100.0
89.1
93.1
Table 1: Comparison with end-to-end, vision-language-action, and world-model/world-action planners on NAVSIM-v1. We report overall PDMS and its safety, road-compliance, comfort, and progress components; higher is better for all metrics. Baseline results follow their reported sensor and training configurations. Using only front-camera observations and ego status, WAC achieves the highest overall PDMS and ego progress among the compared methods.
Method
Spearman ↑
PA (%) ↑
Low-score overestimation ↓
Drive-JEPA
0.468
74.02
44.16
WAC
0.551
77.11
34.02
Table 2: Candidate scoring quality on NAVSIM-v1. PA denotes pairwise ranking accuracy.
Figure 3: Selected trajectories in urban car-following, a signalized urban junction, and a wet urban junction (left to right). Each column pairs the front-camera trajectory projection with its BEV. Red denotes Drive-JEPA, green WAC (Ours), and blue dashed lines the recorded GT. Thin lines show the 32 candidates per method; thick lines show their selections. Endpoint dots in BEV mark four-second predictions.
Variant
AR
Future
Grad
PDMS ↑
Detached critic gradients
✓
✓
×
92.3
One-shot future prediction
×
✓
✓
92.6
No future supervision
✓
×
✓
93.0
Full WAC
✓
✓
✓
93.3
Table 3: WAC structure and supervision ablations on NAVSIM-v1.
Figure 4: Candidate scoring in three selected improvement examples. Each subfigure overlays all 32 Drive-JEPA candidates (red triangles) and all 32 WAC candidates (green circles). Stars mark selections: red outlines for Drive-JEPA and green fills for WAC. The dashed line indicates equality and the shaded band is ±5 PDMS points. Coincident candidates overlap. Each model generates its own candidates.
Poses per step
Rollout steps
Focal
PDMS ↑
2
4
×
92.8
4
2
×
93.3
4
2
✓
93.4
Table 4: Prediction interval and focal-loss ablation on NAVSIM-v1. Every trajectory has eight future poses.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Candidate evaluation and planning decisions. Thin trajectories show candidates and thick trajectories their selections. Below each scene, candidates are sorted by descending predicted PDMS within each method. Curves show predictions, dots evaluator scores, and stars the evaluator score of the selected candidate. Scores use the 0–100 scale.
Parameter
Value
Visual backbone
ViT-L, 24 layers
Joint world–critic layers
2
Trajectory decoder hidden layers
2
Model / FFN hidden width
256 / 1024
Attention heads
8
World–critic activation
SwiGLU
Appendix
Table 5: Key architecture and training hyperparameters.