GuidedWAM: Training-Free Energy Guidance for World Action Models in Autonomous Driving
Authors: Xiyan Su, Finn Rasmus Schäfer, Hojun Lim, Frank Diermeyer, Johannes Betz, Markus Lienkamp
Organizations: Institute of Automotive Technology (FTM), Technical University of Munich, Germany · Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich, Germany · Professorship of Autonomous Vehicle Systems (AVS), Technical University of Munich, Germany
World action models (WAMs) for autonomous driving couple the driving policy and the visual world model in a single generative network whose objectives are fixed post-training. Conditioning on a new driving objective requires retraining. We study energy guidance as a general mechanism and present GuidedWAM, a model-agnostic framework for injecting driving objectives into frozen WAMs at sampling time. Our framework constructs each objective as differentiable energies on the sampler's intermediate clean-action estimate, which needs only forward access to the frozen model, and is stabilized by sampling-time guidance scheduling, per-energy normalization, and velocity-bounded correction. On the NAVSIM v2 hard-split, guidance improves all three frozen, publicly released WAMs by 1.8 to 3.6 EPDMS, averaged over three sampler seeds, and adds less than 10% to the sampling time per step. These results position energy guidance as a practical post-training control channel for WAMs.
Figures & tables
Method
Venue
Stage
NC ↑
DAC ↑
DDC ↑
TLC ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
Diffusion and flow planners
TransFuser [ 4 ]
TPAMI’23
S1
96.2
79.5
99.1
99.5
84.1
95.1
94.2
97.5
79.1
25.1 ± 0.0
S2
77.7
70.2
84.2
98.0
85.1
75.6
45.4
95.7
75.9
DiffusionDrive [ 22 ]
CVPR’25
S1
96.8
86.0
98.8
99.3
84.0
95.8
96.7
97.6
79.6
27.5 ± 0.0
S2
80.1
72.8
84.4
98.4
85.9
76.6
46.4
96.3
72.8
GuideFlow [ 24 ]
CVPR’26
S1
96.6
80.5
96.3
99.3
82.3
94.9
91.5
97.7
67.8
27.1 ± 0.0
Table 1: NAVSIM v2 hard-split. Per-metric scores of stage one (S1, real scenes) and stage two (S2, synthetic scenes) and the combined EPDMS, in points. EG : energy guidance with a single camera-based drivable-area energy and no privileged information at inference, mean and standard deviation over three seeds. Stock rows are single runs. Baseline rows are the numbers published in the respective papers, except TransFuser from the NAVSIM v2 leaderboard, DiffusionDrive as reproduced in SimScale, and ReCogDrive and SGDrive as reproduced in the SimWAM paper. GuideFlow is its no-scorer setting. The stock DriveLaW, DriveWAM and SimWAM rows are our own runs of the released checkpoints.
Figure 1: Energy guidance against post-hoc projection on one NAVSIM v2 hard-split scene where both recover the DAC. Left : front camera with the three plans projected into the image. Right : bird’s-eye view (BEV) with the drivable energy field. The stock plan (grey) leaves the drivable area. The post-hoc projection (blue) follows it until the plan violates the constraint, causing sudden lateral change. Guidance (orange) corrects along the whole plan and reaches the same score with a smooth trajectory.
Model
EPDMS ↑
Δ
Off-road ↓
EC ↑
DriveLaW
stock
30.0
26.0
57.9
guidance
34.8
+4.8
22.7
58.0
projection
37.7
+7.7
16.6
51.2
DriveWAM
stock
35.9
20.9
73.0
guidance
39.2
+3.3
18.5
71.4
projection
40.3
+4.3
15.2
66.0
Table 2: Guidance vs post-hoc gradient projection with the same constraint. Both use the privileged drivable area with a margin of 0.5 m. Off-road is the fraction of synthetic scenes (stage 2) failing the DAC. EC is the stage-two two-frame comfort.
single energy EPDMS ↑
composition EPDMS ↑
Model
stock
LK
DA
DD
LK+DA
DA+DD
LK+DA+DD
DriveLaW
30.0
31.1
34.0
30.6
33.3
34.8
33.0
DriveWAM
35.9
37.5
39.2
36.4
39.5
40.0
39.6
SimWAM
37.8
37.9
39.2
37.8
38.2
39.6
38.9
Table 3: Composition of guidance energies on the privileged information. Lane keeping (LK): centerline attraction; drivable area (DA): drivable-area violation on the vehicle corners; driving direction (DD): vehicle center direction violation. The compositions use the best relative weights.
Figure 2: Guidance strength on single-camera-only drivable-area energy. EPDMS against the strength λ on a log axis. The dashed lines show the stock models. The open marker is the operating point reported in Table 1 .
Model
full-split best
cross-fitted
95% interval
DriveLaW
34.1
33.5
[+1.9,+6.3]
DriveWAM
38.2
37.7
[+0.5,+4.2]
SimWAM
40.0
40.0
[+0.8,+3.8]
Table 4: Cross-fitted selection of strength and margin over two halves of the 450 real frames of stage one, next to the full-split optimum. The last column is the full-split gain of the selected arm with a paired bootstrap interval, 10,000 draws. EPDMS in points.
Figure 3: Window shape against guidance strength on 40 DriveWAM DAC failure scenes under the stock sampler. Top : failure rate, the percentage of scenes whose final plan jitters more than three times the stock sampler. Bottom : ADE/FDE against the stock plan. The operating strength of DriveWAM is λ=4 .
Model
Form
EPDM ↑
guidance / step ↓
forward / step ↓
memory ↓
DriveLaW
Jacobian-free
34.1
0.1 ms
48 ms
8.2 GB
Jacobian
29.9
50.4 ms
42.4 ms
9.8 GB
DriveWAM
Jacobian-free
37.7
1.9 ms
36 ms
13.7 GB
Jacobian
36.3
72.5 ms
35.9 ms
14.3 GB
SimWAM
Jacobian-free
40.0
1.4 ms
16.3 ms
12.7 GB
Jacobian
40.0
14.3 ms
16.2 ms
12.7 GB
Table 5: Jacobian-free guidance against the full Jacobian form of Eq. ( 5 ), at the same strength and window. Times are per sampler step on one NVIDIA RTX 4090 GPU. Memory is the peak during sampling.