World-action models aim to turn visual anticipation into driving decisions, yet typically train action experts on trajectory representations defined independently of visual dynamics. This motivates deriving action targets directly from visual motion. We first examine a simpler setting in which trajectories are reconstructed from recorded future images. Even without the need for future prediction, reconstruction captures the dominant maneuver but leaves centimeter-scale errors. We therefore introduce VITAL (Vision-Induced Tokens for Action Learning), an action tokenizer that couples a visually derived motion scaffold with a compact residual encoding the trajectory error left by that scaffold. This representation makes visual motion the principal prediction target and gives small metric corrections a separate, normalized learning scale. Using flow matching, the action expert jointly predicts both components from the current image, ego state, and navigation command, without generating future images at inference. The same residual coordinates support bounded reinforcement post-training that refines execution while keeping the predicted visual scaffold fixed. With only one action denoising step, VITAL achieves 90.8 PDMS on NAVSIM v1 navtest, within 0.13 points of its ten-step score at 3.29x the end-to-end inference speed. Without retraining, it achieves 91.1 EPDMS on NAVSIM v2 navtest and 38.6 on navhard. Project page: https://worldbench.github.io/vital
Figures & tables
Figure 1: From world prediction to precise action. World-action models aim to translate visual predictions of the physical world into actions. This motivates deriving action targets from visual motion to reduce the visual–action representation gap. Yet even with recorded future images, our study recovers the dominant maneuver while leaving local metric errors. \texorpdfstring \pdfxform @ VITAL jointly addresses visual grounding and execution precision: visual motion defines the principal action target, and a paired residual encodes its remaining trajectory error.
Figure 2: Visual reconstruction and residual precision. vision only, \texorpdfstring \pdfxform @ VITAL, residual. HF denotes the mean x/y high-frequency velocity energy fraction.
Figure 3: \texorpdfstring \pdfxform @ VITAL learns a shared representation for action prediction and refinement. The upper stage constructs visual motion tokens and paired metric residuals from recorded images and trajectories. The lower stage predicts the joint code in one flow step, applies a bounded residual correction, and decodes the executable trajectory. The diagram shows the driving implementation.
\toprule Method
Input
Frames
NFE/plans
NC ↑
DAC ↑
EP ↑
TTC ↑
C ↑
PDMS ↑
Table 1: NAVSIM v1 navtest . “Frame” counts include the current time step; “C/L” denote camera/LiDAR inputs, and “NFE/plans” denotes generator evaluations/candidate counts. Symbol † marks released-checkpoint results; all other baseline values are from the cited papers, respectively. Bold marks the best PDMS, and blue highlights \texorpdfstring \pdfxform @ VITAL@1, the proposed one-trajectory setting, against multi-step methods.