VehDyn: A Driving World Model Benchmark for Vehicle Dynamics

Tianyi Wang; Wangsheng Du; Jiazhou Chen; Tianyi Zeng; Xiangyu Li; Jiseop Byeon; Yujin Wang; Yiming Xu

cs.CVSep 27, 2026

VehDyn: A Driving World Model Benchmark for Vehicle Dynamics

Authors: Tianyi Wang, Wangsheng Du, Jiazhou Chen, Tianyi Zeng, Xiangyu Li, Jiseop Byeon, Yujin Wang, Yiming Xu, +7 more

Abstract

Video world models are emerging as data engines, action planners, and generative simulators for autonomous driving, but existing benchmarks primarily assess visual fidelity and coarse physical plausibility, providing limited evidence on whether generated driving futures obey realistic vehicle kinematics and dynamics. This limitation is further compounded by the lack of datasets in which vehicle, road, maneuver, and speed conditions are independently controlled, and ground-truth vehicle states are recorded in synchrony with videos. We introduce VehDyn, a driving world model benchmark for vehicle dynamics. VehDyn is built on a CARLA-CarSim co-simulation platform where photorealistic rendering is coupled with a validated multi-body dynamics model, and it contains 10,080 configurations from a full factorial design over five vehicle types, four tire-road friction coefficients, three maneuvers, four target speeds, 14 scenes, and three illuminations, each paired with synchronized position, velocity, and attitude sequences. Built on this dataset, VehDyn introduces a hierarchical evaluation framework that measures trajectory alignment, kinematic consistency, and dynamic consistency, and benchmarks 12 state-of-the-art video world models. We further assess the video quality using two established protocols and correlate it with the VehDyn score. Trajectory-level metrics are nearly saturated, with ten of twelve models within 20% of ground truth, while no model reaches 92% of ground truth on dynamic consistency, and visual-quality metrics are only weakly correlated with vehicle-dynamics fidelity. DrivingWorld achieves the highest VehDyn score, followed by Cosmos 3 Nano and LTX-Video 2.5, and the VehDyn score agrees closely with human judgment. VehDyn provides a systematic foundation for developing driving world models that are physically consistent and visually realistic.

Figures & tables

Benchmark	Venue & Year	Data	World Model Types		Vehicle Motion Evaluation			Human
Benchmark	Venue & Year	Data	General	Driving	Trajectory	Kinematics	Dynamics	Human
ACT-Bench	ICLRW ’25	nuScenes	✗	✓	✓	✗	✗	✗
WorldSimBench	ICML ’25	CARLA	✓	✗	$\triangle$	✗	✗	✓
DrivingGen	ICLR ’26	Real-World	✓	✓	✓	$\triangle$	✗	✓
WorldLens	CVPR ’26	nuScenes	✗	✓	✓	$\triangle$	$\triangle$	✓
What-If World	arXiv ’26	nuScenes	✓	✗	✗	✗	$\triangle$	✓

Table 1: Comparison of existing driving world model benchmarks and autonomous driving datasets. ✓: supported;

\triangle

: partial; ✗: not supported.

Figure 1: Overview of the proposed VehDyn benchmark. Left: data construction. A full factorial design over vehicle type, target speed, maneuver, road condition, scene, and illumination is instantiated on a CARLA-CarSim co-simulation platform. Right: evaluation. Vehicle states are recovered from the generated videos by DPVO and compared with ground truth at three levels, trajectory alignment, kinematic consistency, and dynamic consistency, and aggregated into the VehDyn score.

Figure 2: Dynamic characteristics of the VehDyn dataset under different target speeds, road conditions and vehicle types. (a)–(d), emergency-braking condition; (e)–(h), single-lane-changing condition. Curves represent the mean values, and shaded bands denote

\pm 1

standard deviation across scenes and illumination settings.

Figure 3: Overall results of the proposed VehDyn benchmark. (a) VehDyn score of the 12 evaluated models ordered by rank, with the ground-truth video scored through the same pipeline as the reference ceiling (dashed line). (b) Normalized level scores of the 12 evaluated models and ground truth.

VehDyn Metrics Models	Trajectory Alignment		Kinematic Consistency					Dynamic Consistency				Rank
	All		Emergency-Braking		Single-Lane-Changing			All	All
	ADE $\downarrow$	FDE $\downarrow$	$v_{x}$ $\downarrow$	Pitch $\downarrow$	$v_{y}$ $\downarrow$	Roll $\downarrow$	Yaw $\downarrow$	FCCR $\uparrow$	CDC Dir $\uparrow$	CDC Mag $\uparrow$	CDC Trend $\uparrow$
Ground Truth	0.514	0.814	0.897	2.522	0.697	0.602	3.885	0.900	0.722	0.713	0.803	–
General World Models
LTX-Video 2.5	1.665	2.310	3.217	2.077	0.932	0.558	3.330	0.686	0.563	0.610	0.539	3
CogVideoX 1.5	2.804	5.368	5.683	3.018	1.270	0.971	3.543	0.738	0.582	0.557	0.565	6

Table 2: Detailed quantitative results of the VehDyn benchmark for 12 representative video world models. Top: VehDyn metrics. Rank is by VehDyn score. Bottom: Visual evaluation under the WorldScore and DrivingGen benchmarks. Rank is by the sum of the normalized level scores. Best results are in red region , second best in orange region , and third best in blue region . Models fall into three categories: general world models, physical world models, and driving world models.

Figure 4: Cross-dimension correlation analysis. (a) Pearson correlation among the 18 metrics, with black lines separating the visual, trajectory, kinematic, and dynamic groups. (b) Pearson correlation between VehDyn score and human evaluation.

Figure 5: Qualitative comparison of the top-three models with ground truth. (a) Emergency-braking scenario. (b) Single-lane-changing scenario. Frames are shown at 0.5 s to 4.5 s. The right-hand panels plot the trajectory, velocity, and attitude.

Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Factor	Definition	Unit	Levels	Number
Dynamic Factors
Vehicle type	Ego vehicle model	–	Cooper S; Model 3, Grand Tourer; T2, Cybertruck	5
Friction coefficient	Tire-road friction coefficient	–	Asphalt: 0.85 (dry), 0.70 (wet); stone: 0.60 (dry), 0.50 (wet)	4
Driving maneuver	Prescribed ego maneuver	–	Single-lane-changing (left), single-lane-changing (right), emergency-braking	3
Target speed	Cruise speed before maneuver	km/h	20, 54, 72, 108	4
Visual Factors

Appendix

Table 3: Configuration of the VehDyn dataset.

Parameter	Unit	Cooper S	Model 3	Grand Tourer	T2	Cybertruck
Mass and Geometry
Sprung mass	kg	1220	1445	1370	1250	1800
Length	m	3.81	4.79	4.61	4.48	6.27
Width	m	1.97	2.16	2.24	2.07	2.39
Height	m	1.48	1.49	1.67	2.04	2.10
Wheelbase	m	2.13	2.68	2.58	2.51	3.20

Appendix

Table 4: Physical parameters of the five vehicle models.

Figure 6: Route diversity. For each of CARLA Towns 1–7, the map shows two straight road segments used as scenes (blue) and a representative frame from each, illustrating the variation in surroundings, building density, and horizon visibility across scenes at fixed road geometry.

Category	Variable	Definition	Source	Unit	Setting
Vehicle pose	$(x,y,z)$	Position of the vehicle reference point in the CARLA world frame	CarSim	m	30 Hz
Vehicle pose	$(\phi,\theta,\psi)$	Roll, pitch, yaw of the body	CarSim	deg	30 Hz
Vehicle state	$(v_{x},v_{y})$	Planar velocity recorded in the world frame and rotated into the body frame	CarSim	m/s	30 Hz
	$r$	Yaw rate about the body	CarSim	deg/s	30 Hz
	$(u_{\mathrm{thr}},u_{\mathrm{brk}},u_{\mathrm{str}})$	Throttle, brake, and steering commands issued by the controller	CARLA	–	Throttle and brake in $[0,1]$ , steering in $[-1,1]$
Scenario	Route progress	Distance traveled along the reference path	CARLA	m	30 Hz

Appendix

Table 5: Recorded variables, data sources, and acquisition settings of the VehDyn dataset.

Figure 7: Ground-truth vehicle responses at 20 km/h. (a) Braking distance during emergency-braking; (b) Peak pitch angle change during emergency-braking; (c) Peak yaw rate during single-lane-changing; (d) Peak roll angle change during single-lane-changing.

Figure 8: Ground-truth vehicle responses at 54 km/h. (a) Braking distance during emergency-braking; (b) Peak pitch angle change during emergency-braking; (c) Peak yaw rate during single-lane-changing; (d) Peak roll angle change during single-lane-changing.

Figure 9: Ground-truth vehicle responses at 72 km/h. (a) Braking distance during emergency-braking; (b) Peak pitch angle change during emergency-braking; (c) Peak yaw rate during single-lane-changing; (d) Peak roll angle change during single-lane-changing.

Figure 10: Ground-truth vehicle responses at 108 km/h. (a) Braking distance during emergency-braking; (b) Peak pitch angle change during emergency-braking; (c) Peak yaw rate during single-lane-changing; (d) (d) Peak roll angle change during single-lane-changing.

Method	Architecture	Conditioning	Parameters	Resolution	Frames	Inference (s/video)
General World Models
LTX-Video 2.5	DiT	First frame + Text	35.57 B	1280 $\times$ 704	121	173.9
CogVideoX 1.5	3D DiT	First frame + Text	10.55 B	1280 $\times$ 720	81	1082.6
SkyReels-V2-I2V	3D DiT	First frame + Text	20.26 B	1280 $\times$ 720	77	1875.6
Wan 2.2-I2V	3D DiT	First frame + Text	11.39 B	1280 $\times$ 704	77	246.2
HunyuanVideo 1.5-I2V	MMDiT	First frame + Text	17.98 B	1280 $\times$ 720	77	2130.6

Appendix

Table 6: Characteristics of the evaluated video world models.

Method	Vision Encoder / Depth Prior	Feature Dimensions	Geometric Module	Trajectory Estimation
DPVO	Two 4-block CNN encoders	128 matching; 384 context	Patch graph + differentiable BA	Recurrent sparse patch matching with joint pose/depth refinement
DrivingGen	SIFT + UniDepthV2	128-D SIFT descriptor; dense metric depth	RANSAC + PnP	Relative-pose chaining; constant-velocity extrapolation with orientation perturbation
GEM	DROID learned CNN + Depth Anything V2	128 matching; 256 context; dense depth	Recurrent dense update + dense BA	Dense correspondence optimization with depth-assisted scale estimation

Appendix

Table 7: Architectural comparison of the trajectory-reconstruction pipelines.

Maneuver	Response $r$	$s_{r}$	$\tau_{r}$	Unit
Emergency-braking	Braking distance $D_{\mathrm{brake}}$	29.173	1.459	m
	Braking duration $T_{\mathrm{brake}}$	1.469	0.073	s
	Peak pitch excursion $P_{\mathrm{pitch}}$	1.248	0.062	deg
	Peak pitch rate $\Omega_{\mathrm{pitch}}$	3.384	0.169	deg/s
Single-lane-changing	Peak lateral acceleration $A_{\mathrm{lat}}$	1.872	0.094	$\mathrm{m/s^{2}}$
	Peak yaw excursion $\Psi_{\mathrm{yaw}}$	1.639	0.082	deg

Appendix

Table 8: Ground-truth response scales

s_{r}

and meaningful-change thresholds

\tau_{r}

used for condition-aware dynamic consistency.

Metric	Strategy	$a_{m}$	$b_{m}$
ADE	VehDyn empirical	0.137	23.912
FDE	VehDyn empirical	0.061	49.116
$v_{x}$	VehDyn empirical	0.249	20.324
Pitch	VehDyn empirical	0.525	9.854
$v_{y}$	VehDyn empirical	0.030	22.919
Roll	VehDyn empirical	0.042	5.386

Appendix

Table 9: Bounds used for score normalization.

Methods	All Maneuvers		Emergency-Braking		Single-Lane-Changing
Methods	ADE $\downarrow$	FDE $\downarrow$	$v_{x}$ RMSE $\downarrow$	Pitch RMSE $\downarrow$	$v_{y}$ RMSE $\downarrow$	Roll RMSE $\downarrow$	Yaw RMSE $\downarrow$
DPVO	0.514 (0.301)	0.814 (0.346)	0.897 (0.469)	2.522 (2.162)	0.697 (0.521)	0.602 (0.468)	3.885 (3.813)
DrivingGen	2.202 (1.591)	4.536 (2.238)	1.572 (1.248)	8.660 (6.726)	3.552 (2.736)	9.141 (5.250)	9.095 (6.823)
GEM	0.589 (0.283)	1.071 (0.346)	0.982 (0.288)	2.765 (2.037)	1.045 (0.661)	0.624 (0.496)	3.888 (3.863)

Appendix

Table 10: Comparison of trajectory-reconstruction pipelines on the VehDyn dataset. Entries are sequence-level macro means, with medians in parentheses. All metrics are errors, so lower is better. The best mean value in each column is highlighted.

Figure 11: Qualitative comparison of vehicle motion under two representative driving scenarios. Top row: Emergency-braking, (a) vehicle trajectory, (b) longitudinal velocity $v_{x}$ , and (c) pitch angle. Bottom row: Single-lane-changing, (d) vehicle trajectory, (e) lateral velocity $v_{y}$ , and (f) roll angle.

Models	Trajectory Alignment		Kinematic Consistency		Dynamic Consistency				Rank
Models	ADE $\downarrow$	FDE $\downarrow$	$v_{x}$ $\downarrow$	Pitch $\downarrow$	FCCR $\uparrow$	CDC Dir $\uparrow$	CDC Mag $\uparrow$	CDC Trend $\uparrow$	Rank
Ground Truth	0.471	0.584	0.897	2.522	0.890	0.740	0.780	0.716	–
General World Models
LTX-Video 2.5	1.884	2.362	3.217	2.077	0.623	0.649	0.675	0.711	6
CogVideoX 1.5	3.642	7.462	5.683	3.018	0.846	0.699	0.633	0.734	9
SkyReels-V2-I2V	4.415	9.118	7.243	2.415	0.658	0.692	0.654	0.736	10

Appendix

Table 11: Emergency-braking results of the VehDyn benchmark. The rank is obtained from the maneuver-specific VehDyn score: normalized trajectory-alignment, kinematic-consistency, and dynamic-consistency scores are summed to a value in

[0,3]

. Best results are in red region , second best in orange region , and third best in blue region . Models fall into three categories: general world models, physical world models, and driving world models.

VehDyn Metrics Models	Trajectory Alignment		Kinematic Consistency			Dynamic Consistency				Rank
VehDyn Metrics Models	ADE $\downarrow$	FDE $\downarrow$	$v_{y}$ $\downarrow$	Roll $\downarrow$	Yaw $\downarrow$	FCCR $\uparrow$	CDC Dir $\uparrow$	CDC Mag $\uparrow$	CDC Trend $\uparrow$	Rank
Ground Truth	0.586	1.103	0.812	0.652	4.275	0.884	0.711	0.650	0.853	–
General World Models
LTX-Video 2.5	1.365	1.837	0.939	0.547	3.751	0.723	0.526	0.586	0.538	1
Wan 2.2-I2V	2.754	5.010	1.909	0.657	3.746	0.610	0.510	0.553	0.642	3
SkyReels-V2-I2V	1.862	2.225	1.114	0.710	3.771	0.528	0.505	0.556	0.486	4

Appendix

Table 12: Left-lane-changing results of the VehDyn benchmark. All metrics are evaluated only on left single-lane-changing sequences. The final rank is obtained from the maneuver-specific VehDyn score: normalized trajectory-alignment, kinematic-consistency, and dynamic-consistency scores are summed to a value in

[0,3]

. Best results are in red region , second best in orange region , and third best in blue region .

VehDyn Metrics Models	Trajectory Alignment		Kinematic Consistency			Dynamic Consistency				Rank
VehDyn Metrics Models	ADE $\downarrow$	FDE $\downarrow$	$v_{y}$ $\downarrow$	Roll $\downarrow$	Yaw $\downarrow$	FCCR $\uparrow$	CDC Dir $\uparrow$	CDC Mag $\uparrow$	CDC Trend $\uparrow$	Rank
Ground Truth	0.485	0.756	0.581	0.552	3.497	0.925	0.705	0.665	0.841	–
General World Models
LTX-Video 2.5	1.781	2.711	0.925	0.568	2.938	0.698	0.505	0.550	0.483	2
Wan 2.2-I2V	2.711	4.975	1.187	0.658	2.808	0.605	0.526	0.540	0.582	4
CogVideoX 1.5	2.772	4.996	1.164	1.000	2.951	0.681	0.520	0.486	0.528	7

Appendix

Table 13: Right-lane-changing results of the VehDyn benchmark. All metrics are evaluated only on right single-lane-changing sequences. The final rank is obtained from the maneuver-specific VehDyn score: normalized trajectory-alignment, kinematic-consistency, and dynamic-consistency scores are summed to a value in

[0,3]

. Best results are in red region , second best in orange region , and third best in blue region .

Models	Trajectory Alignment		Normalized Score	Rank
	All Maneuvers
	ADE $\uparrow$	FDE $\uparrow$
Ground Truth	0.984	0.985	0.984	–
General World Models
LTX-Video 2.5	0.936	0.954	0.945	1
SkyReels-V2-I2V	0.886	0.904	0.895	5

Appendix

Table 14: Normalized trajectory-alignment results. Best results are in red region , second best in orange region , and third best in blue region . Models fall into three categories: general world models, physical world models, and driving world models.

Models	Kinematic Consistency					Normalized Score	Rank
	Emergency-Braking		Single-Lane-Changing
	$v_{x}$ $\uparrow$	Pitch $\uparrow$	$v_{y}$ $\uparrow$	Roll $\uparrow$	Yaw $\uparrow$
Ground Truth	0.968	0.786	0.971	0.895	0.617	0.847	–
General World Models
LTX-Video 2.5	0.852	0.834	0.961	0.903	0.674	0.845	2
SkyReels-V2-I2V	0.652	0.797	0.952	0.853	0.678	0.786	8

Appendix

Table 15: Normalized kinematic-consistency results. Best results are in red region , second best in orange region , and third best in blue region . Models fall into three categories: general world models, physical world models, and driving world models.

Models	Dynamic Consistency				Normalized Score	Rank
	All Maneuvers	All Maneuvers
	FCCR $\uparrow$	CDC Dir $\uparrow$	CDC Mag $\uparrow$	CDC Trend $\uparrow$
Ground Truth	0.900	0.722	0.713	0.803	0.784	–
General World Models
CogVideoX 1.5	0.738	0.582	0.557	0.565	0.610	4
Wan 2.2-I2V	0.595	0.571	0.602	0.643	0.603	6

Appendix

Table 16: Normalized dynamic-consistency results. Best results are in red region , second best in orange region , and third best in blue region . Models fall into three categories: general world models, physical world models, and driving world models.

Models	VehDyn Benchmark			VehDyn Score	Rank
Models	Trajectory Alignment $\uparrow$	Kinematic Consistency $\uparrow$	Dynamic Consistency $\uparrow$	VehDyn Score	Rank
Ground Truth	0.984	0.847	0.784	2.616	–
General World Models
LTX-Video 2.5	0.945	0.845	0.599	2.389	3
CogVideoX 1.5	0.890	0.777	0.610	2.277	6
SkyReels-V2-I2V	0.895	0.786	0.580	2.261	7

Appendix

Table 17: Overall VehDyn score. Best results are in red region , second best in orange region , and third best in blue region . Models fall into three categories: general world models, physical world models, and driving world models.

Other Benchmarks Models	WorldScore				DrivingGen			Rank
	Quality				Distribution	Temporal Consistency
	3D Consistency $\uparrow$	Photometric Consistency $\uparrow$	Style Consistency $\uparrow$	Subjective Quality $\uparrow$	FVD Score $\uparrow$	Video Consistency $\uparrow$	Trajectory Consistency $\uparrow$
Ground Truth	0.7932	0.1988	0.9471	0.4284	1.0000	0.9102	0.4469	–
General World Models
LTX-Video 2.5	0.7551	0.2736	0.9446	0.2999	0.9196	0.9156	0.4709	1
SkyReels-V2-I2V	0.7190	0.6036	0.8878	0.4085	0.7431	0.9094	0.5130	2

Appendix

Table 18: Normalized visual evaluation results. Best results are in red region , second best in orange region , and third best in blue region . Models fall into three categories: general world models, physical world models, and driving world models.

Figure 12: Qualitative target-speed sweep: Ground Truth. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 13: Qualitative target-speed sweep: Cosmos 3 Nano. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 14: Qualitative target-speed sweep: Vanilla Cosmos 3 Nano. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 15: Qualitative target-speed sweep: Cosmos-Dreams. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 16: Qualitative vehicle-type sweep: Ground Truth. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 17: Qualitative vehicle-type sweep: Epona. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 18: Qualitative vehicle-type sweep: Wan 2.2-I2V. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 19: Qualitative vehicle-type sweep: HunyuanVideo 1.5-I2V. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 20: Qualitative tire-road-friction sweep: Ground Truth. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 21: Qualitative tire-road-friction sweep: Vista. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 22: Qualitative tire-road-friction sweep: LTX-Video 2.5. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 23: Qualitative tire-road-friction sweep: SkyReels-V2-I2V. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 24: Qualitative environment sweep: Ground Truth. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 25: Qualitative environment sweep: DrivingWorld. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 26: Qualitative environment sweep: CogVideoX 1.5. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Figure 27: Qualitative environment sweep: GEM. Top: Emergency-braking scenario; Bottom: Single-lane-changing scenario.

Score	TA: does the vehicle follow the reference path?	KC: does the vehicle move like the reference vehicle?	DC: does the motion respond to the changed condition like the reference?
1.0	Same lane position and heading throughout; stops (braking) or settles in the new lane (lane-changing) at the same point as the reference, within about one vehicle length.	Deceleration or lateral shift begins and ends at the same times as the reference; nose dip, body roll, and heading change match in magnitude and timing; no jitter or implausible sway.	Between the two clips, the generated braking distance, lane-changing duration, and body attitude change in the same direction as the reference and by a visibly comparable amount.
0.8–0.9	Same path with a small lateral offset, or a stopping/settling point one to two vehicle lengths from the reference.	Correct sequence and magnitude of motion with a small timing offset, or a slightly under- or over-stated attitude response.	Correct direction of change for every response; the magnitude of the main response is somewhat under- or over-stated.
0.6–0.7	Maneuver completed on a recognizably similar path, but with a lateral drift of up to about half a lane width or a stop two to four vehicle lengths early or late.	Maneuver recognizable and correctly ordered, but the speed profile or one attitude response is clearly wrong in magnitude.	Correct direction of change for the main response, but the magnitude is clearly wrong or a secondary response changes in the wrong direction.
0.4–0.5	Maneuver completed, but the path drifts by about a lane width or the stop is several vehicle lengths early or late.	Maneuver recognizable, but the motion is visibly jerky, or two or more responses are wrong in magnitude or timing.	The response changes in the right direction for some quantities and not for others, or in the right direction but with negligible magnitude.
0.2–0.3	Maneuver only partly executed (incomplete lane-changing, braking that does not stop the vehicle), or the path diverges over most of the clip.	Motion largely inconsistent with the reference (no deceleration where the reference brakes hard; roll or heading changes that the reference does not show).	The two generated clips are nearly identical although the reference clips differ clearly, or the change is in the wrong direction for the main response.
0.0–0.1	Wrong maneuver, vehicle leaves the road, or the ego vehicle is not identifiable.	Motion physically implausible (teleportation, reversal, the vehicle body deforms) or absent.	The generated response changes in the opposite direction to the reference for the main response, or the clips are unusable for comparison.

Appendix

Table 19: Scoring rubric for the human study. Each level is rated on

[0,1]

in increments of

0.1

. TA: trajectory alignment; KC: kinematic consistency; DC: dynamic consistency.

Explore similar work

May 11, 2026cs.CV

Is Your Driving World Model an All-Around Player?

Today's driving world models can generate remarkably realistic dash-cam videos, yet no single model excels universally. Some generate photorealistic textures but violate basic physics; others maintain geometric consistency but fail when subjected to closed-loop planning. This disconnect exposes a critical gap: the field evaluates how real generated worlds appear, but rarely whether they behave realistically. We introduce WorldLens, a unified benchmark that measures world-model fidelity across the full spectrum, from pixel quality and 4D geometry to closed-loop driving and human perceptual alignment, through five complementary aspects and 24 standardized dimensions. Our evaluation of six representative models reveals that no existing approach dominates across all axes: texture-rich models violate geometry, geometry-aware models lack behavioral fidelity, and even the strongest performers achieve only 2-3 out of 10 on human realism ratings. To bridge algorithmic metrics with human perception, we further contribute WorldLens-26K, a 26,808-entry human-annotated preference dataset pairing numerical scores with textual rationales, and WorldLens-Agent, a vision-language evaluator distilled from these judgments that enables scalable, explainable auto-assessment. Together, the benchmark, dataset, and agent form a unified ecosystem for assessing generated worlds not merely by visual appeal, but by physical and behavioral fidelity.

Lingdong Kong, Ao Liang, Tianyi Yan +20

NUS · UM · USTC +10

Video World Models World Models Generative Video Models Fidelity Rater Texture

Ask PDF

Apr 5, 2026cs.CV

DriveVA: Video Action Models are Zero-Shot Drivers

Generalization is a central challenge in autonomous driving, as real-world deployment requires robust performance under unseen scenarios, sensor domains, and environmental conditions. Recent world-model-based planning methods have shown strong capabilities in scene understanding and multi-modal future prediction, yet their generalization across datasets and sensor configurations remains limited. In addition, their loosely coupled planning paradigm often leads to poor video-trajectory consistency during visual imagination. To overcome these limitations, we propose DriveVA, a novel autonomous driving world model that jointly decodes future visual forecasts and action sequences in a shared latent generative process. DriveVA inherits rich priors on motion dynamics and physical plausibility from well-pretrained large-scale video generation models to capture continuous spatiotemporal evolution and causal interaction patterns. To this end, DriveVA employs a DiT-based decoder to jointly predict future action sequences (trajectories) and videos, enabling tighter alignment between planning and scene evolution. We also introduce a video continuation strategy to strengthen long-duration rollout consistency. DriveVA achieves an impressive PDM-based planning performance of 90.9 PDM score on the NAVSIM benchmark. Extensive experiments also demonstrate the zero-shot capability and cross-domain generalization of DriveVA, which reduces average L2 error and collision rate by 78.9% and 83.3% on nuScenes and 52.5% and 52.4% on the Bench2Drive built on CARLA v2 compared with the state-of-the-art world-model-based planner.

Mengmeng Liu, Diankun Zhang, Jiuming Liu +7

University of Twente, The Netherlands · Xiaomi EV, China · University of Cambridge, United Kingdom +1

Autonomous Driving Drives World Model Planning Generative Video Models

Ask PDF

Apr 23, 2026cs.CV

WorldMark: A Unified Benchmark Suite for Interactive Video World Models

Interactive video generation models such as Genie, YUME, HY-World, and Matrix-Game are advancing rapidly, yet every model is evaluated on its own benchmark with private scenes and trajectories, making fair cross-model comparison impossible. Existing public benchmarks offer useful metrics such as trajectory error, aesthetic scores, and VLM-based judgments, but none supplies the standardized test conditions -- identical scenes, identical action sequences, and a unified control interface -- needed to make those metrics comparable across models with heterogeneous inputs. We introduce WorldMark, the first benchmark that provides such a common playing field for interactive Image-to-Video world models. WorldMark contributes: (1) a unified action-mapping layer that translates a shared WASD-style action vocabulary into each model's native control format, enabling apples-to-apples comparison across six major models on identical scenes and trajectories; (2) a hierarchical test suite of 500 evaluation cases covering first- and third-person viewpoints, photorealistic and stylized scenes, and three difficulty tiers from Easy to Hard spanning 20-60s; and (3) a modular evaluation toolkit for Visual Quality, Control Alignment, and World Consistency, designed so that researchers can reuse our standardized inputs while plugging in their own metrics as the field evolves. We will release all data, evaluation code, and model outputs to facilitate future research. Beyond offline metrics, we launch World Model Arena (warena.ai), an online platform where anyone can pit leading world models against each other in side-by-side battles and watch the live leaderboard.

Xiaojie Xu, Zhengyuan Lin, Kang He +5

Alaya Studio, Shanda AI Research Tokyo · The University of Tokyo · Shanghai Innovation Institute

Video World Models World Models Generative Video Models Interactive Video Generation World Action Sequences+1

Ask PDF