World Embedding Benchmark

Yiqi Liu; Ruifeng Yuan; Yang Wang; Long Li; Fengyu Cai; Hou Pong Chan; Jialin Yu; Hao Zhang

cs.CVOct 2, 2026

World Embedding Benchmark

Authors: Yiqi Liu, Ruifeng Yuan, Yang Wang, Long Li, Fengyu Cai, Hou Pong Chan, Jialin Yu, Hao Zhang,

Organizations: World-Embedding Team

Abstract

Physical fidelity has received increasing attention in world models and video generation, yet how video representations encode physical information remains less understood. We introduce the World Embedding Benchmark, comprising 8,000 controlled simulation cases from 80 families spanning fluid mechanics, solid mechanics, dynamics, and optics & electromagnetism. Each case pairs a rendered video with simulation-derived physical annotations, supporting three complementary tasks: text-video retrieval, physical-property regression, and multiple-choice video-description pair classification. We use these tasks to distinguish cross-modal physical alignment from the recoverability of quantitative physical information. Evaluated pre-trained omnimodal embedding models show weak retrieval and near-chance within-family pair classification, while lightweight probes recover useful physical information from frozen video embeddings. Continual contrastive training with physics-specific video-text pairs improves retrieval and pair classification but degrades physical-property regression, revealing a trade-off between alignment and quantitative information recoverability. Finally, we use the embeddings to retrieve reference videos for retrieval-augmented generation with MiniMax-H3. Retrieved references improve the physical fidelity of generated videos, with stronger retrieval models yielding larger gains in our experiments. Together, these findings highlight the need to evaluate physical alignment and property recoverability jointly, and demonstrate the utility of physical representations for improving video generation.

Figures & tables

Figure 1: Overview of the three evaluation tasks: text-video retrieval, physical-property regression, and video-description pair classification.

Branch	Families	Cases	Computation
Fluid Mechanics	7	700	OpenFOAM
Solid Mechanics	27	2,700	DOLFINx
Dynamics	19	1,900	Chrono + Analytical / ODE integration
Optics & EM	27	2,700	Analytical + HCIPy + Meep (FDTD)
Total	80	8,000	–

Table 1: Simulation families, case counts, and computation methods across physics branches.

Figure 2: Example video frames of representative scene families of the 4 physics branches, illustrating different stages of each simulation sequence. The full dataset contains videos from 80 families.

Model	Modality	Retrieval R@10 $\uparrow$				Regression nRMSE $\downarrow$	Pair Classification Acc. $\uparrow$
Model	Modality	Fluid	Solid	Dynamics	Optics & EM	Avg. ( $\times 100$ )	Within-Fam.	Cross-Fam.
Embedding Models
Video-CLIP-XL	T, V	3.3	1.4	2.3	0.6	0.81	48.1	66.1
Qwen3-VL-Embedding-2B	T, I, V	3.9	1.6	7.3	2.3	1.76	51.5	80.8
Qwen3-VL-Embedding-8B	T, I, V	6.6	1.9	7.4	3.1	1.80	53.1	86.2
Omni-Embed-Nemotron-3B	T, I, V, A	3.1	1.4	3.1	0.7	1.41	52.5	71.2

Table 2: Performance on the World Embedding Benchmark. Retrieval is evaluated globally within each physics branch. Regression reports the macro-average nRMSE across 10 physical-property regression tasks. Pair classification reports within-family and cross-family accuracy. Bold denotes best across all models, underline denotes second-best; * denotes best in embedding models.

Figure 3: Retrieval performance (y-axis) across training steps (x-axis) and training dataset settings (different lines).

Figure 4: Retrieval performance of 3B and 7B backbones after physics adaptation across the four physics branches.

	No RAG	VideoCLIP-XL	LCO-Omni-3B	LCO-Omni-3B + Ft. (general+physics)	LCO-Omni-3B + Ft. (physics-only)
Optics	0.67	0.63	0.65	0.68	0.65
Thermal	0.53	0.54	0.61	0.63	0.64
Average	0.60	0.59	0.63	0.66	0.65

Table 3: Retrieval-augmented video generation with MiniMax-H3 using top-1 retrieved references.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Formulation	Computation	Case parameters	Field output
Fluid Mechanics
buoyancy	equation 1 , equation 2	OpenFOAM (FVM)	cavity dimensions; temperature difference; reference temperature; kinematic viscosity; thermal diffusivity; gravity	$\lVert\mathbf{U}\rVert$
cavity	equation 1	OpenFOAM (FVM)	cavity dimensions; lid velocity; kinematic viscosity; Reynolds number $Re$	$\lVert\mathbf{U}\rVert$
external_aero	equation 1	OpenFOAM (FVM)	airfoil chord; thickness scale; free-stream velocity; kinematic viscosity; Reynolds number $Re$	$\lVert\mathbf{U}\rVert$
external_wake	equation 1	OpenFOAM (FVM)	obstacle diameter or width; free-stream velocity; kinematic viscosity; Reynolds number $Re$	$\lVert\mathbf{U}\rVert$
internal_step	equation 1 , equation 3	OpenFOAM (FVM)	channel dimensions; inlet velocity; Reynolds number $Re$ ; reference viscosity $\nu_{0}$ ; viscosity bounds $\nu_{\min},\nu_{\max}$ ; consistency $K$ ; exponent $n$ ; yield stress $\tau_{0}$	$\lVert\mathbf{U}\rVert$

Appendix

Table 4: Summary of simulation families, their physical formulations, computation methods, case parameters, and field outputs.

Family	Target	Range
pendulum	gravity $g$	$2$ – $18~\mathrm{m\,s^{-2}}$
rigidCollision	mass $m_{2}$	$2$ – $18~\mathrm{kg}$
sphereDrop3D	restitution $e$	$0.35$ – $0.90$
dye2d	Péclet number $Pe$	$30$ – $1200$
flowpastcylinder2d	Reynolds number $Re$	$80$ – $500$
totalInternalReflection	refractive index $n$	$1.44$ – $1.78$

Appendix

Table 5: Continuous physical quantities used in the regression benchmark.

Figure 6: Linear regression performance (y-axis) of different contrastively trained checkpoints (different lines), across different numbers of probe examples (x-axis).

Figure 7: Within-family and cross-family pair classification performance across training steps.

Figure 8: Retrieval performance (y-axis) across training steps (x-axis) and training dataset settings (different lines).

Physical Reasoning in Video

Physical reasoning in video studies how models infer, predict, and generate motion and interactions consistent with physical laws.

See all 74 papers

PhyWorld: Physics-Faithful World Model for Video Generation
May 19, 2026Pu Zhao, Juyi Lin, Timothy Rupprecht +10Physics-Informed Generative Modeling Physical Consistency in Video Generation
Northeastern University · University of Georgia · Tulane University +1
PhyGround: Benchmarking Physical Reasoning in Generative World Models
May 11, 2026Juyi Lin, Arash Akbari, Yumei He +13Physical Consistency in Video Generation World Model Evaluation
1Northeastern University · 2Tulane University · University of Washington +1
Off-Manifold Refinement: Guiding Video Generators with a Frozen World Model
Aug 30, 2026Hai Nguyen-Truong, Tuan-Anh Vu, Dang HuynhPhysics-Informed Generative Modeling Physical Consistency in Video Generation
Fulbright University Vietnam · University of California, Los Angeles

May 19, 2026J/K move · Enter open · S save

PhyWorld: Physics-Faithful World Model for Video Generation

Pu Zhao, Juyi Lin, Timothy Rupprecht, Arash Akbari+9

Northeastern University · University of Georgia · Tulane University+1

Physics-Informed Generative Modeling Physical Consistency in Video Generation Video Generation Temporal Consistency in Video Generation Video World Models

Open paper AskPDF arXiv

World simulators can provide safe and scalable environments for training Physical AI systems before real-world deployment. Large video generation models are emerging as a promising basis for such simulators because they can generate diverse and realistic visual futures. However, using them as world simulators requires physically faithful video continuations, namely, generated videos that preserve the physical state implied by the conditioning input, and evolve in ways consistent with basic physical principles. We propose PhyWorld, a video generation world model designed to produce temporally coherent and physically faithful scene continuations through two-stage post-training. In the first stage, we improve video-to-video continuation with flow matching fine-tuning, encouraging stable visual attributes and coherent motion dynamics across frames. In the second stage, we align generated dynamics with physical principles using Direct Preference Optimization (DPO) over physics preference pairs, guiding the model toward outputs with higher physical plausibility. To evaluate PhyWorld, we use both standard video-quality benchmarks and a dedicated physical-faithfulness benchmark with per-law scoring. Experiments show that PhyWorld improves video consistency, achieving an average score of 0.769 on VBench compared with 0.756 or below for state-of-the-art baselines. PhyWorld also improves physical plausibility, reaching an average score of 3.09 on our physical-faithfulness benchmark compared with 2.99 for the strongest baseline. These results suggest that post-training large video generation models with continuation and physics-preference signals can make them more effective world simulators for Physical AI.

Abstract

Physical Reasoning in Video

Explore similar work

PhyWorld: Physics-Faithful World Model for Video Generation

PhyGround: Benchmarking Physical Reasoning in Generative World Models

Off-Manifold Refinement: Guiding Video Generators with a Frozen World Model