World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models

Shinseong Kim; Jaeseok Jeong; Jianlong Fu; Youngjung Uh

cs.AIOct 12, 2026

World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models

Authors: Shinseong Kim, Jaeseok Jeong, Jianlong Fu, Youngjung Uh

Organizations: Yonsei University · Microsoft Research Asia

Abstract

Video world models must generate physically plausible futures, not just realistic frames. Existing approaches typically use semantic, physical, or predictive information to guide the visual generation stream, leaving a fundamental question: where inside the generator should predictions of scene evolution live? Standard video tokens jointly encode appearance and dynamics, but provide no distinct state for learning and reading predictive dynamics. We answer this question with the Physical Registry: temporally grounded registry slots inside a video DiT that are initialized from an observation-conditioned forecast, co-evolve with video tokens, and receive future-representation supervision, making predictive dynamics an explicit, readable state of the generator. In a controlled comparison of predictive-state interfaces on Physics-IQ Verified, Physical Registry improves vanilla fine-tuning by 9.9/8.7 points on I2V/V2V, the largest gain among the interfaces we compare. Across all Cosmos and WAN backbones, Physical Registry improves Physics-IQ Verified scores. With only 0.45% additional parameters, it enables Cosmos 3 Nano 16B to surpass the published I2V score of Cosmos 3 Super 64B (43.56 vs. 42.7). On WorldModelBench, it also improves physical adherence and the total score for all four evaluated backbones. Because the registry becomes candidate-specific while remaining readable, the Physical Registry Selector (PRS) ranks emerging futures by how far their intermediate registry readouts depart from the shared forecast before full denoising is complete. Across four backbones, this selection improves I2V scores by 5.6-15.9% without completed-video external scoring. Together, these results support a broader principle: dynamics prediction should live as an explicit model state that co-evolves with the video representation. Project page: https://grainsack.github.io/physical-registry/

Figures & tables

Figure 1: Physical Registry during training and inference. The Physical Registry gives predicted future dynamics a dedicated, readable state inside the video DiT. (a) Training: forecast-initialized registry tokens co-evolve with video tokens and receive predictive supervision. (b) Inference: the same registry provides readouts for PRS (Section 3.5 ).

Figure 2: Three-stage prefix selection with PRS. Candidates share the same forecast initialization, are ranked from registry readouts over a $K$ -step prefix, and only the selected trajectory continues for the remaining $D-K$ steps.

Variant	Mechanism	Registry	Alignment	Readout	I2V	$\Delta$	V2V	$\Delta$
Vanilla fine-tuning	–	$\boldsymbol{\times}$	$\boldsymbol{\times}$	$\boldsymbol{\times}$	$41.3\pm 2.1$	–	$46.4\pm 1.0$	–
REPA ( Yu et al., 2025 )	Feature alignment	$\boldsymbol{\times}$	$\checkmark$	$\boldsymbol{\times}$	$41.8\pm 1.7$	+0.5	$46.9\pm 0.9$	+0.5
Cross-attention ( Peebles & Xie, 2023 )	External context	$\boldsymbol{\times}$	$\boldsymbol{\times}$	$\boldsymbol{\times}$	$43.1\pm 1.9$	+1.8	$49.2\pm 1.2$	+2.8
AdaLN ( Peebles & Xie, 2023 )	Feature modulation	$\boldsymbol{\times}$	$\boldsymbol{\times}$	$\boldsymbol{\times}$	$45.1\pm 0.8$	+3.8	$48.3\pm 0.5$	+1.9
Joint generation ( Kouzelis et al., 2025 )	Joint denoising	$\boldsymbol{\times}$	$\checkmark$	$\boldsymbol{\times}$	$45.1\pm 1.8$	+3.8	$48.0\pm 0.4$	+1.6
Registry only	Dedicated predictive state	$\checkmark$	$\boldsymbol{\times}$	$\checkmark$	$45.0\pm 1.3$	+3.7	$50.3\pm 0.9$	+3.9

Table 1: Controlled comparison of predictive-state interfaces. Interface mechanisms and Physics-IQ Verified scores after center-view training and left/right evaluation, with std over three generation seeds and

\Delta

relative to vanilla fine-tuning. Registry only sets

\lambda_{\mathrm{align}}=0

in Eq. ( 10 ).

Model	I2V			V2V
	Plain	$\Delta$	PRS	Plain	$\Delta$	PRS
Cosmos 3 Nano 16B ( NVIDIA et al., 2026 )	30.18 ± 0.84	–		38.78 ± 0.78	–
+ Physical Registry	36.01 ± 0.95	+5.83	40.27 (+4.26)	45.12 ± 0.94	+6.34	46.78 (+1.66)
Cosmos 3 Nano 16B † ( NVIDIA et al., 2026 )	37.3 ± 0.9 *	–		46.45 ± 0.76	–
+ Physical Registry †	43.56 ± 0.95	+6.26	45.81 (+2.25)	49.54 ± 0.63	+3.09	52.06 (+2.52)
Cosmos v2.5 2B ( NVIDIA et al., 2025b )	30.65 ± 0.37	–		37.40 ± 0.53	–

Table 2: Physics-IQ Verified. Plain averages our candidate pool or reports the published value ( * ).

\Delta

is the gain over the backbone above, green parentheses the PRS gain over Plain. Bold and underline mark the best and second-best per task. Gray marks unavailable entries, dashes unreported.

Figure 3: Qualitative comparisons across five Physics-IQ categories. Top: Cosmos 3 Nano 16B backbone; bottom: Physical Registry, with matched conditioning and eight equally spaced frames. Physical Registry better realizes the intended physical event: ball separation after collision, liquid motion in the inverted jar, persistence of the ball and its reflection, preservation of the paper under rising smoke, and wrench attraction toward the magnet. More examples: Figures 16 to 18 .

Model	Instruction following (0–3)	Physical adherence (0–5)	Common sense (0–2)	Total (0–10)
Cosmos 3 Nano 16B ( NVIDIA et al., 2026 )	2.20 ± 0.02	4.50 ± 0.08	1.43 ± 0.06	8.13 ± 0.13
+ Physical Registry	2.20 ± 0.002 (–)	4.82 ± 0.03 (+0.32)	1.81 ± 0.02 (+0.38)	8.83 ± 0.04 (+0.69)
Cosmos v2.5 2B ( NVIDIA et al., 2025b )	2.14 ± 0.03	4.43 ± 0.07	1.55 ± 0.05	8.12 ± 0.14
+ Physical Registry	2.07 ± 0.01 (-0.07)	4.54 ± 0.03 (+0.11)	1.58 ± 0.04 (+0.03)	8.19 ± 0.01 (+0.07)
WAN 2.2 I2V-A14B ( Wan Team, 2025 )	2.23 ± 0.03	4.45 ± 0.04	1.60 ± 0.03	8.28 ± 0.02
+ Physical Registry	2.30 ± 0.04 (+0.07)	4.53 ± 0.04 (+0.07)	1.60 ± 0.02 (+0.004)	8.43 ± 0.05 (+0.15)

Table 3: WorldModelBench results. Values in parentheses indicate the change from the corresponding backbone. * denotes published leaderboard scores ( Li et al., 2025a ) . Table 10 for the full results.

Figure 4: Selection comparison on Physics-IQ Verified. (left) I2V and (right) V2V. Dashed caps mark Oracle scores and hatched bars denote published scores.

Figure 5: Registry attention and activation during balloon inflation. Foreground queries are video tokens inside the Physics-IQ motion mask of the generated video. Labels give query $\rightarrow$ key pairs (e.g., q3 $\rightarrow$ r1: queries in latent frame 3 attending to its time-aligned registry slot r1); r1 alone marks that slot’s activation. Protocol: Appendix A ; more examples: Appendices H.1 and I.3 .

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: The registry projection layer $G_{\phi}$ . The three bands show spatial lifting with 3D convolutions ( Tran et al., 2015 ) , per-position temporal attention ( Ho et al., 2022 ) and channel projection, and backbone input assembly. Shapes are for the Cosmos v2.5 2B configuration.

Model	Pretrained	Registry modules	Batch/update	GPUs
Cosmos 3 Nano 16B	16B	72.63M	16	$8\times$ H100
Cosmos v2.5 2B	2B	35.93M	16	$8\times$ A100 40 GB
WAN 2.2 I2V-A14B	$2\times 14$ B	129.39M	16	$8\times$ H100
WAN 2.2 TI2V-5B	5B	105.02M	16	$8\times$ A100 40 GB

Appendix

Table 6: Model sizes, fine-tuning resources, and optimization settings. Registry modules denote the added projection and readout parameters (e.g., 16.0M + 19.9M for Cosmos v2.5 2B).

Model	Steps	Guidance	Shift	Sampler	Resolution	Gen. frames @ fps
Cosmos 3 Nano 16B	35	6.0	10.0	UniPC	$1280\times 720$	120 @ 24
Cosmos 3 Nano 16B †	50	3.0	4.75	UniPC	$1280\times 720$	120 @ 24
Cosmos v2.5 2B	35	7.0	5.0	UniPC	$1280\times 704$	92 / 80 @ 16
WAN 2.2 I2V-A14B	40	3.5 / 3.5	5.0	UniPC	$1280\times 720$	80 @ 16
WAN 2.2 TI2V-5B	50	5.0	5.0	UniPC	$1280\times 704$	120 @ 24

Appendix

Table 7: Sampling settings. Steps are denoising steps and guidance is the classifier-free guidance scale (WAN I2V-A14B: high-/low-noise experts). Generated frames exclude conditioning frames (Cosmos v2.5: I2V / V2V). † Expanded-prompt setting of Table 2 .

Model	Arm	Sequence (tokens)	s / sampler step	Peak (GiB)
Cosmos v2.5 2B	Backbone	84,480	6.514	6.20
	$+$ Physical Registry	112,640	10.909	7.00
Cosmos 3 Nano 16B	Backbone	28,607 / 31,364	1.384	11.44
	$+$ Physical Registry	35,967 / 38,724	2.148	11.94

Appendix

Table 8: Sampling cost of the registry. Both arms of a backbone are measured in one session on a single H100 80 GB. Times are per sampler step; peak memory is the generation peak above the weights resident before sampling. Two sequence lengths are given for Cosmos 3 Nano 16B because its two classifier-free guidance branches carry different text lengths.

Panel of Figure 8 — Figure 8: Cross-view JEPA feature correspondence. Views of the same scene (blue) are closer than different scenes (red) in L2 distance (left; means 15.7 versus 28.0) and cosine similarity (right; 0.9887 versus 0.9649).

Figure 9: Physics-IQ Verified gains over pretrained WAN TI2V-5B. Shading marks one pretrained standard deviation ( $\pm 0.8$ ) around zero.

I2V	Spatial	Spatiotemp.	Weighted	Inv. MSE	Verified	Original
Nano 16B + PR	46.63	31.01	32.76	33.63	36.01	35.00
Nano 16B † + PR	60.70	33.56	42.46	37.53	43.56	42.95
Cosmos v2.5 2B + PR	44.46	34.44	30.29	25.65	33.71	34.00
WAN I2V-A14B + PR	44.77	42.06	32.67	34.19	38.44	38.90
WAN TI2V-5B + PR	36.82	30.81	22.33	26.99	29.24	27.87
V2V	Spatial	Spatiotemp.	Weighted	Inv. MSE	Verified	Original

Appendix

Table 9: Physics-IQ components and original Physics-IQ score. Component scores, the Verified score, and the original Physics-IQ score for our Physical Registry (PR) models, each normalized to 0–100. Bold / underline mark the highest/second-highest value per column within a task. † Expanded-prompt setting of Table 2 .

Model	Instruction following (0–3)	Physical adherence (0–5)	Common sense (0–2)	Total (0–10)
Cosmos 3 Nano 16B ( NVIDIA et al., 2026 )	2.20 ± 0.02	4.50 ± 0.08	1.43 ± 0.06	8.13 ± 0.13
+ Physical Registry	2.20 ± 0.002 (–)	4.82 ± 0.03 (+0.32)	1.81 ± 0.02 (+0.38)	8.83 ± 0.04 (+0.69)
Cosmos v2.5 2B ( NVIDIA et al., 2025b )	2.14 ± 0.03	4.43 ± 0.07	1.55 ± 0.05	8.12 ± 0.14
+ Physical Registry	2.07 ± 0.01 (-0.07)	4.54 ± 0.03 (+0.11)	1.58 ± 0.04 (+0.03)	8.19 ± 0.01 (+0.07)
WAN 2.2 I2V-A14B ( Wan Team, 2025 )	2.23 ± 0.03	4.45 ± 0.04	1.60 ± 0.03	8.28 ± 0.02
+ Physical Registry	2.30 ± 0.04 (+0.07)	4.53 ± 0.04 (+0.07)	1.60 ± 0.02 (+0.004)	8.43 ± 0.05 (+0.15)

Appendix

Table 10: WorldModelBench reference scores. Backbone pairs report mean

\pm

std, rounded to two decimals. Bold / underline mark the highest/second-highest mean in each displayed column. Green/red parentheses show positive/negative Physical Registry-minus-backbone changes before rounding; (–) marks a tie. Rows from KLING onward are published references under their own generation settings ( Li et al., 2025a ) . Main results: Table 3 .

	Scene	Motion			Semantics			Diversity
Model	Consist.	HSD	NDTW	DYN	BLEU	CLIP	Logic	Div.
Cosmos 3 Nano 16B ( NVIDIA et al., 2026 )	0.855	0.328	0.305	0.383	0.242	0.884	0.921	0.029
$+$ Physical Registry	0.895	0.351	0.378	0.459	0.244	0.897	0.921	0.018
	(+0.040)	(+0.023)	(+0.073)	(+0.076)	(+0.002)	(+0.013)		(-0.011)
EnerVerse-FT * ( Huang et al., 2025 )	0.943	0.536	0.596	0.536	0.180	0.864	0.978	0.069
LTX-FT * ( HaCohen et al., 2024 )	0.944	0.476	0.521	0.620	0.174	0.855	0.967	0.016

Appendix

Table 11: EWMBench ( Hu et al., 2025 ) . All metrics are higher-is-better and shown on a 0–1 scale. Bold/underline mark the highest/second-highest value in each column across all rows, and green parentheses give the Physical Registry change over the backbone above it.

I2V	Plain	WMReward	PRS	Oracle
Cosmos 3 Nano 16B	30.18 ± 0.84	31.69		41.48
+ Physical Registry	36.01 ± 0.95	38.85	40.27	51.88
Cosmos 3 Nano 16B †	37.3 ± 0.9 *	44.28		50.59
+ Physical Registry †	43.56 ± 0.95	45.41	45.81	54.72
Cosmos v2.5 2B	30.65 ± 0.37	30.61		36.07
+ Physical Registry	33.71 ± 1.11	37.74	39.08	49.37

Appendix

Table 12: Selection policies on Physics-IQ Verified. WMReward and PRS each select one candidate from our ten-candidate pool; Oracle is the best in that pool, an upper bound rather than a policy. * marks published Plain values, external to the pool, and † the configuration in Table 2 . Bold marks the strongest non-Oracle score in each row, gray unavailable PRS.

Figure 11: Early selection and decision agreement. (a,b) Prefix-selected Physics-IQ Verified gain over WMReward using the PRS settings in Table 12 ; zero marks the WMReward reference. (c,d) Agreement with the full-schedule PRS decision. Horizontal axes show the fraction of denoising steps used for selection. † denotes the expanded-prompt Nano configuration in Table 2 . Protocol: Appendix F.1 .

Backbone	Policy	Candidate-steps	Decodes	Wall-clock (s)	Speed-up
Cosmos v2.5 2B	Complete BoN $+$ WMReward	350	10	4,031	1.00 $\times$
	PRS ( $K=10$ of $D=35$ )	125	1	1,420	$2.84\times$
Cosmos 3 Nano 16B †	Complete BoN $+$ WMReward	500	10	1,985	1.00 $\times$
	PRS ( $K=10$ of $D=50$ )	140	1	614	$3.23\times$
WAN 2.2 I2V-A14B	Complete BoN $+$ WMReward	400	10	10,748	1.00 $\times$
	PRS ( $K=10$ of $D=40$ )	130	1	3,289	$3.27\times$

Appendix

Table 13: End-to-end cost of selection. One input,

N=10

candidates, both policies run on the same Physical Registry model inside one job on a single H100 80 GB; model loading is excluded. Complete BoN times sampling, all ten VAE decodes, and the ten WMReward evaluations; PRS times the ten prefixes, the continuation of the selected candidate, and its single decode.

K=10

is an illustrative prefix length. † Expanded-prompt setting (Table 7 ).

Figure 12: Registry-source configurations for motion transfer. The native configuration uses the conditioning image’s JEPA context and forecast; the transfer configuration uses video-encoded features from the source video. Both use the same conditioning frame, general text, initial noise, and projection recipe. The exact prompt is shown in a monospace block and contains no motion description.

Figure 13: Motion transfer with matched inputs. The source video supplies JEPA features; A and B are generated videos. Red dots track ball centers and rows share timestamps. The plot shows the upward return $r(t)=\max_{s\leq t}y(s)-y(t)$ , with image $y$ increasing downward, over the second after A’s first landing (0.333 s). B first lands at 0.417 s. Numbers mark A’s three diminishing returns.

Configuration	Valid frames	Vertical span	Upward return	Path length
A: Native forecast	121/121	289.7	80.1	526.9
B: Transfer	121/121	281.1	3.6	290.9

Appendix

Table 14: Image-plane ball-center movement in the matched pair. Both outputs use Cosmos 3 Nano 16B with Physical Registry, over all 121 frames. Image

y

increases downward, and distances are image-plane pixels. Definitions in Figure 13 .

Figure 14: Descent transfer from a slow-descent source. A and B share the prompt, initial noise, sampler, and amplification, and differ only in the registry source. Red dots track ball centers and rows share timestamps. The plot follows each center to first table contact, with image $y$ increasing downward and contact marked on each curve.

Figure 15: Registry feature organization across DiT layers. Columns align five generated frames with their registry slots. Heat rows show hidden channel variance at denoising step 16 for layers 7, 14, 21, and 27. Each layer uses an independent viridis log scale over all eight slots, so colors preserve within-layer slot magnitude.

Figure 16: Qualitative comparisons on Cosmos v2.5. One scenario per physics category, with the layout of Figure 3 ; here the top row is the Cosmos v2.5 backbone.

Figure 17: Additional qualitative comparisons (set A). Layout as in Figure 3 .

Figure 18: Additional qualitative comparisons (set B). Layout as in Figure 3 .

Figure 19: WorldModelBench robotics examples (Cosmos 3 Nano 16B with Physical Registry). Each prompt is followed by eight frames, read left to right and top to bottom; times are in seconds.

Figure 20: WorldModelBench driving, industry, and nature examples (Cosmos 3 Nano 16B with Physical Registry). Layout as in Figure 19 .

Figure 21: Solid mechanics: object occlusion. Entity text: ball . Action text: lowered .

Figure 22: Fluid dynamics: filling a glass. Entity text: glass, red . Action text: filled .

Figure 23: Optics: a mug and its cast shadow. Entity text: mug . Action text: placed, rotating, rotates, casting .

Figure 24: Magnetism: a magnet–domino interaction. Entity text: domino, magnet . Action text: placed, rotating .

Figure 25: Solid mechanics: rolling-ball interaction. Entity text: ball, balls . Action text: rolls .

Figure 26: Fluid dynamics: pouring liquid over a duck. Entity text: liquid, duck . Action text: pours .

Video World Models

Video world models generate and predict action-conditioned visual futures to simulate changing scenes for embodied agents.

See all 257 papers

STRIKE: Learning Visual State Transitions for Physical World Modeling
Oct 7, 2026Wenbin Teng, Tianshuo Xu, Depu Meng +5World Models Video World Models
Applied Intuition · University of Southern California · University of California, Berkeley
PhyWorld: Physics-Faithful World Model for Video Generation
May 19, 2026Pu Zhao, Juyi Lin, Timothy Rupprecht +10Physics-Informed Generative Modeling Physical Consistency in Video Generation
Northeastern University · University of Georgia · Tulane University +1
Distilling Physical Priors into Streaming World Models
Aug 8, 2026Liangliang Zhao, Junying Wang, Danni Yang +5Physics-Informed Generative Modeling Physical Consistency in Video Generation
Fudan University, Shanghai, China · Shanghai AI Laboratory, Shanghai, China

Oct 7, 2026J/K move · Enter open · S save

STRIKE: Learning Visual State Transitions for Physical World Modeling

Wenbin Teng, Tianshuo Xu, Depu Meng, Yuelei Li+4

Applied Intuition · University of Southern California · University of California, Berkeley

World Models Video World Models Physical Consistency in Video Generation

Open paper AskPDF arXiv

Physical world modeling requires predicting how interactions change a scene, not merely generating coherent motion. We propose STRIKE, a framework that separates visual state transition learning from dense video generation. We construct event-aligned supervision by extracting observed states from training videos and pairing them with transition descriptions and temporal offsets. An image-based transition model learns to predict the next scene configuration from the current image, a local transition specification, and elapsed time. At inference, a pretrained vision-language planner predicts time transition specifications, and recursive application of the learned transition model produces a sequence of future visual states. A separately trained dynamic model then generates the complete rollout conditioned on these states and their temporal locations. Experiments on Physics-IQ Verified, PhyGenBench, Pisa-Experiments, and RoboTwin2.0 show improvements of STRIKE over the corresponding video-backbone baselines in benchmark measures of physical consistency and manipulation-video fidelity. These results support learned visual state transitions as an effective intermediate representation for physical world modeling.

Abstract

Video World Models

Explore similar work

STRIKE: Learning Visual State Transitions for Physical World Modeling

PhyWorld: Physics-Faithful World Model for Video Generation

Distilling Physical Priors into Streaming World Models