The Trinity of Consistency as a Defining Principle for General World Models

Jingxuan Wei; Siyuan Li; Yuhang Xu; Zheng Sun; Junjie Jiang; Hexuan Jin; Caijun Jia; Honghao He

Figure 1: The Trinity of Consistency in world models: Modal Consistency (Semantics), Spatial Consistency (Geometry), and Temporal Consistency (Causality).

Figure 2: Performance Comparison of Mainstream Models across Different Tasks. The score has been linearly rescaled from the original range of [0, 10] to a percentage scale of [0, 100].

Symbol	Description	Symbol	Description
$\mathcal{W}$	World Model	$\bm{p}$	3D Position
$\mathcal{S},\mathcal{A}$	State & Action Space	$\bm{P}_{t}$	Camera Pose at $t$
$\bm{s}_{t},\bm{a}_{t}$	State & Action Instance	$\bm{K}$	Intrinsic Matrix
$\pi$	Policy	$\Pi$	Projection Operator
$\tau$	Trajectory	$\mathcal{K}$	Keyframe Set
$\mathcal{T}$	Dynamics Function	$\mathcal{M}_{geo}$	Geometric Manifold

Table 1: Notation and Descriptions

Figure 3: Unified Representation Goal. Modal consistency aims to project heterogeneous inputs (Text, Image, Video, Audio) into a unified, physically-aligned latent space.

Figure 5: The Modal Gap Challenge. (Left) Ideal hypersphere alignment assumes uniform distribution. (Right) In reality, entropy disparity causes visual embeddings to collapse into a narrow ”cone,” leading to topological mismatch with discrete text tokens.

Paradigm	Objective (The Soul)	Error	Topology
Discrete AR	$\displaystyle\mathcal{L}_{AR}=-\mathbb{E}\left[\sum\log P(s_{t}\|s_{<t})\right]$	Exp.	Discrete
Flow Matching	$\displaystyle\mathcal{L}_{FM}=\mathbb{E}\left[\\|v_{\theta}(x_{t})-(x_{1}-x_{0})\\|^{2}\right]$	Linear	Euclidean

Table 2: Mechanism Comparison: Discrete AR vs. Continuous Flow Matching. The formulations highlight the trade-off between optimization objectives and error propagation dynamics.

Figure 6: Evolution of Multimodal Fusion Paradigms. Transitioning from geometric isolation (Dual-Tower) to unstable Early Fusion (Adapter), and finally to the orthogonally decoupled Native unified multimodal model (MM-DiT) in large-scale unified architectures.

Figure 7: Information Asymmetry in LLaVA. The linear projection layer

W_{proj}

acts as a low-rank compressor, prioritizing semantic alignment with the LLM while discarding high-frequency visual textures needed for controllable visual generation.

Figure 8: MM-DiT Architecture. By maintaining independent weight sets for both text and image modalities and interacting only via joint Attention, MM-DiT achieves orthogonal gradient updates, effectively resolving the modality conflict.

Figure 9: Spatial Consistency via Multi-View Constraints. The model ensures that the generated subject (Doge) maintains geometric coherence across Front, Side, and Top-down views, preventing structural distortion and the Janus problem.

Figure 11: Evolution of Spatial Consistency Paradigms. We trace the trajectory from early 2D Proxy Manifolds, to Implicit Continuous Fields like NeRF, moving towards Explicit Lagrangian Primitives like 3DGS, and finally integrating Generative Diffusion Priors.

Figure 12: Key Mechanisms for Advanced Spacetime Modeling. A taxonomy of core techniques underpinning modern models: (a) Full Spacetime Attention enables dense long-range dependencies; (b) Causal Masking ensures temporal causality; (c) 3D Gaussian Splatting offers explicit, differentiable 3D structure; (d) Object-Centric Slots decompose complex scenes into distinct entities.

Figure 14: Temporal Consistency and Identity Preservation. An illustration of the temporal attention mechanism ensuring identical subject features across consecutive frames (

t_{0}\rightarrow t_{n}

). The generation process is governed by two key constraints: Physical Constraints (

\mathcal{L}_{\text{phy}}

) enforce smoothness in motion trajectories to prevent flickering artifacts, while Causal Constraints (

\mathcal{L}_{\text{causal}}

) ensure logical progression of events ( e.g., object permanence) throughout the timeline.

Figure 15: Video Consistency Analysis: Spatial vs. Frequency View. Traditional distribution-based metrics such as FVD primarily assess spatial perceptual quality and smooth motion in feature space, often overlooking high-frequency temporal flickering. In contrast, VCD explicitly models temporal consistency by analyzing the Fourier spectrum of feature embeddings, enabling the detection of subtle high-frequency noise and flicker artifacts invisible to spatial statistics.

Generation Paradigm	Rep. Model	Temporal Consistency $\uparrow$	Physics Compliance $\uparrow$	Causal Reasoning $\uparrow$	Freq. Fidelity (VCD) $\downarrow$
		(VBench Norm.)	(Physics-IQ)	(Task Success Rate)	(Reward Penalty)
Temporal Inflation	AnimateDiff	$0.68$	$0.42$	N/A ( $<10\%$ )	High ( $>1.2$ )
Discrete AR	VideoPoet	$0.79$	$0.55$	Low ( $\sim 25\%$ )	Medium ( $\sim 0.9$ )
Native DiT	HunyuanVideo	$0.88$	$0.78$	Medium ( $\sim 45\%$ )	Low ( $\sim 0.6$ )
World Model Priors	Google Veo 3	0.95 ∗	0.86 ∗	High ( $>70\%$ ) †	Minimal ( $<0.3$ )

Table 3: Empirical Evolution of Cross-Generational Models. Data synthesized from VBench (Temporal), Physics-IQ (Physics), and Veo 3 Technical Report (Reasoning) benchmarks.

Figure 16: Evolution of Video Generation Paradigms. The technical path advances from Temporal Inflation (prone to drift) and Discrete AR (quantization loss) to the current Native DiT. This paradigm achieves full spatiotemporal isomorphism, serving as the foundation for world models.

Figure 17: Tool-use & Closed-loop Verification. Diagram illustrating the ReAct paradigm, where an AI agent cyclically calls external detection and correction tools to refine generation outputs.

Figure 18: Modal + Spatial Consistency: Language Controls Precise Spatial Relations. The model combines modality and spatial consistency, using language instructions to precisely control the spatial relations between the subject (Doge) and objects ( e.g., Sit ON the box, Hide BEHIND the cushion).

Figure 20: Instruction-Driven Image Editing. The diagram illustrating structure-preserving image editing, where a Doge is repositioned via text instruction. It depicts the mechanisms of gradient decoupling (frozen base) and attention injection (geometric gating) to maintain spatial consistency.

Figure 21: Pose-Aligned View Synthesis. Diagram illustrating pose-conditioned generation using a decoupled backbone and epipolar attention. The target view features a split display of RGB texture and a purple-blue Normal Map, representing cross-domain mutual supervision for geometric accuracy.

Figure 22: Multimodal Alignment. The diagram illustrating the convergence of Text, Image, and 3D Point Cloud into a central Unified Embedding. It depicts contrastive alignment for inputs and generative tokenization for 3D data integration.

Figure 23: Modal + Temporal Consistency: Language Controls Time Evolution. The model integrates modality and temporal consistency, using language instructions to control the time evolution process ( e.g., the cherry blossom tree behind the dog blooms and scatters across T1-Winter, T2-Spring, T3-Late Spring), ensuring coherent changes over time.

Figure 25: Autoregressive-Diffusion Model. It depicts an AR Planner constructing a causal temporal skeleton (blue wireframes), which then passes through a Diffusion Refiner (warm mist) for detail injection, resulting in high-quality long-duration video output.

Figure 26: Multi-Condition Decoupling Architecture. The diagram illustrating the Two-Stream Architecture for digital human animation. It depicts the orthogonal decoupling of Reference Appearance (static portrait) and Motion Skeleton (stick figure), which are fused via a Spatial Attention ”Zipper” to generate a spatiotemporally consistent video loop.

Figure 27: Unified Comprehension and Generation Symbiosis Architecture. It features a central LLM converting multimodal inputs into a unified cloud of discrete tokens, enabling seamless ”Any-to-Any” transformation ( e.g., video to text and vice versa).

Figure 28: Temporal + Spatial Consistency: Dynamic Object Permanence across Occlusion. The model combines temporal and spatial consistency to realize dynamic object permanence during occlusion: the subject (Doge) retains consistent features ( e.g., sunglasses, bone) via latent memory when occluded, and re-emerges with unchanged attributes, ensuring the continuity of the object.

Figure 30: Geometric Embedding Injection. It features a Doge on a perspective grid with a trajectory, converting physical rays into geometric tokens to enforce epipolar constraints across frames.

Figure 31: Trajectory-Centric Foundation Models. This illustrates the transformation of a 2D video input into dense trajectory fields (with occlusion handling) and their subsequent lifting into a 4D canonical space for volumetric reconstruction.

Figure 32: The Trinity of Consistency. It depicts a Robot Doge performing embodied tasks (stacking blocks) guided by counterfactual reasoning (visualized as a prediction bubble).

Task	Sub-tasks
Task	I. Basic / Atomic	II. Structured / Dynamic	III. Complex / Constraint
Single-Consistency Dimensions
M	Style/Material transfer	Fine-grained control	Multi-constraint composition
S	Planar layout	Hierarchical occlusion	Multi-view 3D structure
T	Worldline persistence	Rule-guided evolution	Ordered stage transitions
Cross-Consistency Synergies

Table 4: Taxonomy of Tasks in CoW-Bench. The benchmark covers three foundational dimensions: M (Modal Consistency), S (Spatial Consistency), and T (Temporal Consistency). Crucially, it probes their deep synergies required for world simulation: M

\times

S , M

\times

T , and S

\times

T . Each family is further decomposed into three specific sub-tasks to isolate distinct failure modes.

Figure 34: Hierarchical Taxonomy of CoW-Bench. The inner ring represents the main consistency dimensions (Modal, Space, Time), while the outer ring details the 18 fine-grained sub-tasks. The uniform sector sizes visually confirm the rigorously balanced distribution of the dataset.

Mode	Task	Sub-Task	$N$	Scene	Diff	Complexity Metrics (Avg.)
Mode	Task	Sub-Task	$N$	Scene	Diff	Prmpt	ImgR	Act	Elem
Single	Modal	Subj-Attr	91	Obj	Easy	7.1	13.3	22.0	2.1
		Minor-Ctrl	87	Obj	Med	37.4	37.4	37.1	1.6
		Multi-Req	81	Mix	Hard	74.8	13.6	64.7	1.6
	Space	2D-Layed	89	Mix	Med	38.2	24.7	32.8	2.4
		2D-Rel	91	Obj	Easy	42.7	17.1	52.8	1.7

Table 5: Comprehensive Statistics of CoW-Bench.

ID	Atomic check	Operational definition
A1	Identity lock	The intended target entity remains unchanged; no identity swap, duplication, or replacement occurs.
A2	Attribute binding	Key attributes remain bound to the same entity; no attribute migration occurs.
A3	Constraint non-relaxation	Specified constraints are not weakened or substituted with more common but non-equivalent variants.
A4	Evidence clarity	Evidence supporting each constraint judgment is clear and unambiguous.
A5	Mutual exclusivity	Mutually incompatible properties do not co-occur on the same target.
A6	Locality of change	Changes are confined to the designated region or attribute without boundary spillover.

Table 7: Atomic library of CoW-Bench. Each atomic check defines a reusable, observable evaluation criterion with a concise operational one-sentence definition for systematic consistency assessment.

Metric family	A1	A2	A3	A4	A5	A6	A7	A8	A9	A10	A11	A12	A13	A14
Attribute Fidelity (M1)	✓	✓	✓	✓	✓
Local Edit Precision (M2)				✓		✓	✓	✓
Multi-constraint Satisfaction (M3)		✓						✓	✓	✓	✓
Worldline Persistence (T1)	✓	✓		✓			✓					✓	✓
Evolutionary Dynamics (T2)	✓			✓			✓					✓	✓
Ordered Stage Transitions (T3)				✓								✓	✓	✓

Table 8: Compositional definition of metric families via the atomic library. A checkmark indicates that the metric family invokes the corresponding atomic check.

Model	Modal			Temporal			Spatial			Modal-Temporal			Modal-Spatial			Temporal-Spatial			AVG
Model	SUAT	LCED	MCON	WLIN	SLEV	STOR	SEPL	OCCO	MV3D	TREV	LOHO	ATDY	SEPL	OCCO	SEMV	3DLO	OCMO	MAZE	AVG
	Closed-Source Video-Generation Models
Sora [ 27 ]	5.16	8.35	8.38	9.32	5.80	6.22	8.41	6.19	8.51	6.96	8.12	5.25	8.64	5.97	9.49	8.40	9.25	4.17	73.66
Kling [ 314 ]	4.11	8.19	5.63	9.10	5.17	5.53	8.72	7.88	9.32	7.08	8.71	6.58	8.20	6.79	9.44	8.08	9.30	5.30	73.96
	Closed-Source Image-Generation Models
GPT-image-1 [ 247 ]	7.37	8.96	7.96	9.14	7.75	5.68	8.09	7.43	9.26	6.95	9.28	7.22	9.00	6.83	9.79	8.46	8.22	7.24	80.35

Table 9: Main results on CoW-Bench over 18 sub-tasks (higher is better); MEAN averages all sub-tasks. Abbrev: SUAT=Subj-Attr, LCED=Local-Edit, MCON=Multi-Const; WLIN=Worldline, SLEV=Slow-Evol, STOR=Stage-Order; SEPL=Sem-Planar, OCCO=Occ/Contain; MV3D=MV-3D; TREV=Trigger-Event; LOHO=Long-Horizon, ATDY=Attr-Dyn; SEMV=Sem-MV; 3DLO=3D-Loop; OCMO=Occ-Motion; MAZE=Maze-2D. AVG is rescaled from the original [0, 10] to a percentage scale of [0, 100].

Figure 35: Modal–Space consistency on CoW-Bench, shown as a heatmap over sub-metrics (rows) and models (columns), with scores on the 0–2 scale (higher is better). Rows group into Sem-Planar ( Ent-match , Act-align , NT-stab , Attr-bind , Global ), Sem-Hier ( Pos-rel , Neg-rel , Excl. , Vis+Layer , Id-stab ), and Sem-MV ( Anchor , View-stab , Lateral , Scene , Marker ).

Figure 36: Modal–Time consistency on CoW-Bench, shown as a heatmap over sub-metrics (rows) and models (columns), with scores on the 0–2 scale (higher is better). Rows cover Long-Horizon anchoring ( Init-anchor , Long-stab , Cross-scene , Attr-bind , No-unexp ), Attr-Dyn alignment ( Target(E,A) , Follow , Smooth , Rate , Env-stab ), and Trigger-Event compliance ( Pre-hold , Trigger , Post-comp , State-stab , Env-stab ).

Figure 37: Time-Space consistency on CoW-Bench, shown as a heatmap over sub-metrics (rows) and models (columns), with scores on the 0–2 scale (higher is better). Rows cover three metric families: ST1 Maze-2D ( Start/Goal , Traj-cont , Legal , Correct , Struct-stab ), ST2 Occlusion Dynamics under Motion ( Occ-move , Parallax , Rigid , Natural , Env-stab ), and ST3 3D Loop Navigation ( Struct , Rel , View-smooth , Physical , Entity-stab ).

Figure 38: Example diagrams of Modal sub-tasks.

Figure 39: Example diagrams of Spatial sub-tasks.

Figure 40: Example diagrams of Temporal sub-tasks.

Figure 41: Example diagrams of modal and spatial sub-tasks.

Figure 42: Example diagrams of modal and temporal sub-tasks.

Figure 43: Example diagrams of spatial and temporal sub-tasks.

Figure 44: Comparison with different models for modal consistency task.

Figure 45: Comparison with different models for spatial consistency task.

Figure 46: Comparison with different models for temporal consistency task.

Figure 47: Comparison with different models for modal and spatial consistency task.

Figure 48: Comparison with different models for modal and temporal consistency task.

Figure 49: Comparison with different models for spatial and temporal consistency task.

Figure 50: Evolutionary spectrum of World Model paradigms based on interactive action spaces. This figure illustrates the field’s trajectory from the early “Vector-as-Action” paradigm (left: e.g., JEPA, relying on uninterpretable latent space predictions), through the intermediate “Key-as-Action” paradigm (center: e.g., Genie series, constrained by predefined discrete control spaces), and ultimately advancing towards the “Prompt-as-Action” paradigm. In the latter, a semantic compiler translates natural language intents into universal spatiotemporal dynamic simulations.

The Trinity of Consistency as a Defining Principle for General World Models

Abstract

Figures & tables

World Models

World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications

Current World Models Lack a Persistent State Core

A Definition and Roadmap for World Models

Abstract

World Models

Explore similar work

World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications

Current World Models Lack a Persistent State Core

A Definition and Roadmap for World Models