Long-horizon tasks in embodied and web environments involve extended decision sequences and delayed feedback, making accurate planning crucial for success. World models tackle this by learning latent dynamics for imagination-based planning. However, existing approaches merely rely on dense pixel-level observations, which forces the model to absorb redundant information while overlooking relations among task-relevant entities. Consequently, small errors may accumulate over long imagined rollouts and gradually corrupt the predictions. In this paper, we explore world models that imagine over sparse state abstractions, which poses two key challenges: 1) how to remove redundant information and form a compact state abstraction; 2) how to incorporate the relational structure to learn reliable long-horizon dynamics. To address these challenges, we propose Embodied Graph, a relational state abstraction that supports structured dynamics modeling and long-horizon imagination, and the Relational World Model (RWM), which performs imagination directly over graph states. RWM consists of (i) a Task-Aware Embodied Graph Constructor that filters redundant details and maintains graph memory through online updates, and (ii) an Embodied Graph Recurrent State Space Model that learns node-wise latent dynamics while capturing inter-node dependencies. The imagined graph states provide predictions of action consequences to guide decision-making. Experiments on Minecraft and Crafter demonstrate the effectiveness of RWM in embodied environments. Notably, in Minecraft, RWM is the only method that obtains diamonds within the given budget. Experiments on web interaction further illustrate the applicability of relational state prediction to the decision making process of web agents.
Figures & tables
Figure 1. Imagination drift in a pixel-level world model. A minor early inaccuracy (a smoothed sandbar) can accumulate into a severe deviation (a hallucinated oasis) that would lead the agent toward an implausible state.
Figure 2. Overall framework of RWM. A Task-Aware Embodied Graph Constructor builds and continuously updates a compact embodied graph, while an Embodied Graph Recurrent State Space Model learns action-conditioned, node-wise dynamics through relational propagation. The graph imagination model predicts future node attributes and structural outcomes, providing forecasts of action consequences to guide decision-making in dynamic environments.
Steps
Model
Stone
Iron
Gold
Diamond
SR
SL
SR
SL
SR
SL
SR
SL
1M
PPO
92.40 ± 3.99
358 ± 8
28.76 ± 4.68
2628 ± 540
0.00 ± 0.00
–
0.00 ± 0.00
–
Rainbow
95.05 ± 0.70
663 ± 324
29.80 ± 3.77
2320 ± 635
0.00 ± 0.00
–
0.00 ± 0.00
–
DreamerV3
97.03 ± 0.80
264 ± 18
30.82 ± 4.66
1875 ± 255
2.58 ± 2.21
1190 ± 92
0.00 ± 0.00
–
LS-Imagine
96.41 ± 1.78
279 ± 78
25.11 ± 4.64
2103 ± 184
1.89 ± 1.37
5522 ± 721
0.00 ± 0.00
–
RWM (Ours)
95.57 ± 2.76
420 ± 138
32.89 ± 4.62
2115 ± 113
4.19 ± 0.53
3054 ± 1351
1.47 ± 1.11
8198 ± 708
Table 1. Minecraft results after 1M–5M environment steps: success rate (SR ↑ , %) and success length (SL ↓ , steps, “– if none) are reported mean ± std over 3 seeds. Best in bold, second best underlined. Ranks are computed among methods with non-zero SR.
Model
Metric
1M
2M
3M
4M
5M
RWM
SR
1.47 ± 1.11
1.55 ± 1.12
2.97 ± 1.80
5.72 ± 1.78
6.94 ± 2.12
SL
8198 ± 708
5401 ± 297
5306 ± 236
4740 ± 295
4512 ± 1014
Variant 1
SR
0.00 ± 0.00
0.00 ± 0.00
0.00 ± 0.00
0.00 ± 0.00
0.00 ± 0.00
SL
–
–
–
–
–
Variant 2
SR
0.00 ± 0.00
1.33 ± 0.62
1.79 ± 0.71
1.66 ± 0.69
2.48 ± 0.95
SL
–
6750 ± 294
5730 ± 201
5971 ± 337
3945 ± 883
Table 2. Ablation study in the Minecraft diamond task. Variant 1: w/o admission scorer. Variant 2: w/o relational propagation. Success rate (SR, %) and success length (SL, steps) are reported by mean ± std over 3 seeds.
Model
PPO
Rainbow
DreamerV3
RWM
Score (%)
4.6 ± 0.3
4.3 ± 0.2
14.5 ± 1.6
15.8 ± 2.0
Reward
4.2 ± 1.2
6.0 ± 1.3
11.7 ± 1.9
12.2 ± 1.7
Table 3. Experimental results in Crafter after 1M environment steps (mean ± std).
Figure 3. Open-loop imagination drift under different imagination lengths. Compared with DreamerV3, RWM maintains a nearly stable prediction error over long rollouts, showing that our proposed method effectively reduces compounding imagination drift.
Figure 4. Case study on CMS35. Top: Case 1, typing a phone number into the search box. Bottom: Case 2, selecting the report period. Each case is enlarged around the changed element and shows (a) the current screen, (b) the actual next screen after the verifier’s choice, (c) the image RSSM’s prediction for the verifier’s choice, and (d) its prediction for WebRL’s proposal. RWM forecasts the new values exactly ( 8015551212 in the search box and month in the Period drop-down), whereas the image RSSM’s predictions are blurred and nearly identical for the two candidates.
Method
CMS SR
GPT-4-Turbo
14.3
GPT-4o
20.0
AutoWebGLM
28.6
Llama3.1-8B + DigiRL
37.1
GLM-4-9B + WebRL
48.6
WebRL-Llama-3.1-8B
54.3
Table 4. Success rate (%) on WebArena-Lite CMS35. Upper block: published results ( Qi et al., 2025 ) . Lower block: our runs, in which WebRL-Llama-3.1-8B proposes the candidates and the same verifier selects one. The variants differ only in the forecasts given to the verifier.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5. Perception front-end in Minecraft. Each row corresponds to a 64×64 observation (left), the segmentation (middle), and the resulting label map (right).
Hyperparameter
Value
Hyperparameter
Value
Macro-action decision interval ( H )
50
Decoder layers
3
Max nodes in the embodied graph ( N )
128
Policy layers
2
Max observation candidates ( M )
256
Value layers
3
node matching threshold ( δ )
2.0
Imagination length ( K )
15
KNN neighbors ( Knn )
4
Loss scale (dynamics) ( λdyn )
1.0
Replay size
5×105
Loss scale (representation) ( λrep )
0.1
Appendix
Table 5. Key hyperparameters for RWM. Unless otherwise specified, shared hyperparameters follow the default DreamerV3 configuration ( Hafner and et al., 2025 ) .
Figure 6. Agent ability spectrum for Crafter (ordered left to right by number of prerequisites for an achievement). The y-axis is shown in log scale. RWM achieves comparable performance on common early-stage achievements and stronger performance on harder long-horizon achievements involving resource collection and tool crafting.
World models aim to learn representations of real-world environments and predict their future evolution. Recent object-centric world models have made expressive progress by representing visual scenes as sets of object-level latent states, but object-object relations are often captured only implicitly, which limits explicit relational and temporal structure modeling and object-centric dynamic memory modeling. To address such challenges, we propose World-As-Graph (WAG), a graph-based object-centric world model that introduces relational inductive bias into JEPA-style predictive representation learning. The proposed WAG contains two main modules: (1) Relation-aware structure induction, which constructs time-varying latent graphs from object-centric slots and designs relation-aware object masking policies to guide relational object representation learning in latent space; (2) Object-centric memory transition, which maintains and updates object-level dynamic states by combining relational information from neighboring objects with historical memory, enabling effective autoregressive future prediction. Extensive experiments on both visual reasoning and robotic manipulation tasks could demonstrate the superior performance of our proposed WAG.
Yaqi Yang, Shuo Huang, Yujin Huang +3
School of Computing Technologies, RMIT University, Australia · Faculty of Information Technology, Monash University, Australia · KStelrix Star Dynamics Lab, KStelrix, China +1
World models trained via latent dynamics offer a scalable path for embodied planning, yet existing methods suffer from representational collapse, computational overhead, and limited long-horizon capability. We introduce Memory World Model (MemWM), an associative memory-enhanced latent world model that augments JEPA-style prediction with a content-addressable memory bank. MemWM learns high-level historical representations directly from observations and leverages them to condition predictions via adaptive layer normalization, complementing low-level action-conditioned dynamics within a shared latent space. To prevent memory slots from collapsing into identical representations, we propose two complementary losses that encourage structural diversity across slots and balanced utilization across timesteps. On PushT, TwoRoom, Reacher, and OGBCube, MemWM attains an average success rate of 89.9%, matching DINO-WM (89.3%) while using roughly 4× fewer trainable parameters and no pretrained encoder, and improving over LeWM by 4.1%. Additionally, we extend the horizon to 50 timesteps, double the 25 timesteps of the original evaluation, demonstrating MemWM's robustness to long-horizon generalization.
Yuwei Sun, Siyu Zhu
Shanghai Academy of AI for Science · Fudan University
Recently, world models have emerged as a promising paradigm for building intelligent agents by learning predictive models that estimate future environment states conditioned on observations and actions. In particular, JEPA-style latent world models provide an efficient alternative to pixel space prediction by learning action-conditioned dynamics in compact representation spaces. However, existing latent world models typically rely on one-step prediction and must be recursively rolled out for long-horizon planning, which leads to compounding errors and a mismatch between training objectives and downstream planning tasks. To address this limitation, we propose Variable-length Latent World Models (VLWMs), a framework that learns to predict future latent states conditioned on action sequences of variable lengths. Instead of training only on one-step transitions, VLWMs directly model temporally extended dynamics, allowing the same predictor to evaluate action plans over different horizons. We further introduce a curriculum training strategy that progressively expands the action horizon, stabilizing optimization from short-range dynamics to long-range prediction. At test time, we design planning methods tailored to VLWMs to better exploit their variable-length predictive capabilities. Experiments on long-horizon control tasks show that VLWMs significantly improve latent space world models, achieving 13% average improvement over the state-of-the-art LeWM across different datasets, with especially large gains on tasks requiring extended planning. These results suggest that VLWM provides a simple yet effective paradigm for improving long-horizon prediction and planning in latent world models.
Tianqi Du, Qi Zhang, Yifei Wang +1
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Amazon AGI SF Lab · Institute for Artificial Intelligence, Peking University