Model-based reinforcement learning (MBRL) agents operating in high-dimensional observation spaces, such as Dreamer, rely on learning abstract representations for effective planning and control. Existing approaches typically employ reconstruction-based objectives in the observation space, which can render representations sensitive to task-irrelevant details. Recent alternatives trade reconstruction for auxiliary action prediction heads or view augmentation strategies, but perform worse in the Crafter environment than reconstruction-based methods. We close this gap between Dreamer and reconstruction-free models by introducing a JEPA-style predictor defined on continuous, deterministic representations. Our method matches Dreamer's performance on Crafter, demonstrating effective world model learning on this benchmark without reconstruction objectives.
Figures & tables
Method
Reconstruction-free
Non-contrastive
No action prediction
No view augmentation
Dreamer
∘
∘
∙
∙
DreamerPro
∙
∙
∙
∘
MuDreamer
∙
∘
∘
∙
Dreamer-CDP
∙
∙
∙
∙
Table 1: Overview of Dreamer variants. Dreamer relies on pixel-based reconstruction. Other methods are reconstruction-free: MuDreamer uses action prediction to train the sequence model. DreamerPro utilizes augmented views. Dreamer-CDP (this article) relies solely on internal prediction.
Figure 1: a) Schematic of Dreamer-CDP. The hidden state is passed through a predictor (green) trained to approximate the next continuous representation u^t≈ut . In Dreamer, the hidden state and the input embedding are used to predict the next input xt (dashed gray). b) Graphical model of Dreamer (left) and Dreamer-CDP (right) with losses in red. c) Visual examples when Lrecon (Dreamer), LCDP (Dreamer-CDP) or neither is applied. For the latter two, the decoder was trained independently with detached gradients for visualization purposes.
Metrics
Dreamer
DreamerPro
MuDreamer
Dreamer-CDP (ours)
Crafter score
14.5±1.6%†
4.7±0.5%†
7.3±2.6%
16.2±2.1%
Cum. reward
11.7±1.9†
—
5.6±1.6
9.8±0.4
Table 2: Crafter score and cum. reward ± std ( n=7 ) of different Dreamer variants (cf. Sec. 5 ). † published results by Hafner et al. (2023) and Kauvar et al. (2023) .
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1: Achievement success rate for DreamerV3 (Blue) and Dreamer-CDP (purple) sorted by DreamerV3 success rates.
Figure A.2: Comparison between Dreamer-CDP (Purple) and several ablations. Orange: Ablation of the reward and value gradient to train the world model. Most of the learning signal comes from the latent space predictive loss. Green: Ablation of LCDP and Ldyn/rep . The KL balancing is not sufficient to train the world model in the latent space. Black: CDP loss alone also results in lower cumulative return.