A desirable property of a world model is the ability to learn continually across tasks, adapting to new environments without forgetting what the agent has already learnt. In particular, the ability to retain and reuse knowledge obtained from prior experiences underpins an agent's ability to efficiently adapt to novel environments, as the dynamics of the physical world can often be described in recurring mechanisms. However, the world model's measure of adaptation entangles two abilities: the speed and capacity to learn unseen tasks, and the reuse of knowledge already acquired, since incoming tasks carry novel content alongside what recurs. In order to isolate knowledge reuse from prior experiences, we propose a compositional continual learning benchmark for world models in robot manipulation. Specifically, we design each task curriculum with compositional tasks that combine aspects of the tasks seen in the sequence. We further factorise this composition along the axes of action and perception to better understand how different input modalities bottleneck knowledge reuse. We evaluate state-of-the-art world models under canonical continual learning methods, alongside a modular world model whose dynamics backbone contains explicitly reusable components. Results show that modularity balances reuse against forgetting better than conventional methods, but none solve the problem fully, leaving clear room for continual world models built to reuse without forgetting. More details are available on our project website: https://object814.github.io/Compositional-Continual-Learning/.
Figures & tables
Fig. 1: Overview of our compositional continual learning benchmark for world models in robot manipulation. Top : Each task suite presents a sequence of primitive manipulation tasks, ending with a composition task that recombines them across action, perception, or full composition axes. Bottom : We evaluate the continual learning ability of world models to efficiently adapt to the composition tasks through knowledge reuse, without forgetting previously learnt primitives along the sequence.
Task Suite
Compositional
Continual Learning
Dense, Monolithic
Expert
Structure
Structure
Protocol
Reward
Demo/Policy
BEHAVIOR-1K
✗
✗
✗
✗
✓
CALVIN
✓
✓
✗
✗
✓
CausalWorld
✓
✗
✗
✓
✗
Colosseum
✗
✗
✗
✗
✓
CompoSuite
✗
✓
✗
✓
✓
TABLE I: Comparison of robot manipulation benchmark designs. Task Suite Structure groups related tasks into suites, Compositional Structure exposes the model with primitive tasks then test compositional generalisation, Continual Learning Protocol forces the models to learn in a curriculum and evaluate forgetting while adaptation happens, Dense, Monolithic Reward enables RL benchmarking, and Expert Demo/Policy provide expert demonstration for offline training. While many benchmarks look at different bits, ours evaluates them comprehensively.
Suite
Primitive Tasks → Composition task
Action composition
Reach
reach in xy (plane) → reach in xz → reach in yz → reach in xyz
Grasp
pick cube → place cube → pick and place cube
Perception composition
BinPnP
pick and place cube from red bin to blue bin → yellow to blue → red to purple → yellow to purple
PnPBlock
pick and place red block → pick and place blue block → stack the two blocks
TABLE II: The six composition task suites. primitives are trained in the listed order, final composition task is shown in bold
Fig. 2: World model separation for continual learning. The task-agnostic part captures how the world evolves and is learnt continually across the curriculum. The task-specific heads consume the dynamics latent and are re-initialised for each new task.
Fig. 3: Continual learning performance on all six task suites. Task suites are shaded by composition type. Lower BWT and higher FWT are better, black outlines mark the best method within a backbone. Left : the monolithic backbones DreamerV3 (blue) and TD-MPC2 (orange) under four continual learning methods. Right : modular PWM (teal) against monolithic TD-MPC2 (orange), with a scratch or a pretrained frozen encoder. Bars show the mean over three random seeds with minimum and maximum scores indicated.
Action
Perception
Full
Backbone
BWT ↓
FWT ↑
BWT ↓
FWT ↑
BWT ↓
FWT ↑
DreamerV3
5.94
21.56
11.79
24.52
30.12
15.54
TD-MPC2
32.15
16.76
57.81
−1.88
68.68
1.54
TABLE III: Performance by composition axis. BWT ↓ and FWT ↑ grouped by composition type, each averaged over all four continual learning methods and both task suites of that modality. Bold indicates the best performance of each backbone across the composition axis.
Fig. 4: Task composition structure and expert routing dynamics. Left : Task-composition structure for each curriculum. For a matrix with rows i and columns j , entry (i,j) indicates whether task i contains a reusable component from task j along the corresponding action, perception, or full-composition axis. Middle : Relative change in the average routing weight assigned to the three experts introduced by each task. Each entry is expressed as a ratio to that task’s weight immediately after learning the corresponding task itself; 1× denotes no change, values below 1× indicate down-weighting, and values above 1× indicate increased weighting. If PWM reuses experts according to the task-composition structure, the pattern in this panel should broadly correspond to that on the left . Right : Detailed routing weights over individual experts at each curriculum stage, normalised by the uniform allocation 1/Kactive , such that 1.0 denotes uniform weighting. Each row shows the expert weight allocation after training on a task, while each column tracks the routing weight of a specific expert over the curriculum. The boxed final row shows routing after learning the composition task.
Fig. 5: Expert ablation on the composition task. Closed-loop return and open-loop latent prediction fidelity on the composition tasks when activating only prior experts, only new composition task experts, or uniformly weighting all, relative to the full learnt mixture. Prior experts retain substantial performance, while new experts alone and uniform routing perform worse.
Continual learning treats degradation on previously seen data as evidence of failure, a convention inherited from settings with a stationary prediction target, where a correct label remains correct indefinitely. World models do not satisfy this condition. Their prediction target is the environment, which changes, so knowledge that was accurate when acquired may later become false, and discarding it is required behavior rather than a defect. Non-stationary ground truth is well studied in the concept drift literature and in the temporal factuality of language models, but has not been formulated for world models, which are distinctive in that they also encode knowledge that must never be revised. We argue that continual world models require retention stratified by invariance timescale, separating invariants such as physics and object permanence, which must never be revised, from instance-level facts that should be revised as soon as the environment changes. Standard forgetting metrics cannot distinguish a world model that has correctly revised outdated knowledge from one that has suffered catastrophic forgetting, and consequently rank a frozen model highest, while existing physical-reasoning benchmarks evaluate only frozen checkpoints. We propose differential retention, which reports invariant regression testing across the adaptation stream jointly with revision latency, without aggregation.
Learning and planning in imagination using world models provides an effective paradigm for training agents for decision-making. However, existing approaches often rely on high-dimensional latent spaces or generic visual embeddings that retain many factors irrelevant to control, limiting efficiency and generalization across tasks. To this end, we study how agents can learn world models with representations that are task-specific, minimal, and sufficient for decision-making. We achieve this via a closed-loop synergy between the agent and the world model, in which structured world-model learning distills task-sufficient representations from informative interaction data. On the agent side, agents actively probe the environment to collect informative trajectories that expose task-relevant latent factors, guided by an adaptive curriculum. On the world-model side, we learn structured representations over observations to distill compact, task-sufficient latent states from the collected interaction data. This synergy enables the empirical recovery of task-sufficient latent representations that capture all control-relevant factors. Leveraging these representations, the resulting policies achieve improved sample efficiency and generalization, including generalization across skills, object-skill compositions, and previously unseen tasks on standard continuous-control and robotic-manipulation benchmarks.
Foundation world models have made remarkable progress in planning, simulation, and embodied intelligence. However, optimizing a pretrained world model toward a user-specified objective remains difficult: each campaign typically rediscovers optimization strategies from scratch, and the resulting knowledge rarely transfers to the next model. Existing research agents automate the optimization loop but treat successful strategies as directly reusable recipes, without principled safeguards for when transfer is appropriate. We argue instead that retrieval is not transfer: a strategy validated on one model is at best an optimization hypothesis for another, and becomes transferable knowledge only after target-side experimental valida- tion. Guided by this principle, we propose VERDI , a continual framework for evidence-licensed world model optimization. VERDI characterizes each world model through shared inference-time probes to construct an Optimization Fin- gerprint, retrieves relevant prior experience as ranked hypotheses, and validates every candidate under a frozen target-side verifier before admitting it as reusable evidence; contradictions among nearby fingerprints further trigger probe evolution, continually refining the diagnostic representation itself. Experiments on Ctrl-World, the Cosmos family, and RoboCoin show that VERDI reduces search cost by 68%, GPU cost by 69%, and negative transfer from 0.34 to 0.06, while predicting transfer outcomes with 83% sign accuracy.