cs.LGSep 11, 2026

Measuring the Value of World-Model Updates: A Counterfactual Utility Protocol for Continual Adaptation

Authors: Anqi Peter LiKaden Kim

Abstract

Continual world models must decide whether new data justify changing the model. Fixed replay schedules and prediction-error triggers specify when to update, but neither reveals the value of an individual update: one deployment run cannot show how the same model would have performed at that moment had it held its parameters. We introduce the fork ledger, which branches a deployment stream at pre-registered decision points into matched update and hold continuations under common random numbers. It evaluates both continuations on the same episodes and records ΔR=RupdateRhold\Delta R = R_{\mathrm{update}} - R_{\mathrm{hold}}. Always applying one fixed update mechanism lowers return on all three simulated control tasks: CartPole (144.0-144.0; checkpoint-bootstrap 95%95\% CI [185.4,116.1][-185.4,-116.1], against a converged return near 650650), Walker (82.8-82.8; [101.1,61.7][-101.1,-61.7]) and Cheetah (18.6-18.6; [29.0,6.6][-29.0,-6.6]). Divergence is an outcome of applying the update, so the estimand counts every attempted fork; restricted to the 693693 of 720720 that did not collapse, CartPole and Walker are unchanged in sign (113.4-113.4 and 82.1-82.1) and Cheetah becomes unresolved (3.9-3.9; [17.5,+13.0][-17.5,+13.0]). The task is the unit of inference: each contributes 240240 attempted forks over five pretrained checkpoints crossed with two drift directions. The ledger makes counterfactual utility observable for a fixed mechanism, allowing triggers to be judged by the updates they select rather than by surprise detection alone.

Explore similar work

CardsList