On the Divergence of Accuracy and Mechanism Consistency in Time Series World Models
Organizations: University of North Carolina at Chapel Hill · Foci Labs · Stevens Institute of Technology
Abstract
A time series world model (TSWM) predicts a controlled system's state from its observed history and planned actions and exogenous inputs. Current approaches build forecasters with actions as covariates, trained and evaluated on prediction error under the executed plan. Yet world models compare unexecuted plans, but their responses to changed plans remain untested. We ask which design choices matter and whether accurate forecasters respond to changed plans as real systems do. We address both with a formalization and benchmark. The formalization separates state, actions and exogenous inputs, distinguishes continuous, mode and event actions, and introduces mechanism consistency, a metric built on declared action-state relations with known directions, such as a vasopressor raising blood pressure: it checks whether shifting an action moves the forecast in the declared direction. The benchmark consolidates eight public datasets with real actions from engineered infrastructure and clinical care, varying prediction space, plan fusion and plan encoding across seven backbones and five seeds. First, a frozen latent prediction space lowers MAE by 9.9% over observation space and gated output fusion lowers it by 12.7% over input concatenation on average, with both improving all eight datasets; temporal plan encoding changes average MAE by at most 2.2%. Second, prediction error and mechanism consistency diverge: the lowest-error configuration is at or below chance in consistency on four of five datasets with declared mechanisms, and no design choice avoids this. Finally, directional supervision, a loss penalizing the wrong-signed part of the response to a shifted action, significantly raises consistency on penalized mechanisms with no change in MAE. Together they give TSWMs a recipe: a frozen latent space and output-side fusion for accuracy, and a training objective for mechanism consistency.
Figures & tables
| scale | series dim. | actions | |||||||
| Dataset | Domain | systems | windows | state | exog. | cont. | mode ( ) | event | |
| Greenhouse | horticulture | 5 min | 5/1 | 163 | 4 | 8 | 7 | – | – |
| PleiaData | building HVAC | 10 min | 50/12 | 7,548 | 2 | 8 | 1 | 2 (2,6) | – |
| PREDIST | district heating | 10 min | 46/12 | 6,109 | 5 | 2 | 2 | 3 (2,3,2) | – |
| Wastewater | water treatment | 2 min | 1 plant | 1,311 | 3 | 2 | 3 | 2 (2,2) | – |
| VitalDB | anesthesia | 2 s | 2,754/689 | 40,193 | 4 | – | 2 | – | – |
| Dataset | Action | State | Sign |
| Greenhouse | heating-pipe temp. | air temp. | |
| leeward vent | air temp. | ||
| windward vent | air temp. | ||
| CO 2 dosing | CO 2 level | ||
| VitalDB | propofol | BIS | |
| propofol | MAP |
| Option | Greenhouse | PleiaData | PREDIST | Wastewater | VitalDB | CGMacros | Shanghai | MIMIC-Cardio | Rank | |
| Space | Observation | .0821 ±.0269 | .0476 ±.0306 | .1866 ±.0099 | .1718 ±.0116 | .0377 ±.0090 | .0131 ±.0054 | .1330 ±.0132 | .0562 ±.0029 | 2.52 |
| AE | .0761 ±.0200 | .0337 ±.0252 | .1821 ±.0047 | .1695 ±.0123 | .0329 ±.0026 | .0114 ±.0049 | .1249 ±.0096 | .0552 ±.0021 | 1.93 | |
| VAE | .0956 ±.0134 | .0339 ±.0244 | .1836 ±.0054 | .1672 ±.0106 | .0336 ±.0039 | .0105 ±.0025 | .1206 ±.0048 | .0552 ±.0020 | 1.95 | |
| JEPA | .1041 ±.0271 | .0607 ±.0359 | .2143 ±.0327 | .1824 ±.0266 | .0403 ±.0112 | .0270 ±.0233 | .1418 ±.0208 | .0621 ±.0095 | 3.61 | |
| Fusion | None | .0751 ±.0364 | .0292 ±.0172 | .1858 ±.0219 | .1928 ±.0085 | .0396 ±.0187 | .0174 ±.0187 | .1257 ±.0155 | .0608 ±.0174 | 4.89 |
| Concat | .0761 ±.0200 | .0337 ±.0252 | .1821 ±.0047 | .1695 ±.0123 | .0329 ±.0026 | .0114 ±.0049 | .1249 ±.0096 | .0552 ±.0021 | 5.70 |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Adjusted forecast | Details |
| None | Plan discarded. | |
| Concat | , backbone input | The history plan enters with the latent history; the future plan enters through the native covariate interface of TimeXer and TiDE, and through otherwise. |
| Res | Standard initialization. | |
| Res-0 | Output layer of zero-initialized. | |
| FiLM-0 | , output layer zero-initialized. | |
| Gate | , the logistic function. |
| Encoder | Encoded plan | Details |
| Instant | Identity. | |
| Decay | with learned per channel; the gate is zero-initialized. | |
| Conv | , with | Six residual blocks with : depthwise kernels of eight taps with dilation , zero-initialized; receptive field steps. |
| Attn | with position embeddings ; causal mask; four heads, width ; zero-initialized. |
| Forecasting protocol (Experiments 1–3, the diagnostic of Appendix G and Section 5 , whose added term is in Appendix F ) | |
| Window | context , horizon , stride |
| Optimizer | Adam, learning rate , no weight decay, batch size |
| Schedule | halve the learning rate after epochs without held-out improvement |
| Stopping | at most epochs; stop after epochs without held-out improvement; the checkpoint with the lowest held-out loss is kept |
| Loss | MSE between the decoded forecast and the observed future states; gradients clipped at norm |
| Seeds | training seeds – (initialization, minibatch order, dropout); the split is fixed at seed with a held-out fraction of |
| Sweep | Method | Mean rank | MAE vs. ref. | Params. |
| Prediction space | Observation | 2.52 | – | – |
| AE | 1.93 | – | ||
| VAE | 1.95 | – | ||
| JEPA | 3.61 | – | ||
| Plan fusion | None | 4.89 | 0 | |
| Concat | 5.70 | – | – |
| Exp. | Contrast | MAE | CI (bootstrap over datasets) | Datasets |
| 1 | AE vs. observation | 8/8 | ||
| 1 | VAE vs. observation | 7/8 | ||
| 1 | JEPA vs. observation | 0/8 | ||
| 1 | VAE vs. AE | 3/8 | ||
| 2 | Gate vs. Concat | 8/8 | ||
| 2 | Gate vs. None | 8/8 |
| Option | Greenhouse | PleiaData | PREDIST | VitalDB | MIMIC-Cardio | Mean | |
| Space | Observation | .61 ±.13 | .49 ±.10 | .97 ±.03 | .51 ±.10 | .35 ±.14 | .59 |
| AE | .55 ±.17 | .51 ±.11 | .92 ±.15 | .55 ±.10 | .33 ±.17 | .57 | |
| VAE | .47 ±.26 | .53 ±.15 | .98 ±.03 | .47 ±.05 | .33 ±.08 | .55 | |
| JEPA | .55 ±.15 | .44 ±.16 | .88 ±.13 | .51 ±.11 | .33 ±.10 | .54 | |
| Fusion | Concat | .55 ±.17 | .51 ±.11 | .92 ±.15 | .55 ±.10 | .33 ±.17 | .57 |
| Res | .49 ±.11 | .46 ±.10 | .97 ±.02 | .35 ±.09 | .48 ±.09 | .55 |
| Dataset | Mechanism | Sign | Concat | Gate | FiLM-0 | seed range |
| Greenhouse | Tpipe Tair | .83 | .97 | .99 | .03 | |
| CO2dosing CO2air | .48 | .08 | .17 | .29 | ||
| VentLee Tair | .61 | .64 | .36 | .77 | ||
| VentWind Tair | .28 | .19 | .22 | .66 | ||
| PleiaData | setpoint_norm@mode=1 indoor_temp | .80 | .60 | .50 | .75 | |
| setpoint_norm@mode=2 indoor_temp | .21 | .37 | .42 | .57 |
| lowest cell | ||||||||
| Dataset | Action State | Sign | Pen. | Gate | DS | Gate | DS | DS, |
| Greenhouse | heating-pipe temp. air temp. | 0.973 | 1.000 | 0.974 | 1.000 | 1.000 | ||
| Greenhouse | leeward vent air temp. | 0.653 | 1.000 | 0.634 | 1.000 | 0.991 | ||
| Greenhouse | windward vent air temp. | 0.196 | 1.000 | 0.181 | 1.000 | 1.000 | ||
| Greenhouse | CO 2 dosing CO 2 level | 0.097 | 0.154 | 0.068 | 0.126 | 0.000 | ||
| PleiaData | setpoint (heating) indoor temp. | 0.606 | 1.000 | 0.601 | 1.000 | 1.000 | ||
| Dataset | Pen. | Gate | DS | MAE (median) | interval |
| Greenhouse | 3 | 0.0632 | 0.0639 | +0.03% | |
| PleiaData | 1 | 0.0205 | 0.0198 | 1.54% | |
| PREDIST | 2 | 0.1641 | 0.1641 | +0.35% | |
| VitalDB | 2 | 0.0313 | 0.0312 | 0.07% | |
| MIMIC-Cardio | 4 | 0.0543 | 0.0542 | 0.15% | |
| Wastewater | 0 | 0.1500 | 0.1498 | +0.00% |
| Dataset | |||
| MAE (fused, ) | |||
| Greenhouse | 0.0632 ±.0005 | 0.0651 ±.0004 | 0.0658 ±.0009 |
| VitalDB | 0.0313 ±.0002 | 0.0312 ±.0001 | 0.0312 ±.0001 |
| CGMacros | 0.0086 ±.0001 | 0.0095 ±.0001 | 0.0098 ±.0002 |
| Shanghai | 0.1165 ±.0010 | 0.1174 ±.0010 | 0.1177 ±.0007 |
| MIMIC-Cardio | 0.0543 ±.0001 | 0.0545 ±.0001 | 0.0545 ±.0001 |
| Backbone dropped | MAE | MAE bypass | ||
| none (full set) | ||||
| TimeXer | ||||
| TiDE | ||||
| DUET | ||||
| PatchTST | ||||
| Dataset | Backbone | Observation | AE | VAE | JEPA |
| Greenhouse | TimeXer | 0.0627 ±.0008 | 0.0604 ±.0006 | 0.1051 ±.0009 | 0.0882 ±.0206 |
| TiDE | 0.0635 ±.0026 | 0.0658 ±.0027 | 0.1094 ±.0007 | 0.1158 ±.0152 | |
| DUET | 0.0920 ±.0084 | 0.0709 ±.0016 | 0.0962 ±.0134 | 0.1453 ±.0261 | |
| PatchTST | 0.0986 ±.0334 | 0.0924 ±.0334 | 0.0921 ±.0025 | 0.1019 ±.0152 | |
| TimeKAN | 0.0620 ±.0007 | 0.0642 ±.0011 | 0.0770 ±.0022 | 0.0894 ±.0282 | |
| CrossLinear | 0.1147 ±.0327 | 0.1066 ±.0024 | 0.0996 ±.0142 | 0.0827 ±.0065 |
| Dataset | Backbone | None | Concat | Res | Res-0 | FiLM-0 | Gate | X-attn |
| Greenhouse | TimeXer | 0.0587 ±.0006 | 0.0604 ±.0006 | 0.0625 ±.0019 | 0.0621 ±.0013 | 0.0617 ±.0008 | 0.0608 ±.0012 | 0.0651 ±.0021 |
| TiDE | 0.0609 ±.0005 | 0.0658 ±.0027 | 0.0636 ±.0017 | 0.0635 ±.0020 | 0.0629 ±.0012 | 0.0615 ±.0006 | 0.0641 ±.0011 | |
| DUET | 0.0657 ±.0006 | 0.0709 ±.0016 | 0.0670 ±.0013 | 0.0678 ±.0013 | 0.0642 ±.0006 | 0.0649 ±.0017 | 0.0694 ±.0014 | |
| PatchTST | 0.0563 ±.0008 | 0.0924 ±.0334 | 0.0606 ±.0013 | 0.0612 ±.0013 | 0.0605 ±.0026 | 0.0590 ±.0014 | 0.0694 ±.0039 | |
| TimeKAN | 0.0587 ±.0004 | 0.0642 ±.0011 | 0.0656 ±.0020 | 0.0650 ±.0022 | 0.0645 ±.0014 | 0.0636 ±.0017 | 0.0659 ±.0026 | |
| CrossLinear | 0.1626 ±.0056 | 0.1066 ±.0024 | 0.0731 ±.0009 | 0.0732 ±.0016 | 0.0721 ±.0039 | 0.0703 ±.0019 | 0.1310 ±.0060 |
| Dataset | Backbone | Instant | Decay | Conv | Attn |
| Greenhouse | TimeXer | 0.0608 ±.0012 | 0.0610 ±.0017 | 0.0611 ±.0018 | 0.0608 ±.0010 |
| TiDE | 0.0615 ±.0006 | 0.0614 ±.0007 | 0.0616 ±.0009 | 0.0614 ±.0011 | |
| DUET | 0.0649 ±.0017 | 0.0657 ±.0011 | 0.0656 ±.0017 | 0.0646 ±.0009 | |
| PatchTST | 0.0590 ±.0014 | 0.0586 ±.0016 | 0.0596 ±.0009 | 0.0589 ±.0018 | |
| TimeKAN | 0.0636 ±.0017 | 0.0637 ±.0016 | 0.0635 ±.0018 | 0.0634 ±.0015 | |
| CrossLinear | 0.0703 ±.0019 | 0.0706 ±.0028 | 0.0705 ±.0013 | 0.0713 ±.0007 |
| Dataset | Backbone | Instant | Decay | Conv | Attn |
| Greenhouse | TimeXer | 0.0617 ±.0008 | 0.0613 ±.0007 | 0.0623 ±.0013 | 0.0618 ±.0011 |
| TiDE | 0.0629 ±.0012 | 0.0629 ±.0012 | 0.0631 ±.0013 | 0.0627 ±.0015 | |
| DUET | 0.0642 ±.0006 | 0.0642 ±.0014 | 0.0649 ±.0005 | 0.0655 ±.0014 | |
| PatchTST | 0.0605 ±.0026 | 0.0596 ±.0009 | 0.0609 ±.0024 | 0.0598 ±.0010 | |
| TimeKAN | 0.0645 ±.0014 | 0.0654 ±.0026 | 0.0648 ±.0015 | 0.0637 ±.0013 | |
| CrossLinear | 0.0721 ±.0039 | 0.0724 ±.0038 | 0.0722 ±.0033 | 0.0720 ±.0029 |