From Forecasting to World Modeling: Can Temporal Foundation Models Recover Causal Response?
Organizations: Seoul National University Seoul, South Korea · Microsoft San Francisco, USA · Cox School of Business Southern Methodist University Dallas, Texas, USA
Abstract
A model that forecasts sales accurately can still mislead a retailer about how sales respond to a price change. We evaluate released tabular and time-series foundation models, as well as a foundation model pretrained specifically to estimate causal effects. We report three findings. First, forecast accuracy does not indicate the accuracy of the predicted effect of price, both on simulated data in which the true effect is known and on real retail sales records. Second, for the tabular foundation models, additional context improves forecasts but not the predicted effect of price. Third, instrumental variables, the standard remedy in econometrics, reduce this error only when supplied in a specific form and increase it for every model under confounding when supplied as raw instruments.
Figures & tables
| Category | Snack crackers | Breakfast cereals | ||
|---|---|---|---|---|
| Metric | Forecast WMAPE | Own-sign accuracy | Forecast WMAPE | Own-sign accuracy |
| Chronos-2 | 0.368 [0.357, 0.381] | 0.834 [0.806, 0.865] | 0.499 [0.472, 0.531] | 0.942 [0.931, 0.952] |
| TabPFN | 0.383 [0.370, 0.396] | 0.844 [0.825, 0.863] | 0.380 [0.371, 0.389] | 0.788 [0.774, 0.801] |
| TabICLv2 | 0.437 [0.425, 0.450] | 0.852 [0.823, 0.878] | 0.413 [0.403, 0.423] | 0.892 [0.877, 0.907] |
| TabFM | 0.484 [0.468, 0.501] | 0.844 [0.824, 0.864] | 0.494 [0.477, 0.512] | 0.869 [0.859, 0.878] |
| LightGBM | 0.389 [0.380, 0.398] | 0.915 [0.894, 0.935] | 0.367 [0.359, 0.375] | 0.838 [0.820, 0.853] |
| Forecast WMAPE | Counterfactual WMAPE | ||||||
| no instruments | residuals | raw instrument | no instruments | residuals | raw instrument | ||
| Panel A. Confounded | |||||||
| LightGBM | 0.335 0.012 | 0.301 0.018 | 0.306 0.020 | 0.092 0.009 | 0.064 0.005 | 0.153 0.024 | |
| TabICLv2 | 0.409 0.035 | 0.381 0.013 | 0.354 0.014 | 0.077 0.010 | 0.052 0.009 | 0.179 0.023 | |
| TabPFN | 0.353 0.018 | 0.325 0.016 | 0.329 0.019 | 0.094 0.009 | 0.052 0.009 | 0.193 0.014 | |
| TabFM | 0.411 0.040 | 0.365 0.031 | 0.363 0.043 | 0.064 0.011 | 0.059 0.007 | 0.123 0.043 | |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| model | implementation | settings |
|---|---|---|
| Tree model | ||
| LightGBM | lightgbm 4.7.0 | 300 trees; learning rate 0.05; 63 leaves; tuned on the real data (Appendix C.2.1 ) |
| Tabular foundation models | ||
| TabPFN | tabpfn 8.5.0, package-default weights | package defaults (8 estimators); fitted with cache |
| TabICLv2 | tabicl 2.2.0, checkpoint tabicl-regressor-v2-20260212 | package defaults (8 estimators); key-value cache |
| TabFM | tabfm 1.0.1, weights google/tabfm-1.0.0 | 1 estimator; bfloat16 precision |
Causal Effect Estimation
Causal effect estimation measures how outcomes would change under interventions, using experiments or observational data and explicit assumptions.
See all 285 papers