cs.LGNov 8, 2026

Data-informed Adaptation of Foundation Models for Causal Effect Estimation

Authors: Chunwei Ma, Russell D Wolfinger

Organizations: JMP Statistical Discovery Cary, NC, USA

Abstract

Due to the fundamental problem of causal inference -- the impossibility of observing counterfactual outcomes -- there is always a gap between the training goal (outcome, propensity) and the inference target (CATE, ATE). Traditional causal effect estimation revolves around setting assumptions, selecting identification strategies, and choosing causal estimators, while more recently, Causal Foundation Models (CFMs), built on Prior-Data-Fitted Networks (PFNs), ease this process by pretraining on massive synthetic prior datasets. Despite their success, such amortization makes causal inference less transparent, obscuring which assumptions the model implicitly relies on for a given dataset. The high diversity of real-world data-generating processes (DGPs) and Structural Causal Models (SCMs) makes it hard to know, a priori, which causal inference strategy is compatible with a given dataset. In this work, we focus on this compatibility between an observational dataset and method-specific assumptions, and find that geometric characteristics of data embeddings -- specifically, the variogram -- provide a useful signal of whether a given causal estimation strategy is appropriate. We develop MoE-Learner, a mixture-of-causal-experts framework that embeds a dataset under multiple assumption-specific views and uses the variogram as a counterfactual-free router to assess each expert's validity. Rather than eliminating causal assumptions, this shifts their validation from a fixed, a priori choice to a criterion that can be checked directly from data geometry. MoE-Learner achieves the best average rank on benchmark datasets (IHDP, ACIC, Lalonde). Being orthogonal to (causal) foundation models, our method builds a bridge from data characteristics to causal estimation strategy rather than proposing another CFM.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Causal Foundation Models

    Sep 2, 2026Christopher Stith, Hossein Rahmani, Jesse C. CresswellCausal Foundation ModelsCausal Effect Estimation

  2. Use What You Know: Causal Foundation Models with Partial Graphs

    Feb 16, 2026Arik Reuter, Anish Dhir, Cristiana Diaconu +6Causal Foundation ModelsCausal Discovery

  3. What You Observe Determines How You Identify Causal Effects: Evaluating Causal Models across Observational Views

    Sep 29, 2026Heejin Jung, Gyeongdeok Seo, Hoyoon Byun +2Causal Effect EstimationCausal Foundation Models