stat.MLNov 15, 2026

Joint Learning of Factual and Counterfactual Worlds for Causal Prior-data Fitted Networks

Authors: Furkan Danisman, Luke Zhang, Zhijing Jin, Dehan Kong

Organizations: University of Toronto · Jinesis Lab & Vector Institute · Max Planck Institute for Intelligent Systems · EuroSafeAI

Abstract

Estimating treatment effects is central to science across many disciplines and requires counterfactual reasoning about alternative worlds. In the real world, however, each unit reveals its outcome only under the factual world. Thus, the factual and counterfactual outcomes can never be jointly learned by any model trained on real data. The problem is therefore tractable only in synthetic settings, where both worlds can be generated. Recent progress on prior-data fitted networks (PFNs) has shown that models pre-trained purely on synthetic data can transfer to real-world prediction. This paradigm has since carried over to causal effect estimation. Despite this fully synthetic setup, existing causal PFNs carry the real-world constraint into their training objective, confining learning to a single world's outcome per unit. The dependence between the two worlds is thereby lost, and the density of the treatment effect relies by default on independence, which is implausible in practice. We therefore propose joint learning, a training setup in which a causal PFN learns the joint distribution of the factual and counterfactual outcomes of each unit. From the joint distribution, we obtain both the response and the individual treatment effect densities, and at inference, we apply optimal transport across units to obtain an approximation of the average treatment effect density. Ours is therefore the first causal PFN approach to provide all three densities together while accounting for the cross-world dependence. We apply joint learning and optimal transport to existing causal PFNs and evaluate it across semi-synthetic, synthetic, and real benchmarks spanning diverse causal scenarios. The jointly trained models significantly improve upon their baselines in point estimation and uncertainty quantification in most settings, indicating that joint learning is a preferable training setup for causal PFNs.

Figures & tables

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Causal Longitudinal Prior-Fitted Networks for Counterfactual Outcome Prediction

    Jun 4, 2026Amirhossein Zare, Amirhessam Zare, Herlock Rahimi +2Structural Causal ModelsCounterfactual Prediction

  2. Frequentist Consistency of Prior-Data Fitted Networks for Causal Inference

    Mar 12, 2026Valentyn Melnychuk, Vahid Balazadeh, Stefan Feuerriegel +1Causal Effect EstimationPrior-Data Fitted Networks

  3. Foundation Models for Partial Causal Identification

    Aug 21, 2026Alexis Bellot, Anish DhirCausal Foundation ModelsCausal Effect Estimation