MediPFN: Amortized In-Context Nuisance Learning and Inference for Causal Mediation Analysis
Organizations: Department of Mathematics and Statistics, Boston University
Abstract
Causal mediation analysis decomposes a treatment effect into indirect pathways through mediators and direct pathways through other processes. Tabular foundation models perform classification and regression through in-context learning, providing estimates of conditional means and propensity scores used in causal mediation analysis. Constructing confidence intervals from these estimates requires accounting for uncertainty introduced by fitting and combining several nuisance functions. We introduce MediPFN, an amortized inference framework that adapts a pretrained tabular foundation model to nuisance prediction and combines its predictions through a cross-fitted influence-function estimator. A separate head learns quantiles of the estimator's standardized error from simulated tables with known indirect effects. Supervised fine-tuning, symmetrization, and independent calibration then produce a confidence interval. We establish coverage bounds for the resulting intervals under standard assumptions. Across experiments, MediPFN attains coverage closer to the nominal level than several comparators. It also reduces the computational cost of uncertainty quantification, running 59.1 times faster than bootstrap in matched comparisons.
Figures & tables
| Method | ID | Source shift | Composition shift | Real covariates | ||||
|---|---|---|---|---|---|---|---|---|
| MediPFN | 94.01 | 0.607 | 92.97 | 1.752 | 91.67 | 1.074 | 94.92 | 0.555 |
| Linear SEM | 78.48 | 0.375 | 49.74 | 1.278 | 70.83 | 0.605 | 91.21 | 0.302 |
| RFF | 91.43 | 0.856 | 89.58 | 1.759 | 85.42 | 1.234 | 89.45 | 2.334 |
| RF | 87.77 | 2.776 | 85.94 | 4.651 | 78.13 | 1.358 | 89.65 | 0.778 |
| RF bootstrap | 61.09 | 1.481 | 46.09 | 1.718 | 22.92 | 0.710 | 77.34 | 0.413 |
| Method | cnae-9 | adult | eucalyptus | car | ||||
|---|---|---|---|---|---|---|---|---|
| MediPFN | 96.88 | 0.456 | 94.53 | 0.526 | 93.75 | 0.695 | 94.53 | 0.541 |
| Linear SEM | 89.84 | 0.299 | 91.41 | 0.309 | 89.84 | 0.287 | 93.75 | 0.312 |
| RFF | 96.88 | 2.187 | 91.41 | 3.338 | 83.59 | 2.616 | 85.94 | 1.194 |
| RF | 91.41 | 1.096 | 87.50 | 0.608 | 90.62 | 0.793 | 89.06 | 0.615 |
| RF bootstrap | 81.25 | 0.358 | 68.75 | 0.409 | 82.81 | 0.445 | 76.56 | 0.442 |
| Setting | Bias | RMSE | SD | Interval endpoints | Interval length | Coverage (%) |
|---|---|---|---|---|---|---|
| ID, all | 0.369 | 0.385 | 0.607 | 94.01 | ||
| 0.593 | 0.631 | 0.954 | 92.11 | |||
| 0.022 | 0.182 | 0.161 | 0.472 | 95.39 | ||
| 0.150 | 0.147 | 0.395 | 94.52 | |||
| Source shift | 0.076 | 1.039 | 1.211 | 1.752 | 92.97 | |
| Composition shift | 0.427 | 0.365 | 1.074 | 91.67 |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Set | OpenML source | Version | Population rows | Covariates |
|---|---|---|---|---|
| 1 | cnae-9 (1468) | 1 | 1,080 | 30 |
| 1 | adult (1590) | 2 | 48,842 | 30 |
| 1 | eucalyptus (188) | 1 | 736 | 30 |
| 1 | car (40975) | 3 | 1,728 | 21 |
| 2 | pc1 (1068) | 1 | 1,109 | 21 |
| 2 | MagicTelescope (1120) | 1 | 19,020 | 10 |
| Setting | Tables | Coverage (%) | Interval length |
|---|---|---|---|
| ID | 768 | 94.01 | 0.607 |
| Source shift | 288 | 92.97 | 1.752 |
| Composition shift | 96 | 91.67 | 1.074 |
| Real covariates, evaluation set 1 | 512 | 94.14 | 0.554 |
| Real covariates, evaluation set 2 | 512 | 95.90 | 3.656 |
| Setting | Method | 90% | 95% | 99% |
|---|---|---|---|---|
| ID | MediPFN | 90.12 / 0.551 | 94.01 / 0.607 | 98.70 / 0.784 |
| Source shift | MediPFN | 89.84 / 1.609 | 92.97 / 1.752 | 97.40 / 2.208 |
| Composition shift | MediPFN | 89.58 / 0.999 | 91.67 / 1.074 | 94.79 / 1.352 |
| Nominal level | Before calibration | After calibration |
|---|---|---|
| 90% | 90.23 / 0.490 | 90.23 / 0.490 |
| 95% | 93.16 / 0.530 | 94.14 / 0.554 |
| 99% | 96.29 / 0.603 | 97.46 / 0.668 |
| Method | cnae-9 | adult | eucalyptus | car | ||||
|---|---|---|---|---|---|---|---|---|
| MediPFN | 96.09 | 0.417 | 89.06 | 0.487 | 90.62 | 0.652 | 92.97 | 0.498 |
| Linear SEM | 84.38 | 0.251 | 83.59 | 0.259 | 88.28 | 0.241 | 85.94 | 0.262 |
| RFF | 87.50 | 1.835 | 81.25 | 2.801 | 73.44 | 2.195 | 79.69 | 1.002 |
| RF | 86.72 | 0.920 | 76.56 | 0.510 | 82.03 | 0.665 | 79.69 | 0.516 |
| RF bootstrap | 78.91 | 0.305 | 64.06 | 0.354 | 79.69 | 0.384 | 71.09 | 0.380 |
| Method | cnae-9 | adult | eucalyptus | car | ||||
|---|---|---|---|---|---|---|---|---|
| MediPFN | 99.22 | 0.580 | 100.00 | 0.659 | 95.31 | 0.866 | 97.66 | 0.686 |
| Linear SEM | 96.88 | 0.393 | 96.88 | 0.406 | 96.09 | 0.377 | 97.66 | 0.411 |
| RFF | 100.00 | 2.874 | 98.44 | 4.387 | 96.09 | 3.438 | 96.09 | 1.569 |
| RF | 96.09 | 1.441 | 94.53 | 0.798 | 96.88 | 1.042 | 94.53 | 0.808 |
| RF bootstrap | 84.38 | 0.429 | 77.34 | 0.485 | 88.28 | 0.529 | 85.16 | 0.540 |
| DGP source | 90% | 95% | 99% | |||
|---|---|---|---|---|---|---|
| Cheng and Li (2026) | 89.58 | 1.894 | 93.75 | 2.126 | 100.00 | 2.829 |
| Díaz and Hejazi (2020) | 89.58 | 0.482 | 91.67 | 0.512 | 95.83 | 0.650 |
| Díaz et al. (2021) | 91.67 | 0.997 | 93.75 | 1.048 | 97.92 | 1.279 |
| Tchetgen Tchetgen and Shpitser (2012) | 70.83 | 0.771 | 77.08 | 0.821 | 91.67 | 1.064 |
| He et al. (2024) | 89.58 | 0.241 | 91.67 | 0.261 | 97.92 | 0.333 |
| Setting | Tables | Coverage (%) | Interval length |
| Additional noise | 192 | 3.046 | |
| Source shift | – | 92.36 | – |
| Tchetgen Tchetgen and Shpitser (2012) | 48 | 0.821 | |
| He et al. (2024) , added noise | 16 | 0.207 |
| Stratum | Tables | Coverage (%) | Length |
|---|---|---|---|
| Positive effect | 208 | 93.27 | 0.609 |
| Zero effect | 130 | 93.85 | 0.355 |
| Negative effect | 174 | 92.53 | 0.566 |
| 166 | 88.55 | 0.769 | |
| 178 | 94.38 | 0.486 | |
| 168 | 96.43 | 0.340 |
| Method | ID | Source shift | Composition shift | Real covariates | ||||
|---|---|---|---|---|---|---|---|---|
| MediPFN | 91.26 | 0.591 | 92.97 | 1.859 | 92.56 | 0.968 | 95.03 | 1.697 |
| Linear SEM | 79.53 | 0.359 | 62.50 | 1.212 | 71.87 | 0.615 | 93.11 | 0.451 |
| RFF | 89.94 | 0.870 | 90.10 | 1.669 | 85.42 | 1.413 | 91.99 | 6.032 |
| RF | 87.74 | 2.976 | 86.72 | 5.266 | 73.96 | 1.435 | 82.05 | 0.831 |
| RF bootstrap | 59.79 | 1.566 | 47.14 | 1.788 | 14.58 | 0.671 | 78.04 | 0.545 |
| Coverage (%) | Interval length | |||
|---|---|---|---|---|
| Setting | Original | Expanded | Original | Expanded |
| No confounding | 96.88 | 96.88 | 0.629 | 0.629 |
| Moderate positive bias | 93.75 | 100.00 | 0.597 | 0.809 |
| Strong positive bias | 59.38 | 100.00 | 0.668 | 1.368 |
| Strong negative bias | 62.50 | 100.00 | 0.475 | 1.211 |
| Nonlinear positive bias | 87.50 | 96.88 | 0.660 | 1.135 |
Causal Mediation Analysis
Causal mediation analysis separates an intervention’s effects into pathways that act directly and those transmitted through mediators.
See all 19 papers