Treatment effects estimated from a randomized controlled trial are local not only to the study population but also to the time at which the trial was conducted. The literature on generalizing experimental findings to new populations is extensive, yet transporting effects across time has received far less attention, and even defining the target estimand is nonobvious. We formalize the transported average treatment effect under a separable temporal effects assumption, derive two identification strategies: replicated trials and common arm, and develop doubly robust, semiparametrically efficient estimators for each. Applied to a large archive of headline A/B tests, the common arm strategy is substantially more precise but exhibits systematic bias when the temporal factor depends on the gap between intervention and measurement rather than on measurement time alone, while the replicated trials strategy, which allows this dependence, tracks the ground truth more faithfully. Simulation studies investigate when each strategy is reliable and when it silently fails.
Figures & tables
Figure 1: Temporal transportation uses anchor trials to extrapolate treatment effects across time. Rows are treatment comparisons; columns are times. Solid boxes are ATEs directly estimable from each trial’s data. Solid arrows denote information flow within a time period (from the target trial to contemporary anchors). The target TATE is identified by chaining anchor ATEs across time without requiring the primary comparison at the target time.
DGP A : assumptions hold
DGP B : lag-dependent Λ
DGP C : trt–time interaction
n
Est.
Bias
RMSE
Cov.
Bias
RMSE
Cov.
Bias
RMSE
Cov.
600
Oracle
−.00
.13
.98
−.00
.11
.95
—
—
—
S1
+.03
.34
.93
+.03
.32
.92
+.06
.46
.92
S2-C
−.00
.16
.96
−.05
.14
.91
−.66
.67
.00
S2-T
+.00
.16
.98
−.04
.13
.90
−.24
.30
.78
S2-M
−.02
.15
.97
−.06
.13
.90
−.64
.65
.01
Table 1: Simulation results across three DGPs ( B=500 Monte Carlo replications per cell). Each entry reports Bias / RMSE / Coverage of the 95% CI. Bold marks the lower-RMSE strategy among non-Oracle; red shading marks coverage below 90% (the failure regime). Oracle is undefined under DGP C (separability itself is violated). The red shading is the paper’s central empirical finding: Strategy 2 fails silently under assumption violations, with worsening coverage at larger n because its tighter intervals center on a biased estimand.
Figure 2: Degradation under four assumption violations ( n=1800 , B=60 ). (a) ∣Bias∣ vs. lag-decay rate β (violates Assumption 2 ). (b) Strategy 1 ∣Bias∣ vs. contamination radius γ (violates Assumption 1 ) with the Proposition 1 bound B(γ) as upper envelope. (c) RMSE vs. second-factor strength δ (rank misspecification) vs. no-adjustment baseline. (d) 95% CI coverage vs. target-time covariate drift δdrift .
Trial A ( n=11 months)
Trial B ( n=12 months)
S1
S2
S1
S2
RMSE
.0027
.0105
.0134
.0081
Bias
+.0002
+.0074
+.0040
−.0014
Avg Std Err
.0054
.0019
.0052
.0018
Corr
+0.97
−0.12
−0.87
+0.26
Table 2: Estimation performance against the ground-truth TATE (per-month pooled across the target cluster pair). “Corr” is the Pearson correlation across target months; source months excluded.
Figure 3: TATE estimates by target month for both trials. Each panel shows Strategy 1 (blue circles), Strategy 2 (orange squares), and true TATE (green diamonds) with 95% confidence bands; dashed red line marks the observed ATE at the source time. (a) Trial A (Cluster 17 vs. 10), source time February 2014. (b) Trial B (Cluster 46 vs. 12), source time January 2014.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Embedding
K
RMSE
Corr. with ground truth
MiniLM-L6-v2
30
< similar
≈0.93 – 0.97
MiniLM-L6-v2
50 (default)
0.0027
+0.97
MiniLM-L6-v2
75
< similar
≈0.93 – 0.97
MiniLM-L6-v2
100
< similar
≈0.93 – 0.97
MPNet-base-v2
50
< similar
≈0.95 – 0.97
Appendix
Table 3: Robustness of Strategy 1 Trial A estimates to clustering configuration (representative; exact values for the default K=50 configuration match Table 2 ).
Figure 4: Breakdown frontier analysis for the Upworthy TATE estimates. Each panel shows the point estimate (horizontal line), sampling confidence interval (gray band), and the identified set under contamination γ (colored band). The vertical dashed line marks the breakdown value γ∗ at which the identified set first includes zero. Rows correspond to Trial A (top) and Trial B (bottom); columns to Strategy 1 (left) and Strategy 2 (right).
Table 8
Figure 5: Additional robustness studies complementing main-body Figure 2 . (a,b) Reproduced from main body for appendix self-containedness: rank- 2 misspecification RMSE and population-drift coverage. (c) Double robustness: bias when the outcome model, propensity scores, or both are misspecified. (d) Λ -shape invariance: RMSE of Strategy 1 and Strategy 2-M across sinusoidal, linear, step, and exponential temporal effects.
We present a regression-adjustment framework designed for the estimation of longitudinal treatment effects in randomized experiments under static regimes. While regression-adjustment methods are useful for variance reduction in randomized experiments by using pre-treatment covariates, they usually focus only on average effects, from which we cannot obtain valuable insights into when the effects appear and how long they continue. To address this issue, we consider intermediate outcomes and evolving post-treatment covariates over time, and we represent such dynamic trajectories using transition kernels. Furthermore, we establish the asymptotic normality and the semiparametric efficiency bound for our estimator, enabling more powerful statistical inference. Simulation studies and empirical analysis using A/B test data from a streaming platform in Japan show the practical advantages of our method.
Naoki Chihara, Tatsushi Oka, Yasuko Matsubara +2
SANKEN, The University of Osaka, Osaka, Japan · CyberAgent AI Lab · Keio University, Tokyo, Japan +1
Randomized controlled trials often do not represent the populations where decisions are made, and covariate shift across studies can invalidate standard IPD meta-analysis and transport estimators. We propose a placebo-anchored transport framework that treats source-trial outcomes as abundant proxy signals and target-trial placebo outcomes as scarce, high-fidelity gold labels to calibrate baseline risk. A low-complexity (sparse) correction anchors proxy outcome models to the target population, and the anchored models are embedded in a cross-fitted doubly robust learner, yielding a Neyman-orthogonal, target-site doubly robust estimator for patient-level heterogeneous treatment effects when target treated outcomes are available. We distinguish two regimes: in connected targets (with a treated arm), the method yields target-identified effect estimates; in disconnected targets (placebo-only), it reduces to a principled screen--then--transport procedure under explicit working-model transport assumptions. Experiments on synthetic data and a semi-synthetic IHDP benchmark evaluate pointwise CATE accuracy, ATE error, ranking quality for targeting, decision-theoretic policy regret, and calibration. Across connected settings, the proposed method is best or near-best and improves substantially over proxy-only, target-only, and transport baselines at small target sample sizes; in disconnected settings, it retains strong ranking performance for targeting while pointwise accuracy depends on the strength of the working transport condition.
Zilong Wang, Ali Abdeen, Turgay Ayer
Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia, USA
Estimating causal effects under network interference typically assumes that the network used for training and the network used for deployment coincide. In practice, an intervention is run on one population while the question of interest concerns a different population, and the two generally differ in topology, node-covariate composition, and spillover pathways. Transporting a causal effect across networks is therefore a data-fusion problem that no existing algorithm solves. We employ a selection diagram, extended to the network setting so that covariate shift and structural network shift enter as separate selectors, and derive from it a transport formula for the direct, spillover, and total effects in the deployment population. Each formula makes explicit which interventional mechanism is assumed invariant and which observational distribution must be reweighted. We then turn the formulas into TranCE (Transported Causal Effects), a doubly-robust algorithm combining an interventional outcome model, a domain density-ratio correction, and cross-fitted inference. Extensive experiments on two semi-synthetic benchmarks derived from real-world social networks and on a fully real weather-insurance field experiment, where the transported effects are checked against held-out randomized estimates, confirm the effectiveness of our approach. Our findings have the potential to improve intervention strategies in networked systems, particularly in social networks and public health.
Xiaojing Du, Jiuyong Li, Lin Liu +3
Adelaide University, Adelaide, Australia · Hainan University, Haikou, China