TEA-Time: Transporting Effects Across Time

Harsh Parikh; Gabriel Levin-Konigsberg; Dominique Perrault-Joncas; Alexander Volfovsky

stat.MEMar 7, 2026

TEA-Time: Transporting Effects Across Time

Authors: Harsh Parikh, Gabriel Levin-Konigsberg, Dominique Perrault-Joncas, Alexander Volfovsky

Organizations: Amazon SCOT, Seattle, USA · Yale University, New Haven, USA · Duke University, Durham, USA

Abstract

Treatment effects estimated from a randomized controlled trial are local not only to the study population but also to the time at which the trial was conducted. The literature on generalizing experimental findings to new populations is extensive, yet transporting effects across time has received far less attention, and even defining the target estimand is nonobvious. We formalize the transported average treatment effect under a separable temporal effects assumption, derive two identification strategies: replicated trials and common arm, and develop doubly robust, semiparametrically efficient estimators for each. Applied to a large archive of headline A/B tests, the common arm strategy is substantially more precise but exhibits systematic bias when the temporal factor depends on the gap between intervention and measurement rather than on measurement time alone, while the replicated trials strategy, which allows this dependence, tracks the ground truth more faithfully. Simulation studies investigate when each strategy is reliable and when it silently fails.

Figures & tables

Panel of Figure 1 — Figure 1: Temporal transportation uses anchor trials to extrapolate treatment effects across time. Rows are treatment comparisons; columns are times. Solid boxes are ATEs directly estimable from each trial’s data. Solid arrows denote information flow within a time period (from the target trial to contemporary anchors). The target TATE is identified by chaining anchor ATEs across time without requiring the primary comparison at the target time.

		DGP A : assumptions hold			DGP B : lag-dependent $\Lambda$			DGP C : trt–time interaction
$n$	Est.	Bias	RMSE	Cov.	Bias	RMSE	Cov.	Bias	RMSE	Cov.
600	Oracle	$-.00$	.13	.98	$-.00$	.11	.95	—	—	—
	S1	$+.03$	.34	.93	$+.03$	.32	.92	$+.06$	.46	.92
	S2-C	$-.00$	.16	.96	$-.05$	.14	.91	$-.66$	.67	.00
	S2-T	$+.00$	.16	.98	$-.04$	.13	.90	$-.24$	.30	.78
	S2-M	$-.02$	.15	.97	$-.06$	.13	.90	$-.64$	.65	.01

Table 1: Simulation results across three DGPs (

B=500

Monte Carlo replications per cell). Each entry reports Bias / RMSE / Coverage of the 95% CI. Bold marks the lower-RMSE strategy among non-Oracle; red shading marks coverage below 90% (the failure regime). Oracle is undefined under DGP C (separability itself is violated). The red shading is the paper’s central empirical finding: Strategy 2 fails silently under assumption violations, with worsening coverage at larger

n

because its tighter intervals center on a biased estimand.

Figure 2: Degradation under four assumption violations (

n=1800

B=60

). (a)

|\mathrm{Bias}|

vs. lag-decay rate

\beta

(violates Assumption 2 ). (b) Strategy 1

|\mathrm{Bias}|

vs. contamination radius

\gamma

(violates Assumption 1 ) with the Proposition 1 bound

B(\gamma)

as upper envelope. (c) RMSE vs. second-factor strength

\delta

(rank misspecification) vs. no-adjustment baseline. (d)

95\%

CI coverage vs. target-time covariate drift

\delta_{\mathrm{drift}}

	Trial A ( $n=11$ months)		Trial B ( $n=12$ months)
	S1	S2	S1	S2
RMSE	.0027	.0105	.0134	.0081
Bias	$+.0002$	$+.0074$	$+.0040$	$-.0014$
Avg Std Err	.0054	.0019	.0052	.0018
Corr	$\mathbf{+0.97}$	$-0.12$	$-0.87$	$+0.26$

Table 2: Estimation performance against the ground-truth TATE (per-month pooled across the target cluster pair). “Corr” is the Pearson correlation across target months; source months excluded.

Panel of Figure 3 — Figure 3: TATE estimates by target month for both trials. Each panel shows Strategy 1 (blue circles), Strategy 2 (orange squares), and true TATE (green diamonds) with 95% confidence bands; dashed red line marks the observed ATE at the source time. (a) Trial A (Cluster 17 vs. 10), source time February 2014. (b) Trial B (Cluster 46 vs. 12), source time January 2014.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Embedding	$K$	RMSE	Corr. with ground truth
MiniLM-L6-v2	30	$<$ similar	$\approx 0.93$ – $0.97$
MiniLM-L6-v2	50 (default)	0.0027	+0.97
MiniLM-L6-v2	75	$<$ similar	$\approx 0.93$ – $0.97$
MiniLM-L6-v2	100	$<$ similar	$\approx 0.93$ – $0.97$
MPNet-base-v2	50	$<$ similar	$\approx 0.95$ – $0.97$

Appendix

Table 3: Robustness of Strategy 1 Trial A estimates to clustering configuration (representative; exact values for the default

K=50

configuration match Table 2 ).

Figure 4: Breakdown frontier analysis for the Upworthy TATE estimates. Each panel shows the point estimate (horizontal line), sampling confidence interval (gray band), and the identified set under contamination

\gamma

(colored band). The vertical dashed line marks the breakdown value

\gamma^{*}

at which the identified set first includes zero. Rows correspond to Trial A (top) and Trial B (bottom); columns to Strategy 1 (left) and Strategy 2 (right).

Table 8

Figure 5: Additional robustness studies complementing main-body Figure 2 . (a,b) Reproduced from main body for appendix self-containedness: rank-

2

misspecification RMSE and population-drift coverage. (c) Double robustness: bias when the outcome model, propensity scores, or both are misspecified. (d)

\Lambda

-shape invariance: RMSE of Strategy 1 and Strategy 2-M across sinusoidal, linear, step, and exponential temporal effects.

Causal Effect Estimation

Causal effect estimation measures how outcomes would change under interventions, using experiments or observational data and explicit assumptions.

See all 257 papers

Modeling Covariate Transition for Efficient Estimation of Longitudinal Treatment Effects in Randomized Experiments
May 29, 2026Naoki Chihara, Tatsushi Oka, Yasuko Matsubara +2Causal Effect Estimation
SANKEN, The University of Osaka, Osaka, Japan · CyberAgent AI Lab · Keio University, Tokyo, Japan +1
Transfer Learning for Meta-analysis Under Covariate Shift
Apr 3, 2026Zilong Wang, Ali Abdeen, Turgay AyerCausal Inference Individualized Treatment Effect Estimation
Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia, USA
Transportable Causal Effect Estimation across Networks under Interference
Aug 19, 2026Xiaojing Du, Jiuyong Li, Lin Liu +3Causal Effect Estimation Causal Inference
Adelaide University, Adelaide, Australia · Hainan University, Haikou, China

May 29, 2026J/K move · Enter open · S save

Modeling Covariate Transition for Efficient Estimation of Longitudinal Treatment Effects in Randomized Experiments

Naoki Chihara, Tatsushi Oka, Yasuko Matsubara, Yasushi Sakurai+1

SANKEN, The University of Osaka, Osaka, Japan · CyberAgent AI Lab · Keio University, Tokyo, Japan+1

Causal Effect Estimation

Open paper AskPDF arXiv

We present a regression-adjustment framework designed for the estimation of longitudinal treatment effects in randomized experiments under static regimes. While regression-adjustment methods are useful for variance reduction in randomized experiments by using pre-treatment covariates, they usually focus only on average effects, from which we cannot obtain valuable insights into when the effects appear and how long they continue. To address this issue, we consider intermediate outcomes and evolving post-treatment covariates over time, and we represent such dynamic trajectories using transition kernels. Furthermore, we establish the asymptotic normality and the semiparametric efficiency bound for our estimator, enabling more powerful statistical inference. Simulation studies and empirical analysis using A/B test data from a streaming platform in Japan show the practical advantages of our method.

Abstract

Causal Effect Estimation

Explore similar work

Modeling Covariate Transition for Efficient Estimation of Longitudinal Treatment Effects in Randomized Experiments

Transfer Learning for Meta-analysis Under Covariate Shift

Transportable Causal Effect Estimation across Networks under Interference