Online Causal Configuration for Networked Systems via Doubly Robust Steady-State Learning

Yuli Liu; Zhiheng Zhang

cs.LGNov 6, 2026

Online Causal Configuration for Networked Systems via Doubly Robust Steady-State Learning

Authors: Yuli Liu, Zhiheng Zhang

Organizations: Smart Ocean Technology, University of Macau, Macao SAR, China · School of Statistics and Data Science, Shanghai University of Finance and Economics Shanghai 200433, P.R. China · Institute of Big Data Research, Shanghai University of Finance and Economics Shanghai 200433, P.R. China

Abstract

Measurement exploration reveals how interacting local controllers behave but can alter the steady-state regime a network operator seeks to optimize. Exploratory rewards may therefore fail to identify a configuration's no-exploration value: under full exploration, identical logged distributions can conceal different target values. Identification requires accounting for both action-law mismatch and shifts in the steady-state context distribution. We introduce CAUSAL-UCB, which selects configurations using optimistic bounds on target causal values. Its interference-aware clipped doubly robust evaluator produces predecision certificates separating statistical variation from calibrated identification, transport, and temporal-dependence errors. For any finite horizon and predetermined schedule of measurement-exploration rates $\{η_t\}$ , we prove lower bounds of $Ω((1+\sum_tη_t)^{-1/2})$ on minimax inference error and $Ω(\sum_tη_t)$ on measurement-induced deployment loss in a stationary, potentially slow-mixing family. We bound regret under valid certificates and selected-information growth, and establish an anytime finite-sample certificate with a shared radius for estimation, selection, and reporting in a restricted known-local-interference model. In a high-mismatch benchmark with 24 paired seeds and 200 rounds, CAUSAL-UCB attains target regret of $1.732$ versus $6.326$ for certified IPS-UCB, with deployment gaps of $30.607$ and $23.217$ , respectively. Independent-audit coverage tests and trace-driven semi-synthetic studies further assess certificate coverage and deployment trade-offs.

Figures & tables

Figure 1: The CAUSAL-UCB deployment loop. A global configuration induces interacting local actions; post-burn-in measurements enter an exposure-aware evaluator, whose value certificate guides the next optimistic configuration choice. Visual elements are conceptual.

Research line	I	L	G	S	R
Interf. estimands	$\checkmark$	$\circ$	–	–	–
DR OPE	–	$\checkmark$	–	$\circ$	–
Network bandits	$\checkmark$	$\circ$	$\circ$	–	$\checkmark$
Design bandits	$\checkmark$	$\circ$	$\circ$	–	$\checkmark$
Temporal overlap	–	$\checkmark$	–	$\circ$	–
CAUSAL-UCB	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

Table 1: Representative research scope.

Figure 2: Identification–deployment trade-off. Blue: error; orange: cost; purple: combined; dashed: illustrative balance.

Panel of Figure 3 — Figure 3: Certified online selection (a) Conceptual spatial-network schematic; (b) cumulative target regret; (c) maximum pairwise ATE error; (d) decision-active certificate diameter for Certified DR/IPS. Curves show 24-paired-seed means with pointwise 95% Student- $t$ intervals.

Method	Reg. $\downarrow$	Gap $\downarrow$	ATE $\downarrow$	$W\downarrow$	Dep. $\downarrow$
CAUSAL-UCB (ours)	1.732	.000	.0230	.552	30.607
Certified IPS-UCB	6.326	.022	.1718	1.541	23.217
Legacy DR-UCB	16.647	.060	.0081	1.961	12.279
Legacy IPS-UCB	16.647	.060	.0610	1.961	12.279
Raw-UCB	19.235	.085	.1595	1.919	8.016
Greedy-Raw	16.207	.080	.1626	2.000	4.120

Table 2: High-mismatch controlled comparison. Means over 24 paired seeds and 200 rounds. Low-mismatch results are in Appendix F.3 .

Figure 4: Interference sensitivity. (a) Cumulative target regret and (b) maximum pairwise ATE error across five interference levels. Curves show means over 24 paired seeds with pointwise 95% Student-

t

intervals.

Panel of Figure 5 — Figure 5: Logging–target support and context shift. (a–c) Action–exposure support; (d–f) logging–target context distributions; (g–i) joint clean-neighborhood matches at $\eta=0.55,0.30,0.08$ . Blue/red indicate target/logging draws in (a–f); green/yellow indicate match/no-match observations in (g–i).

Figure 6: Mechanism ablations. (a) Cumulative target regret and (b) maximum pairwise ATE error across controlled variants. Results illustrate the distinction between sequential decision quality and causal-estimation accuracy.

Figure 7: Exploration and deployment trade-off. (a) Cumulative target regret versus exploration rate

\eta

and (b) high-mismatch Pareto frontier comparing cumulative target regret with deployment gap.

Method	Regret $\downarrow$	Gap $\downarrow$	ATE $\downarrow$	Dep. $\downarrow$
METR-LA
CAUSAL-UCB	0.754 $\pm$ 0.035	0.000	0.0466	12.681
IPS	0.774 $\pm$ 0.036	0.000	0.0670	12.663
Raw	8.438 $\pm$ 0.090	0.026	0.0703	4.177
Greedy	5.640 $\pm$ 0.097	0.030	0.0743	3.567
Switch	8.745 $\pm$ 0.000	0.055	0.0713	4.116

Table 3: High-mismatch semi-synthetic comparison. Mean cumulative target regret

\pm

95% Student-

t

CI half-width over eight seeds (

T=180

). Low-mismatch results are summarized in Appendix G.2 .

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Research line	Core contribution	Distinction in this work
Interference and designs [ 2 , 1 , 31 ]	Potential outcomes, exposure-based inference, and temporal treatment designs.	A global configuration induces local actions and a post-burn-in no-exploration target regime.
DR and orthogonal OPE [ 4 , 5 , 15 ]	Logged-action correction and nuisance-robust value estimation.	The certificate separately budgets exposure, target-context transport, and dependence errors.
Longitudinal / latent evaluation [ 20 , 19 ]	Evaluation under partial observability and overlap-weighted temporal treatment effects.	The estimand here is a global configuration’s steady-state value without measurement exploration.
Contextual and network bandits [ 26 , 7 , 8 ]	Online learning with unknown context laws or interacting rewards.	Selection uses certified causal values from exploration-perturbed network telemetry.
Design-based inference–regret [ 9 ]	Pareto trade-offs and anytime asymptotic inference under network interference.	Our lower bound concerns identification versus measurement-induced deployment loss in a specified stationary, potentially slow-mixing family.

Appendix

Table 4: Representative connections and precise scope distinctions.

Symbol(s)	Meaning	Symbol(s)	Meaning
Network, logging, and causal target
$N,\mathcal{Z},M$	Nodes, configuration set, and its size.	$i,t,T$	Node, deployment round, horizon.
$z,z_{t},z^{\star}$	Candidate, chosen, and optimal configuration ( $\arg\max_{z}W_{\infty}(z;0)$ ).	$\tau,T_{\mathrm{burn}},s,T_{0}$	Fast time, burn-in, retained-window index/count.
$Y_{\tau}^{(t)},H_{t-1}$	Latent network state and predecision history.	$\mathcal{D}_{t},O_{i,s}^{(t)}$	Round- $t$ logs and raw $(A,C,R)$ tuple.
$A,R,B_{u},C^{-}$	Local action, response ( $\|R\|\leq B_{u}$ ), and pre-action context.	$E,\phi,\phi^{+}$	Exposure and baseline/enriched maps.
$\widetilde{A},\widetilde{O}$	Joint treatment $(A,E)$ and score record $(\widetilde{A},C^{-},R)$ .	$\eta_{t},\eta,\xi$	Exploration rates and exploratory action law.

Appendix

Table 5: Notation and experimental protocols.

Figure 8: Conceptual comparison of raw-reward and certified causal-value maximizers.

Method	Regret	Gap	ATE error	Width $W$	Dep. gap
CAUSAL-UCB (ours)	1.018	.000	.0176	.323	4.622
Certified IPS-UCB	1.052	.000	.0597	.644	4.615
Legacy DR-UCB	11.984	.027	.0057	1.105	2.545
Legacy IPS-UCB	11.984	.027	.0197	1.105	2.545
Raw-UCB	15.098	.071	.0235	.968	1.993
Greedy-Raw	1.000	.000	.0291	2.000	4.626

Appendix

Table 6: Low-mismatch controlled comparison. Means over 24 paired seeds and 200 rounds; smaller is better in every numeric column.

Panel of Figure 9 — Figure 9: Supplementary deployment-cost and Pareto diagnostics. (a) Deployment gap by ablation variant; (b) cost–regret map for the same variants; (c) low-mismatch sample-mean Pareto frontier. Panels (a)–(b) use the controlled ablation protocol described in the main text and are not pooled with the five-metric benchmark table. Panel (c) reports the low-mismatch counterpart of the high-mismatch frontier in main-text Figure 7 (b). Regret and simulator-based deployment gap are separate minimized objectives. The cost–regret and Pareto comparisons are descriptive and do not establish statistically significant dominance.

Panel of Figure 10 — Figure 10: Supplementary interference sensitivity across the seven levels shown in the panels. The four outcomes are cumulative target regret, terminal target-value gap, maximum pairwise ATE error, and decision-active certificate width. Widths for non-certified methods remain diagnostic and do not imply simultaneous target coverage.

Panel of Figure 11 — Figure 11: Exploration-rate and uncalibrated-drift stress diagnostics. Panels (a)–(b) report target regret and certificate width across the fixed exploration rates shown. Panels (c)–(d) report regret and empirical path coverage under deliberately uncalibrated post-burn-in drift. The drift panels violate the calibrated-drift condition and are descriptive diagnostics, not theorem-certified coverage evidence.

Figure 12: Measurement-exploration cost and certificate width. (a) Cumulative deployment gap across exploration rates and methods; (b) Certified DR deployment gap (bars) and regret (line) on separate axes; (c) Decision-active certificate width across fixed exploration rates. Panels (a)–(b) correspond to the controlled fixed-rate sweep, not the minimax lower-bound construction. Panel (c) reports a radius diagnostic for the displayed certified methods; width alone does not establish coverage, which remains conditional on the calibration requirements of Theorem 1 .

Panel of Figure 13 — Figure 13: Trace-driven semi-synthetic comparisons by dataset and mismatch regime. Panels (a)–(d) show METR-LA/high mismatch, METR-LA/low mismatch, FlockLab/high mismatch, and FlockLab/low mismatch, respectively. Whiskers are 95% Student- $t$ intervals across eight independent seeds. The high-mismatch cells are also summarized in Table 3 ; this figure adds the low-mismatch cells. These runs are not live deployments and are not certified by the homogeneous ring-model confidence theorem.

Panel of Figure 14 — Figure 14: Seed-level outcomes under high mismatch. Panels (a)–(d) report cumulative target regret, local audit width, terminal target-value gap, and final selected-configuration frequency. All panels use the frozen 24 paired seeds. Widths for non-certified methods are diagnostic only, and the final panel reports configuration IDs rather than unobserved true-value ranks.

Figure 15: Fixed-rate sensitivity sweeps: (a) exploration rate, (b) importance-weight clipping cap, and (c) retained-window budget. A nonzero clipping bias must be accounted for before an interval is treated as certified.

Figure 16: Network-size sensitivity across the displayed node counts. Panels report target regret, maximum pairwise ATE error, and deployment gap, respectively.

Panel of Figure 17 — Figure 17: Known-radius scalability sweep. Panels (a)–(c) report target regret, maximum pairwise ATE error, and deployment gap for radius $r=1$ ; panels (d)–(f) report the same metrics for radius $r=2$ .

Figure 18: Clipping-cap sensitivity across the displayed cap values. Panels report target regret, maximum pairwise ATE error, and certificate width. Clipping without a valid bias allowance is a stress diagnostic, not a certified guarantee.

Figure 19: Retained-window-budget sensitivity. Panels report target regret, maximum pairwise ATE error, and deployment gap across the window counts shown.

Panel of Figure 20 — Figure 20: Supplementary overlap quantification for Figure 5 . Panel (a) reports empirical joint clean-neighborhood match probability and the conditional 10th-percentile match probability; panel (b) reports descriptive node-level IPW effective counts across overlap regimes. These counts are not the dependence-calibrated effective sample size used in Theorem 1 .

Figure 21: Matched-log DR/IPS-by-exposure comparison across the interference levels shown. Panels report maximum pairwise ATE error, ATE RMSE, and diagnostic pairwise ATE coverage. The local representation omits neighboring actions; the displayed coverage is a diagnostic and is not the general Theorem 1 guarantee.

Figure 22: Online exposure-versus-local stress test across the interference levels shown. Panel (a) reports target regret; panel (b) reports observed all-time coverage. The local estimator is heuristic and the coverage curve is diagnostic, not a certificate under the general theorem.

Panel of Figure 23 — Figure 23: Logging–target alignment and mismatch sensitivity across the exploration rates shown. The panels report absolute value error, naive interval coverage, and interval width. The sampling-only intervals are diagnostic: high coverage or narrow width does not establish target identification under context shift.

Regime	$n_{\rm mis}$	Certified-OFU	Certified-Greedy	Decaying-Forced
Easy, balanced	0	0.327	0.000	1.728
Ambiguous, balanced	17	0.330	0.051	0.299
Starved optimal	33	0.241	0.467	0.268
Starved middle	24	0.297	0.019	0.238
Starved low	30	0.314	0.024	0.241

Appendix

Table 7: Certified OFU versus Greedy under five fixed initial-information regimes. Mean cumulative target regret across 96 paired seeds;

n_{\rm mis}

is the number of seeds whose initial point estimates rank the true best configuration incorrectly. Lower regret is better. All regimes are reported.

Figure 24: Correction under initially insufficient optimal-configuration information. Mean trajectories and pointwise 95% seed-level intervals on the 33 initial-misranking paths. Left: cumulative target regret. Right: number of certified scoring windows accrued on the truly optimal configuration. All policies start from the same pilot counts and receive 400 new scored blocks per online round.

Figure 25: Decision value is conditional on the information structure. Left: paired Greedy-minus-OFU target regret, with 95% seed-level intervals, in all five fixed regimes (positive favors OFU). Right: a valid conservative uniform seven-term bias envelope preserves OFU’s advantage on initially misranked paths but incurs additional exploration cost.

Panel of Figure 26 — Figure 26: Gap–noise–budget sensitivity of certified optimism. (a) Mean paired regret differences across the $3\times 3\times 2$ stress grid, n.2/B200 denotes noise amplitude $\sigma=0.2$ and a budget of 200 scored windows per decision; the remaining labels follow the same convention; (b)–(d) paired regret differences across noise amplitudes at target gaps $0.005$ , $0.016$ , and $0.030$ , respectively, under 200 and 400 scored windows. Differences are OFU minus Greedy; negative values favor OFU. Each cell uses 24 paired seeds and 80 online rounds. Error bars denote 95% paired Student- $t$ intervals; the heatmap reports cell means.

Figure 28: Full-path certificate coverage and the OFU path inequality over 300 independently seeded runs, each with 160 online rounds. Points show the empirical fractions and intervals show 95% Wilson uncertainty.

Figure 29: Certificate informativeness under the 300-seed, 160-round audit. Panel (a) reports the seed-averaged predecision radius and its statistical and calibrated-bias components; panel (b) reports the seven-term budget ledger; panel (c) shows the fraction of runs whose selected upper confidence bound reaches the utility cap. The separate 24-seed pilot diagnostics are reported in Figure 30 .

Figure 30: Supplementary certificate diagnostics from the panel-source archive. Panel (a) reports radius components in the 24-seed, 80-round pilot; panel (b) shows the configuration-specific bias-budget composition in the separate 300-seed audit; panel (c) reports capped-optimism frequency in the 24-seed pilot. The protocols are intentionally distinguished and their outcomes are not pooled. The 300-seed audit’s aggregate informativeness summary appears in Figure 29 .

Violation	False issuance	Full-path covered	Coverage (95% Wilson CI)	Mean final radius (95% $t$ CI)
Target action law	200/200	200/200	100.0% ( $[98.1,100]$ )	0.323 ( $[0.318,0.328]$ )
Target context law	200/200	200/200	100.0% ( $[98.1,100]$ )	0.324 ( $[0.319,0.329]$ )
Target response	200/200	199/200	99.5% ( $[97.2,99.9]$ )	0.317 ( $[0.313,0.321]$ )
Exposure/transport	200/200	200/200	100.0% ( $[98.1,100]$ )	0.324 ( $[0.320,0.329]$ )
Dependence envelope	200/200	200/200	100.0% ( $[98.1,100]$ )	0.324 ( $[0.320,0.329]$ )
Audit independence	200/200	200/200	100.0% ( $[98.1,100]$ )	0.239 ( $[0.238,0.240]$ )

Appendix

Table 9: Full-path coverage conditional on deliberately false certificates being issued. Each condition has 200 seeds.

Panel of Figure 31 — Figure 31: Certificate diagnostics under hidden assumption violations. (a) Full-path coverage conditional on false certificate issuance under six hidden assumption violations, with Wilson 95% confidence intervals. (b) Mean final average certificate radius under the same violations, with 95% Student- $t$ intervals across seeds.

Causal Bandits

See all 8 papers

Causal Bayesian Optimization: Foundations, Methods, and Applications
Sep 21, 2026Chenfeng Huang, Thuy T. Le, Zixuan Ma +1Bayesian Optimization Causal Inference
Department of Statistics & Data Science University of California, Los Angeles · Department of Mathematics and Statistics California State University, Long Beach · Department of Mathematics North Carolina State University
Uncertainty-aware Causal Decision Making via Effect Bound Decomposition
Jan 30, 2026Md Musfiqur Rahman, Ziwei Jiang, Hilaf Hasson +1Causal Effect Estimation Parameter Identifiability
Electrical and Computer Engineering, Purdue University · Computer Science, Johns Hopkins University · Cohesity
Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Risk
May 28, 2026Tim Woydt, Paul-David ZuercherPAC-Bayesian Generalization Bounds Multi-Armed Bandits
ProdAxon · Department of Computer Science Technical University Darmstadt · Institute for Manufacturing Department of Engineering University of Cambridge

Sep 21, 2026J/K move · Enter open · S save

Causal Bayesian Optimization: Foundations, Methods, and Applications

Chenfeng Huang, Thuy T. Le, Zixuan Ma, Hien Tran

Department of Statistics & Data Science University of California, Los Angeles · Department of Mathematics and Statistics California State University, Long Beach · Department of Mathematics North Carolina State University

Bayesian Optimization Causal Inference Causal Intervention

Open paper AskPDF arXiv

Causal Bayesian Optimization (CBO) combines causal inference with Bayesian optimization to enable sample-efficient intervention selection in systems with causal structure. This survey provides a systematic review of CBO through a unified BO-loop perspective, showing how causal assumptions shape intervention search spaces, surrogate models, acquisition functions, and decision policies. We organize existing methods by graph and system-knowledge assumptions, environment, intervention representation, surrogate architecture, and decision rule, and connect CBO to causal bandits, Bayesian experimental design, safe optimization, policy search, and causal abstraction. We also introduce a reproducibility-oriented benchmark spanning hard- and soft-intervention settings, with standardized GAP and a new trajectory-aware Path-Aware GAP (PA-GAP), evaluating seven CBO methods and a non-causal BO baseline across thirteen datasets, three budgets, and two metrics. Results show that no method dominates uniformly: rankings depend on dataset, budget, metric, and how causal information is used, while strong non-causal baselines remain competitive in several settings. Controlled graph-misspecification and omitted-variable stress tests further show that rankings can change substantially when learner-side causal information is perturbed. We conclude by identifying key open challenges, including robustness to causal-assumption violations, scalable unknown-graph optimization, mixed intervention types, realistic cost models, stronger theoretical guarantees, and integration with modern representation learning and causal abstractions.

Abstract

Causal Bandits

Explore similar work

Causal Bayesian Optimization: Foundations, Methods, and Applications

Uncertainty-aware Causal Decision Making via Effect Bound Decomposition

Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Risk