Policy Learning with Weak Signals

Benedikt Koch; Winston Chou; Aurélien Bibaut; Nathan Kallus

stat.MEOct 7, 2026

Policy Learning with Weak Signals

Authors: Benedikt Koch, Winston Chou, Aurélien Bibaut, Nathan Kallus

Organizations: Harvard University · Netflix

Abstract

Policy learning in digital experimentation faces three challenges: weak signal-to-noise ratios, rich covariate spaces, and massive data volumes. We formalize this regime by modeling treatment-effect estimates from increasingly fine covariate partitions as Gaussian observations with bounded signal-to-noise ratios. We establish that, in general, the optimal treatment policy is not learnable in this setting. Even learning the optimal policy value suffers from impractically slow rates. However, when treatment effects vary smoothly, we derive minimax-adaptive policies based on linear smoothers that achieve vanishing welfare regret. We demonstrate the practical value of our framework by applying it to large-scale real-world experiments at Netflix, showing that personalized linear-smoothing policies can dominate unpersonalized policies even in this challenging empirical setting.

Figures & tables

Smoother	Weight $w_{r,j}(x)$	Conditions
Partition pooling	$\dfrac{p_{j}\,\mathbf{1}{\left\{x_{j}\in A_{r}(x)\right\}}}{\sum_{\ell:\,x_{\ell}\in A_{r}(x)}p_{\ell}}$	$\mathcal{A}_{r}$ a finite partition of $\mathcal{X}$ ; $A_{r}(x)\in\mathcal{A}_{r}$ the cell containing $x$ ; $\max_{A\in\mathcal{A}_{r}}\sup_{x,y\in A}\\|x-y\\|_{2}\leq r$ ;
Kernel smoothing	$\dfrac{\mathsf{K}\left((x_{j}-x)/r\right)}{\sum_{\ell=1}^{K}\mathsf{K}\left((x_{\ell}-x)/r\right)}$	$\mathsf{K}:\mathbb{R}^{d}\to[0,\infty)$ bounded, supported on $B(0,1)$ , continuous at $0$ , $\mathsf{K}(0)>0$ .
Series regression	$a_{j}\,b_{r}(x)^{\top}G_{r,\bm{a}}^{-1}b_{r}(x_{j})$	$a_{k}>0$ , $\sum_{k}a_{k}=1$ ; $b_{r}:\mathcal{X}\to\mathbb{R}^{q_{r}}$ with $q_{r}\leq K$ , $q_{r}\asymp r^{-d}$ ; constants $\in\operatorname{span}(b_{r})$ ; $G_{r,\bm{a}}:=\sum_{k}a_{k}b_{r}(x_{k})b_{r}(x_{k})^{\top}$ nonsingular.

Table 1: Linear smoothers at resolution

r

Figure 1: The first three panels show the true curved CATE $g$ , one draw of the rescaled estimates $Z_{k}=\sqrt{n}\,\widehat{\tau}_{k}$ , and the plug-in policy $\mathbf{1}{\left\{Z_{k}>0\right\}}$ that these estimates imply. The remaining panels show representative linear smoothers. The dashed line is the true frontier at which $g=0$ , while the solid line represents the learner’s decision boundary, $\widehat{g}_{w}=0$ .

Figure 2: The top row shows the observed CATEs and full-sample fits of a plane, tensor-product B-spline, regression tree, and local-linear Epanechnikov kernel applied to Experiment 1. Each displayed fit uses the candidate selected most often across outer folds in nested CV. The lower-left panel shows the average held-out policy gains over unpersonalized policies of the family with the lowest estimated policy loss. The remaining panels show the calibration of cross-fitted CATE predictions after centering within each fold.

Figure 3: Panels are laid out as in Figure 2 , but for Experiment 2.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: The first three panels show the true linear CATE $g$ , one draw of the rescaled estimates $Z_{k}=\sqrt{n}\,\widehat{\tau}_{k}$ , and the plug-in policy $\mathbf{1}{\left\{Z_{k}>0\right\}}$ that these estimates imply. The remaining panels show representative linear smoothers. The dashed line is the true frontier at which $g=0$ , while the solid line represents the learner’s decision boundary, $\widehat{g}_{w}=0$ .

		Linear signal		Curved signal
Learner	$\tilde{r}$	$R_{K}$	$W/V^{\ast}$	$R_{K}$	$W/V^{\ast}$
Plug-in	$1$	$0.085$	$0.27$	$0.052$	$0.14$
Pooled	$10$	$0.117$	$0.00$	$0.060$	$0.00$
Tree ( $J=4$ )	$5.0$	$0.035$	$0.70$	$0.034$	$0.43$
Tree ( $J=9$ )	$3.3$	$0.040$	$0.65$	$0.034$	$0.44$
Tree ( $J=25$ )	$2.0$	$0.063$	$0.46$	$0.043$	$0.30$

Appendix

Table 2: Exact welfare regret and proportional welfare for representative learners. The resolution

\tilde{r}

is measured in units of

K^{-1/d}

, so

\tilde{r}=1

corresponds to the plug-in rule and

\tilde{r}=10

corresponds to full pooling, that is, a single treatment decision is made for all cells jointly.

Figure 5: Approximation error

\mathsf{A}_{g}(w_{r})

, estimation error

\gamma^{\ast}\mathsf{E}_{\bm{\sigma}}(w_{r})

, their sum, and exact regret

R_{K}

as functions of resolution

\tilde{r}

. Rows correspond to signal regimes, while columns correspond to the smoother families considered in the main text.

Figure 6: Every candidate is one point in terms of approximation error

\mathsf{A}_{g}(w)

and estimation error

\gamma^{\ast}\mathsf{E}_{\bm{\sigma}}(w)

. Note that the axis are on different scales. Better smoothers lie closer to the bottom-left corner.

Explore similar work

Adaptive Policy Learning Under Unknown Network Interference
May 11, 2026Aidan Gleich, Eric Laber, Alexander VolfovskyTreatment Allocation Thompson Sampling
Department of Statistical Science Duke University
A Hierarchy of Policy Learning Problems
Jul 3, 2026Hamsa Bastani, Osbert Bastani, Shihan ChenHierarchical
Wharton School, University of Pennsylvania · University of Pennsylvania · Graduate Group in Applied Mathematics and Computational Science, University of Pennsylvania
Wasserstein Policy Learning for Distributional Outcomes
Jun 17, 2026Yiyan Huang, Cheuk Hang Leung, Qi Wu +1Wasserstein Distance Causal Inferences
School of Statistics and Data Science & Institute of Big Data Research, Shanghai University of Finance and Economics, Shanghai, China

May 11, 2026, stat.MLJ/K move · Enter open · S save

Adaptive Policy Learning Under Unknown Network Interference

Aidan Gleich, Eric Laber, Alexander Volfovsky

Department of Statistical Science Duke University

Treatment Allocation Thompson Sampling Interference

Open paper AskPDF arXiv

Adaptive experimentation under unknown network interference requires solving two coupled problems: (i) learning the underlying dynamics of interference among units and (ii) using these dynamics to inform treatment allocation in order to maximize a cumulative outcome of interest (e.g. revenue). Existing adaptive experimentation methods either assume the interference network is fully known or bypass the network by operating on coarse cluster-level randomizations. We develop a Thompson sampling algorithm that jointly learns the interference network and adaptively optimizes individual-level treatment allocations via a Gibbs sampler. The algorithm returns both an optimized treatment policy and an estimate of the interference network; the latter supports downstream causal analyses such as estimation of direct, indirect, and total treatment effects. For additive spillover models, we show that total reward is linear in the treatment vector with coefficients given by an

n

-dimensional latent score. We prove a Bayesian regret bound of order

\sqrt{nT \cdot B \log(en/B)}

for exact posterior sampling; empirically, our Gibbs-based approximate sampler achieves regret consistent with this rate and remains sublinear when the additive spillovers assumption is violated. For general Neighborhood Interference, where this reduction is unavailable, we analyze an explore-then-commit variant with

O(n^2 \log T)

graph-discovery cost. An information-theoretic

Ω(n \log T)

lower bound complements both results. Empirically, our method achieves more than an order-of-magnitude reduction in regret in head-to-head comparisons. On two real-world networks, the algorithm achieves sublinear regret and yields downstream effect estimates with small RMSE relative to the truth.

Abstract

Explore similar work

Adaptive Policy Learning Under Unknown Network Interference

A Hierarchy of Policy Learning Problems

Wasserstein Policy Learning for Distributional Outcomes