cs.LGNov 9, 2026

Variational Approach to Optimal IPS Estimator for Multi-logger Off-Policy Evaluation

Authors: Joon Suk Huh, Junghoon Seo

Organizations: Department of Computer Science University of Wisconsin–Madison, USA · PIT IN Corp., Republic of Korea

Abstract

We study off-policy evaluation (OPE) in contextual bandits with data collected from multiple logging policies. Inverse propensity scoring (IPS) is a standard approach to OPE and extends naturally to the multi-logger setting. However, as highlighted by Agarwal et al. [2017], neither balanced nor weighted IPS consistently outperforms the other in this setting. We resolve this dilemma by deriving an optimal IPS estimator with sample-dependent weights that minimize variance subject to unbiasedness. Using a variational calculus approach, we obtain closed-form optimal weights, yielding an estimator that is unbiased and achieves asymptotically optimal variance within a generalized weighted-IPS estimator class. Instead of a reward model, the estimator needs only a K×KK\times K matrix and a KK-vector of moments, where KK is the number of logging policies, and we quantify how much efficiency this costs relative to the semiparametric oracle. Experiments on benchmark datasets show that our estimator generally improves on existing multi-logger IPS methods. We also extend our estimator to a doubly robust form by incorporating a conditional reward estimator. We compare the resulting DR extension with the semiparametrically efficient DR estimator of Kallus et al. [2021], both theoretically and empirically. We show that the two estimators achieve the same asymptotic variance when the conditional reward estimator is consistent, while our estimator can retain semiparametric efficiency under certain forms of reward misspecification. Empirically, the proposed DR estimator often achieves lower estimation error than the existing DR baseline.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Variance-Optimal Off-Policy Evaluation with Conjunct Effect Modeling

    Oct 6, 2026Nicolò Felicioni, Michael Benigni, Maurizio Ferrari Dacrema +1Causal Effect EstimationOff-Policy Evaluation

  2. Off-Policy Evaluation for Ranking Policies under Deterministic Logging Policies

    Mar 23, 2026Koichi Tanaka, Kazuki Kawamura, Takanori Muroi +6Off-Policy EvaluationInverse Probability Weighting

  3. From Weighting to Modeling: A Nonparametric Estimator for Off-Policy Evaluation

    Mar 10, 2026Rong J. B. ZhuOff-Policy EvaluationInverse Probability Weighting