Overview

Policy evaluation asks how well a decision rule performs and whether it improves on alternatives in settings such as reinforcement learning, marketplaces, healthcare, and public policy. Common approaches include temporal-difference and off-policy value estimation, propensity-score and inverse-probability methods, randomized or adaptive experiments, and simulation with agent-based models or digital twins. Researchers also develop confidence intervals, risk-sensitive measures, and benchmarks to compare policies when observations are limited or outcomes are uncertain. A central challenge is that offline estimates and benchmark rankings can mislead under confounding, distribution shift, or adaptive behavior, so reliable assessment must connect metrics to deployment goals and often requires prospective validation.

Latest papers

64 papers12 papers in the last four weeks, up 20% on the four weeks before. 0.1% of all new papers.

Feb 9, 2026

Winner's Curse Drives False Promises in Data-Driven Decisions

A major challenge in data-driven decision-making is accurate policy evaluation---i.e., guaranteeing that a learned decision-making policy achieves the promised benefits. A popular strategy is model-based policy evaluation, which estimates a model from data to infer counterfactual outcomes. This strategy is known to produce unwarrantedly optimistic estimates of the true benefit due to the winner's curse. We searched the recent literature on data-driven decision-making, identifying a sample of 55 papers published in Management Science in the past decade; all but two relied on this flawed methodology. Several common justifications are provided: (1) the estimated models are accurate, stable, and well-calibrated, (2) the historical data uses random treatment assignment, (3) the model family is well-specified, and (4) the evaluation methodology uses sample splitting. Unfortunately, we show that no combination of these justifications avoids the winner's curse. First, we provide a theoretical analysis demonstrating that the winner's curse can cause large, spurious reported benefits even when all these justifications hold. Second, we perform a simulation study based on the recent and consequential data-driven refugee matching problem. We construct a synthetic refugee matching environment (calibrated to closely match the real setting) but designed so that no assignment policy can improve expected employment compared to random assignment. Model-based methods report large, stable gains of around 60% even when the true effect is zero; these gains are on par with improvements of 22-75% reported in the literature. Our results provide strong evidence against model-based evaluation.
Jan 31, 2026

Persistent-Transient Policy Evaluation for Markov Chains via Minimal Peripheral Quotients

We study fixed-policy evaluation for finite Markov chains that may be reducible and periodic. Classical evaluation methods with gain and bias decomposition are not always diagnostic: the gain records only invariant Cesàro averages, while persistent phase-dependent behavior is absorbed into the bias together with genuinely transient effects. We identify the real peripheral invariant subspace K(P)\mathcal{K}(P) of the transition matrix PP as the source of this ambiguity. Quotienting by K(P)\mathcal{K}(P) is the minimal exact quotient that removes all non-decaying modes and makes the remaining dynamics strictly stable. After choosing a gauge projection ΠΠ with kernel K(P)\mathcal{K}(P), the reward admits a unique decomposition r=gΠ⋆+(I−P)vΠ⋆r = g_Π^\star + (I-P)v_Π^\star, where gΠ⋆g_Π^\star is a persistent regime profile and vΠ⋆v_Π^\star is a gauge-fixed transient component. An exact comparison with classical normalized gain and bias shows that the new pair reallocates the same information into a persistent profile and a gauge-fixed representative of the transient quotient. This decomposition reconstructs finite-horizon returns, recovers statewise average reward, admits a transient-cost interpretation, and yields a stable estimator under a generative model.
Nov 24, 2025

Neural Evidence and Behavioural Welfare Economics: When Can Neuroeconomics Inform Policy?

Neuroeconomics appears to offer welfare economics an evidential supplement, since neural data may help distinguish choices that express genuine preferences from choices distorted by addiction, framing or self-control failure. This paper asks when such evidence can legitimately inform welfare judgements for policy. It develops a conceptual framework linking three levels of analysis, namely neural signals, computational decision models and normative welfare criteria. The central argument is that neural data can constrain welfare analysis only under four conditions, covering the validity of the neural measure, the identifiability of the decision model, the explicit statement of the welfare criterion, and the institutional context of any policy use. Applying this framework to addiction, neuromarketing and environmental policy, and working through the United Kingdom's Tobacco and Vapes Act 2026, a major intervention justified without any neural measurement, the paper develops a Neuroeconomic Welfare Inference Checklist for neural evidence. Neural data can improve behavioural welfare economics by clarifying the mechanisms behind choice, and they cannot replace normative argument about what counts as welfare. The same reasoning applies to artificial agents trained on human feedback, whose internal reward signals are computational quantities and not welfare measures.
Oct 21, 2024

Statistical Inference for Policy Evaluation with Temporal Difference Learning

We investigate the statistical properties of Temporal Difference (TD) learning with Polyak-Ruppert averaging, arguably one of the most widely used algorithms in reinforcement learning, for the task of estimating the parameters of the optimal linear approximation to the value function. Assuming independent samples, we make three theoretical contributions that improve upon the current state-of-the-art results: (i) we establish refined high-dimensional Berry-Esseen bounds over the class of convex sets, achieving faster rates than the best known results, and (ii) we propose and analyze a novel, computationally efficient online plug-in estimator of the asymptotic covariance matrix; (iii) we derive sharper high probability convergence guarantees that depend explicitly on the asymptotic variance and hold under weaker conditions than those adopted in the literature. These results enable the construction of confidence regions and simultaneous confidence intervals for the linear parameters of the value function approximation, with guaranteed finite-sample coverage. We demonstrate the applicability of our theoretical findings through numerical experiments.