Policy Evaluation

Follow this topic Ask about this topic

Policy evaluation asks how well a decision rule performs and whether it improves on alternatives in settings such as reinforcement learning, marketplaces, healthcare, and public policy. Common approaches include temporal-difference and off-policy value estimation, propensity-score and inverse-probability methods, randomized or adaptive experiments, and simulation with agent-based models or digital twins. Researchers also develop confidence intervals, risk-sensitive measures, and benchmarks to compare policies when observations are limited or outcomes are uncertain. A central challenge is that offline estimates and benchmark rankings can mislead under confounding, distribution shift, or adaptive behavior, so reliable assessment must connect metrics to deployment goals and often requires prospective validation.

Latest papers

55 papers11 papers in the last four weeks, up 120% on the four weeks before. 0.1% of all new papers.

Online Policy Evaluation for MDPs with Dynamic UBSR Measures
Jul 25, 2026Weikai Wang, Erick DelagePolicy Evaluation Risk-Sensitive RL
GERAD & Department of Decision Sciences, HEC Montr´eal, Canada
A Formally Grounded ODRL Evaluator: Implementation and Comparison
Jul 17, 2026Jaime Osvaldo Salas, Paolo Pareti, Adeel Aslam +2Policy Evaluation Access Control
University of Southampton, University Road, Southampton, SO17 1BJ, UK
Comparing Socially-Equitable Renewable Energy Budget Allocation MDP Policies in Mature and Emerging Economies
Jul 11, 2026Riya Kinnarkar, Mansur M. Arief, Yan Pratama Akhra +1Fair Resource Allocation Policy Evaluation
Management & Technology University of Pennsylvania Philadelphia, PA, USA
When Aggregate Alignment Misleads: Auditing Policy Repair Without Per-State Expert Actions
Jul 3, 2026Peiying Zhu, Sidi ChangLLM Agent Evaluation Policy Evaluation
Blossom AI San Francisco, USA · Blossom AI Labs Tokyo, Japan
Critical Interval MSE: Toward Reliable Offline Validation for Robot Manipulation Policies
Jun 29, 2026Haoxu Huang, Tongsam Zheng, Yifan Chen +2Robot Policy Learning Off-Policy Evaluation
Tsinghua University · Shanghai Qi Zhi Institute · Spirit AI
Metric Aggregation Divergence: A Hidden Validity Threat in Agent-Based Policy Optimization and a Contractual Remedy
Jun 27, 2026Ruiyu Zhang, Lin Nie, Xin ZhaoSoftware Engineering Policy Evaluation
Department of Politics and Public Administration · Department of Applied Social Sciences
Minimax PAC Bounds for Learning in Exogenous Contextual MDPs
Jun 23, 2026Corentin Pla, Hugo Richard, Marc Abeille +1Markov Decision Processes Reinforcement Learning
CREST, ENSAE · Criteo AI Lab · FairPlay Joint Team
Fed-CausalDiff: Decoupled Synchronization for Federated Do-Simulation and Policy Evaluation
Jun 21, 2026Pengfei Li, Mohammad KhalilPolicy Evaluation Causal Inference
Centre for the Science of Learning & Technology (SLATE), University of Bergen Bergen, Norway
Causal Gaussian Processes for Robust Treatment Effect Evaluation with Unobserved Confounding
Jun 20, 2026Junzhe Zhang, Jingyuan Chen, Elias BareinboimCausal Effect Estimation Causal Inference with Unobserved Confounding
Department of Electrical Engineering and Computer Science, Syracuse University · Department of Computer Science, Columbia University
AI for Social Good: An Investigation of the Causal Relationship Between Environmental Regulations and Their Effects on Air Pollution in London, UK
Jun 13, 2026Yang Han, Jacqueline CK Lam, Victor OK Li +1Causal Effect Estimation Policy Evaluation
Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong
Evaluating AI Investment Strategies
Jun 7, 2026Irene AldridgeAlgorithmic Auditing Policy Evaluation
PiL-World: A Chunk-Wise World Model for VLA Policy-in-the-Loop Evaluation
Jun 4, 2026Chong Ma, Taiyi Su, Jian Zhu +4Action-Conditioned World Models Robotic Policy Evaluation
1Tongji University · 2AIRC, Midea Group
X4Val: Learning Neural Surrogates for Variance-Reduced Policy Evaluation
Jun 3, 2026Rachel Luo, Michael Watson, Apoorva Sharma +6Off-Policy Evaluation Robotic Policy Evaluation
NVIDIA Research, Santa Clara, CA, USA · Harvard University, Cambridge, MA, USA · Stanford University, Stanford, CA, USA
Margin Play: A Multi-Agent System For Public Policy Analysis In The Brazilian Equatorial Margin
May 26, 2026Antonio de Sousa Leitão Filho, Fabrício Saul Lima, Selby Mykael Lima dos Santos +4Policy Evaluation
Aia Context, São Luís, MA, Brazil · Universidade Federal do Maranhão — UFMA, São Luís, MA, Brazil · Universidade Estadual de Campinas — UNICAMP, Campinas, SP, Brazil
Decision Support for Marketplace Policies under Incomplete Evidence: From Replay to Launch Readiness
May 13, 2026Prashant Shekhar, Caroline HowardAI-Assisted Decision Making Off-Policy Evaluation
Department of Mathematics Embry-Riddle Aeronautical University, Daytona Beach, FL, USA
Bridging the Gap Between Average and Discounted TD Learning
May 3, 2026Haoxing Tian, Zaiwei Chen, Ioannis Ch. Paschalidis +1Reinforcement Learning Average-Reward RL
Department of Electrical and Computer Engineering Boston University · Edwardson School of Industrial Engineering Purdue University
Structural Consequences of Policy-Based Interventions on the Global Supply Chain Network
Apr 13, 2026Lea Karbevska, Liming Xu, Zehui Dai +2Supply Chain Management Policy Evaluation
Department of Engineering, University of Cambridge, Cambridge, United Kingdom · The Alan Turing Institute, London, United Kingdom
LLM-Augmented Digital Twin for Policy Evaluation in Short-Video Platforms
Mar 11, 2026Haoting Zhang, Yunduan Lin, Jinghai He +3Policy Evaluation Digital Twins
University of California, Berkeley · The Chinese University of Hong Kong · New York University +1
Accelerated Online Risk-Averse Policy Evaluation in POMDPs with Theoretical Guarantees and Novel CVaR Bounds
Feb 26, 2026Yaacov Pariente, Vadim IndelmanPartially Observable Markov Decision Processes Policy Evaluation
Technion - Israel Institute of Technology, Israel
PPCR-IM: A System for Multi-layer DAG-based Public Policy Consequence Reasoning and Social Indicator Mapping
Feb 25, 2026Zichen Song, Weijia LiPolicy Evaluation LLM Reasoning with Graphs
Lanzhou University
Denotational Semantics for ODRL: Knowledge-Based Constraint Conflict Detection
Feb 23, 2026Daham Mustafa, Diego Collarana, Yixin Peng +4Policy Evaluation Knowledge Representation
RWTH Aachen University, Germany · Fraunhofer FIT, Sankt Augustin, Germany · University of Galway, Ireland
Axis-Aligned Semantics for ODRL: Resolving Dimensional Ambiguity in Policy Constraints
Feb 23, 2026Daham Mustafa, Diego Collarana, Sabrina Kirrane +5Policy Evaluation Access Control
RWTH Aachen University, Aachen, Germany · Fraunhofer FIT, Sankt Augustin, Germany · Vienna University of Economics and Business (WU), Vienna, Austria +1
Decision Quality Evaluation Framework at Pinterest
Feb 17, 2026Yuqi Tian, Robert Paine, Attila Dobi +3LLM Safety Evaluation Content Moderation
Pinterest, San Francisco, CA, USA · Pinterest, New York, NY, USA · Pinterest, Richmond, VA, USA +1
Neural Evidence and Behavioural Welfare Economics: When Can Neuroeconomics Inform Policy?
Nov 24, 2025Louis Yiven ZhuPolicy Evaluation
Student (BSc, 2023-26), Dept. of Science and Technology Studies, UCL, UK
Statistical Inference for Policy Evaluation with Temporal Difference Learning
Oct 21, 2024Weichen Wu, Gen Li, Yuting Wei +1Parameter Estimation Confidence Region Estimation
The Voleon Group, New York, NY, USA · Department of Statistics, The Chinese University of Hong Kong, Hong Kong SAR. · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, Philadelphia, PA, USA. +1

Jul 25, 2026J/K move · Enter open · S save

Online Policy Evaluation for MDPs with Dynamic UBSR Measures

Weikai Wang, Erick Delage

GERAD & Department of Decision Sciences, HEC Montr´eal, Canada

Policy Evaluation Risk-Sensitive RL Temporal-Difference Learning

Open paper AskPDF arXiv

Developing efficient function-approximation methods for policy evaluation is a fundamental challenge in risk-aware reinforcement learning. Existing approaches either focus on restrictive classes of risk measures or rely on access to a simulator, limiting their applicability in fully online settings. In this work, we propose computationally efficient online learning algorithms for policy evaluation in Markov decision processes (MDPs) with dynamic utility-based shortfall risk (UBSR) measures under linear function approximation. Specifically, we introduce the UBSR-TD algorithm, establish conditions under which it converges almost surely, and develop several variants designed to accelerate convergence. Our formulation shows that existing policy evaluation algorithms for risk-neutral MDPs can be readily adapted to dynamic UBSR settings by incorporating a loss function into the temporal-difference error. Numerical experiments support our theoretical findings, and an application to a perishable inventory management problem with shelf-life uncertainty demonstrates the practical effectiveness of the proposed methods.