Overview

Off-policy evaluation (OPE) estimates how a new decision policy would perform using data collected by a different policy, avoiding costly or risky deployment trials. Common approaches reweight logged actions with importance sampling, combine weights with reward models in doubly robust estimators, fit value functions with fitted Q-evaluation, or learn dynamics and occupancy ratios; bootstrap and conformal methods help quantify uncertainty. Applications span contextual bandits and sequential decision processes, including recommendation, healthcare, robotics, and language-model agents, with benchmarks testing policy shift, sparse support, and missing outcomes. Estimates can become biased or unstable when target actions or histories are poorly represented, so overlap diagnostics, sensitivity checks, and calibrated uncertainty matter alongside point estimates.

Latest papers

CardsList

82 papers10 papers in the last four weeks, down 44% on the four weeks before. 0.1% of all new papers.

  1. Action-outcome coupling: a pre-investment feasibility audit for offline reinforcement learning in supply chain operations

    Nov 17, 2026Tung Dang, The Hung Phung, Son Lam Nguyen +1Offline RLAlgorithmic Auditing

  2. Off-policy Learning with Excursion Policies

    Nov 9, 2026Jiamin He, Mark Rowland, Zhaohan Daniel Guo +2Off-Policy RLOff-Policy Evaluation

  3. Variational Approach to Optimal IPS Estimator for Multi-logger Off-Policy Evaluation

    Nov 9, 2026Joon Suk Huh, Junghoon SeoOff-Policy EvaluationInverse Probability Weighting

  4. JUBILEE: A Benchmark Environment for Off-Policy Evaluation in Debt Collections

    Nov 7, 2026Diego Silva Prado, Tiago Amaral Penteado de Jesus, Bruno Caldas de Souza +1Off-Policy EvaluationDoubly Robust Estimation

  5. Intra-Option Fitted Q-Evaluation: Evaluating Hierarchical Policies from Non-Hierarchical Data

    Nov 7, 2026Yunfu Deng, Josiah P. HannaOff-Policy EvaluationHierarchical RL

  6. Adaptive Budgeted Data Acquisition for Robust Off-Policy Evaluation from Multiple Loggers

    Nov 6, 2026Amitakshar Biswas, Ruoqing ZhuOff-Policy EvaluationAdaptive Sampling

  7. High-Dimensional Off-Policy Evaluation without Sample Splitting using an Onsager Correction

    Nov 5, 2026Saksham Jain, Guillaume Braun, Masaaki ImaizumiOff-Policy EvaluationReward Model Evaluation

  8. Reinforcement Learning Under Runtime Confounding: Policy Learning and Off-Policy Evaluation

    Oct 29, 2026Raphael Kim, Michele Santacatterina, Ramin Zabih +1Off-Policy EvaluationCausal RL

  9. Model Selection for Off-Policy Evaluation via Bellman Tournaments

    Oct 26, 2026Lars van der Laan, Vasilis Syrgkanis, Nathan KallusOff-Policy EvaluationValue Function Estimation

  10. Prior-Data Fitted Networks for In-Context Off-Policy Evaluation

    Oct 25, 2026Shuguang Yu, Ting Li, Fan ZhouOff-Policy EvaluationPrior-Data Fitted Networks

  11. Doubly Robust Distributionally Robust Offline Contextual Pricing

    Oct 22, 2026Min Xu, Xinyi Yin, Yunfan Zhang +3Off-Policy EvaluationDistributionally Robust Optimization

  12. Explore Once, Evaluate Many: Multiple Policy Evaluation for Stochastic Linear Bandits

    Oct 15, 2026Spyros Dragazis, Yilei Chen, Aldo PacchianoPolicy EvaluationLinear Contextual Bandits

  13. RoboFQE: Rollout-Free Evaluation of Visual Robot Policies from Offline Trajectories

    Oct 13, 2026Yunfu Deng, Sangwoo Shin, Brahma S. Pavse +1Robotic Policy EvaluationOff-Policy Evaluation

  14. agent-risk: Risk Assessment for Changes to AI Agents

    Oct 12, 2026Yue Zhao, Zixiang Xu, Angela Huyue Zhang +2AI Risk ManagementAI Agent Safety

  15. What a Data Release Must Preserve for Off-Policy Evaluation

    Oct 11, 2026Kyungbok Lee, Yating Zou, Michael R. KosorokOff-Policy EvaluationPartial Identification

  16. Budgeted Multi-Source Counterfactual Annotation for Off-Policy Evaluation

    Oct 7, 2026Biao Xiang, Ali Eshragh, Yuexing Li +1Off-Policy EvaluationCounterfactual Evaluation

  17. Amortized Off-Policy Evaluation for LLMs

    Oct 7, 2026Younwoo Choi, Leo Feng, Vincent Liu +1Off-Policy EvaluationLLM Evaluation

  18. Efficient Best-of-N policy evaluation for inference-time alignment

    Oct 7, 2026Jonas Schweisthal, Yuxin Wang, Athiya Deviyani +2Best-of-N SamplingOff-Policy Evaluation

  19. Variance-Optimal Off-Policy Evaluation with Conjunct Effect Modeling

    Oct 6, 2026Nicolò Felicioni, Michael Benigni, Maurizio Ferrari Dacrema +1Causal Effect EstimationOff-Policy Evaluation

  20. Evaluation of Active Feature Acquisition Policies with Tabular Foundation Models

    Oct 5, 2026Yuta Kobayashi, Divyam Madaan, Shalmali JoshiOff-Policy EvaluationValue Function Estimation

  21. Revealing After Overwriting: An Exponential POMDP OPE Lower Bound under History-Dependent Logging

    Oct 4, 2026Youyu Luo, Pengzhan Zhou, Zhida Qin +4Off-Policy EvaluationPartially Observable Markov Decision Processes

  22. OTROPE: Optimal Transport-based Robust Off-policy Evaluation for Large Language Models

    Sep 28, 2026Liner Xiang, Wenbo Zhang, Hengrui CaiLLM EvaluationOff-Policy Evaluation

  23. Recommendation Ranking Off-Policy Evaluation under Ranking-Dependent Examination via Examination-Relevance Decomposition

    Sep 28, 2026Riki Okamura, Toshiharu SugawaraOff-Policy EvaluationDoubly Robust Estimation

  24. Deep Weighted Bellman Residual Minimization for Q∗Q^* Estimation

    Sep 28, 2026Lican Kang, Jerry Zhijian Yang, Cheng Yuan +1Off-Policy EvaluationValue Function Estimation

  25. Offline Policy Evaluation via Mixed Bellman Residuals and Adaptive Critic Representations

    Sep 27, 2026Amitakshar Biswas, Yuhan Li, Ruoqing ZhuOff-Policy EvaluationRepresentation Learning for RL

  26. BranchDrive: A Branch-Structured Dataset for Action-Conditioned Driving Prediction

    Sep 23, 2026Feeza Khan Khanzada, Sudarshan Sridhar, Jaerock KwonAutonomous Driving DatasetsAutonomous Driving Benchmarks

  27. Optimal Expert Annotation for Evaluating Sequential Decision Policies from Text Records

    Sep 22, 2026Woojin Chae, Ezinne Nwankwo, Haitong Qin +1Off-Policy EvaluationLearning with Missing Data