Policy Prediction

Policy prediction aims to estimate the performance of a new policy using data collected under a different policy, a crucial task in scenarios where deploying a new policy directly is risky or impractical. Current research emphasizes robust off-policy prediction methods, often employing conformal prediction frameworks to provide prediction intervals with guaranteed confidence levels, and exploring advanced importance weighting techniques to mitigate the variance inherent in off-policy learning. These advancements are significant for improving the safety and reliability of reinforcement learning applications in diverse fields, from robotics and autonomous systems to personalized medicine and resource management.

Papers

March 25, 2024

Conformal Off-Policy Prediction for Multi-Agent Systems
Tom Kuipers, Renukanandan Tumu, Shuo Yang, Milad Kazemi, Rahul Mangharam, Nicola Paoletti
Multi Agent System Agent Trajectory Policy Prediction

November 2, 2023

Conformal Policy Learning for Sensorimotor Control Under Distribution Shifts
Huang Huang, Satvik Sharma, Antonio Loquercio, Anastasios Angelopoulos, Ken Goldberg, Jitendra Malik
Reinforcement Learning Distribution Shift Conformal Prediction Conformalized Quantile Regression Policy Prediction Sensorimotor Control

June 27, 2023

Value-aware Importance Weighting for Off-policy Reinforcement Learning
Kristopher De Asis, Eric Graves, Richard S. Sutton
Policy Reinforcement Learning Importance Sampling Importance Weighting Importance Weight Policy Prediction

April 5, 2023

Conformal Off-Policy Evaluation in Markov Decision Processes
Daniele Foffano, Alessio Russo, Alexandre Proutiere
Reinforcement Learning Markov Decision Process Conformal Prediction Policy Evaluation Efficient Control Policy Prediction

April 4, 2023

Off-Policy Action Anticipation in Multi-Agent Reinforcement Learning
Ariyan Bighashdel, Daan de Geus, Pavol Jancura, Gijs Dubbelman
Multi Agent Reinforcement Learning Action Anticipation Anticipation Model Policy Prediction

June 14, 2022

Conformal Off-policy Prediction
Yingying Zhang, Chengchun Shi, Shikai Luo
Conformal Prediction Policy Evaluation Prediction Interval Policy Prediction Interval Estimation

June 9, 2022

Conformal Off-Policy Prediction in Contextual Bandits
Muhammad Faaiz Taufiq, Jean-Francois Ton, Rob Cornish, Yee Whye Teh, Arnaud Doucet
Conformal Prediction Contextual Bandit Policy Evaluation Policy Prediction

May 17, 2022

Robust Losses for Learning Value Functions
Andrew Patterson, Victor Liao, Martha White
Reinforcement Learning Value Function Saddle Point Gradient Update Robust Loss Policy Prediction

January 17, 2022

Chaining Value Functions for Off-Policy Learning
Simon Schmitt, John Shawe-Taylor, Hado van Hasselt
Reinforcement Learning Policy Learning Value Function Policy Prediction