Policy Learning

Policy learning, a core area of reinforcement learning, aims to develop algorithms that enable agents to learn optimal decision-making strategies from data, often without explicit reward functions. Current research emphasizes improving sample efficiency and robustness, particularly in offline settings, using techniques like generative adversarial imitation learning (GAIL), transformer-based architectures, and model-based methods that incorporate world models or causal representations to handle noisy or incomplete data. These advancements are crucial for scaling reinforcement learning to complex real-world problems, such as robotics and personalized recommendations, where online learning is impractical or unsafe. The development of more efficient and robust policy learning algorithms has significant implications for various fields, improving the performance and generalizability of AI agents in diverse applications.

Papers

July 4, 2024

July 1, 2024

Coordination Failure in Cooperative Offline MARL
Callum Rhys Tilbury, Claude Formanek, Louise Beyers, Jonathan P. Shock, Arnu Pretorius
Multi Agent Reinforcement Learning Policy Learning Action Free Offline Multi Agent Policy Coordination Problem Multi Agent Optimal Control

June 23, 2024

Bounding-Box Inference for Error-Aware Model-Based Reinforcement Learning
Erin J. Talvitie, Zilei Shao, Huiying Li, Jinghan Hu, Jacob Boerma, Rory Zhao, Xintong Wang
Reinforcement Learning Model Based Reinforcement Learning Policy Learning Efficient Planning

June 16, 2024

Improving Reward-Conditioned Policies for Multi-Armed Bandits using Normalized Weight Functions
Kai Xu, Farid Tajaddodianfar, Ben Allison
Reinforcement Learning Multi Armed Bandit Policy Gradient Policy Learning Weight Normalization Reward Conditioned

June 13, 2024

Is Value Learning Really the Main Bottleneck in Offline RL?
Seohong Park, Kevin Frans, Sergey Levine, Aviral Kumar
Reinforcement Learning Imitation Learning Offline Reinforcement Learning Policy Learning Offline Reinforcement Learning Algorithm Major Challenge Bottleneck Value Learning

May 29, 2024

Efficient Preference-based Reinforcement Learning via Aligned Experience Estimation
Fengshuo Bai, Rui Zhao, Hongming Zhang, Sijia Cui, Ying Wen, Yaodong Yang, Bo Xu, Lei Han
Reinforcement Learning Policy Learning Preference Based Reinforcement Learning Policy Regularization Experience Pool Reward Estimation Accuracy

May 25, 2024

Diffusion-Reward Adversarial Imitation Learning
Chun-Mao Lai, Hsiang-Chun Wang, Ping-Chun Hsieh, Yu-Chiang Frank Wang, Min-Hung Chen, Shao-Hua Sun
Diffusion Model Policy Learning Generative Training Generative Adversarial Imitation Learning

May 21, 2024

Efficient Imitation Learning with Conservative World Models
Victor Kolev, Rafael Rafailov, Kyle Hatch, Jiajun Wu, Chelsea Finn
Imitation Learning Policy Learning Adversarial Imitation Policy Sampling

May 20, 2024

Feasibility Consistent Representation Learning for Safe Reinforcement Learning
Zhepeng Cen, Yihang Yao, Zuxin Liu, Ding Zhao
Safe Reinforcement Learning Policy Learning Consistent Representation Learning Safety Learning

May 9, 2024

Optimal Baseline Corrections for Off-Policy Contextual Bandits
Shashank Gupta, Olivier Jeunen, Harrie Oosterhuis, Maarten de Rijke
Policy Learning Optimal Estimation Control Variate Unbiased Prediction Unbiased Estimator Optimal Baseline

May 6, 2024

Out-of-Distribution Adaptation in Offline RL: Counterfactual Reasoning via Causal Normalizing Flows
Minjae Cho, Jonathan P. How, Chuangchuang Sun
Offline Reinforcement Learning Policy Learning Counterfactual Reasoning Offline Policy Evaluation Causal Reinforcement Learning Graphical Normalizing Flow Graph Out of Distribution Adaptation

April 24, 2024

ViViDex: Learning Vision-based Dexterous Manipulation from Human Videos
Zerui Chen, Shizhe Chen, Etienne Arlaud, Ivan Laptev, Cordelia Schmid
Reinforcement Learning Vision Based Policy Learning Visual Policy

April 23, 2024

Hyperparameter Optimization Can Even be Harmful in Off-Policy Learning and How to Deal with It
Yuta Saito, Masahiro Nomura
Hyperparameter Optimization Policy Evaluation Policy Learning Optimal Hyperparameters Hyperparameter OPtimization

April 18, 2024

Enabling Stateful Behaviors for Diffusion-based Policy Learning
Xiao Liu, Fabian Weigend, Yifan Zhou, Heni Ben Amor
Imitation Learning Policy Learning Diffusion Policy Action Generation State Action

April 15, 2024

Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learning
Tidiane Camaret Ndir, André Biedenkapp, Noor Awad
Reinforcement Learning Zero Shot Context Learning Policy Learning Contextual Representation Behavior Style

April 12, 2024

Hindsight PRIORs for Reward Learning from Human Preferences
Mudit Verma, Katherine Metcalf
Reinforcement Learning Policy Learning Human Preference Reward Learning Hindsight Instruction Reward Engineering

March 31, 2024

Learning Off-policy with Model-based Intrinsic Motivation For Active Online Exploration
Yibo Wang, Jiang Zhao
Deep Reinforcement Learning Policy Learning Sparse Reward Sample Efficient Intrinsic Motivation

March 22, 2024

Planning with a Learned Policy Basis to Optimally Solve Complex Tasks
Guillermo Infante, David Kuric, Anders Jonsson, Vicenç Gómez, Herke van Hoof
Reinforcement Learning Task Planning Policy Learning Sequential Decision Making Problem Complex Task Stochastic Environment Global Optimality Non Markovian Reward

March 21, 2024

Constrained Reinforcement Learning with Smoothed Log Barrier Function
Baohe Zhang, Yuan Zhang, Lilli Frison, Thomas Brox, Joschka Bödecker
Reinforcement Learning Policy Learning Reward Shaping Constrained Reinforcement Learning Soft Actor Critic Control Constraint Logarithmic Barrier