Policy Learning

Policy learning, a core area of reinforcement learning, aims to develop algorithms that enable agents to learn optimal decision-making strategies from data, often without explicit reward functions. Current research emphasizes improving sample efficiency and robustness, particularly in offline settings, using techniques like generative adversarial imitation learning (GAIL), transformer-based architectures, and model-based methods that incorporate world models or causal representations to handle noisy or incomplete data. These advancements are crucial for scaling reinforcement learning to complex real-world problems, such as robotics and personalized recommendations, where online learning is impractical or unsafe. The development of more efficient and robust policy learning algorithms has significant implications for various fields, improving the performance and generalizability of AI agents in diverse applications.

Papers

June 16, 2023

Fairness in Preference-based Reinforcement Learning
Umer Siddique, Abhinav Sinha, Yongcan Cao
Procedural Fairness Policy Learning Vector Valued Preference Based Reinforcement Learning Preference Based Reward

June 15, 2023

June 13, 2023

Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective
Zeyu Zhang, Yi Su, Hui Yuan, Yiran Wu, Rishab Balasubramanian, Qingyun Wu, Huazheng Wang, Mengdi Wang
Offline Reinforcement Learning Policy Learning Stable Rank Reinforcement Learning Approach Offline Reinforcement Learning Algorithm Click Model

June 11, 2023

Learning World Models with Identifiable Factorization
Yu-Ren Liu, Biwei Huang, Zhengmao Zhu, Honglong Tian, Mingming Gong, Yang Yu, Kun Zhang
Ground Truth Policy OpTimization World Model Policy Learning Efficient Reinforcement Learning

June 5, 2023

May 29, 2023

DoMo-AC: Doubly Multi-step Off-policy Actor-Critic Algorithm
Yunhao Tang, Tadashi Kozuno, Mark Rowland, Anna Harutyunyan, Rémi Munos, Bernardo Ávila Pires, Michal Valko
Policy Gradient Policy Learning Policy Improvement Policy Actor Critic Multi Step Off Policy

May 27, 2023

Query-Policy Misalignment in Preference-Based Reinforcement Learning
Xiao Hu, Jianxiong Li, Xianyuan Zhan, Qing-Shan Jia, Ya-Qin Zhang
Reinforcement Learning Policy Learning Preference Based Reinforcement Learning Policy Alignment

May 25, 2023

Exponential Smoothing for Off-Policy Learning
Imad Aouali, Victor-Emmanuel Brunel, David Rohde, Anna Korba
Policy Learning PAC Bayesian Inverse Propensity Exponential Smoothing Smoothness Regularization Full Bandit

May 21, 2023

Federated Offline Policy Learning with Heterogeneous Observational Data
Aldo Gael Carranza, Susan Athey
Policy Learning Regret Analysis Federated Offline Reinforcement Learning

May 8, 2023

The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation
Jan Ole von Hartz, Eugenio Chisari, Tim Welschehold, Wolfram Burgard, Joschka Boedecker, Abhinav Valada
Robotic Manipulation Policy Learning Sparse Keypoints Real Robot Experiment Keypoint Tracking Betrayal Detection

April 12, 2023

Exploiting Symmetry and Heuristic Demonstrations in Off-policy Reinforcement Learning for Robotic Manipulation
Amir M. Soufi Enayati, Zengjie Zhang, Kashish Gupta, Homayoun Najjaran
Reinforcement Learning Robotic Manipulation Policy Learning Behavior Cloning Policy Reinforcement Learning

April 5, 2023

Learn to Grasp via Intention Discovery and its Application to Challenging Clutter
Chao Zhao, Chunli Jiang, Junhao Cai, Hongyu Yu, Michael Yu Wang, Qifeng Chen
Application Proficiency Policy Learning Functional Grasping Grasp Anything Dense Clutter Grasping Policy Exploration Based Learning

March 20, 2023

A Unified Framework of Policy Learning for Contextual Bandit with Confounding Bias and Missing Observations
Siyu Chen, Yitan Wang, Zhaoran Wang, Zhuoran Yang
Optimal Policy Policy Learning Unobserved Confounders Partial Observation Confounding Bias Offline Contextual Bandit

March 11, 2023

Uncertainty-Aware Instance Reweighting for Off-Policy Learning
Xiaoying Zhang, Junpu Chen, Hongning Wang, Hong Xie, Yang Liu, John C. S. Lui, Hang Li
Policy OpTimization Policy Learning Recommendation Datasets Inverse Propensity Instance Weighting

March 5, 2023

Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching
Lantao Yu, Tianhe Yu, Jiaming Song, Willie Neiswanger, Stefano Ermon
Policy Learning Distribution Matching Offline Imitation Offline Imitation Learning Suboptimal Demonstration Divergence Regularization State Action Distribution

February 20, 2023

Backstepping Temporal Difference Learning
Han-Dong Lim, Donghwan Lee
Reinforcement Learning Policy Learning Temporal Difference Learning Temporal Difference TD Learning

February 18, 2023

Natural Language-conditioned Reinforcement Learning with Inside-out Task Language Development and Translation
Jing-Cheng Pang, Xin-Yu Yang, Si-Hang Yang, Yang Yu
Reinforcement Learning Policy Learning View Translation Language Conditioned Task Language

February 7, 2023

Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability
Hanlin Zhu, Amy Zhang
Policy Learning Goal Conditioned Reinforcement Learning Offline Goal Conditioned Reinforcement Learning Single Policy Concentrability

Policy Learning

Papers

Fairness in Preference-based Reinforcement Learning

Simplified Temporal Consistency Reinforcement Learning

ChessGPT: Bridging Policy Learning and Language Modeling

Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective

Learning World Models with Identifiable Factorization

Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic

Tackling Non-Stationarity in Reinforcement Learning via Causal-Origin Representation

DoMo-AC: Doubly Multi-step Off-policy Actor-Critic Algorithm

Query-Policy Misalignment in Preference-Based Reinforcement Learning

Exponential Smoothing for Off-Policy Learning

Federated Offline Policy Learning with Heterogeneous Observational Data

The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation

Exploiting Symmetry and Heuristic Demonstrations in Off-policy Reinforcement Learning for Robotic Manipulation

Learn to Grasp via Intention Discovery and its Application to Challenging Clutter

A Unified Framework of Policy Learning for Contextual Bandit with Confounding Bias and Missing Observations

Uncertainty-Aware Instance Reweighting for Off-Policy Learning

Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching

Backstepping Temporal Difference Learning

Natural Language-conditioned Reinforcement Learning with Inside-out Task Language Development and Translation

Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability