Sub Optimal Policy

Suboptimal policies in reinforcement learning (RL) and related fields represent policies that, while functional, do not achieve optimal performance. Current research focuses on mitigating the negative impacts of suboptimal policies through techniques like improved model architectures (e.g., incorporating adversarial and contrastive learning, backpropagation through agents), refined algorithms (e.g., generalized policy improvement prioritization, Wasserstein belief updates), and leveraging supplementary data or pre-trained policies to enhance learning efficiency. Addressing suboptimal policies is crucial for improving the sample efficiency and robustness of RL algorithms, ultimately leading to more effective applications in diverse areas such as resource allocation, epidemic control, and robotic control.

Papers

October 29, 2024

Solving Minimum-Cost Reach Avoid using Reinforcement Learning
Oswin So, Cheng Ge, Chuchu Fan
Reinforcement Learning Reach Avoid Sub Optimal Policy

May 3, 2024

Metalearners for Ranking Treatment Effects
Toon Vanderschueren, Wouter Verbeke, Felipe Moraes, Hugo Manuel Proença
Causal Inference Treatment Effect Big Gain Sub Optimal Policy Budget Allocation Metalearning Algorithm Allocation Policy

March 16, 2024

Game and Reference: Policy Combination Synthesis for Epidemic Prevention and Control
Zhiyi Tan, Bingkun Bao
External Control Game Content Higher Quality Reference Pandemic Control Policy Synthesis Sub Optimal Policy Policy Model Epidemic Decision

January 23, 2024

Backpropagation Through Agents
Zhiyuan Li, Wenshuai Zhao, Lijun Wu, Joni Pajarinen
Multi Agent Reinforcement Learning Multi Agent Back Propagation Agent Smith Proximal Policy Optimization Sub Optimal Policy Joint Policy

March 24, 2023

Causality Detection for Efficient Multi-Agent Reinforcement Learning
Rafael Pina, Varuna De Silva, Corentin Artaud
Multi Agent Reinforcement Learning Causal Effect Estimation Sub Optimal Policy Temporal Causal Causality Detection Lazy Agent

March 9, 2023

Beware of Instantaneous Dependence in Reinforcement Learning
Zhengmao Zhu, Yuren Liu, Honglong Tian, Yang Yu, Kun Zhang
Reinforcement Learning Optimal Policy Model Based Reinforcement Learning Linear Dependence Sub Optimal Policy

March 6, 2023

The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Models
Raphael Avalos, Florent Delgrange, Ann Nowé, Guillermo A. Pérez, Diederik M. Roijers
Latent Space Markov Decision Process Observable Environment POMDP Policy Wasserstein Dictionary Sub Optimal Policy Belief Update

January 27, 2023

Theoretical Analysis of Offline Imitation With Supplementary Dataset
Ziniu Li, Tian Xu, Yang Yu, Zhi-Quan Luo
Feature Imitation Behavior Cloning Theoretical Analysis Offline Imitation Sub Optimal Policy

January 18, 2023

Sample-Efficient Multi-Objective Learning via Generalized Policy Improvement Prioritization
Lucas N. Alegre, Ana L. C. Bazzan, Diederik M. Roijers, Ann Nowé, Bruno C. da Silva
Active Learning Multi Objective Multi Objective Reinforcement Learning Sequential Decision Making Problem Sub Optimal Policy Generalized Policy Improvement

November 21, 2022

Improving TD3-BC: Relaxed Policy Constraint for Offline Learning and Stable Online Fine-Tuning
Alex Beeson, Giovanni Montana
Reinforcement Learning Behavior Cloning Optimal Behavior Policy Constraint Sub Optimal Policy

November 8, 2022

Doubly Inhomogeneous Reinforcement Learning
Liyuan Hu, Mengbing Li, Chengchun Shi, Zhenke Wu, Piotr Fryzlewicz
Policy Learning Change Point Detection Sub Optimal Policy Testing Stationarity

September 17, 2022

Sub-optimal Policy Aided Multi-Agent Reinforcement Learning for Flocking Control
Yunbo Qiu, Yue Jin, Jian Wang, Xudong Zhang
Multi Agent Reinforcement Learning Traditional Reinforcement Learning Flocking Control Sub Optimal Policy

August 13, 2022

Lyapunov Design for Robust and Efficient Robotic Reinforcement Learning
Tyler Westenbroek, Fernando Castaneda, Ayush Agrawal, Shankar Sastry, Koushil Sreenath
Reinforcement Learning Robust Version Lyapunov Function Model Based Efficient Reinforcement Learning Control Lyapunov Function Sub Optimal Policy

July 11, 2022

Grounding Aleatoric Uncertainty for Unsupervised Environment Design
Minqi Jiang, Michael Dennis, Jack Parker-Holder, Andrei Lupu, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel, Jakob Foerster
Reinforcement Learning Optimal Policy Covariate Shift Aleatoric Uncertainty Minimax Optimal Regret Unsupervised Environment Design Sub Optimal Policy

June 2, 2022

Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm
Semih Cayci, Niao He, R. Srikant
Markov Decision Process Policy OpTimization Finite Time Global Optimality Entropy Regularized Regularized Reinforcement Learning Sub Optimal Policy

May 6, 2022

How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation
Alex X. Lee, Coline Devin, Jost Tobias Springenberg, Yuxiang Zhou, Thomas Lampe, Abbas Abdolmaleki, Konstantinos Bousmalis
Reinforcement Learning Offline Reinforcement Learning Reinforcement Learning Algorithm Vision Based Sub Optimal Policy Action Distribution Manipulation Benchmark Robot Time