Standard Reinforcement Learning

Standard reinforcement learning (RL) aims to train agents to make optimal decisions in an environment by learning a policy that maximizes cumulative rewards. Current research emphasizes improving RL's efficiency and safety, focusing on areas like hierarchical RL for complex tasks, safe exploration techniques to prevent unsafe actions during training, and incorporating human feedback to align agent behavior with human preferences. These advancements are crucial for deploying RL in real-world applications, particularly in robotics and human-computer interaction, where safety and efficient learning are paramount.

Papers

November 7, 2024

Enabling Adaptive Agent Training in Open-Ended Simulators by Targeting Diversity
Robby Costales, Stefanos Nikolaidis
Open Source Diversity Awareness Adaptive Behavior Standard Reinforcement Learning Adaptive Agent Agent Training Meta Reinforcement

October 24, 2024

Learning Transparent Reward Models via Unsupervised Feature Selection
Daulet Baimukashev, Gokhan Alcan, Kevin Sebastian Luck, Ville Kyrki
Inverse Reinforcement Learning Reward Model Behavior Cloning Dimensional State Space Standard Reinforcement Learning Unsupervised Feature Selection

October 15, 2024

Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents
Federico Pizarro Bejarano, Lukas Brunke, Angela P. Schoellig
Reinforcement Learning Training Data High Efficiency System Performance Reinforcement Learning Agent Safety Filter Standard Reinforcement Learning

September 2, 2024

Revisiting Safe Exploration in Safe Reinforcement learning
David Eckel, Baohe Zhang, Joschka Bödecker
Safe Reinforcement Learning Safe Exploration Standard Reinforcement Learning

July 21, 2024

Temporal Abstraction in Reinforcement Learning with Offline Data
Ranga Shaarad Ayyagari, Anurita Ghosh, Ambedkar Dukkipati
Reinforcement Learning Hierarchical Reinforcement Learning Offline Data Multi Agent MuJoCo Standard Reinforcement Learning Temporal Abstraction

June 16, 2024

Toward Optimal LLM Alignments Using Two-Player Games
Rui Zheng, Hongyi Guo, Zhihan Liu, Xiaoying Zhang, Yuanshun Yao, Xiaojun Xu, Zhaoran Wang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Hang Li, Yang Liu
LLM Alignment Adversarial Agent Two Player Standard Reinforcement Learning Two Agent Iterative Interaction

May 29, 2024

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF
Shicong Cen, Jincheng Mei, Katayoon Goshvadi, Hanjun Dai, Tong Yang, Sherry Yang, Dale Schuurmans, Yuejie Chi, Bo Dai
Reinforcement Learning Unified Framework Preference Optimization Reinforcement Learning From Human Feedback Online Service Standard Reinforcement Learning Implicit Reward Policy Parametrization

March 21, 2024

Task-optimal data-driven surrogate models for eNMPC via differentiable simulation and optimization
Daniel Mayfrank, Na Young Ahn, Alexander Mitsos, Manuel Dahmen
Optimization Purpose Surrogate Model Differentiable Simulation End to End Learning Mechanistic Model Data Driven Surrogate Standard Reinforcement Learning Surrogate Dynamic

March 18, 2024

Reinforcement Learning from Delayed Observations via World Models
Armin Karamzade, Kyungmin Kim, Montek Kalsi, Roy Fox
Reinforcement Learning World Model Partial Observability C POMDPs Standard Reinforcement Learning Perception Latency

March 13, 2024

Optimizing Risk-averse Human-AI Hybrid Teams
Andrew Fuchs, Andrea Passarella, Marco Conti
AI System Human AI Team Heterogeneous Team Standard Reinforcement Learning Delegation Decision

March 11, 2024

The Cram Method for Efficient Simultaneous Learning and Evaluation
Zeyang Jia, Kosuke Imai, Michael Lingzhi Li
Global Evaluation Standard Reinforcement Learning Simultaneous Learning Generic Machine Learning

December 15, 2023

Peer Learning: Learning Complex Policies in Groups from Scratch via Action Recommendations
Cedric Derstroff, Mattia Cerrato, Jannis Brugger, Jan Peters, Stefan Kramer
Multi Armed Bandit Group Setting Scratch Project Policy Design Standard Reinforcement Learning Peer Learning Single Agent Learning Action Recommendation

November 29, 2023

Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
Lei Zhao, Mengdi Wang, Yu Bai
Reward Function Inverse Reinforcement Learning Standard Reinforcement Learning Intermediate Target

November 9, 2023

Accelerating Exploration with Unlabeled Prior Data
Qiyang Li, Jason Zhang, Dibya Ghosh, Amy Zhang, Sergey Levine
Unlabeled Data Sparse Reward Standard Reinforcement Learning Rapid Exploration

October 5, 2023

Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithms
Akifumi Wachi, Wataru Hashimoto, Xun Shen, Kazumune Hashimoto
Reinforcement Learning Practical Algorithm Safe Exploration Standard Reinforcement Learning

September 27, 2023

Evaluation of Constrained Reinforcement Learning Algorithms for Legged Locomotion
Joonho Lee, Lukas Schroth, Victor Klemm, Marko Bjelonic, Alexander Reske, Marco Hutter
Deep Reinforcement Learning Global Evaluation Constrained Reinforcement Learning Legged Locomotion Constrained Policy Optimization Standard Reinforcement Learning Constrained Reinforcement Learning Algorithm

August 30, 2023

Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMO
Yangkun Chen, Joseph Suarez, Junjie Zhang, Chenghui Yu, Bo Wu, Hanmo Chen, Hengman Zhu, Rui Du, Shanliang Qian, Shuai Liu, Weijun Hong, Jinke He, Yibing Zhang, Liang Zhao, Clare Zhu, Julian Togelius, Sharada Mohanty, Jiaxin Chen, Xiu Li, Xiaolong Zhu, Phillip Isola
Strong Generalization Case Study Multi Agent System Robustness Benchmark Standard Reinforcement Learning Neural MMO

June 25, 2023

Is RLHF More Difficult than Standard RL?
Yuanhao Wang, Qinghua Liu, Chi Jin
Reinforcement Learning Reinforcement Learning From Human Feedback Robust Reinforcement Learning Standard Reinforcement Learning

March 24, 2023

Learning to Operate in Open Worlds by Adapting Planning Models
Wiktor Piotrowski, Roni Stern, Yoni Sher, Jacob Le, Matthew Klenk, Johan deKleer, Shiwali Mohan
LeArning Abstract Open World Heuristic Search Action Selection Domain Model Standard Reinforcement Learning Agent Planning

February 1, 2023

Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learning
Zun Li, Marc Lanctot, Kevin R. McKee, Luke Marris, Ian Gemp, Daniel Hennes, Paul Muller, Kate Larson, Yoram Bachrach, Michael P. Wellman
Generative Model Game Theoretic Tree Search Standard Reinforcement Learning Negotiation Game Nash Bargaining Policy Space Response Oracle Meta Solver

Standard Reinforcement Learning

Papers

Enabling Adaptive Agent Training in Open-Ended Simulators by Targeting Diversity

Learning Transparent Reward Models via Unsupervised Feature Selection

Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents

Revisiting Safe Exploration in Safe Reinforcement learning

Temporal Abstraction in Reinforcement Learning with Offline Data

Toward Optimal LLM Alignments Using Two-Player Games

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF

Task-optimal data-driven surrogate models for eNMPC via differentiable simulation and optimization

Reinforcement Learning from Delayed Observations via World Models

Optimizing Risk-averse Human-AI Hybrid Teams

The Cram Method for Efficient Simultaneous Learning and Evaluation

Peer Learning: Learning Complex Policies in Groups from Scratch via Action Recommendations

Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective

Accelerating Exploration with Unlabeled Prior Data

Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithms

Evaluation of Constrained Reinforcement Learning Algorithms for Legged Locomotion

Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMO

Is RLHF More Difficult than Standard RL?

Learning to Operate in Open Worlds by Adapting Planning Models

Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learning