Model Based Reinforcement Learning

Model-based reinforcement learning (MBRL) aims to improve the sample efficiency and robustness of reinforcement learning agents by learning a model of the environment's dynamics. Current research focuses on enhancing model accuracy and robustness through techniques like incorporating expert knowledge, using bisimulation metrics for state representation, and employing adversarial training to handle uncertainties. These advancements are leading to more efficient and reliable control policies in various applications, including robotics, autonomous driving, and even protein design, by reducing the need for extensive real-world interactions during training.

Papers

May 22, 2022

Should Models Be Accurate?
Esra'a Saleh, John D. Martin, Anna Koop, Arash Pourzarabi, Michael Bowling
Full Model Meta Learning Model Based Reinforcement Learning Model Generated Environment Dynamic Dynamic Planning

May 20, 2022

Planning with Diffusion for Flexible Behavior Synthesis
Michael Janner, Yilun Du, Joshua B. Tenenbaum, Sergey Levine
Task Planning Diffusion Explainer Model Based Reinforcement Learning Trajectory Optimization Diffusion Planner Behavior Synthesis

May 9, 2022

Accelerated Reinforcement Learning for Temporal Logic Control Objectives
Yiannis Kantaros
Reinforcement Learning Markov Decision Process Model Based Reinforcement Learning Temporal Logic Temporal Logic Task

April 25, 2022

Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methods
Yi Wan, Ali Rahimi-Kalahroudi, Janarthanan Rajendran, Ida Momennejad, Sarath Chandar, Harm van Seijen
Model Based Reinforcement Learning Model Based Deep Learning Local Adaptivity Model Based Deep Reinforcement Learning Second Order Method

April 18, 2022

INFOrmation Prioritization through EmPOWERment in Visual Model-Based RL
Homanga Bharadhwaj, Mohammad Babaeizadeh, Dumitru Erhan, Sergey Levine
Model Based Reinforcement Learning Latent State Variational Empowerment Visual Model Based

April 4, 2022

Value Gradient weighted Model-Based Reinforcement Learning
Claas Voelcker, Victor Liao, Animesh Garg, Amir-massoud Farahmand
Model Based Reinforcement Learning Model Learning Valuation Model Value Learning

March 28, 2022

Revisiting Model-based Value Expansion
Daniel Palenicek, Michael Lutter, Jan Peters
Model Based Reinforcement Learning Value Function Value Expansion

March 17, 2022

Investigating Compounding Prediction Errors in Learned Dynamics Models
Nathan Lambert, Kristofer Pister, Roberto Calandra
Model Based Reinforcement Learning Robot Control State Action Pair Prediction Error Learned Dynamic Data Driven Control

March 9, 2022

SAGE: Generating Symbolic Goals for Myopic Models in Deep Reinforcement Learning
Andrew Chester, Michael Dann, Fabio Zambetta, John Thangarajah
Deep Reinforcement Learning Model Based Reinforcement Learning Free Counterpart Symbolic Planning Continual Task Hybrid Planning Weak Model Goal Generation

February 19, 2022

TransDreamer: Reinforcement Learning with Transformer World Models
Chang Chen, Yi-Fu Wu, Jaesik Yoon, Sungjin Ahn
Reinforcement Learning Model Based Reinforcement Learning Transformer Based Policy Transformer Based World Model

February 14, 2022

Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization
Mirco Mutti, Riccardo De Santi, Emanuele Rossi, Juan Felipe Calderon, Michael Bronstein, Marcello Restelli
Markov Decision Process Model Based Reinforcement Learning Sequential Decision Systematic Generalization Reward Free Novel Causal Model

February 12, 2022

Neural NID Rules
Luca Viano, Johanni Brea
Neural Network Model Based Reinforcement Learning Transition Dynamic

February 7, 2022

Reward-Respecting Subtasks for Model-Based Reinforcement Learning
Richard S. Sutton, Marlos C. Machado, G. Zacharias Holland, David Szepesvari, Finbarr Timbers, Brian Tanner, Adam White
Reinforcement Learning Model Based Reinforcement Learning Temporal Abstraction Average Reward Option Discovery

January 29, 2022

Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processes
Chao Qu, Xiaoyu Tan, Siqiao Xue, Xiaoming Shi, James Zhang, Hongyuan Mei
Reinforcement Learning Model Based Reinforcement Learning Sequential Decision Making Problem Temporal Point Process Hawkes Process Bellman Operator Asynchronous Stochastic

January 28, 2022

Joint Differentiable Optimization and Verification for Certified Reinforcement Learning
Yixuan Wang, Simon Zhan, Zhilu Wang, Chao Huang, Zhaoran Wang, Zhuoran Yang, Qi Zhu
Reinforcement Learning Proximal Policy Optimization Model Based Reinforcement Learning Verification Task Joint Optimization Learned Controller Novel Control

January 15, 2022

Physical Derivatives: Computing policy gradients by physical forward-propagation
Arash Mehrjou, Ashkan Soleymani, Stefan Bauer, Bernhard Schölkopf
Policy Gradient Model Based Reinforcement Learning Model Free Dynamic Model Physical System Backward Propagation Derivative Process Custom Robot

January 8, 2022

Assessing Policy, Loss and Planning Combinations in Reinforcement Learning using a New Modular Architecture
Tiago Gaspar Oliveira, Arlindo L. Oliveira
Reinforcement Learning Deep Reinforcement Learning Model Based Reinforcement Learning Interest Loss Health Policy Recommendation Reinforcement Learning Research Modular Architecture Planning Perspective

December 28, 2021

Exponential Family Model-Based Reinforcement Learning via Score Matching
Gene Li, Junbo Li, Anmol Kabra, Nathan Srebro, Zhaoran Wang, Zhuoran Yang
Model Based Reinforcement Learning O$ Regret Score Matching Ridge Regression Episodic Reinforcement Learning Efficient Estimation Optimistic Algorithm

December 19, 2021

Exploration-exploitation trade-off for continuous-time episodic reinforcement learning with linear-convex models
Lukasz Szpruch, Tanut Treetanthiploet, Yufei Zhang
Model Based Reinforcement Learning Finite Horizon Episodic Reinforcement Learning Exploration Exploitation Trade Performance Gap Convex Model

December 16, 2021

Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Critic
Zhihai Wang, Jie Wang, Qi Zhou, Bin Li, Houqiang Li
Model Based Reinforcement Learning Model Based Sample Efficient Reinforcement Learning Free Counterpart Accurate Learning Conservative Objective Model