Model Based Reinforcement Learning

Model-based reinforcement learning (MBRL) aims to improve the sample efficiency and robustness of reinforcement learning agents by learning a model of the environment's dynamics. Current research focuses on enhancing model accuracy and robustness through techniques like incorporating expert knowledge, using bisimulation metrics for state representation, and employing adversarial training to handle uncertainties. These advancements are leading to more efficient and reliable control policies in various applications, including robotics, autonomous driving, and even protein design, by reducing the need for extensive real-world interactions during training.

Papers

May 22, 2023

TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matching
Yecheng Jason Ma, Kausik Sivakumar, Jason Yan, Osbert Bastani, Dinesh Jayaraman
LeArning Abstract Model Based Reinforcement Learning Continuous Control Task State Occupancy Matching Reinforcement Learning Objective

May 8, 2023

Sense, Imagine, Act: Multimodal Perception Improves Model-Based Reinforcement Learning for Head-to-Head Autonomous Racing
Elena Shrestha, Chetan Reddy, Hanxi Wan, Yulun Zhuang, Ram Vasudevan
Model Based Reinforcement Learning Autonomous Racing Multimodal Agent Multiple Sens Multimodal Perception Head to Head Autonomous Racing

May 5, 2023

A Survey on Offline Model-Based Reinforcement Learning
Haoyang He
Timely Survey Offline Reinforcement Learning Model Based Reinforcement Learning Model Based Action Free Offline Model Based Offline Reinforcement Learning

May 3, 2023

Human Machine Co-adaption Interface via Cooperation Markov Decision Process System
Kairui Guo, Adrian Cheng, Yaqi Li, Jun Li, Rob Duffield, Steven W. Su
Markov Decision Process Model Based Reinforcement Learning Co Adaptation

April 29, 2023

Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learning
Liangyu Zhang, Yang Peng, Wenhao Yang, Zhihua Zhang
Model Based Reinforcement Learning Efficient Reinforcement Learning Constrained Markov Decision Process

April 26, 2023

FLEX: an Adaptive Exploration Algorithm for Nonlinear Systems
Matthieu Blanke, Marc Lelarge
Model Based Reinforcement Learning Nonlinear Dynamic Nonlinear Model Active Exploration Dynamic Control Adaptive Exploration

April 19, 2023

April 6, 2023

Decision-Focused Model-based Reinforcement Learning for Reward Transfer
Abhishek Sharma, Sonali Parbhoo, Omer Gottesman, Finale Doshi-Velez
Model Based Reinforcement Learning Multiple Objective MDP Model

April 1, 2023

Tracker: Model-based Reinforcement Learning for Tracking Control of Human Finger Attached with Thin McKibben Muscles
Daichi Saito, Eri Nagatomo, Jefferson Pardomuan, Hideki Koike
Model Based Reinforcement Learning External Tracker Exoskeleton Technology Human Finger MuSCLe Length Tracking Control Link Manipulator Hand Exoskeleton

March 31, 2023

Models as Agents: Optimizing Multi-Step Predictions of Interactive Local Models in Model-Based Multi-Agent Reinforcement Learning
Zifan Wu, Chao Yu, Chen Chen, Jianye Hao, Hankz Hankui Zhuo
Full Model Multi Agent Multi Agent System Agent Smith Model Based Reinforcement Learning Local Model Step Prediction Model Rollouts Model Based Multi Agent Reinforcement

March 27, 2023

Model-Based Reinforcement Learning with Isolated Imaginations
Minting Pan, Xiangming Zhu, Yitao Zheng, Yunbo Wang, Xiaokang Yang
Autonomous Driving Model Based Reinforcement Learning Long Horizon Reinforcement Learning Model Transition Model Latent Imagination

March 22, 2023

EDGI: Equivariant Diffusion for Planning with Embodied Agents
Johann Brehmer, Joey Bose, Pim de Haan, Taco Cohen
Model Based Reinforcement Learning Conditional Generative Interaction Generation Embodied Agent Equivariant Diffusion Equivariant Filter

March 17, 2023

Dynamic Update-to-Data Ratio: Minimizing World Model Overfitting
Nicolai Dorka, Tim Welschehold, Wolfram Burgard
Reinforcement Learning Model Based Reinforcement Learning World Model Related Hyperparameters DeepMind Control Suite Dynamic Update to Data Ratio

March 15, 2023

March 13, 2023

Transformer-based World Models Are Happy With 100k Interactions
Jan Robine, Marc Höftmann, Tobias Uelwer, Stefan Harmeling
Reinforcement Learning Model Based Reinforcement Learning World Model Interaction Strategy Transformer Based World Model

March 12, 2023

Predictive Experience Replay for Continual Visual Control and Forecasting
Wendong Zhang, Geng Chen, Xiangming Zhu, Siyu Gao, Yunbo Wang, Xiaokang Yang
Continual LEArning Model Based Reinforcement Learning State of the Art Forecasting Experience Replay Object Prior Visual Dynamic Visual Control

March 9, 2023

Beware of Instantaneous Dependence in Reinforcement Learning
Zhengmao Zhu, Yuren Liu, Honglong Tian, Yang Yu, Kun Zhang
Reinforcement Learning Optimal Policy Model Based Reinforcement Learning Linear Dependence Sub Optimal Policy

March 7, 2023

Diminishing Return of Value Expansion Methods in Model-Based Reinforcement Learning
Daniel Palenicek, Michael Lutter, Joao Carvalho, Jan Peters
Model Based Reinforcement Learning Value Function Customer Service Value Expansion

Model Based Reinforcement Learning

Papers

TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matching

Sense, Imagine, Act: Multimodal Perception Improves Model-Based Reinforcement Learning for Head-to-Head Autonomous Racing

A Survey on Offline Model-Based Reinforcement Learning

Human Machine Co-adaption Interface via Cooperation Markov Decision Process System

Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learning

FLEX: an Adaptive Exploration Algorithm for Nonlinear Systems

Model Based Reinforcement Learning for Personalized Heparin Dosing

Sample-efficient Model-based Reinforcement Learning for Quantum Control

Decision-Focused Model-based Reinforcement Learning for Reward Transfer

Tracker: Model-based Reinforcement Learning for Tracking Control of Human Finger Attached with Thin McKibben Muscles

Models as Agents: Optimizing Multi-Step Predictions of Interactive Local Models in Model-Based Multi-Agent Reinforcement Learning

Model-Based Reinforcement Learning with Isolated Imaginations

EDGI: Equivariant Diffusion for Planning with Embodied Agents

Dynamic Update-to-Data Ratio: Minimizing World Model Overfitting

On the Benefits of Leveraging Structural Information in Planning Over the Learned Model

Replay Buffer with Local Forgetting for Adapting to Local Environment Changes in Deep Model-Based Reinforcement Learning

Transformer-based World Models Are Happy With 100k Interactions

Predictive Experience Replay for Continual Visual Control and Forecasting

Beware of Instantaneous Dependence in Reinforcement Learning

Diminishing Return of Value Expansion Methods in Model-Based Reinforcement Learning