Model Based Reinforcement Learning

Model-based reinforcement learning (MBRL) aims to improve the sample efficiency and robustness of reinforcement learning agents by learning a model of the environment's dynamics. Current research focuses on enhancing model accuracy and robustness through techniques like incorporating expert knowledge, using bisimulation metrics for state representation, and employing adversarial training to handle uncertainties. These advancements are leading to more efficient and reliable control policies in various applications, including robotics, autonomous driving, and even protein design, by reducing the need for extensive real-world interactions during training.

Papers

February 1, 2024

Control in Stochastic Environment with Delays: A Model-based Reinforcement Learning Approach
Zhiyuan Yao, Ionut Florescu, Chihoon Lee
External Control Optimal Policy Model Based Reinforcement Learning Policy OpTimization Stochastic Environment Probabilistic Planning Significant Delay Change

January 23, 2024

Locality Sensitive Sparse Encoding for Learning World Models Online
Zichen Liu, Chao Du, Wee Sun Lee, Min Lin
Continual Learning Online Learning Model Based Reinforcement Learning World Model Sparse Coding

December 30, 2023

Laboratory Experiments of Model-based Reinforcement Learning for Adaptive Optics Control
Jalo Nousiainen, Byron Engler, Markus Kasper, Chang Rajani, Tapio Helin, Cédric T. Heritier, Sascha P. Quanz, Adrian M. Glauser
Reinforcement Learning Model Based Reinforcement Learning Physic Experiment Adaptive Optic

December 15, 2023

Sample-Efficient Learning to Solve a Real-World Labyrinth Game Using Data-Augmented Model-Based Reinforcement Learning
Thomas Bi, Raffaello D'Andrea
Model Based Reinforcement Learning Sample Efficient Augmented Data

December 7, 2023

Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization
Carlos E. Luis, Alessandro G. Bottero, Julia Vinogradska, Felix Berkenkamp, Jan Peters
Model Based Reinforcement Learning Epistemic Uncertainty Policy Value Bellman Equation Inefficient Exploration Risk Averse Policy

December 5, 2023

Score-Aware Policy-Gradient Methods and Performance Guarantees using Local Lyapunov Conditions: Applications to Product-Form Stochastic Networks and Queueing Systems
Céline Comte, Matthieu Jonckheere, Jaron Sanders, Albert Senen-Cerda
Policy Gradient Model Based Reinforcement Learning Lyapunov Function Performance Guarantee Stochastic Network

November 17, 2023

Autonomous Port Navigation With Ranging Sensors Using Model-Based Reinforcement Learning
Siemen Herremans, Ali Anwar, Arne Troch, Ian Ravijts, Maarten Vangeneugden, Siegfried Mercelis, Peter Hellinckx
Model Based Reinforcement Learning Model Free Reinforcement Learning Sensor System Autonomous Vessel Port Operation Navigation Algorithm

November 2, 2023

DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothing
Vint Lee, Pieter Abbeel, Youngwoon Lee
Model Based Reinforcement Learning Sparse Reward Reward Prediction Differentiable Reward

October 31, 2023

The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback
Nathan Lambert, Roberto Calandra
Reinforcement Learning Human Feedback Model Based Reinforcement Learning Reward Model Human Alignment Model Mismatch Reward Model Training

October 30, 2023

Efficient Exploration in Continuous-time Model-based Reinforcement Learning
Lenart Treven, Jonas Hübotter, Bhavya Sukhija, Florian Dörfler, Andreas Krause
Reinforcement Learning Model Based Reinforcement Learning Efficient Exploration Discrete Time Continuous Time Model Continuous Time Dynamic

October 25, 2023

TD-MPC2: Scalable, Robust World Models for Continuous Control
Nicklas Hansen, Hao Su, Xiaolong Wang
Model Based Reinforcement Learning World Model Continuous Control Hybrid MPC Td MPC

October 23, 2023

Mind the Model, Not the Agent: The Primacy Bias in Model-based RL
Zhongjian Qiao, Jiafei Lyu, Xiu Li
Deep Reinforcement Learning Model Based Reinforcement Learning Model Free World Modeling Primacy Bias

October 20, 2023

Tree Search in DAG Space with Model-based Reinforcement Learning for Causal Discovery
Victor-Alexandru Darvariu, Stephen Hailes, Mirco Musolesi
Causal Discovery Model Based Reinforcement Learning Causal Structure Bayesian Network Model Free Reinforcement Learning Tree Search

October 17, 2023

Value-Biased Maximum Likelihood Estimation for Model-based Reinforcement Learning in Discounted Linear MDPs
Yu-Heng Hung, Ping-Chun Hsieh, Akshay Mete, P. R. Kumar
Model Based Reinforcement Learning Optimal Regret O$ Regret Infinite Horizon Linear Markov Decision Process

October 14, 2023

STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning
Weipu Zhang, Gang Wang, Jian Sun, Yetian Yuan, Gao Huang
Reinforcement Learning Variational Autoencoders Model Based Reinforcement Learning World Model Transformer Based World Model

October 12, 2023

Pay Attention to How You Drive: Safe and Adaptive Model-Based Reinforcement Learning for Off-Road Driving
Sean J. Wang, Honghao Zhu, Aaron M. Johnson
Adaptive Importance Model Based Reinforcement Learning Robot Dynamic Driving Context Model Predictive Path Integral Control Pay Attention Dynamic Simulation Dynamic Target

October 11, 2023

COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RL
Xiyao Wang, Ruijie Zheng, Yanchao Sun, Ruonan Jia, Wichayaporn Wongkamjan, Huazhe Xu, Furong Huang
Reinforcement Learning Model Based Reinforcement Learning Model Based High Level Plan Learned Dynamic Optimistic Exploration

October 10, 2023

A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning
Ran Wei, Nathan Lambert, Anthony McDonald, Alfredo Garcia, Roberto Calandra
Model Based Reinforcement Learning Policy OpTimization Unified View Learned Dynamic Reward Structure Model Mismatch

October 9, 2023

Multi-timestep models for Model-based Reinforcement Learning
Abdelhakim Benechehab, Giuseppe Paolo, Albert Thomas, Maurizio Filippone, Balázs Kégl
Model Based Reinforcement Learning Batch Reinforcement Learning Step to Step Dynamic Multi Timestep

October 6, 2023

Amortized Network Intervention to Steer the Excitatory Point Processes
Zitao Song, Wendi Ren, Shuang Li
Model Based Reinforcement Learning Dynamic Graph Hidden Dynamic Node Intervention

Model Based Reinforcement Learning

Papers

Control in Stochastic Environment with Delays: A Model-based Reinforcement Learning Approach

Locality Sensitive Sparse Encoding for Learning World Models Online

Laboratory Experiments of Model-based Reinforcement Learning for Adaptive Optics Control

Sample-Efficient Learning to Solve a Real-World Labyrinth Game Using Data-Augmented Model-Based Reinforcement Learning

Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization

Score-Aware Policy-Gradient Methods and Performance Guarantees using Local Lyapunov Conditions: Applications to Product-Form Stochastic Networks and Queueing Systems

Autonomous Port Navigation With Ranging Sensors Using Model-Based Reinforcement Learning

DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothing

The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback

Efficient Exploration in Continuous-time Model-based Reinforcement Learning

TD-MPC2: Scalable, Robust World Models for Continuous Control

Mind the Model, Not the Agent: The Primacy Bias in Model-based RL

Tree Search in DAG Space with Model-based Reinforcement Learning for Causal Discovery

Value-Biased Maximum Likelihood Estimation for Model-based Reinforcement Learning in Discounted Linear MDPs

STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning

Pay Attention to How You Drive: Safe and Adaptive Model-Based Reinforcement Learning for Off-Road Driving

COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RL

A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning

Multi-timestep models for Model-based Reinforcement Learning

Amortized Network Intervention to Steer the Excitatory Point Processes