Model Free Reinforcement Learning

Model-free reinforcement learning (MFRL) aims to learn optimal policies for controlling systems without explicit knowledge of the system's dynamics, focusing on direct interaction and reward maximization. Current research emphasizes improving sample efficiency and robustness through techniques like residual policy learning, adaptive horizon control, and the integration of physics-based models to guide learning, often employing algorithms such as PPO, SAC, TD3, and actor-critic methods with various neural network architectures (e.g., CNNs, RNNs, Transformers). These advancements are significant for applications requiring efficient learning in complex, high-dimensional environments, such as robotics, autonomous navigation, and financial portfolio management, where detailed models are unavailable or computationally intractable.

Papers

May 26, 2024

Multi-State TD Target for Model-Free Reinforcement Learning
Wuhao Wang, Zhiyong Chen, Lepeng Zhang
Reinforcement Learning Model Free Reinforcement Learning Actor Critic Algorithm Deep Deterministic Policy Gradient State Action Pair Temporal Difference

May 6, 2024

$\epsilon$-Policy Gradient for Online Pricing
Lukasz Szpruch, Tanut Treetanthiploet, Yufei Zhang
Gradient Descent Optimal Policy Model Free Reinforcement Learning Gradient Method Curious Price Epsilon Greedy

April 25, 2024

April 23, 2024

Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems
Haozhe Tian, Homayoun Hamedmoghadam, Robert Shorten, Pietro Ferraro
Reinforcement Learning Safe Reinforcement Learning Model Free Reinforcement Learning Safe Control Safe Policy Behavior Regularization

April 4, 2024

Integrating Hyperparameter Search into Model-Free AutoML with Context-Free Grammars
Hernán Ceferino Vázquez, Jorge Sanchez, Rafael Carrascosa
Machine Learning Model Free Reinforcement Learning Online autoML Context Free End to End AutoML Step Gram

March 7, 2024

Learning Agility Adaptation for Flight in Clutter
Guangyu Zhao, Tianyue Wu, Yeke Chen, Fei Gao
Mobile Robot Model Free Reinforcement Learning Dense Clutter Flight Pattern Animal Level Agility

March 6, 2024

Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
Wesley A. Suttle, Vipul K. Sharma, Krishna C. Kosaraju, S. Sivaranjani, Ji Liu, Vijay Gupta, Brian M. Sadler
Obstacle Avoidance Nonlinear Dynamic Safe Reinforcement Learning Safety Constraint Model Free Reinforcement Learning Safety Filter

March 1, 2024

SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation
Noriaki Hirose, Dhruv Shah, Kyle Stachowicz, Ajay Sridhar, Sergey Levine
Reinforcement Learning Model Free Reinforcement Learning Robot Behavior Social Navigation Self Improvement Selfie Image Robot Experience Self Improving

February 24, 2024

Analysis of Off-Policy Multi-Step TD-Learning with Linear Function Approximation
Donghwan Lee
General Analysis Gradient Descent Model Free Reinforcement Learning Linear Function Approximation Deterministic Algorithm Multi Step Off Policy

February 8, 2024

January 27, 2024

R$\times$R: Rapid eXploration for Reinforcement Learning via Sampling-based Reset Distributions and Imitation Pre-training
Gagan Khandate, Tristan L. Saidi, Siqi Shang, Eric T. Chang, Yang Liu, Seth Dennis, Johnson Adams, Matei Ciocarlie
Reinforcement Learning Model Free Reinforcement Learning Dexterous Manipulation Based Imitation Rapid Exploration Dexterous in Hand Manipulation Based Reset

December 28, 2023

Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexity
Guhao Feng, Han Zhong
Reinforcement Learning Camera Lens Model Free Reinforcement Learning Value Based Reinforcement Learning Representation Complexity

December 21, 2023

Multi-Agent Probabilistic Ensembles with Trajectory Sampling for Connected Autonomous Vehicles
Ruoqi Wen, Jiahao Huang, Rongpeng Li, Guoru Ding, Zhifeng Zhao
Reinforcement Learning Autonomous Vehicle Model Free Reinforcement Learning Connected Autonomous Vehicle Trajectory Sampling Agent Ensemble

December 14, 2023

ReCoRe: Regularized Contrastive Representation Learning of World Model
Rudra P. K. Poudel, Harit Pandya, Stephan Liwicki, Roberto Cipolla
World Model Model Free Reinforcement Learning Visual Navigation Distribution Detection Task

November 17, 2023

Autonomous Port Navigation With Ranging Sensors Using Model-Based Reinforcement Learning
Siemen Herremans, Ali Anwar, Arne Troch, Ian Ravijts, Maarten Vangeneugden, Siegfried Mercelis, Peter Hellinckx
Model Based Reinforcement Learning Model Free Reinforcement Learning Sensor System Autonomous Vessel Port Operation Navigation Algorithm

November 3, 2023

Towards model-free RL algorithms that scale well with unstructured data
Joseph Modayil, Zaheer Abbas
Reinforcement Learning Reward Function Reinforcement Learning Algorithm Unstructured Data Model Free Reinforcement Learning Deep RL Single Agent Reinforcement Learning

October 20, 2023

Tree Search in DAG Space with Model-based Reinforcement Learning for Causal Discovery
Victor-Alexandru Darvariu, Stephen Hailes, Mirco Musolesi
Causal Discovery Model Based Reinforcement Learning Causal Structure Bayesian Network Model Free Reinforcement Learning Tree Search

October 9, 2023

DSAC-T: Distributional Soft Actor-Critic with Three Refinements
Jingliang Duan, Wenxuan Wang, Liming Xiao, Jiaxin Gao, Shengbo Eben Li
Reinforcement Learning Policy Reinforcement Learning Model Free Reinforcement Learning Task Specific Reward

Model Free Reinforcement Learning

Papers

Multi-State TD Target for Model-Free Reinforcement Learning

$\epsilon$-Policy Gradient for Online Pricing

Distilling Privileged Information for Dubins Traveling Salesman Problems with Neighborhoods

A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints

Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems

Integrating Hyperparameter Search into Model-Free AutoML with Context-Free Grammars

Learning Agility Adaptation for Flight in Clutter

Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems

SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation

Analysis of Off-Policy Multi-Step TD-Learning with Linear Function Approximation

Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning

Multi-Timescale Ensemble Q-learning for Markov Decision Process Policy Optimization

R$\times$R: Rapid eXploration for Reinforcement Learning via Sampling-based Reset Distributions and Imitation Pre-training

Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexity

Multi-Agent Probabilistic Ensembles with Trajectory Sampling for Connected Autonomous Vehicles

ReCoRe: Regularized Contrastive Representation Learning of World Model

Autonomous Port Navigation With Ranging Sensors Using Model-Based Reinforcement Learning

Towards model-free RL algorithms that scale well with unstructured data

Tree Search in DAG Space with Model-based Reinforcement Learning for Causal Discovery

DSAC-T: Distributional Soft Actor-Critic with Three Refinements