MaxEnt RL

Maximum Entropy Reinforcement Learning (MaxEnt RL) aims to find optimal policies that balance maximizing expected rewards with maximizing policy entropy, leading to more robust and exploratory behavior. Current research focuses on developing efficient algorithms, particularly for continuous action spaces, using architectures like Energy-Based Models and Normalizing Flows, and improving the estimation of entropy for expressive policies. These advancements are improving the performance of MaxEnt RL in complex tasks, such as robotics and wildfire prediction, by addressing challenges like high dimensionality and non-smoothness in the underlying models.

Papers

July 19, 2024

On Maximum Entropy Linear Feature Inversion
Paul M Baggenstoss
Unified Framework Dimension Reduction Maximum Entropy Entropy Measure MaxEnt RL

May 22, 2024

Maximum Entropy Reinforcement Learning via Energy-Based Normalizing Flow
Chen-Hao Chao, Chien Feng, Wei-Fang Sun, Cheng-Kuang Lee, Simon See, Chun-Yi Lee
Reinforcement Learning Normalizing Flow Action Space Maximum Entropy Reinforcement Learning Actor Critic Framework MaxEnt RL

May 2, 2024

S$^2$AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic
Safa Messaoud, Billel Mokeddem, Zhenghai Xue, Linsey Pang, Bo An, Haipeng Chen, Sanjay Chawla
Soft Actor Critic Energy Based Stein Variational Gradient Descent Maximum Entropy Reinforcement Learning Expressive Policy MaxEnt RL

March 11, 2024

Efficient first-order algorithms for large-scale, non-smooth maximum entropy models with application to wildfire science
Gabriel P. Langlois, Jatan Buch, Jérôme Darbon
Application Proficiency High Efficiency Science Journalism Kullback Leibler Divergence Maximum Entropy First Order Algorithm MaxEnt RL

February 15, 2024

Discrete Probabilistic Inference as Control in Multi-path Environments
Tristan Deleu, Padideh Nouri, Nikolay Malkin, Doina Precup, Yoshua Bengio
Markov Decision Process External Control Inference Problem Stochastic Policy Maximum Entropy Reinforcement Learning MaxEnt RL

November 29, 2023

Maximum Entropy Model Correction in Reinforcement Learning
Amin Rakhsha, Mete Kemertas, Mohammad Ghavamzadeh, Amir-massoud Farahmand
Reinforcement Learning Value Iteration Approximate Model MaxEnt RL

January 5, 2023

Extreme Q-Learning: MaxEnt RL without Entropy
Divyansh Garg, Joey Hejna, Matthieu Geist, Stefano Ermon
Offline Reinforcement Learning Q Learning Q Function Stable Entropy Maximum Entropy Reinforcement Learning MaxEnt RL Modern Deep Reinforcement Learning

November 28, 2021

Count-Based Temperature Scheduling for Maximum Entropy Reinforcement Learning
Dailin Hu, Pieter Abbeel, Roy Fox
Maximum Entropy Reinforcement Learning Policy Entropy Soft Q Learning MaxEnt RL