Path Entropy

Path entropy, a measure of the uncertainty or randomness in the sequence of states and actions an agent takes, is a burgeoning area of research in reinforcement learning and related fields. Current work focuses on using path entropy to improve the predictability, robustness, and efficiency of reinforcement learning agents, often through entropy minimization or regularization within algorithms like Soft Actor-Critic (SAC) and policy gradient methods. This research aims to create more interpretable, reliable, and adaptable agents, with applications ranging from robotics to complex systems modeling, by addressing the exploration-exploitation dilemma and promoting efficient learning.

Papers

April 15, 2024

Effective Reinforcement Learning Based on Structural Information Principles
Xianghua Zeng, Hao Peng, Dingli Su, Angsheng Li
Reinforcement Learning Multi Agent Reinforcement Learning Structural Information Effective Reinforcement Learning Semantic Entropy Structural Entropy Path Entropy

November 30, 2023

Predictable Reinforcement Learning Dynamics through Entropy Rate Minimization
Daniel Jarne Ornia, Giannis Delimpaltadakis, Jens Kober, Javier Alonso-Mora
Reinforcement Learning Path Entropy Entropy Reward

March 31, 2023

Environmental path-entropy and collective motion
Harvey L. Devereux, Matthew S. Turner
Resilient Swarm Collective Motion Feather Flock Fitness Evolution Path Entropy

February 2, 2023

A general Markov decision process formalism for action-state entropy-regularized reward maximization
Dmytro Grytskyy, Jorge Ramírez-Ruiz, Rubén Moreno-Bote
Markov Decision Process Unconstrained Optimization Entanglement Entropy Dual Formulation Policy Entropy Path Entropy

December 14, 2022

Robust Policy Optimization in Deep Reinforcement Learning
Md Masudur Rahman, Yexiang Xue
Deep Reinforcement Learning Policy Gradient Policy OpTimization Robust Policy Policy Entropy Path Entropy

October 1, 2022

Boosting Exploration in Actor-Critic Algorithms by Incentivizing Plausible Novel States
Chayan Banerjee, Zhiyong Chen, Nasimul Noman
Policy OpTimization Actor Critic Algorithm Intrinsic Reward Model Free Deep Reinforcement Learning Exploration Performance Policy Actor Critic Path Entropy

May 20, 2022

Complex behavior from intrinsic motivation to occupy action-state path space
Jorge Ramírez-Ruiz, Dmytro Grytskyy, Chiara Mastrogiuseppe, Yamen Habib, Rubén Moreno-Bote
Goal Oriented Intrinsic Motivation Reward Maximization Complex Behavior State Action Path Entropy

February 7, 2022

Soft Actor-Critic with Inhibitory Networks for Faster Retraining
Jaime S. Ide, Daria Mićović, Michael J. Guarino, Kevin Alcedo, David Rosenbluth, Adrian P. Pope
Deep Reinforcement Learning Faster Training Soft Actor Critic Retraining Method Auto Tuning Path Entropy Inhibitory Neuron