Actor Critic Algorithm

Actor-critic algorithms are a class of reinforcement learning methods that learn optimal policies by iteratively improving a policy (the actor) and estimating its value (the critic). Current research focuses on improving the stability and efficiency of these algorithms, particularly through advancements in model architectures like neural networks (including those using ReLU networks and moment neural networks), and addressing challenges such as bias in gradient estimation, sample efficiency, and robustness to uncertainty in the environment. These improvements are driving progress in various applications, including robotics, resource management, and even LLM alignment, where actor-critic methods are increasingly used for policy optimization.

Papers

October 10, 2024

Variational Inequality Methods for Multi-Agent Reinforcement Learning: Performance and Stability Gains
Baraah A. M. Sidahmed, Tatjana Chavdarova
Multi Agent Reinforcement Learning System Performance Actor Critic Algorithm Variational Inequality Multi Agent Setting Multi Agent Particle Multi Agent Deep Deterministic Policy

October 2, 2024

Stochastic Gradient Descent with Adaptive Data
Ethan Che, Jing Dong, Xin T. Tong
Stochastic Gradient Descent Lyapunov Function Actor Critic Algorithm Stochastic System Data Adaptation

September 24, 2024

Agent-state based policies in POMDPs: Beyond belief-state MDPs
Amit Sinha, Aditya Mahajan
Q Learning Actor Critic Algorithm Prior Policy C POMDPs Sub Optimal Belief MDP Agent State

September 16, 2024

Robust Reinforcement Learning with Dynamic Distortion Risk Measures
Anthony Coache, Sebastian Jaimungal
Reinforcement Learning Actor Critic Algorithm Robust Reinforcement Learning Distortion Metric Risk Measure Risk Aware Reinforcement Learning

September 11, 2024

The Role of Deep Learning Regularizations on Actors in Offline RL
Denis Tarasov, Anja Surina, Caglar Gulcehre
Reinforcement Learning Neural Network Offline Reinforcement Learning Integral Role Actor Critic Algorithm Regularization Technique Regularization Method Actor Loss

September 8, 2024

Soft Actor-Critic with Beta Policy via Implicit Reparameterization Gradients
Luca Della Libera
Deep Reinforcement Learning Actor Critic Algorithm Soft Actor Critic Reparameterization Method Frank Wolfe Policy Optimization

September 2, 2024

Compatible Gradient Approximations for Actor-Critic Algorithms
Baturay Saglam, Dionysis Kalogerias
Actor Critic Actor Critic Algorithm Gradient Approximation Deterministic Policy Gradient

August 14, 2024

q-exponential family for policy optimization
Lingwei Zhu, Haseeb Shah, Han Wang, Yukie Nagai, Martha White
Policy OpTimization Actor Critic Algorithm Exponential Family Heavy Tailed Policy

July 29, 2024

A Method for Fast Autonomy Transfer in Reinforcement Learning
Dinuka Sahabandu, Bhaskar Ramasubramanian, Michail Alexiou, J. Sukarno Mertoguno, Linda Bushnell, Radha Poovendran
Reinforcement Learning Practical Method Value Function Actor Critic Algorithm Novel Reinforcement Learning Autonomy Transfer

July 24, 2024

Sublinear Regret for a Class of Continuous-Time Linear--Quadratic Reinforcement Learning Problems
Yilie Huang, Yanwei Jia, Xun Yu Zhou
Reinforcement Learning Optimal Policy Sublinear Regret Actor Critic Algorithm Time Varying Control

July 11, 2024

Gradient Boosting Reinforcement Learning
Benjamin Fuhrer, Chen Tessler, Gal Dalal
Reinforcement Learning Actor Critic Algorithm Gradient Boosted Tree

July 1, 2024

To Switch or Not to Switch? Balanced Policy Switching in Offline Reinforcement Learning
Tao Ma, Xuzhi Yang, Zoltan Szabo
Reinforcement Learning Offline Reinforcement Learning Optimal Transport Actor Critic Algorithm Optimal Behavior Policy Switching

June 13, 2024

Current applications and potential future directions of reinforcement learning-based Digital Twins in agriculture
Georg Goldenits, Kevin Mallinger, Sebastian Raubitzek, Thomas Neubauer
Reinforcement Learning Digital Twin Future Direction Positive Reinforcement Actor Critic Algorithm Agricultural Domain Future Application

June 12, 2024

Finite Time Analysis of Temporal Difference Learning for Mean-Variance in a Discounted MDP
Tejaram Sangadi, L. A. Prashanth, Krishna Jagannathan
Markov Decision Process Actor Critic Algorithm Linear Function Approximation Temporal Difference Learning Finite Time Risk Sensitive Reinforcement Learning Markov Reward Process Mean Variance Optimization

June 5, 2024

Physics-Guided Actor-Critic Reinforcement Learning for Swimming in Turbulence
Christopher Koh, Laurent Pagnier, Michael Chertkov
Reinforcement Learning Actor Critic Algorithm Physical Reasoning Turbulent Flow Stratified Turbulence Actor Critic Reinforcement Learning

June 3, 2024

Value Improved Actor Critic Algorithms
Yaniv Oren, Moritz A. Zanger, Pascal R. van der Vaart, Matthijs T. J. Spaan, Wendelin Bohmer
Actor Critic Algorithm State of the Art Reinforcement Simultaneous Improvement Policy Based Algorithm

June 2, 2024

Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradient
Zechu Li, Rickmer Krohn, Tao Chen, Anurag Ajay, Pulkit Agrawal, Georgia Chalvatzaki
Deep Reinforcement Learning Q Learning Actor Critic Algorithm Scratch Project Diffusion Policy Multimodal Policy

May 26, 2024

Multi-State TD Target for Model-Free Reinforcement Learning
Wuhao Wang, Zhiyong Chen, Lepeng Zhang
Reinforcement Learning Model Free Reinforcement Learning Actor Critic Algorithm Deep Deterministic Policy Gradient State Action Pair Temporal Difference

May 22, 2024

Learning to sample fibers for goodness-of-fit testing
Ivan Gvozdanović, Sonja Petrović
Actor Critic Actor Critic Algorithm Discrete Optimization Optimal Sampling Convex Polytope Sampling Framework

May 8, 2024

Novel Actor-Critic Algorithm for Robust Decision Making of CAV under Delays and Loss of V2X Data
Zine el abidine Kherroubi
Autonomous Vehicle Actor Critic Algorithm Autonomous Driving System Significant Delay Change Robust Decision