Actor Critic Framework

The actor-critic framework is a reinforcement learning approach where a "critic" network evaluates the actions of an "actor" network, guiding its learning to maximize rewards. Current research focuses on improving sample efficiency and stability, particularly in high-dimensional visual environments, through techniques like prioritized experience replay, double actor-critic architectures, and incorporating diffusion models for policy representation. These advancements are enhancing the performance of actor-critic methods in various applications, including robotics, game playing, and recommendation systems, by addressing challenges such as out-of-distribution actions and exploration-exploitation trade-offs. The framework's flexibility and adaptability make it a significant tool for solving complex decision-making problems.

Papers

September 28, 2024

June 11, 2024

Augmenting Offline RL with Unlabeled Data
Zhao Wang, Briti Gangopadhyay, Jia-Fong Yeh, Shingo Takamatsu
Offline Reinforcement Learning Unlabeled Data State Action Pair Actor Critic Framework Policy Similarity

June 5, 2024

Object Manipulation in Marine Environments using Reinforcement Learning
Ahmed Nader, Muhayy Ud Din, Mughni Irfan, Irfan Hussain
Reinforcement Learning Deep Reinforcement Learning Underwater Environment Object Manipulation Marine Environment Actor Critic Framework

May 31, 2024

Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning
Linjiajie Fang, Ruoxue Liu, Jing Zhang, Wenjia Wang, Bing-Yi Jing
Offline Reinforcement Learning Diffusion Policy Policy Regularization Actor Critic Framework Diffusion Noise

May 24, 2024

Diffusion Actor-Critic with Entropy Regulator
Yinuo Wang, Likun Wang, Yuxuan Jiang, Wenjun Zou, Tong Liu, Xujie Song, Wenxuan Wang, Liming Xiao, Jiang Wu, Jingliang Duan, Shengbo Eben Li
Reinforcement Learning Diffusion Policy Actor Critic Framework Expressive Policy

May 22, 2024

Maximum Entropy Reinforcement Learning via Energy-Based Normalizing Flow
Chen-Hao Chao, Chien Feng, Wei-Fang Sun, Cheng-Kuang Lee, Simon See, Chun-Yi Lee
Reinforcement Learning Normalizing Flow Action Space Maximum Entropy Reinforcement Learning Actor Critic Framework MaxEnt RL

February 1, 2024

SLIM: Skill Learning with Multiple Critics
David Emukpere, Bingbing Wu, Julien Perez, Jean-Michel Renders
Skill Learning Skill Discovery Self Supervised Task Multiple Critic Actor Critic Framework

November 6, 2023

Virtual Action Actor-Critic Framework for Exploration (Student Abstract)
Bumgeun Park, Taeyoung Kim, Quoc-Vinh Lai-Dang, Dongsoo Har
Reinforcement Learning Environment Exploration Actor Critic Virtual World Actor Critic Framework Policy Entropy

May 24, 2023

Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees
Sharan Vaswani, Amirreza Kazemi, Reza Babanezhad, Nicolas Le Roux
Reinforcement Learning Policy Gradient Function Approximation Theoretical Guarantee Policy Improvement Actor Critic Framework Critic Regularization

April 20, 2023

Topological Guided Actor-Critic Modular Learning of Continuous Systems with Temporal Objectives
Lening Li, Zhentian Qian
Reinforcement Learning Algorithm Continuous Time Temporal Goal Policy Synthesis Actor Critic Framework Online Policy Customization Topological Guidance

January 14, 2023

Opponent-aware Role-based Learning in Team Competitive Markov Games
Paramita Koley, Aurghya Maiti, Niloy Ganguly, Sourangshu Bhattacharya
Multi Agent Reinforcement Learning Multi Agent Actor Critic Actor Critic Framework Role Based

January 10, 2023

Actor-Director-Critic: A Novel Deep Reinforcement Learning Framework
Zongwei Liu, Yonghong Song, Yuanlin Zhang
Reinforcement Learning Deep Reinforcement Learning Actor Critic Actor Critic Framework

October 3, 2022

Latent State Marginalization as a Low-cost Approach for Improving Exploration
Dinghuai Zhang, Aaron Courville, Yoshua Bengio, Qinqing Zheng, Amy Zhang, Ricky T. Q. Chen
Reinforcement Learning Environment Exploration Latent State Latent Action Actor Critic Framework Policy Distribution Cost Effective Approach

July 5, 2022

Ask-AC: An Initiative Advisor-in-the-Loop Actor-Critic Framework
Shunyu Liu, Kaixuan Chen, Na Yu, Jie Song, Zunlei Feng, Mingli Song
Actor Critic Actor Critic Framework Interactive Reinforcement Learning User in the Loop

May 26, 2022

Constrained Reinforcement Learning for Short Video Recommendation
Qingpeng Cai, Ruohan Zhan, Chi Zhang, Jie Zheng, Guangwei Ding, Pinghua Gong, Dong Zheng, Peng Jiang
Recommender System Constrained Reinforcement Learning Recommendation Algorithm Actor Critic Framework