Actor-Critic Methods

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 62

All topics

Structure-Conditioned Actor-Critic Branches for Quality-Diversity Reinforcement Learning
Jun 7, 2026Lianrong Zuo, Peilan Xu, Yong Liu +1Reinforcement Learning Policy Optimization
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen 518055, Guangdong, China
Learning Predictive Control with Deep Koopman Operators for Autonomous Vehicle Motion Planning
Jun 6, 2026Xinglong Zhang, Yongqian Xiao, Haotian Cao +3Model Predictive Control Autonomous Driving Planning
Learning to replenish: A hybrid deep reinforcement learning for dynamic inventory management in the pharmaceutical supply chains
Jun 4, 2026Amandeep Kaur, Gyan PrakashInventory Control Actor-Critic Methods
Department of Management Studies ABV-Indian Institute of Information Technology and Management Gwalior-474015, Madhya Pradesh, India
Retry Policy Gradients in Continuous Action Spaces
Jun 4, 2026Soichiro Nishimori, Paavo ParmasPolicy Optimization RL Exploration
The University of Tokyo, Japan
Failure Modes of Deep Multi-Agent RL in Asynchronous Pricing: Reproducible Triggers, Trace Diagnostics, and a Partial Fix
Jun 3, 2026Shree Murthy, Rohan PandeyAsynchronous RL Algorithmic Collusion
DigitalOcean, USA
From Ticks to Flows: Dynamics of Neural Reinforcement Learning in Continuous Environments
Jun 2, 2026Saket Tiwari, Tejas Kotwal, George KonidarisRL Control Reinforcement Learning
Department of Computer Science Brown University · Department of Applied Mathematics Brown University
Zero Collapse: A Failure Mode of Policy Gradient Methods in Discontinuous Reward Environments
May 29, 2026Nishant Kumar, Enrique Areyan Viqueira, Amy GreenwaldReinforcement Learning Policy Gradient Methods
Refined Analysis of Entropy-Regularized Actor-Critic
May 23, 2026Safwan Labbi, Paul Mangold, Daniil Tiapkin +1Reinforcement Learning Maximum Entropy RL
CMAP, CNRS, École Polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France · Université Paris-Saclay, CNRS, LMO, 91405, Orsay, France · Google DeepMind +2
Second-Order Actor-Critic Methods for Discounted MDPs via Policy Hessian Decomposition
May 14, 2026Sanjeev Manivannan, Shuban VReinforcement Learning Policy Gradient Methods
Indian Institute of Technology Madras Chennai, India
Collaborative Yet Personalized Policy Training: Single-Timescale Federated Actor-Critic
May 14, 2026Leo Muxing Wang, Pengkun Yang, Lili SuFederated RL Actor-Critic Methods
Northeastern University · Tsinghua University
Achieving $ε^{-2}$ Sample Complexity for Single-Loop Actor-Critic under Minimal Assumptions
May 13, 2026Ishaq Hamza, Zaiwei ChenPolicy Gradient Last-Iterate Convergence
IISc · Edwardson School of Industrial Engineering, Purdue University · Purdue IE
Debiased Model-based Representations for Sample-efficient Continuous Control
May 12, 2026Jiafei Lyu, Zichuan Lin, Scott Fujimoto +5Representation Learning Continuous Control
1Tencent Hunyuan · 2McGill University · School of Computer
TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing
May 12, 2026Yuanpeng Li, Gefei Lin, Annie Qu +1Multi-Task RL Proximal Policy Optimization
UC Irvine · George Washington University · UC Santa Barbara +1
XQCfD: Accelerating Fast Actor-Critic Algorithms with Prior Data and Prior Policies
May 11, 2026Daniel Palenicek, Florian Vogt, Joe Watson +3Robotic RL Learning from Demonstration
Technical University of Darmstadt · KTH Royal Institute of Technology · University of Oxford +2
DelAC: A Multi-agent Reinforcement Learning of Team-Symmetric Stochastic Games
May 11, 2026Duan-Shin Lee, Yu-Hsiu HungNash Equilibrium Game Theory
Department of Computer Science, National Tsing Hua University · MediaTek Inc.
ACSAC: Adaptive Chunk Size Actor-Critic with Causal Transformer Q-Network
May 10, 2026Qian Chen, Junqiao Zhao, Hongtu Zhou +4Reinforcement Learning Transformer-Based RL
Tongji University
Actor-Critic Algorithm for Dynamic Expectile and CVaR
May 8, 2026Yudong Luo, Erick DelageReinforcement Learning Risk-Sensitive RL
1GERAD & Department of Decision Sciences, HEC Montr´eal, Canada · 2Mila - Quebec AI Institute, Canada
Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
May 8, 2026Yunho Choi, Jongwon Lim, Woojin Ahn +3Value Function Estimation RL for Language Model Reasoning
Graduate School of Data Science Seoul National University · Computer Science and Engineering Seoul National University
Actor-Critic with Active Importance Sampling
May 8, 2026Majid Molaei, Gabor Paczolay, Matteo Papini +2Reinforcement Learning Policy Gradient Methods
Politecnico di Milano Milan, Italy
AdaGamma: State-Dependent Discounting for Temporal Adaptation in Reinforcement Learning
May 7, 2026Yaomin Wang, Jianting Pan, Ran Tian +4Reinforcement Learning Temporal-Difference Learning
School of Data Science, The Chinese University of Hong Kong, Shenzhen
Breaking the Computational Barrier: Provably Efficient Actor-Critic for Low-Rank MDPs
May 2, 2026Ruiquan Huang, Donghao Li, Yingbin Liang +1Markov Decision Processes Reinforcement Learning
University of Kentucky. · University of Virginia. · Ohio State University..
SAVGO: Learning State-Action Value Geometry with Cosine Similarity for Continuous Control
May 1, 2026Stavros Orfanoudakis, Pedro P. VergaraRepresentation Learning Policy Optimization
Delft University of Technology, The Netherlands
Bayesian policy gradient and actor-critic algorithms
Apr 30, 2026Mohammad Ghavamzadeh, Yaakov Engel, Michal ValkoBayesian RL Temporal-Difference Learning
Adobe Research & Inria · Rafael Advanced Defence System, Israel · Inria Lille — SequeL team, France
Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithms
Apr 30, 2026Zhenjie Ren, Xiaoli Wei, Xiang Yu +1Reinforcement Learning Stochastic Optimal Control
LaMME, Universit´e ´Evry Paris-Saclay, ´Evry, France · Department of Applied Mathematics, The Hong Kong Polytechnic University, Kowloon, Hong Kong · Department of Industrial Engineering and Operations Research, Columbia University, New York, USA
Efficient Reinforcement Learning using Linear Koopman Dynamics for Nonlinear Robotic Systems
Apr 21, 2026Wenjian Hao, Yuxuan Fang, Zehui Lu +1Robotic Control Reinforcement Learning
School of Aeronautics and Astronautics, Purdue University, West Lafayette, IN 47907, USA · independent researcher
Scalable Neighborhood-Based Multi-Agent Actor-Critic
Apr 20, 2026Tim Goppelsroeder, Rasmus JensenMulti-Agent Systems Actor-Critic Methods
Department of Computer Science, ETH Zürich.
Pretraining in Actor-Critic Reinforcement Learning for Locomotion
Oct 14, 2025Jiale Fan, Andrei Cramariuc, Tifanny Portela +1RL for Robotics Reinforcement Learning
Section of Microengineering, EPFL, 1015 Lausanne, Switzerland · Robotic Systems Lab, ETH Zurich, 8092 Zürich, Switzerland · ETH AI Center, ETH Zurich, 8092 Zürich, Switzerland
Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning
Jun 8, 2025Yang Xu, Swetha Ganesh, Vaneet AggarwalRobust Markov Decision Processes Reinforcement Learning
Purdue University, West Lafayette, Indiana, USA 47907
Streaming Deep Reinforcement Learning Finally Works
Oct 18, 2024Mohamed Elsayed, Elena Sorina Lupu, Gautham Vasan +1Reinforcement Learning Actor-Critic Methods
Department of Computing Science, University of Alberta, Edmonton, Canada · Alberta Machine Intelligence Institute (Amii) · Canada CIFAR AI Chair
Topology-Guided Modular Actor-Critic Learning for Continuous Systems under Temporal Objectives
Apr 20, 2023Lening Li, Zhentian Qian, Jianan Xia +7RL Control Reinforcement Learning
Ningbo University, Ningbo, China · Robotics Engineering Program, Worcester Polytechnic Institute, Worcester, MA, USA · Ningbo Zhenhai Silver-Ball Bearing Co., Ltd., Ningbo, China

Jun 7, 2026, cs.AIJ/K move · Enter open · S save

Structure-Conditioned Actor-Critic Branches for Quality-Diversity Reinforcement Learning

Lianrong Zuo, Peilan Xu, Yong Liu, Wenjian Luo

Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen 518055, Guangdong, China

Reinforcement Learning Policy Optimization Actor-Critic Methods

Open paper AskPDF arXiv

Quality-diversity reinforcement learning (QD-RL) aims to construct policy repertoires that contain both high-performing and behaviorally diverse policies. Existing QD-RL methods mainly diversify policy instances after rollout evaluation or use learned value information to improve policy quality and behavior targeting, while the learning branches that generate candidate policies remain less explored. This paper proposes SV-QD-RL, a structure-value coupled framework that represents each candidate as a structure-conditioned actor-critic branch. Each branch contains an actor, a structural mask, a branch-specific critic, a replay state, and evaluation attributes including behavior, return, sparsity, and value profile. The structural mask defines the actor subspace in which the branch learns, while the branch-specific critic and replay state shape its value-learning trajectory. A branch-aware QD archive then evaluates and retains branches according to behavioral quality, structural footprint, and value-profile information. Experiments on MuJoCo continuous-control tasks show that SV-QD-RL constructs policy repertoires with strong archive quality and behaviorally useful diversity. Ablation and diagnostic analyses further indicate that structural conditioning, critic differentiation, and memory-consistent refinement make complementary contributions to behavioral specialization. Schedule-aware repertoire evaluation shows that the learned archive provides selectable policy alternatives under changing behavior-level requirements. These results suggest that coupling actor structure with branch-specific value learning is an effective mechanism for generating diverse QD-RL policy repertoires.

Momentum

Related topics

Latest papers 62