Overview

Actor-critic methods pair an actor, which chooses actions, with a critic, which estimates their value and helps improve the policy. On-policy methods such as A2C and PPO and off-policy methods such as SAC, TD3, and DDPG are widely used for continuous control, robotics, multi-agent learning, and language-model agents. Researchers also adapt critics for offline learning, risk and safety constraints, partial observability, and long-horizon credit assignment, while developing convergence and sample-complexity guarantees. A central challenge is that inaccurate or unstable critic estimates can misdirect policy updates, so better value estimation and training stability must be balanced against data efficiency and computational cost.

Latest papers

CardsList

120 papers13 papers in the last four weeks, down 41% on the four weeks before. 0.1% of all new papers.

  1. Robust Actor-Critic Learning under Transport and Divergence Uncertainty

    Nov 17, 2026Kunqi Huang, Lican Kang, Hongrui Shu +1Robust Markov Decision ProcessesDistributionally Robust RL

  2. Learning to Act with World Models by Valuing What Comes Next

    Nov 9, 2026Chenghao Gao, Zhi Zhang, Dian Jin +2Model-Based RLSparse-Reward RL

  3. Future-Informed Actor-Critic

    Nov 6, 2026Samy Mokeddem, Gaspard Lambrechts, Adrien Bolland +1Actor-Critic MethodsCredit Assignment in RL

  4. CRESiST: Self-Enhancing Explorative Policy Learning for Simultaneous Speech Translation

    Nov 4, 2026Chenyang Le, Pei Zhang, Yuxiang Chen +4Simultaneous Speech TranslationLLM Inference Acceleration

  5. Finite-Sample Convergence in Networked Average Reward MARL: Decentralization Pitfalls and Entropy Remedies

    Nov 2, 2026Yizhou Zhang, Yashaswini Murthy, Laixi Shi +1Decentralized MARLMulti-Agent Reinforcement Learning

  6. Neural Actor--Critic Learning with Minimax Critics and Reflected Langevin Sampling

    Nov 2, 2026Kunqi Huang, Lican Kang, Hongrui Shu +1Actor-Critic MethodsContinuous Control

  7. Improved Sample Complexity Analysis For First Order Bilevel Reinforcement Learning

    Nov 1, 2026Mudit Gaur, Naman Saxena, Vaneet AggarwalReinforcement LearningSample-Efficient RL

  8. AC-SMPC: Differentiable Actor--Critic MPC with State Constraints for Safe and Fast Robot Navigation

    Oct 30, 2026Chaoran Wu, Jiayu Fan, Qichen Tan +2Model Predictive ControlConstrained Motion Planning

  9. FB-MOAC: A Reinforcement Learning Algorithm for Forward-Backward Markov Decision Processes

    Oct 29, 2026Mohsen Amidzadeh, Mario Di FrancescoMarkov Decision ProcessesMulti-Objective Reinforcement Learning

  10. A Unified Analysis of Importance-Corrected Momentum Actor-Critic over a Markovian Trajectory

    Oct 24, 2026Yanjie Dong, Zhou Su, Wei Wang +1Reinforcement LearningActor-Critic Methods

  11. Financial Context-Rich Strategy for Actor-Critic based Agents: A Deep Reinforcement Learning Approach for Stock Trading

    Oct 23, 2026Hrishikesh Singh, Aarti Sharma, Millie PantAlgorithmic TradingActor-Critic Methods

  12. Continuous-Time Policy Gradient from Discrete Data: Convergence Analysis and Model-Free Algorithms

    Oct 20, 2026Martin Hernandez, Xinyan Qing, Mo Zhou +1Continuous-Time RLModel-Free RL

  13. From Outcomes to Steps: Reliable Success-Aware Critic Learning for Long-Horizon Language Agents

    Oct 20, 2026Jiaxi Liu, Shixiong Kai, Zhentao Tang +4Reinforcement LearningRL for LLM Agents

  14. Sample-Efficient Reinforcement Learning with General Utilities via Neural Dual Critics

    Oct 17, 2026Ankur Naskar, Vaneet AggarwalReinforcement LearningSample-Efficient RL

  15. ACRO: Actor-Critic Rollout Orchestration

    Oct 14, 2026Junhoo Lee, Seungyeon Kim, Baekseung Kim +4Robotic ManipulationRobot Policy Learning

  16. FOCUS: From Privileged States to RGB-D with Controlled Modality Switching and Representation Alignment

    Oct 8, 2026Filip Grigorov, Kourosh Darvish, Nandita VijaykumarRobotic RLVisual RL

  17. A Closed-Loop Non-Asymptotic Convergence Analysis of PPO with Learned Critics and Clipping

    Oct 7, 2026Junwei Su, Mengfan Liu, Yanyong Zhang +1Proximal Policy OptimizationActor-Critic Methods

  18. Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization

    Oct 6, 2026Myeung Suk Oh, Zhiyao Zhang, Alvaro Velasquez +2Wireless Resource AllocationDecentralized MARL

  19. Taylor Representations for Model-Free RL in Networked MDPs

    Oct 4, 2026Salah Chikhi, Abdelhaq Chaoui, Asuman Ozdaglar +1Markov Decision ProcessesRepresentation Learning for RL

  20. Policy as Data: Replay-Based Policy Dual Averaging via Advantage Regression

    Oct 3, 2026Nianli Peng, Geoffrey J. Gordon, Kianté BrantleyActor-Critic MethodsExperience Replay

  21. Amortized Score-Hamiltonian Policy Iteration: A Grid-Free Scheme for Relaxed Stochastic Control Problems

    Oct 3, 2026Qi Feng, Gu WangStochastic Optimal ControlPolicy Iteration

  22. Memory-State Critic for Asymmetric Actor-Critic with Application to Vision-Based Pursuit-Evasion

    Oct 2, 2026Arthur Louette, Alejandro Sánchez Roncero, Gaspard Lambrechts +4Asynchronous Advantage Actor-CriticActor-Critic Methods

  23. Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agents

    Oct 1, 2026Ankur Samanta, Yonathan Efroni, Paul Sajda +2Autonomous Research AgentsLong-Horizon Agent Tasks

  24. Trust the Critic More

    Sep 30, 2026Kaiyue Wen, Luke Bailey, Arvind Mahankali +1RL for Language Model ReasoningCredit Assignment in RL

  25. Privy to the Foil: Recasting Value Estimation with a Self-Privileged Critic for RLVR

    Sep 29, 2026Kun Liang, Chenming Tang, Clive Bai +5Value Function EstimationRL for Language Model Reasoning

  26. EasyPPO: Stabilizing the Critic Is Key

    Sep 29, 2026Xuanyi Zhou, Qiuyang Mang, Huanzhi Mao +7RL for Language ModelsProximal Policy Optimization

  27. ABC: Advantage-Based Control Variates for Reinforcement Learning with Verifiable Rewards

    Sep 28, 2026Hsiao-Ru Pan, Florent Draye, Bernhard SchölkopfReinforcement LearningActor-Critic Methods