stat.MLNov 2, 2026

Neural Actor--Critic Learning with Minimax Critics and Reflected Langevin Sampling

Authors: Kunqi Huang, Lican Kang, Hongrui Shu, Jincheng Ying

Organizations: School of Mathematics and Statistics, Wuhan University, Wuhan, 430072, China. · Institute for Math and AI, Hubei Key Laboratory of Computational Science, and School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China. · School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China.

Abstract

Actor--critic methods rely on critic gradients to improve continuous-action policies, yet finite-sample guarantees must control both value and gradient errors and their propagation through the actor. We develop a neural actor-critic procedure that combines a regularized minimax Bellman estimator over sigmoid networks with a finite reflected Langevin sampler driven by the fitted critic gradient. Under uniform Bellman regularity and covered reset sampling, a positive high-order action-derivative penalty yields value and action-gradient risk bounds with classical nonparametric minimax exponents up to logarithmic factors. We propagate estimation, mixing, discretization, temperature, and policy-iteration errors to obtain return guarantees, with refinements for bounded multimodal potentials and strongly concave regimes. Empirical studies across controlled simulations, logged continuous-control data, and a simulated decision task show favorable performance relative to the considered baselines.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Refined Analysis of Entropy-Regularized Actor-Critic

    May 23, 2026Safwan Labbi, Paul Mangold, Daniil Tiapkin +1Reinforcement LearningMaximum Entropy RL

  2. Schrödinger--Föllmer Actor--Critic: Diffusion Policy Improvement with Finite-Sample Analysis

    Sep 27, 2026Yuling Jiao, Lican Kang, Jerry Zhijian Yang +1Diffusion PolicyRL for Diffusion Models

  3. From Ticks to Flows: Dynamics of Neural Reinforcement Learning in Continuous Environments

    Jun 2, 2026Saket Tiwari, Tejas Kotwal, George KonidarisRL ControlReinforcement Learning