Maximum Entropy RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, with none the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 45

All topics

Revisiting Mixture Policies in Entropy-Regularized Actor-Critic
May 9, 2026Jiamin He, Samuel Neumann, Jincheng Mei +2RL Control Policy Optimization
University of Alberta & Amii · Google DeepMind · Canada CIFAR AI Chair
Generative Actor-Critic with Soft Bridge Policies
May 9, 2026Ke He, Le He, Shunpu Tang +2Policy Optimization Maximum Entropy RL
Guangzhou University · Zhejiang University · Southeast University
Entropy-Regularized Adjoint Matching for Offline Reinforcement Learning
May 7, 2026Abdelghani Ghanem, Mounir GhoghoFlow Matching Offline RL
College of Computing Mohammed VI Polytechnic University Rabat 11103, Morocco
Global Optimality for Constrained Exploration via Penalty Regularization
Apr 30, 2026Florian Wolf, Ilyas Fatkhullin, Niao HeConstrained RL Maximum Entropy RL
The Computing & Mathematical Sciences Department, California Institute of Technology, Pasadena, CA. · Department of Computer Science, ETH Zurich, Switzerland. · 3ETH AI Center, ETH Zurich, Switzerland.
Continuous-time q-learning for mean-field control with common noise, part-I: Theoretical foundations
Apr 30, 2026Zhenjie Ren, Xiaoli Wei, Xiang Yu +1Stochastic Optimal Control Q-Learning
LaMME, Universit´e ´Evry Paris-Saclay, ´Evry, France · Department of Applied Mathematics, The Hong Kong Polytechnic University, Kowloon, Hong Kong · Department of Industrial Engineering and Operations Research, Columbia University, New York, USA
Addressing Performance Saturation for LLM RL via Precise Entropy Curve Control
Apr 29, 2026Bolian Li, Yifan Wang, Yi Ding +3RL for Language Models Reinforcement Learning
Purdue University, West Lafayette, IN, USA
Maximum Entropy Semi-Supervised Inverse Reinforcement Learning
Apr 22, 2026Julien Audiffren, Michal Valko, Alessandro Lazaric +1Reinforcement Learning Maximum Entropy RL
1CMLA, ENS Cachan · 2SequeL team, INRIA Lille - Nord Europe · 3Adobe Research
Planning in entropy-regularized Markov decision processes and games
Apr 21, 2026Jean-Bastien Grill, Omar Darwiche Domingues, Pierre Ménard +2Value Function Estimation Markov Models
DeepMind Paris · SequeL team, Inria Lille
Targeted Exploration via Unified Entropy Control for Reinforcement Learning
Apr 16, 2026Chen Wang, Lai Wei, Yanzhi Zhang +5Reinforcement Learning RL for Language Model Reasoning
College of Software, Nankai University · Zhongguancun Academy · Shanghai Jiao Tong University +3
Tracking Drift: Variation-Aware Entropy Scheduling for Non-Stationary Reinforcement Learning
Jan 27, 2026Tongxi Wang, Zhuoyang Xia, Xinran Chen +1Non-Stationary RL Sequential Decision Making
School of Future Technology, Southeast University, Nanjing, China · School of Automation, Southeast University, Nanjing, China
Soft Fitted Q-Iteration without Bellman Completeness: Occupancy Reweighting and Temperature Annealing
Dec 30, 2025Lars van der Laan, Nathan KallusReinforcement Learning KL-Regularized RL
Department of Statistics, University of Washington · Netflix · Cornell University
Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning
Dec 1, 2025Sebastian Sanokowski, Kaustubh Patil, Majid KhadivRL Control Diffusion Policy
Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University Munich · Practical Project Student Exchange Program, Technical University Munich · MIT World Peace University
Action-Driven Processes for Continuous-Time Control
Oct 30, 2025Ruimin He, Shaowei LinMaximum Entropy RL Continuous Control
Ukusan Pte Ltd, Singapore
Discrete Compositional Generation via General Soft Operators and Robust Reinforcement Learning
Jun 20, 2025Marco Jiralerspong, Esther Derman, Danilo Vucetic +5Scientific Discovery Generative Modeling
1Université de Montréal · 2Mila · 5Independent +1
ELEMENT: Episodic and Lifelong Exploration via Maximum Entropy
Dec 5, 2024Hongming Li, Zhao Yang, Xiaoxuan Liang +2Intrinsic Reward Methods Reinforcement Learning
Zhejiang Lab, Huangzhou, China · Vrije Universiteit Amsterdam, Amsterdam, the Netherlands · Computational NeuroEngineering Laboratory at the University of Florida, Gainesville, USA

May 9, 2026, cs.LGJ/K move · Enter open · S save

Revisiting Mixture Policies in Entropy-Regularized Actor-Critic

Jiamin He, Samuel Neumann, Jincheng Mei, Adam White+1

University of Alberta & Amii · Google DeepMind · Canada CIFAR AI Chair

RL Control Policy Optimization Maximum Entropy RL Soft Actor-Critic Policy Learning

Open paper AskPDF arXiv

Mixture policies theoretically offer greater flexibility than unimodal policies in continuous action reinforcement learning, but the practical benefits of this complexity remain elusive. Mixture policies are notably absent from most state-of-the-art algorithms, raising a fundamental question: Is the added representational overhead useful? We show that increased flexibility can theoretically enhance solution quality and entropy robustness. Yet standard algorithms like SAC do not leverage these advantages. A core issue is the lack of a low-variance reparameterization trick for mixtures, a luxury Gaussian policies enjoy. We propose a marginalized reparameterization (MRP) estimator to address this, proving it offers lower variance than the standard likelihood-ratio (LR) approach. Our experiments across Gym MuJoCo, DeepMind Control Suite, and MetaWorld show that MRP mixture policies significantly outperform their LR ones, and reach parity (sometimes better) with Gaussian counterparts. In addition, we do find several cases where MRP mixture policies exhibit clear empirical advantages. In this paper, we provide a clearer understanding of the trade-offs involved, elevating MRP mixture policies from theoretical curiosity to a practical tool.

Momentum

Related topics

Latest papers 45