Maximum Entropy RL

RL: Reinforcement Learning

Overview

Maximum entropy reinforcement learning adds a preference for stochastic, high-entropy actions to reward maximization, yielding soft value functions and Boltzmann-like policies that can improve exploration and retain multiple useful behaviors. Common approaches include soft actor-critic, entropy-regularized policy gradients and Q-learning, plus temperature schedules and state-, task-, or token-level entropy control. Current work extends these methods to expressive diffusion, flow, and mixture policies, offline and multi-agent learning, and continuous-time control, while theory examines convergence and robustness. A central tension is preserving enough entropy to discover diverse high-value solutions without weakening reward optimization, destabilizing critics, or making policy densities and training too costly.

Latest papers

CardsList

72 papers3 papers in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

  1. Not All Entropy Is Equal: Reward-Conditioned Entropy Allocation in Reinforcement Learning with Verifiable Rewards

    Nov 6, 2026Song Yu, Li Li, Zhenming WuEntropy Collapse in RLRL for Language Model Reasoning

  2. Entropy-Regularized Reinforcement Learning for the Regime-Switching Consumption-Investment Problem

    Oct 31, 2026Congde Hu, Danping Li, Xun Li +1Financial Decision-MakingContinuous-Time RL

  3. Conditional Flow Models for Offline Maximum Entropy Reinforcement Learning

    Oct 23, 2026Hao Cheng, Yuling Jiao, Lican Kang +2Offline RLMaximum Entropy RL

  4. DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement Learning

    Oct 12, 2026Ruslan Ageev, Huaze Tang, Runqing Miao +2Flow-Based Policy LearningReinforcement Learning

  5. Drift, Then Spread: Learning One-Step Generative Policies for Online Reinforcement Learning

    Oct 11, 2026Chunyu Luo, Hongyu Zhao, Guangyi Ni +2Reinforcement LearningMaximum Entropy RL

  6. Reinforcement Learning via Fixed-point Flow Matching

    Oct 9, 2026Denis Blessing, Onur Celik, Gerhard NeumannFlow-Based Policy LearningReinforcement Learning

  7. LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL

    Oct 6, 2026Songyuan Zhang, Oswin So, Eric Yang Yu +3Reinforcement LearningOffline RL

  8. Amortized Score-Hamiltonian Policy Iteration: A Grid-Free Scheme for Relaxed Stochastic Control Problems

    Oct 3, 2026Qi Feng, Gu WangStochastic Optimal ControlPolicy Iteration

  9. FERPO: Forward Entropy-Regularized Policy Optimization

    Oct 1, 2026Sebastian Sanokowski, Alireza Sarmadi, Majid KhadivKL-Regularized RLPolicy Optimization

  10. An analysis of Mirror-Descent Soft Actor-Critic

    Sep 28, 2026Denis Zorba, Michal ValkoReinforcement LearningOnline Mirror Descent

  11. RLVR landscapes for iterated multiplications can be benign: Insights from spin-glass theory

    Sep 23, 2026Noa Rubin, Zohar RingelReinforcement LearningMaximum Entropy RL

  12. Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence

    Sep 14, 2026Jin Ma, Gaozhan Wang, Jianfeng Zhang +1Reinforcement LearningStochastic Optimal Control

  13. SCQ: Stabilizing Conservative Q-Learning with Sigmoid-Bounded Entropy

    Sep 11, 2026Xiefeng Wu, Shu Zhang, Zhaojie Chu +1Offline-To-Online RLConservative RL

  14. Learning Complementary Routes for Scouting Persimmon Orchards during Shoot Growth with CoverShare

    Aug 27, 2026Junhao Cao, Tao Wang, Zipeng Xie +3RL ExplorationCredit Assignment in RL

  15. Where Entropy Is Measured Matters: Policy Geometry in Bounded Continuous-Control PPO

    Aug 25, 2026Yiyang He, Zhichun Zhou, Ziwei Wang +2Policy OptimizationProximal Policy Optimization

  16. Wasserstein Policy Gradient for Entropy-Regularized Linear-Quadratic Control

    Aug 7, 2026Zhaoyu Zhu, Rui Gao, Shuang LiMaximum Entropy RLPolicy Gradient Methods

  17. Distributional Soft Bellman Operator under the Cramér Geometry

    Jul 20, 2026Keru Wang, Yixin Deng, Yao Lyu +2Distributional RLMaximum Entropy RL

  18. Group Entropy-Controlled Policy Optimization

    Jul 18, 2026Guangran Cheng, Chengqi Lyu, Songyang Gao +2RL for Language ModelsGroup Relative Policy Optimization

  19. Entropy Pacing Policy Optimization for Multi-Task Agentic Reinforcement Learning

    Jul 8, 2026Zetian Hu, Shunyu Liu, Junjie Zhang +4Agentic RLGroup Relative Policy Optimization

  20. Entropy Regularization Improves Policy Robustness in Continuous-Time Reinforcement Learning

    Jul 3, 2026Jialun Cao, Fernando Acero, David Šiška +1Robust Markov Decision ProcessesMaximum Entropy RL

  21. ACPO: Adaptive Credit Policy Optimization via Fine-Grained Surrogate Entropy

    Jul 3, 2026Zijun Xie, Yuyang You, Yongzhi Li +6Token-Level Credit AssignmentRL for Language Model Reasoning

  22. Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Models

    Jun 27, 2026Congde Hu, Danping Li, Lin Xu +1Stackelberg GamesMaximum Entropy RL

  23. Entropy Regularized Reinforcement Learning for Zero-Sum Stochastic Differential Games in a Regime-Switching Jump-Diffusion Process

    Jun 27, 2026Congde Hu, Zhuo Jin, Danping Li +1Zero-Sum GamesMaximum Entropy RL

  24. Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching

    Jun 21, 2026Serge Thilges, Onur Celik, Denis Blessing +2Diffusion PolicyMaximum Entropy RL

  25. STARE: Surprisal-Guided Token-Level Advantage Reweighting for Policy Entropy Stability

    Jun 17, 2026Haipeng Luo, Qingfeng Sun, Songli Wu +4RL for Language Model ReasoningMaximum Entropy RL

  26. Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward

    Jun 15, 2026Şevket Kaan Alkır, Naci Saldı, Berkay Anahtarcı +1Reinforcement LearningAverage-Reward RL

  27. Mitigating Bias in Low-SNR Financial Reinforcement Learning via Quantum Representations

    Jun 9, 2026Zeyu Liu, Xuanzhi Feng, Sing Kwong Lai +6Quantitative FinanceQuantum Machine Learning

  28. PAEC: Position-Aware Entropy Calibration for LLM Reasoning in RLVR

    Jun 7, 2026Shumeng Yang, Yisu Liu, Jiayi Zheng +2RL for Language Model ReasoningMaximum Entropy RL