Maximum Entropy RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, with none the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 45

All topics

LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL
Oct 6, 2026Songyuan Zhang, Oswin So, Eric Yang Yu +3Reinforcement Learning Offline RL
MIT · MIT Lincoln Laboratory
FERPO: Forward Entropy-Regularized Policy Optimization
Oct 1, 2026Sebastian Sanokowski, Alireza Sarmadi, Majid KhadivKL-Regularized RL Policy Optimization
Applied and Theoretical Aspects of Robot Intelligence (ATARI) Lab, Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich
An analysis of Mirror-Descent Soft Actor-Critic
Sep 28, 2026Denis Zorba, Michal ValkoReinforcement Learning Online Mirror Descent
School of Mathematics, University of Edinburgh, United Kingdom · INRIA, Paris, France
Surprising Success, Repeated Failure: Entropy-Guided Credit Assignment for Exploration in LLM Reasoning
Sep 27, 2026Woongyeong Yeo, Minki Kang, Chanuk Lee +3RL for Language Model Reasoning Credit Assignment in RL
KAIST
RLVR landscapes for iterated multiplications can be benign: Insights from spin-glass theory
Sep 23, 2026Noa Rubin, Zohar RingelReinforcement Learning Maximum Entropy RL
Racah Institute of Physics The Hebrew University of Jerusalem
SCQ: Stabilizing Conservative Q-Learning with Sigmoid-Bounded Entropy
Sep 14, 2026Xiefeng Wu, Shu Zhang, Zhaojie Chu +1Robotic RL Offline RL
School of Computer Science, Wuhan University, Wuhan, China · School of Electronic Information, Wuhan University, Wuhan, China · School of Internet, Anhui University, Hefei, China
Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence
Sep 14, 2026Jin Ma, Gaozhan Wang, Jianfeng Zhang +1Reinforcement Learning Stochastic Optimal Control
Department of Mathematics, University of Southern California, Los Angeles, CA 90089 · Department of Industrial Engineering and Operations Research, Columbia University, New York, NY 10027
Wasserstein Policy Gradient for Entropy-Regularized Linear-Quadratic Control
Aug 7, 2026Zhaoyu Zhu, Rui Gao, Shuang LiMaximum Entropy RL Policy Gradient Methods
University of Illinois Urbana-Champaign · The University of Texas at Austin · The Chinese University of Hong Kong, Shenzhen
Distributional Soft Bellman Operator under the Cramér Geometry
Jul 20, 2026Keru Wang, Yixin Deng, Yao Lyu +2Distributional RL Maximum Entropy RL
School of Electrical and Electronic Engineering, University College Dublin, Dublin, Ireland · School of Vehicle and Mobility, Tsinghua University, Beijing, China · College of Artificial Intelligence, Tsinghua University, Beijing, China
Rationalizing Boltzmann Rationality: An Axiomatic Characterization of Entropy-Regularized Policies
Jul 19, 2026Silviu PitisMarkov Decision Processes Maximum Entropy RL
University of Michigan
Group Entropy-Controlled Policy Optimization
Jul 18, 2026Guangran Cheng, Chengqi Lyu, Songyang Gao +2RL for Language Models Group Relative Policy Optimization
Shanghai AI Laboratory
Entropy Pacing Policy Optimization for Multi-Task Agentic Reinforcement Learning
Jul 8, 2026Zetian Hu, Shunyu Liu, Junjie Zhang +4Agentic RL Group Relative Policy Optimization
Generative AI Lab, College of Computing and Data Science, Nanyang Technological University, Singapore 639798 · Tongyi Lab, Alibaba Group
Entropy Regularization Improves Policy Robustness in Continuous-Time Reinforcement Learning
Jul 3, 2026Jialun Cao, Fernando Acero, David Šiška +1Robust Markov Decision Processes Maximum Entropy RL
School of Mathematics, University of Edinburgh, United Kingdom. · J.P. Morgan AI Research, United Kingdom. · Department of Mathematics, Imperial College London, United Kingdom.
ACPO: Adaptive Credit Policy Optimization via Fine-Grained Surrogate Entropy
Jul 3, 2026Zijun Xie, Yuyang You, Yongzhi Li +6Token-Level Credit Assignment RL for Language Model Reasoning
Peking University · Baidu Inc. · Kuaishou Inc.
Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Models
Jun 27, 2026Congde Hu, Danping Li, Lin Xu +1Stackelberg Games Maximum Entropy RL
School of Mathematics and Statistics, Anhui Normal University, Wuhu, Anhui 241002, China · School of Statistics, Key Laboratory of Advanced Theory and Application in Statistics and Data Science-MOE, Faculty of Economics and Management, East China Normal University, Shanghai, China · School of Mathematics, Southeast University, Nanjing, Jiangsu 211189, China
Entropy Regularized Reinforcement Learning for Zero-Sum Stochastic Differential Games in a Regime-Switching Jump-Diffusion Process
Jun 27, 2026Congde Hu, Zhuo Jin, Danping Li +1Zero-Sum Games Maximum Entropy RL
School of Mathematics and Statistics, Anhui Normal University, Wuhu, 241002, Anhui, China. · Department of Actuarial Studies and Business Analytics, Macquarie University, Sydney, 2109, NSW, Australia. · School of Statistics, KLATASDS-MOE, China Inclusive Ageing Finance Research Center, East China Normal University, Shanghai, 200062, Shanghai, China.
Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching
Jun 21, 2026Serge Thilges, Onur Celik, Denis Blessing +2Diffusion Policy Maximum Entropy RL
Autonomous Learning Robots, Karlsruhe Institute of Technology
STARE: Surprisal-Guided Token-Level Advantage Reweighting for Policy Entropy Stability
Jun 17, 2026Haipeng Luo, Qingfeng Sun, Songli Wu +4RL for Language Model Reasoning Maximum Entropy RL
Shenzhen International Graduate School, Tsinghua University · Tencent Hunyuan
Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward
Jun 15, 2026Şevket Kaan Alkır, Naci Saldı, Berkay Anahtarcı +1Reinforcement Learning Average-Reward RL
Department of Mathematics Bilkent University 06800 Ankara, Turkey · Department of Mathematical Engineering Özyeğin University 34794 Istanbul, Turkey
Mitigating Bias in Low-SNR Financial Reinforcement Learning via Quantum Representations
Jun 9, 2026Zeyu Liu, Xuanzhi Feng, Sing Kwong Lai +6Quantitative Finance Quantum Machine Learning
The Hong Kong University of Science and Technology
PAEC: Position-Aware Entropy Calibration for LLM Reasoning in RLVR
Jun 7, 2026Shumeng Yang, Yisu Liu, Jiayi Zheng +2RL for Language Model Reasoning Maximum Entropy RL
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · School of Artificial Intelligence, Beijing University of Posts and Telecommunications +2
Selective-Advantage Entropy-Adaptive Horizon GRPO: Asymmetric Token-Level Discounting for Efficient Reinforcement Learning of Language Models
Jun 3, 2026Chirag Chawla, Rohan Charudatt Salvi, Madhav S. BaidyaReinforcement Learning RL for Language Model Reasoning
Indian Institute of Technology (BHU), Varanasi, India · Department of Computer Science, University of Illinois Chicago, IL 60607, USA
FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance
May 29, 2026Sungha Kim, Gawon Lee, Jusuk Lee +3RL Control Maximum Entropy RL
Seoul National University · Georgia Institute of Technology
Global Convergence of Wasserstein Policy Gradient for Entropy-Regularized Reinforcement Learning
May 25, 2026Zhaoyu Zhu, Rui Gao, Shuang LiOptimization Convergence Analysis Wasserstein Gradient Flows
Shanghai Jiao Tong University · The University of Texas at Austin · The Chinese University of Hong Kong, Shenzhen
Global linear convergence of entropy-regularized softmax policy gradient beyond tabular MDPs
May 24, 2026Ziyue Chen, David Šiška, Lukasz SzpruchPolicy Gradient Reinforcement Learning
Refined Analysis of Entropy-Regularized Actor-Critic
May 23, 2026Safwan Labbi, Paul Mangold, Daniil Tiapkin +1Reinforcement Learning Maximum Entropy RL
CMAP, CNRS, École Polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France · Université Paris-Saclay, CNRS, LMO, 91405, Orsay, France · Google DeepMind +2
A note on convergence of Wasserstein policy optimization
May 21, 2026David Šiška, Yufei ZhangWasserstein Gradient Flows Policy Optimization
School of Mathematics, University of Edinburgh, United Kingdom · Department of Mathematics, Imperial College London, United Kingdom
Stochastic MeanFlow Policies: One-Step Generative Control with Entropic Mirror Descent
May 20, 2026Zeyuan Wang, Da Li, Yulin Chen +6Reinforcement Learning Maximum Entropy RL
Laboratory for Big Data and Decision, National University of Defense Technology, China · Samsung AI Center Cambridge, UK · Queen Mary University of London, UK +3
Behavior-Consistent Deep Reinforcement Learning
May 20, 2026Marcel Hussing, Liv G. d'Aliberti, Claas Voelcker +2Reinforcement Learning Policy Optimization
University of Pennsylvania · Princeton University · University of Texas at Austin
Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control
May 12, 2026Jiazheng Zhang, Ziche Fu, Junrui Shen +17RL for Language Model Reasoning Policy Optimization
Fudan NLP Group, Honor Device Co., Ltd · University of Hong Kong, Shanghai Jiao Tong University and Tencent Hunyuan

Oct 6, 2026, cs.LGJ/K move · Enter open · S save

LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL

Songyuan Zhang, Oswin So, Eric Yang Yu, Matthew Cleaveland+2

MIT · MIT Lincoln Laboratory

Reinforcement Learning Offline RL Maximum Entropy RL

Open paper AskPDF arXiv

While offline reinforcement learning (RL) enables policy optimization from static datasets without costly online interaction, it remains bottlenecked by the risk of executing out-of-distribution (OOD) actions. Recent approaches mitigate this by learning a behavior-cloning policy through flow matching and then performing RL within its constrained latent space. However, naively optimizing the latent policy can easily cause the policy to collapse into a brittle mode or exploit sharp artifacts of the learned critic. In this work, we find that entropy regularization is essential in latent-space RL for addressing these challenges. We introduce LASER, a novel offline RL algorithm that applies latent-space adjoint matching to achieve entropy-regularized latent-space RL with expressive flow policies while avoiding backpropagation through time. Through comprehensive experiments on 40 challenging OGBench tasks with varying dataset qualities, we show that LASER achieves state-of-the-art performance. Notably, LASER uses fixed method-specific hyperparameters across all tasks and outperforms the evaluated baselines, including those with task- and dataset-specific tuning, which highlights the robust applicability of LASER. Project website: https://mit-realm.github.io/laser/.

Momentum

Related topics

Latest papers 45