Distributional RL

RL: Reinforcement Learning

Ask about this topic

Overview

Distributional reinforcement learning studies the probability distribution of future rewards, rather than only their expected value, which can reveal uncertainty and rare high-cost outcomes. Common approaches represent returns with categorical supports, quantiles, particles, or learned flows, and train them using distributional Bellman updates; risk-sensitive methods then use measures such as quantiles or conditional value at risk to guide decisions. The methods are tested in settings including Atari, MuJoCo, D4RL offline control, robotics, and safety-critical navigation, while theory increasingly addresses convergence, sample complexity, and statistical inference. A key open tension is whether learned return distributions reliably capture environmental risk: distributional critics can improve robustness, but their risk claims require careful validation and do not follow automatically from modeling a distribution.

Latest papers

CardsList

49 papers4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

  1. Revisiting Adam for Streaming Reinforcement Learning

    May 7, 2026Florin Gogianu, Adrian Catalin Lutu, Razvan PascanuDistributional RLReinforcement Learning

    Bitdefender, Bucharest, Romania · Mila, Montreal, Canada

  2. Quantile Geometry Regularization for Distributional Reinforcement Learning

    May 5, 2026Zhaofan Zhang, Minghao Yang, Rufeng Chen +2Distributional RLQuantile Regression

    Information Hub, AI Thrust Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China

  3. Towards Efficient and Expressive Offline RL via Flow-Anchored Noise-conditioned Q-Learning

    May 3, 2026Sungyoung Lee, Dohyeong Kim, Eshan Balachandar +2Distributional RLQ-Learning

    The University of Texas at Austin, Austin, TX, USA · Independent Researcher, Seoul, South Korea.

  4. Distributional Reinforcement Learning via the Cramér Distance

    Apr 26, 2026Vanya Aziz, Ivo Nowak, E. M. T HendrixDistributional RLRobotic RL

    HAW Hamburg · de Málaga

  5. Distributional Value Estimation Without Target Networks for Robust Quality-Diversity

    Apr 22, 2026Behrad Koohy, Jamie BayneDistributional RLReinforcement Learning

    Luffy.AI Culham, Oxfordshire, United Kingdom

  6. Distributional Off-Policy Evaluation with Deep Quantile Process Regression

    Apr 20, 2026Qi Kuang, Chao Wang, Yuling Jiao +1Distributional RLQuantile Regression

    School of Statistics and Data Science, and Philosophy and Social Sciences Laboratory of Data Science in Finance and Economics at the Ministry of Education,2026 Jiangxi University of Finance and Economics · School of Statistics and Data Science, Shanghai University of Finance and EconomicsApr · School of Artificial Intelligence, and Hubei Key Laboratory of Computational Science, 24 Wuhan University

  7. Unleashing Implicit Rewards: Prefix-Value Learning for Distribution-Level Optimization

    Apr 14, 2026Shiping Gao, Hongzhan Chen, Xiaojun Quan +2Process Reward ModelsRL for Language Model Reasoning

    School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, Guangdong, China · Shenzhen Loop Area Institute, Shenzhen, Guangdong, China · Meta AI, Menlo Park, California, USA +1

  8. StructRL: Recovering Dynamic Programming Structure from Learning Dynamics in Distributional Reinforcement Learning

    Apr 9, 2026Ivo NowakReinforcement LearningDistributional RL

    HAW-Hamburg

  9. Boosted Distributional Reinforcement Learning: Analysis and Healthcare Applications

    Apr 6, 2026Zequn Chen, Wesley J. MarreroReinforcement LearningDistributional RL

    Thayer School of Engineering, Dartmouth College

  10. Flow-based Policy With Distributional Reinforcement Learning in Trajectory Optimization

    Apr 1, 2026Ruijie Hao, Longfei Zhang, Yang Dai +3RL ControlDistributional RL

    College of Systems Engineering, National University of Defense Technology, Changsha 410073, China · Aviation University of Air Force, Changchun 130000, China

  11. Federated Distributional Reinforcement Learning with Distributional Critic Regularization

    Mar 18, 2026David Millard, Cecilia Alm, Rashid Ali +2Federated RLDistributional RL

    Dept. of Mechanical Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA · Dept. of Psychology and School of Information, Rochester Institute of Technology, Rochester, NY 14623, USA · Department of Engineering Science, University West, Trollhättan 461 30, Sweden +1

  12. FastDSAC: Unlocking the Potential of Maximum Entropy RL in High-Dimensional Humanoid Control

    Mar 13, 2026Jun Xue, Junze Wang, Shanze Wang +3Maximum Entropy RLRobotic RL

    College of Information Science and Technology, Eastern Institute of Technology, Ningbo, China · School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · College of Computer Science and Technology, China University of Petroleum(East China), Qingdao, China

  13. A Spectral Revisit of the Distributional Bellman Operator under the Cramér Metric

    Mar 13, 2026Keru Wang, Yixin Deng, Yao Lyu +2Distributional RL

    School of Electrical and Electronic Engineering, University College Dublin, Dublin, Ireland · School of Vehicle and Mobility, Tsinghua University, Beijing, China · College of Artificial Intelligence, Tsinghua University, Beijing, China

  14. Joint MDPs and Reinforcement Learning in Coupled-Dynamics Environments

    Mar 6, 2026Ege C. Kaya, Mahsa Ghasemi, Abolfazl HashemiSemi-Markov Decision ProcessMarkov Decision Processes

    Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA

  15. Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Driving

    Mar 6, 2026Ahmed Abouelazm, Jonas Michel, Daniel Bogdoll +2Multi-Objective Reinforcement LearningRL for Autonomous Driving

    FZI Research Center for Information Technology, Germany · Karlsruhe Institute of Technology, Germany

  16. Distributional Reinforcement Learning with Information Bottleneck for Uncertainty-Aware DRAM Equalization

    Mar 5, 2026Muhammad Usama, Dong Eui ChangRisk-Sensitive RLDistributional RL

    Control Laboratory, School of Electrical Engineering, KAIST, Daejeon, 34141, Republic of Korea

  17. LAD: Learning Advantage Distribution for Reasoning

    Feb 23, 2026Wendi Li, Sharon LiRL for Language Model ReasoningRL Post-Training

    Department of Computer Sciences, University of Wisconsin–Madison

  18. FlowCritic: Bridging Value Estimation with Flow Matching in Reinforcement Learning

    Oct 26, 2025Shan Zhong, Shutong Ding, He Diao +3Value Function EstimationDistributional RL

    School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China · School of Information Science and Technology, ShanghaiTech University · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore +1

  19. On the Approximation and Convergence of Distributional Policy Gradient Algorithms for Risk-Sensitive Reinforcement Learning

    May 23, 2024Xian Yu, Minheng Xiao, Lei YingStochastic OptimizationDistributional RL

    Department of Integrated Systems Engineering, The Ohio State University, Columbus, OH, USA · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA