Distributional RL

RL: Reinforcement Learning

Ask about this topic

Overview

Distributional reinforcement learning studies the probability distribution of future rewards, rather than only their expected value, which can reveal uncertainty and rare high-cost outcomes. Common approaches represent returns with categorical supports, quantiles, particles, or learned flows, and train them using distributional Bellman updates; risk-sensitive methods then use measures such as quantiles or conditional value at risk to guide decisions. The methods are tested in settings including Atari, MuJoCo, D4RL offline control, robotics, and safety-critical navigation, while theory increasingly addresses convergence, sample complexity, and statistical inference. A key open tension is whether learned return distributions reliably capture environmental risk: distributional critics can improve robustness, but their risk claims require careful validation and do not follow automatically from modeling a distribution.

Latest papers

CardsList

49 papers4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

  1. Near Minimax Rate for Quantile Distributional Reinforcement Learning

    Oct 21, 2026Zijie Cheng, Zhihua ZhangDistributional RLPolicy Evaluation

    School of Mathematical Sciences, Peking University

  2. Stein Variational Distributional Reinforcement Learning: Score-Guided Particles for Multivariate Return Distributions

    Oct 13, 2026Manuel Haußmann, Melih KandemirDistributional RLRisk-Sensitive RL

    University of Southern Denmark

  3. Posterior-Averaged Bellman Flow Matching for Distributional Critics

    Oct 13, 2026Zeqiao Li, Yijing Wang, Haoyu Wang +2Distributional RLFlow Matching

    Tianjin Key Laboratory of Intelligent Unmanned Swarm Technology and System, School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China · Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China

  4. Variance-Averse nn-Step Offline Reinforcement Learning for Sparse Long-Horizon Environments

    Oct 6, 2026Guhyeon Kang, Minhae KwonDistributional RLRisk-Sensitive RL

    Department of Electrical and Computer Engineering Sungkyunkwan University, Republic of Korea

  5. Retimed Bellman Flows: Escaping the Impossible Triangle of Velocity Bootstrapping

    Sep 26, 2026Boyang Xu, Shengzhe Chen, Hao YanReinforcement LearningDistributional RL

    Arizona State University

  6. Reinforcement Learning under State and Outcome Uncertainty: A Foundational Distributional Perspective

    Sep 21, 2026Larry Preuett, Qiuyi Zhang, Muhammad Aurangzeb AhmadDistributional RLMarkov Decision Processes

    University of Washington, Tacoma, USA · Google DeepMind, California, USA · University of Washington, Bothell, USA

  7. Deep Reinforcement Learning with Buffered Quantile Objectives

    Sep 18, 2026Mohammad Alipour-vaezi, Sajad KhodadadianRisk-Sensitive RLDistributional RL

    Virginia Tech

  8. A Finite-Sample Analysis of Quantile Temporal-Difference Learning

    Aug 27, 2026Zijie Cheng, Xiang Li, Yang Peng +1Distributional RLStochastic Approximation

    School of Mathematical Sciences, Peking University · Perelman School of Medicine, University of Pennsylvania · Yau Mathematical Sciences Center, Tsinghua University

  9. Learning to Control Coupled-Dynamics Environments with Joint Markov Decision Processes

    Aug 24, 2026Ege C. Kaya, Aliasghar Pourghani, Mahsa Ghasemi +2Semi-Markov Decision ProcessMarkov Decision Processes

    Elmore Family School of Electrical and Computer Engineering Purdue University

  10. Risk-Sensitive Reinforcement Learning with Smoothed Quantile Objectives

    Aug 23, 2026Mohammad Alipour-Vaezi, Huaiyang Zhong, Sajad KhodadadianRisk-Sensitive RLRegret Minimization in RL

    Grado Department of Industrial & Systems Engineering, Virginia Tech, Blacksburg, VA 24061, USA

  11. Online Inference in Distributional Temporal-Difference Learning

    Aug 14, 2026Yang Peng, Liangyu ZhangDistributional RLTemporal-Difference Learning

    Yau Mathematical Sciences Center, Tsinghua University. · School of Statistics and Data Science, Shanghai University of Finance and Economics.

  12. Online Inference for Quantile Distributional Reinforcement Learning

    Aug 13, 2026Zijie Cheng, Yang Peng, Zhihua ZhangDistributional RLReinforcement Learning

    School of Mathematical Sciences, Peking University · Yau Mathematical Sciences Center, Tsinghua University

  13. Distributional Soft Bellman Operator under the Cramér Geometry

    Jul 20, 2026Keru Wang, Yixin Deng, Yao Lyu +2Distributional RLMaximum Entropy RL

    School of Electrical and Electronic Engineering, University College Dublin, Dublin, Ireland · School of Vehicle and Mobility, Tsinghua University, Beijing, China · College of Artificial Intelligence, Tsinghua University, Beijing, China

  14. Active Beyond-Diagonal RIS Empowered Heterogeneous Edge Computing: A Distributional Reinforcement Learning Approach

    Jul 14, 2026Tianyu Pang, Hongyu LiDistributional RLReconfigurable Intelligent Surfaces

    Internet of Things Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

  15. Auditing the Risk Claims of Distributional Reinforcement Learning

    Jul 13, 2026Hari PrasadDistributional RLAlgorithmic Auditing

  16. Statistical Efficiency and Inference of Quantile Distributional Reinforcement Learning

    Jul 9, 2026Zijie Cheng, Yang Peng, Zhihua ZhangDistributional RLReinforcement Learning

    School of Mathematical Sciences, Peking University · Yau Mathematical Sciences Center, Tsinghua University

  17. Learning the Supports for Categorical Critic in Reinforcement Learning

    Jul 2, 2026Jen-Yen Chang, Takayuki Osa, Tatsuya HaradaDistributional RL

    Graduate School of Information Science and Technology, The University of Tokyo · RIKEN AIP

  18. FastDSAC: Enhancing Policy Plasticity via Constrained Exploration for Scalable Humanoid Locomotion

    Jun 30, 2026Guanchen Lu, Yajuan Dun, Yi Zhou +4Distributional RLReinforcement Learning

    Chongqing University, Chongqing 400044, China · Ministry of Industry and Information Technology, Beijing 100804, China · Tsinghua University, Beijing 100084, China +1

  19. Dual-Flow Reinforcement Learning with State-Aware Exploration

    Jun 29, 2026Qijun Li, Zheng Fu, Qi Song +4Distributional RLReinforcement Learning

    School of Vehicle and Mobility and the State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing 100084, China

  20. RS-Diffuser: Risk-Sensitive Diffusion Planning with Distributional Value Guidance

    Jun 26, 2026Shiqiang GongDistributional RLDiffusion-Based Planning

    Northwestern Polytechnical University

  21. DRIVE: Distributional and Retrieval-Augmented Bidding with Value Evaluation

    Jun 12, 2026Miduo Cui, Haochen Wang, Shangqin Mao +6Distributional RLAutomated Bidding

    Shandong University, Jinan, China · Meituan, Beijing, China

  22. QnRL: Quantum-Native Reinforcement Learning

    Jun 6, 2026Alexander DeRieux, Walid SaadDistributional RLQuantum Generative Modeling

    Bradley Department of Electrical and Computer Engineering, Virginia Tech Institute for Advanced Computing, Alexandria, VA 22305 USA

  23. Reinforcement Learning from Rich Feedback with Distributional DAgger

    Jun 3, 2026Rishabh Agrawal, Jacob Fein-Ashley, Paria RashidinejadDistributional RLRL for Language Model Reasoning

    University of Southern California

  24. Multivariate Distributional Reinforcement Learning Using Sliced Divergences

    May 29, 2026Baptiste Debes, Tinne TuytelaarsDistributional RL

    Department of Electrical Engineering (ESAT), KU Leuven, Leuven, Belgium.

  25. On Distributional Reinforcement Learning in Chaotic Dynamical Systems

    May 28, 2026James Rudd-Jones, Mirco Musolesi, María Pérez-OrtizDistributional RLReinforcement Learning

    Centre for Artificial Intelligence Department of Computer Science University College London London, UK · Department of Computer Science and Engineering, University of Bologna Bologna, Italy

  26. Reinforcement Learning over Predictive Distributions for LLM Regression

    May 20, 2026Jungsoo Park, Hyungjoo Chae, Ethan Mendes +4Distributional RLRL for Language Models

    Georgia Institute of Technology · Allen Institute for AI

  27. Quotient-Categorical Representations for Bellman-Compatible Average-Reward Distributional Reinforcement Learning

    May 11, 2026Ege C. Kaya, Aliasghar Pourghani, Vijay Gupta +1Distributional RLReinforcement Learning

    Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47906

  28. Quantile-Coupled Flow Matching for Distributional Reinforcement Learning

    May 8, 2026Michael Groom, Victor-Alexandru Darvariu, Lars Kunze +2Distributional RLConditional Flow Matching

    Oxford Robotics Institute, University of Oxford, Oxford, United Kingdom · Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom · Dyson Institute of Engineering & Technology, Malmesbury, United Kingdom

  29. A Sharp Finite-Iteration Theory for Asynchronous Categorical Distributional Temporal-Difference Learning

    May 7, 2026Ege C. Kaya, Abolfazl HashemiDistributional RLReinforcement Learning

    Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47906

  30. Path-Coupled Bellman Flows for Distributional Reinforcement Learning

    May 7, 2026Boyang Xu, Qing Zou, Siqin Yang +1Distributional RLFlow Matching

    School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA