Distributional Reinforcement Learning

Distributional reinforcement learning (DRL) aims to learn the entire distribution of future rewards, rather than just the expected value, offering a more nuanced understanding of uncertainty in decision-making. Current research focuses on developing efficient algorithms, often employing quantile regression, generative models (like energy-based models and diffusion models), and various distributional Bellman operators, to accurately estimate and utilize these reward distributions. This approach enhances robustness and allows for risk-sensitive decision-making, finding applications in diverse fields such as finance, robotics, and wireless network management, where handling uncertainty is crucial for optimal and safe performance.

Papers

July 15, 2022

The Nature of Temporal Difference Errors in Multi-step Distributional Reinforcement Learning
Yunhao Tang, Mark Rowland, Rémi Munos, Bernardo Ávila Pires, Will Dabney, Marc G. Bellemare
Distributional Reinforcement Learning Landscape Image Multi Step Off Policy

June 28, 2022

Risk Perspective Exploration in Distributional Reinforcement Learning
Jihwan Oh, Joonkee Kim, Se-Young Yun
Distributional Reinforcement Learning Multi Agent Setting

June 14, 2022

Robust Reinforcement Learning with Distributional Risk-averse formulation
Pierre Clavier, Stéphanie Allassonière, Erwan Le Pennec
Reinforcement Learning Native Robustness Distributional Reinforcement Learning Robust Reinforcement Learning

June 13, 2022

IGN : Implicit Generative Networks
Haozheng Luo, Tianyi Wu, Colin Feiyu Han, Zhijun Yan
GAN Model Policy OpTimization Distributional Reinforcement Learning Distributional Learning Implicit Generative State Action Distribution

May 30, 2022

A Simulation Environment and Reinforcement Learning Method for Waste Reduction
Sami Jullien, Mozhdeh Ariannezhad, Paul Groth, Maarten de Rijke
Reinforcement Learning Distributional Reinforcement Learning Reinforcement Learning Environment Reinforcement Learning Task Advanced Textile Recycling

May 14, 2022

Interpretable Stochastic Model Predictive Control using Distributional Reinforced Estimation for Quadrotor Tracking Systems
Yanran Wang, James O'Keeffe, Qiuchen Qian, David Boyle
Distributional Reinforcement Learning Aerodynamic Force Disturbance Estimation Quadrotor Navigation Stochastic Model Predictive

March 28, 2022

Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learning
Cheng Liu, Erik-Jan van Kampen, Guido C. H. E. de Croon
Reinforcement Learning Distributional Reinforcement Learning Cluttered Environment Nano Drone Adaptive Risk Risk Based Decision

March 3, 2022

On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiency
Thanh Nguyen-Tang
Reinforcement Learning Optimization Purpose Potential Scalability Distributional Reinforcement Learning Robust Reinforcement Learning Sample Efficiency

March 1, 2022

Distributional Reinforcement Learning for Scheduling of Chemical Production Processes
Max Mowbray, Dongda Zhang, Ehecatl Antonio Del Rio Chanona
Reinforcement Learning Task Scheduling Distributional Reinforcement Learning Optimal Scheduling Production Scheduling

February 6, 2022

Exploration with Multi-Sample Target Values for Distributional Reinforcement Learning
Michael Teng, Michiel van de Panne, Frank Wood
Environment Exploration Distributional Reinforcement Learning Multi Target Quantile Temporal Difference Distributional Perspective

February 1, 2022

Distributional Reinforcement Learning with Regularized Wasserstein Loss
Ke Sun, Yingnan Zhao, Wulong Liu, Bei Jiang, Linglong Kong
Distributional Reinforcement Learning Sinkhorn Algorithm

January 31, 2022

On solutions of the distributional Bellman equation
Julian Gerstenberg, Ralph Neininger, Denis Spiegel
Solution Path Distributional Reinforcement Learning Distributional Bellman

January 18, 2022

Conservative Distributional Reinforcement Learning with Safety Constraints
Hengrui Zhang, Youfang Lin, Sheng Han, Shuo Wang, Kai Lv
Policy Reinforcement Learning Safety Constraint Policy Optimization Distributional Reinforcement Learning Constrained Markov Decision Process Policy Algorithm

December 28, 2021

Robustness and risk management via distributional dynamic programming
Mastane Achab, Gergely Neu
Reinforcement Learning Native Robustness Optimal Policy Dynamic Programming Distributional Reinforcement Learning Risk Management Distributional Reduction

December 14, 2021

Conjugated Discrete Distributions for Distributional Reinforcement Learning
Björn Lindenberg, Jonas Nordqvist, Karl-Olof Lindahl
Reinforcement Learning Single Agent Distributional Reinforcement Learning Discrete Distribution Finite State Markov Distributional Reduction

November 12, 2021

Two steps to risk sensitivity
Chris Gagne, Peter Dayan
Distributional Reinforcement Learning Sample STEP Distributional Assumption Network Sensitivity Prospect Theory Risk Preference Distributional Approach

November 2, 2021

Robust Dynamic Bus Control: A Distributional Multi-agent Reinforcement Learning Approach
Jiawei Wang, Lijun Sun
Multi Agent Reinforcement Learning Distributional Reinforcement Learning Public Transit Future Transportation