cs.LGFeb 4, 2026

Decoupling Time and Risk: Risk-Sensitive Reinforcement Learning with General Discounting

Authors: Mehrdad Moghimi, Anthony Coache, Hyejin Ku

Organizations: Dept. of Mathematics and Statistics, York University, Toronto, Canada · Dept. of Mathematics, Imperial College London, London, United Kingdom

Abstract

Distributional reinforcement learning (RL) is a powerful framework increasingly adopted in safety-critical domains for its ability to optimize risk-sensitive objectives. However, the role of the discount factor is often overlooked, as it is typically treated as a fixed parameter of the Markov decision process or tunable hyperparameter, with little consideration of its effect on the learned policy. In the literature, it is well-known that the discounting function plays a major role in characterizing time preferences of an agent, which an exponential discount factor cannot fully capture. Building on this insight, we propose a novel framework that supports flexible discounting of future rewards and optimization of risk measures in distributional RL. We provide a technical analysis of the optimality of our algorithms, show that our multi-horizon extension fixes issues raised with existing methodologies, and validate the robustness of our methods through extensive experiments. Our results highlight that discounting is a cornerstone in decision-making problems for capturing more expressive temporal and risk preferences profiles, with potential implications for real-world safety-critical applications.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Finite-Time Analysis of Discounted Exponential-Utility Reinforcement Learning

    Aug 3, 2026Ankur Naskar, Vivek T A, Aditya Kumar +2Reinforcement LearningRisk-Sensitive RL

  2. Adaptive Multi-Horizon Reinforcement Learning

    Jul 22, 2026Manoosh Samiei, Doina Precup, Paul MassetContinual RL