cs.LGJan 25, 2026

Entropic Risk-Aware Monte Carlo Tree Search

Authors: Pedro P. Santos, Jacopo Silvestrin, Manuel Lopes, Alberto Sardinha, Francisco S. Melo

Abstract

We propose a provably correct Monte Carlo tree search (MCTS) algorithm for solving risk-aware Markov decision processes (MDPs) with entropic risk measure (ERM) objectives. We provide a non-asymptotic analysis of our proposed algorithm, showing that the algorithm: (i) is correct in the sense that the empirical ERM obtained at the root node converges to the optimal ERM; and (ii) enjoys polynomial regret concentration. Our algorithm successfully exploits the dynamic programming formulations for solving risk-aware MDPs with ERM objectives introduced by previous works in the context of an upper confidence bound-based tree search algorithm. Finally, we provide a set of illustrative experiments to evaluate our risk-aware MCTS method, showing it successfully trades off between risk-neutral and risk-averse behavior.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Risk-Aware General-Utility Markov Decision Processes

    Jul 10, 2026Pedro P. Santos, Fábio Vital, Alberto Sardinha +1Markov Decision ProcessesMarkov Models

  2. Beyond Expected Values: Risk-Sensitive Planning with Distributional Monte-Carlo Tree Search

    Oct 16, 2026Quynh Trinh, Trung Nguyen, Nam Nguyen +1Risk-Sensitive PlanningMonte Carlo Tree Search

  3. Long-Term Sequential Decision Making under Risk

    Jul 22, 2026Irmaan, Mirzanejad, Nadjet Bourdache +1Markov Decision ProcessesDynamic Programming