stat.MLSep 9, 2024

Bridging Rested and Restless Bandits with Graph-Triggering: Rising and Rotting

Authors: Gianmarco GenaltiMarco MussiNicola GattiMarcello RestelliMatteo CastiglioniAlberto Maria Metelli

Organizations: Politecnico di Milano2026 Piazza Leonardo da Vinci 32, Milan, 20133, Italy

Abstract

Rested and Restless Bandits are two well-known bandit settings that are useful to model real-world sequential decision-making problems in which the expected reward of an arm evolves over time due to the actions we perform or due to the nature. In this work, we propose Graph-Triggered Bandits (GTBs), a unifying framework to generalize and extend rested and restless bandits. In this setting, the evolution of the arms' expected rewards is governed by a graph defined over the arms. An edge connecting a pair of arms (i,j)(i,j) represents the fact that a pull of arm ii triggers the evolution of arm jj, and vice versa. Interestingly, rested and restless bandits are both special cases of our model for some suitable (degenerated) graph. As relevant case studies for this setting, we focus on two specific types of monotonic bandits: rising, where the expected reward of an arm grows as the number of triggers increases, and rotting, where the opposite behavior occurs. For these cases, we study the optimal policies. We provide suitable algorithms for all scenarios and discuss their theoretical guarantees, highlighting the complexity of the learning problem concerning instance-dependent terms that encode specific properties of the underlying graph structure.

Explore similar work

CardsList
  1. A single algorithm for both restless and rested rotting bandits

    Apr 23, 2026Julien Seznec, Pierre Ménard, Alessandro Lazaric +1BanditsNon-Stationarity

  2. Learning from Local Walks on Dynamic Graphs with Bandit Feedback

    Jul 12, 2026Sourav Chakraborty, Amit Kiran Rege, Claire Monteleoni +1BanditsEfficient Exploration