cs.LGFeb 9, 2026

Breaking the Grid: Distance-Guided Reinforcement Learning in Large Discrete Action Spaces

Authors: Heiko Hoppe, Fabian Akkerman, Wouter van Heeswijk, Maximilian Schiffer

Organizations: Technical University of Munich · University of Twente

Abstract

Reinforcement Learning (RL) is increasingly applied to large-scale decision-making problems like logistics, scheduling, and recommender systems, but existing algorithms struggle with the curse of dimensionality in such large discrete action spaces. We propose Distance-Guided Reinforcement Learning (DGRL), combining Sampled Dynamic Neighborhoods and Distance-Based Updates to enable efficient RL in problems with up to 103610^{36} actions. Unlike prior methods, DGRL performs stochastic volumetric exploration and transforms policy optimization into a stable regression task, decoupling gradient variance from action space cardinality. On structured tasks, DGRL provably guarantees local value improvement. DGRL naturally handles action spaces without regular semantic structure and hybrid continuous-discrete action spaces. We demonstrate performance improvements of up to 66% against state-of-the-art benchmarks across regularly and irregularly structured environments, while simultaneously improving convergence speed and computational complexity.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Mesh-RL: Coupled subgrid reinforcement learning

    Jun 24, 2026Behnam Gheshlaghi, Bahador Rashidi, Shahin AtakishiyevReinforcement LearningCredit Assignment in RL

  2. FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance

    May 29, 2026Sungha Kim, Gawon Lee, Jusuk Lee +3RL ControlMaximum Entropy RL

  3. When to Stop Reusing: Dynamic Gradient Gating for Sample-Efficient RLVR

    May 19, 2026Yuchun Miao, Sen Zhang, Yuqi Zhang +4RL for Language Model ReasoningPolicy Optimization