cs.MANov 14, 2026

Delay-Aware Reinforcement Learning for O-RAN Control under Stochastic Reward Feedback

Authors: Xingqi Wu, Junaid Farooq, Tao Li, Juntao Chen, Ying Wang

Organizations: Department of Electrical and Computer Engineering, University of Michigan-Dearborn, MI 48128 USA · Department of Systems Engineering, City University of Hong Kong, Hong Kong. · Department of Computer and Information Science, Fordham University, New York, NY USA · Department of Systems Engineering, Stevens Institute of Technology, Hoboken, NJ USA

Abstract

Reinforcement learning (RL) is increasingly employed for real-time control in Open Radio Access Networks (O-RANs). However, the asynchronous nature of O-RAN feedback introduces stochastic delays between actions and their observable rewards, leading to misaligned credit assignment and unstable training. This letter formulates the O-RAN control problem as a Markov decision process (MDP) with delayed and stochastic rewards, where the true reward corresponding to an action is received after an unknown delay. To address this, we propose a delay-aware deep RL framework that integrates a long short-term memory (LSTM)-based delay predictor and a temporal reward realignment module within an off-policy learning pipeline. Using empirical delay traces collected from the Open AI Cellular (OAIC) testbed, we demonstrate that the proposed framework achieves faster convergence and significantly reduces latency violations and packet drops compared to conventional RL methods that ignore feedback delay. The results confirm that reward-delay compensation is critical for reliable AI-driven O-RAN control.

Figures & tables

Explore similar work

CardsList
  1. Residual Reinforcement Learning for Robot Teleoperation under Stochastic Delays

    May 14, 2026Kaize Deng, Zewen YangRobotic ControlRL Control

  2. Diffusion-Guided Uncertainty-Aware Delayed Policy Optimization

    Jul 6, 2026Junqi Tu, Zejiao Liu, Fangfei Li +1Reinforcement LearningRobotic RL

  3. Adaptive Reinforcement Learning for Unobservable Random Delays

    Jun 17, 2025John Wikman, Alexandre Proutiere, David BromanReinforcement LearningRobotic RL