Multi-Agent Off-Policy Deep Reinforcement Learning for Smart Campus Coverage

Omar Rady; Mohamed Ayman; Ali Arafa; Mohamed Shalma

cs.LGAug 19, 2026

Multi-Agent Off-Policy Deep Reinforcement Learning for Smart Campus Coverage

Authors: Omar Rady, Mohamed Ayman, Ali Arafa, Mohamed Shalma

Organizations: Faculty of Information Engineering and Technology The German University in Cairo Cairo, Egypt

Abstract

Deep reinforcement learning (DRL) has recently gained a great attention due to its real-time adaptation and effectiveness in complex optimization problems. This paper investigates the optimal deployment of millimeter-wave (mmWave) base stations (BSs) in a realistic, non-convex campus topology. The optimization problem is NP-hard, due to the non-convex, non-smooth nature of the max-min fairness objective. To overcome these constraints, we formulate the BS placement as a Markov Decision Process (MDP) and systematically benchmark four DRL schemes: a discrete single-agent Deep Q-Network (DQN), a spatially partitioned Multi-Agent DQN, a continuous single-agent Deep Deterministic Policy Gradient (DDPG), and a geographically partitioned multi-agent DDPG framework. Numerical evaluations reveal that the multi-agent DDPG approach substantially outperforms single-agent in dense scenarios. Additionally full coverage is achieved, and a fairness Jain's index of 0.94 is obtained. Finally, the multi-agent demonstrates highly efficient computational convergence of dense scenarios with $400$ users.

Figures & tables

Fig. 1: Aerial view of the full campus environment at the German University in Cairo (GUC) showing all main buildings.

Fig. 2: System model showing the 4 BSs and multiple users

Parameter Description	Value
Total Number of Users ( $K$ )	$400$ ( $100$ per zone)
Transmit Power ( $P_{\text{tx}}$ )	$1.0$ W
Noise Floor Power ( $N_{0}$ )	$1\times 10^{-10}$ W
Path Loss Exponent ( $\alpha$ )	$4.0$
BS Antenna Height	$12.0$ m
User Equipment Height	$1.7$ m

TABLE I: Simulation Configuration Parameters

Optimization Technique	Total Users ( $n$ )	Mean Rate (bps/Hz)	Min SNR (dB)	Jain’s Index ( $J$ )
Single-Agent DQN	$400$	3.17	0.01	0.71
Multi-Agent DQN	$400$	9.05	14.01	0.82
Single-Agent DDPG	$400$	$9.64$	$19.19$	$0.94$
Multi-Agent DDPG	$400$	$9.65$	$19.18$	$0.9486$

TABLE II: Coverage of Campus and Fairness Evaluation via Jain’s Index

Fig. 3: Training reward convergence profile for the Single-Agent DQN.

Fig. 4: Training reward convergence profile for the Multi-Agent DQN.

Fig. 5: Time-based convergence comparison of average rates between Single-Agent and Multi-Agent DQN frameworks.

Fig. 6: Episodic training reward progression for the continuous Single-Agent DDPG.

Fig. 7: Time-based convergence comparison of average achievable rates for Single-Agent and Multi-Agent DDPG.

Fig. 8: Optimal spatial mapping and resulting SNR distribution generated by the Multi-Agent DDPG framework.

Millimeter-Wave Communications

See all 16 papers

Energy Saving for Cell-Free Massive MIMO Networks: A Multi-Agent Deep Reinforcement Learning Approach
Apr 8, 2026Qichen Wang, Keyu Li, Ozan Alp Topal +3Wireless Communications
Department of Communication Systems, KTH Royal Institute of Technology, Sweden · Department of Electrical and Electronics Engineering, Bilkent University, Turkiye · Department of Electronic Systems, Aalborg University, Denmark
Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning
Feb 23, 2026Shan Yang, Yang LiuCooperative MARL Multi-Agent Reinforcement Learning
National University of Singapore
Heterogeneous Multi-Agent Reinforcement Learning for Radio Resource Management under Coupled Finite-Horizon Constraints
Aug 3, 2026Yeonseo Jeong, Wonhyeok Ko, Sungweon Hong +1Stochastic Optimization Fair Resource Allocation
Department of Electronic Engineering, Hanyang University, Seoul, Korea · Department of AI-Semiconductor, Hanyang University, Seoul, Korea

Apr 8, 2026J/K move · Enter open · S save

Energy Saving for Cell-Free Massive MIMO Networks: A Multi-Agent Deep Reinforcement Learning Approach

Qichen Wang, Keyu Li, Ozan Alp Topal, Özlem Tugfe Demir+2

Department of Communication Systems, KTH Royal Institute of Technology, Sweden · Department of Electrical and Electronics Engineering, Bilkent University, Turkiye · Department of Electronic Systems, Aalborg University, Denmark

Wireless Communications

Open paper AskPDF arXiv

This paper focuses on energy savings in downlink operation of cell-free massive MIMO (CF mMIMO) networks under dynamic traffic conditions. We propose a multi-agent deep reinforcement learning (MADRL) algorithm that enables each access point (AP) to autonomously control antenna re-configuration and advanced sleep mode (ASM) selection. After the training process, the proposed framework operates in a fully distributed manner, eliminating the need for centralized control and allowing each AP to dynamically adjust to real-time traffic fluctuations. Simulation results show that the proposed algorithm reduces power consumption (PC) by 56.23% compared to systems without any energy-saving scheme and by 30.12% relative to a non-learning mechanism that only utilizes the lightest sleep mode, with only a slight increase in drop ratio. Moreover, compared to the widely used deep Q-network (DQN) algorithm, it achieves a similar PC level but with a significantly lower drop ratio.

Abstract

Millimeter-Wave Communications

Explore similar work

Energy Saving for Cell-Free Massive MIMO Networks: A Multi-Agent Deep Reinforcement Learning Approach

Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning

Heterogeneous Multi-Agent Reinforcement Learning for Radio Resource Management under Coupled Finite-Horizon Constraints