Enhancing User Throughput in Multi-panel mmWave Radio Access Networks for Beam-based MU-MIMO Using a DRL Method

Ramin Hashemi; Vismika Ranasinghe; Teemu Veijalainen; Petteri Kela; Risto Wichman

cs.ITMar 3, 2026

Enhancing User Throughput in Multi-panel mmWave Radio Access Networks for Beam-based MU-MIMO Using a DRL Method

Authors: Ramin Hashemi, Vismika Ranasinghe, Teemu Veijalainen, Petteri Kela, Risto Wichman

Organizations: Nokia, Espoo, Finland · Department of Information and Communications Engineering, Aalto University, Espoo, Finland

Abstract

Millimeter-wave (mmWave) communication systems, particularly those leveraging multi-user multiple-input and multiple-output (MU-MIMO) with hybrid beamforming, face challenges in optimizing user throughput and minimizing latency due to the high complexity of dynamic beam selection and management. This paper introduces a deep reinforcement learning (DRL) approach for enhancing user throughput in multi-panel mmWave radio access networks in a practical network setup. Our DRL-based formulation utilizes an adaptive beam management strategy that models the interaction between the communication agent and its environment as a Markov decision process (MDP), optimizing beam selection based on real-time observations. The proposed framework exploits spatial domain (SD) characteristics by incorporating the cross-correlation between the beams in different antenna panels, the measured reference signal received power (RSRP), and the beam usage statistics to dynamically adjust beamforming decisions. As a result, the spectral efficiency is improved and end-to-end latency is reduced. The numerical results demonstrate an increase in throughput of up to 16% and a reduction in latency by factors 3-7x compared to baseline (legacy beam management).

Figures & tables

Figure 1: Beam selection approach in beam-based MU-MIMO with $M_{p}=4$ antenna panels. The activated beams in each panel are spatially separated from beams within other panels, however, spatially closer beams could have a non-zero cross-correlation.

Figure 2: Possible dimensions to be considered in beam-based MU-MIMO beam management procedure apart from traditional RSRP-based legacy approach.

Parameter	Value
Deployment scenario	3GPP Dense Urban Macro [ 17 ] ,
	3D channel model in [ 12 ]
gNB transmit power	$40$ dBm
Number of MTs ( $U$ )	$210$
Number of gNBs ( $M$ )	$21$ (7 sites, 3 sectors/site)
Antenna panels per gNB ( $M_{p}$ )	$3$ panels per sector

Table I : Simulation configuration.

Figure 3: Impact of exploration during training on the average MT throughput (total no. of MTs in network $U=210$ ).

Figure 4: The CDF of user throughput ( $U=210$ ).

Figure 5: Comparison between geometric mean user throughput, and the end-to-end latency with RL-DDQN vs. baseline.

Figure 6: The CDF of co-scheduled users in spatial domain.

Millimeter-Wave Communications

See all 16 papers

Spatio-Temporal Attention Enhanced Multi-Agent DRL for UAV-Assisted Wireless Networks with Limited Communications
Mar 23, 2026Che Chen, Lanhua Li, Shimin Gong +3Spatio-Temporal Attention Multi-Agent Reinforcement Learning
School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518000, China, and Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai), Zhuhai 519082, China · Department of Equipment Management and Unmanned Aerial Vehicle Engineering, Air Force Engineering University, Xi’an · School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics, Nanchang 330032, China +1
Multi-Agent Off-Policy Deep Reinforcement Learning for Smart Campus Coverage
Aug 19, 2026Omar Rady, Mohamed Ayman, Ali Arafa +1Millimeter-Wave Communications Wireless Communications
Faculty of Information Engineering and Technology The German University in Cairo Cairo, Egypt
Energy Saving for Cell-Free Massive MIMO Networks: A Multi-Agent Deep Reinforcement Learning Approach
Apr 8, 2026Qichen Wang, Keyu Li, Ozan Alp Topal +3Wireless Communications
Department of Communication Systems, KTH Royal Institute of Technology, Sweden · Department of Electrical and Electronics Engineering, Bilkent University, Turkiye · Department of Electronic Systems, Aalborg University, Denmark

Mar 23, 2026J/K move · Enter open · S save

Spatio-Temporal Attention Enhanced Multi-Agent DRL for UAV-Assisted Wireless Networks with Limited Communications

Che Chen, Lanhua Li, Shimin Gong, Yu Zhao+2

School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518000, China, and Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai), Zhuhai 519082, China · Department of Equipment Management and Unmanned Aerial Vehicle Engineering, Air Force Engineering University, Xi’an · School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics, Nanchang 330032, China+1

Spatio-Temporal Attention Multi-Agent Reinforcement Learning Wireless Resource Allocation UAV Trajectory Optimization Sequential MARL

Open paper AskPDF arXiv

In this paper, we employ multiple UAVs to accelerate data transmissions from ground users (GUs) to a remote base station (BS) via the UAVs' relay communications. The UAVs' intermittent information exchanges typically result in delays in acquiring the complete system state and hinder their effective collaboration. To maximize the overall throughput, we first propose a delay-tolerant multi-agent deep reinforcement learning (MADRL) algorithm that integrates a delay-penalized reward to encourage information sharing among UAVs, while jointly optimizing the UAVs' trajectory planning, network formation, and transmission control strategies. Additionally, considering information loss due to unreliable channel conditions, we further propose a spatio-temporal attention based prediction approach to recover the lost information and enhance each UAV's awareness of the network state. These two designs are envisioned to enhance the network capacity in UAV-assisted wireless networks with limited communications. The simulation results reveal that our new approach achieves over 50% reduction in information delay and 75% throughput gain compared to the conventional MADRL. Interestingly, it is shown that improving the UAVs' information sharing will not sacrifice the network capacity. Instead, it significantly improves the learning performance and throughput simultaneously. It is also effective in reducing the need for UAVs' information exchange and thus fostering practical deployment of MADRL in UAV-assisted wireless networks.

Abstract

Millimeter-Wave Communications

Explore similar work

Spatio-Temporal Attention Enhanced Multi-Agent DRL for UAV-Assisted Wireless Networks with Limited Communications

Multi-Agent Off-Policy Deep Reinforcement Learning for Smart Campus Coverage

Energy Saving for Cell-Free Massive MIMO Networks: A Multi-Agent Deep Reinforcement Learning Approach