Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization

Myeung Suk Oh; Zhiyao Zhang; Alvaro Velasquez; Nathaniel D. Bastian; Jia Liu

cs.LGOct 6, 2026

Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization

Authors: Myeung Suk Oh, Zhiyao Zhang, Alvaro Velasquez, Nathaniel D. Bastian, Jia Liu

Organizations: The Ohio State University Columbus, Ohio, USA · University of Colorado Boulder Boulder, Colorado, USA · Pathfinder Applied Research, LLC Ashburn, Virginia, USA

Abstract

Random access (RA) is one of the most foundational medium access control (MAC) layer scheduling schemes for handling unpredictable data traffic from multiple terminals. While multi-agent reinforcement learning (MARL) has been explored to optimize RA-based wireless networks, its reliance on experience-driven, distributed policy learning incurs significant training overhead for each optimization task, limiting its feasibility in real-world applications. In this work, we propose to leverage a foundation model (FM) to improve MARL efficiency across diverse RA network optimization tasks. Specifically, we design an FM-aided actor-critic algorithm within a consensus-based decentralized MARL architecture and provide its convergence analysis under local reward exchanges and nonlinear value function approximations to show that our algorithm achieves the same convergence order as the conventional MARL with critic model exchanges and linear approximations. Our numerical results show that our FM-based approach significantly enhances MARL speed for RA network optimization.

Figures & tables

Figure 1 . An RA-based wireless network, where devices contend for channel access. Each device makes its own decision on when to transmit. Communication links are established across the devices for exchanging local information. .

Figure 2 . The architecture of FM as a reward-agnostic critic backbone. The model takes a state-action history and extracts embeddings via state and action encoders. Transformer blocks are used to process the embeddings. The output of a prediction head is sent to the reward-specific critic head. .

Figure 3 . A visual representation of our fully decentralized MARL framework. Both the actor and critic are trained in a decentralized manner. The critic contains two distinct parts: a reward-agnostic backbone for forward dynamics prediction and a reward-specific head for task-specific finetuning. .

Panel of Figure 4 — Figure 4 . Throughput and AoI under end-to-end learning with $K=2$ devices across different RA optimization tasks.

Panel of Figure 5 — Figure 5 . Collision and reward performance under end-to-end learning with $K=2$ devices across RA optimization tasks. .

Panel of Figure 6 — Figure 6 . Performance comparison of conventional, end-to-end (E-E), and FM-aided MARL across different RA optimization tasks with $K=4$ . The FM-aided MARL achieves faster convergence than the end-to-end baseline. .

Figure 7 . Fair-AoI and Max-rate performance comparison with non-MARL baselines across the number of devices

K

. .

Task	Alg.	Mean	Min.	Max.	N-Gap	95%-T (s)
Fair-AoI	E-E	0.731 $\pm$ 0.010	0.726	0.734	0.011	335
(ms)	FM	0.769 $\pm$ 0.032	0.757	0.772	0.019	108
Max-rate	E-E	17.044 $\pm$ 0.266	4.092	41.225	0.901	380
(Mbps)	FM	17.924 $\pm$ 0.345	3.918	43.865	0.911	159
Fair-rate	E-E	9.049 $\pm$ 0.186	6.903	11.956	0.423	373
(Mbps)	FM	8.083 $\pm$ 0.131	6.143	11.590	0.463	104

Table 1 . Performance Comparison of End-to-End (E-E) and FM-aided MARL for

K=4

(top) and

K=8

(bottom). AoI is measured for the fair-AoI task, and rate is measured for both max-rate and fair-rate tasks.

Task	Alg.	Mean	Min.	Max.	N-Gap	95%-T (s)
Fair AoI	E-E	0.366 $\pm$ 0.014	0.364	0.368	0.003	723
(ms)	FM	0.368 $\pm$ 0.025	0.365	0.370	0.013	231
Max Rate	E-E	7.235 $\pm$ 0.375	3.895	30.175	0.871	812
(Mbps)	FM	7.152 $\pm$ 0.236	4.100	29.685	0.862	345
Fair Rate	E-E	6.365 $\pm$ 0.352	5.851	6.864	0.148	768
(Mbps)	FM	6.335 $\pm$ 0.244	5.642	7.025	0.197	225

Table 9

Explore similar work

Heterogeneous Multi-Agent Reinforcement Learning for Radio Resource Management under Coupled Finite-Horizon Constraints
Aug 3, 2026Yeonseo Jeong, Wonhyeok Ko, Sungweon Hong +1Power Allocation Open Radio Access Networks
Department of Electronic Engineering, Hanyang University, Seoul, Korea · Department of AI-Semiconductor, Hanyang University, Seoul, Korea
Hierarchical Wireless Foundation Model for Multi-Task Optimization
Jul 18, 2026Yangjing Wang, Ouya Wang, Shenglong Zhou +1Wireless Communications Multi--Task Learning
Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London, SW7 2BX London, U.K. · School of Mathematics and Statistics, Beijing Jiaotong University, China
Optimization-as-a-Service via Multi-Agent Large Language Model for Radio Access Networks
May 15, 2026Chaoqun You, Yueyue Dai, Xingqiu He +3Open Radio Access Networks Agentic Optimization
Fudan University · Huazhong University of Science and Technology · University of Surrey +1

Aug 3, 2026, cs.LGJ/K move · Enter open · S save

Heterogeneous Multi-Agent Reinforcement Learning for Radio Resource Management under Coupled Finite-Horizon Constraints

Yeonseo Jeong, Wonhyeok Ko, Sungweon Hong, Songnam Hong

Department of Electronic Engineering, Hanyang University, Seoul, Korea · Department of AI-Semiconductor, Hanyang University, Seoul, Korea

Power Allocation Open Radio Access Networks Multi-Agent Reinforcement Learning Handoff Algorithmic Fairness

Open paper AskPDF arXiv

Maximizing throughput under proportional fairness in dense wireless networks requires jointly managing user association, scheduling, base station (BS) activation, and handover control under hard finite-horizon energy and handover budgets, which induces a fundamental tension between BS-side energy management and user-side handover regulation. While multi-agent reinforcement learning (MARL) is a natural framework for such distributed sequential control, its application here faces two difficulties: finite-horizon budget constraints cannot be evaluated at each time slot, and the nonlinear proportional fairness utility admits no principled per-slot decomposition. We propose HeLyMARL, a Lyapunov-embedded heterogeneous MARL framework that resolves both via drift-plus-penalty decomposition with virtual queues. The energy and handover constraint pressures are internalized directly into a unified per-slot reward, converting the constrained finite-horizon problem into an unconstrained MARL problem. Comparison against two Lagrangian-based alternatives reveals a timescale separation: Lagrangian relaxation regulates constraints only across training episodes, whereas the virtual queues of HeLyMARL bound cumulative budget consumption at every partial horizon within an episode, a pacing guarantee beyond the reach of greedy Lyapunov-based control. Simulations show that HeLyMARL is the only method that sustains the throughput-fairness balance together with uninterrupted service throughout the horizon, outperforming conventional MARL, Lyapunov-based, and constrained MARL benchmarks without premature budget exhaustion.

Abstract

Explore similar work

Heterogeneous Multi-Agent Reinforcement Learning for Radio Resource Management under Coupled Finite-Horizon Constraints

Hierarchical Wireless Foundation Model for Multi-Task Optimization

Optimization-as-a-Service via Multi-Agent Large Language Model for Radio Access Networks