Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management

Yiming Zhang; Kun Yang; Cong Shen; Dongning Guo

eess.SYAug 28, 2026

Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management

Authors: Yiming Zhang, Kun Yang, Cong Shen, Dongning Guo

Organizations: Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL 60208 USA · Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904 USA

Abstract

This paper develops a hybrid offline-online multi-agent reinforcement learning framework based on decision transformers. The policy is first pretrained offline via supervised sequence modeling of trajectories generated by existing policies, providing a safe and sample-efficient initialization. It is then fine-tuned online using a hybrid objective that incorporates critic-guided gradients, enabling performance improvements beyond the offline policy. To facilitate stable offline-to-online transfer and effective multi-agent coordination, the framework incorporates return-weighted sampling, a critic conditioned on neighbors' actions, and neighborhood-correlated exploration. The approach is fully distributed: both training and execution rely only on local observations and limited information exchange among neighboring agents. Evaluations with dynamic traffic arrivals in two settings: (i) joint scheduling and power allocation and (ii) coordinated beamforming, show that the proposed method achieves quality-of-service (QoS) performance comparable to centralized methods. Moreover, when pretrained on lower-quality datasets, online fine-tuning is also observed to surpass the initial offline policy. These results demonstrate a promising learning-based alternative for wireless resource management.

Figures & tables

Fig. 1: An example of Dec-POMDP-IR.

Fig. 2: Three-phase hybrid offline-online MADT framework for wireless resource allocation.

Fig. 3: Nineteen hexagonal cells, each serving one active device.

Fig. 4: Beamforming deployment: six APs with inward-facing

120^{\circ}

sectors form one cluster of mutually-interfering agents.

Parameter	Power allocation	Beamforming
Cell radius	500 m	160 m
Path loss ( $d$ in km)	$128.1+37.6\log_{10}d$ dB	exponent $=2$
Transmit power	$(P_{\min},\overline{P})=(0,23)$ dBm	fixed $P$
Noise power	$-114$ dBm	10 dB cell-edge SNR
Action space $\mathcal{A}_{k}$	7 power levels	8 beams $+$ silent
Channel variation	$f_{d}=10$ Hz, $\rho=0.64$	user random walk

TABLE I: Simulation Parameters of the Two Instantiations

Fig. 5: Learning curves when pretraining on the expert (ideal WMMSE) dataset: (a) offline pretraining and (b) online fine-tuning. The dashed line marks the average episode return of the offline dataset. Each curve is read against both vertical axes: average episode return (left) and the corresponding mean end-to-end delay (right, inverted so that smaller delay is at the top).

Fig. 6: Learning curves when pretraining on the medium (delayed WMMSE) dataset: (a) offline pretraining and (b) online fine-tuning. Axes as in Fig. 5 . After the initial exploration phase, the fine-tuned policy surpasses the average return of the offline dataset

Fig. 7: CCDF of packet delays for the 19-device network (log scale), highlighting the delay tail.

Method	Mean	90th pct.	99th pct.
full power	96.4	273.2	1029.2
random power	468.2	1984.5	5362.8
ITLinQ	104.1	210.2	1501.6
ideal WMMSE (Expert)	30.5	72.4	151.7
delayed WMMSE (Medium)	44.1	109.7	252.3
DT pretrained (Expert)	31.8	77.3	182.6

TABLE II: Average Packet Delay (ms)

Fig. 8: Beamforming learning curves when pretraining on the medium (max-gain) dataset: (a) offline pretraining and (b) online fine-tuning. The dashed line marks the average episode return of the offline dataset.

Fig. 9: CDF of per-packet delays for the beamforming network. The DT pretrained (Expert) curve is hidden beneath the DT fine-tuned (Expert) curve, with which it exactly coincides.

Method	Mean	90th pct.	99th pct.
random	3097.8	6241.9	7324.9
max-gain (Medium)	76.2	186.4	344.3
ideal WMMSE (Expert)	17.6	53.8	121.5
DT pretrained (Expert)	31.6	86.1	184.4
DT fine-tuned (Expert)	31.6	86.1	184.4
DT pretrained (Medium)	82.7	204.2	363.3

TABLE III: Beamforming Packet Delay (ms)

Wireless Resource Allocation

Wireless resource allocation assigns spectrum, power, beams, and computing capacity to meet throughput, latency, fairness, and energy goals.

See all 172 papers

WiSDoM: Wireless Sparse Decision Transformer with Mixture-of-Experts for Multi-Task Mobile Network Optimization
Aug 31, 2026Fatih Temiz, Shavbo Salehi, Melike Erol-KantarciSparse Mixture-of-Experts Decision Transformer
School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada
Generalizable Multi-Task Learning for Wireless Networks Using Prompt Decision Transformers
Jun 3, 2026Fatih Temiz, Shavbo Salehi, Melike Erol-KantarciReinforcement Learning Transformer
School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada
Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control
Mar 4, 2026Nicolas Helson, Pegah Alizadeh, Anastasios GiovanidisConservative RL Offline RL
Ericsson Research, Massy, France

Aug 31, 2026J/K move · Enter open · S save

WiSDoM: Wireless Sparse Decision Transformer with Mixture-of-Experts for Multi-Task Mobile Network Optimization

Fatih Temiz, Shavbo Salehi, Melike Erol-Kantarci

School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada

Sparse Mixture-of-Experts Decision Transformer Offline RL Multi-Task RL Wireless Resource Allocation

Open paper AskPDF arXiv

Emerging 6G wireless networks are expected to operate across diverse deployment scenarios, where variations in network topology, user mobility, traffic demand, and radio conditions challenge the scalability of conventional radio resource management (RRM). While offline reinforcement learning (RL) methods have demonstrated strong decision-making capabilities, learning a single policy that performs consistently across heterogeneous wireless environments remains difficult due to conflicting optimization objectives and limited model specialization. These challenges become particularly pronounced in coordinated multipoint (CoMP) transmission, where selecting the optimal serving-cell combination requires sequential decision-making under evolving network conditions. This paper presents the Wireless Sparse Decision Transformer with Mixture of Experts (WiSDoM), a sparse multi-task offline RL framework for adaptive multi-cell selection. WiSDoM combines Decision Transformers (DTs) with a Mixture-of-Experts (MoE) architecture that dynamically activates specialized experts according to task characteristics. This MoE mechanism improves model capacity without proportionally increasing inference cost, mitigates negative transfer, and enables expert specialization across tasks. WiSDoM is trained jointly on diverse network configurations spanning multiple base station and user equipment densities, mobility levels, and scheduler policies. Experimental results show that WiSDoM consistently outperforms heuristic methods, single-task models, and conventional multi-task DTs, improving quality of experience (QoE) by up to 55% while activating approximately one-third of the parameters of its dense counterpart during inference. Furthermore, WiSDoM exhibits strong task generalization and efficiently adapts to unseen wireless scenarios through few-shot prompting without retraining or fine-tuning.

Abstract

Wireless Resource Allocation

Explore similar work

WiSDoM: Wireless Sparse Decision Transformer with Mixture-of-Experts for Multi-Task Mobile Network Optimization

Generalizable Multi-Task Learning for Wireless Networks Using Prompt Decision Transformers

Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control