cs.AIOct 27, 2025

Multi-Agent Conditional Diffusion Planning for Decentralized Wireless Resource Allocation

Authors: Kechen Meng, Sinuo Zhang, Rongpeng Li, Yansha Deng, Chan Wang, Zhifeng Zhao

Organizations: the College of Information Science and Electronic Engineering, Zhejiang University · the Department of Engineering, King’s College London, London WC2R 2LS, U.K. · Zhejiang Lab

Abstract

In wireless communication systems, efficient and adaptive resource allocation is essential for enhancing overall Quality of Service (QoS). Compared with conventional Model-Free Reinforcement Learning (MFRL), Model-Based RL (MBRL) first learns a generative world model for subsequent planning, enabling more stable training through the reuse of historical experience. However, deploying MBRL in large-scale wireless networks remains challenging due to high-dimensional stochastic dynamics, strong inter-agent cooperation, and stringent communication constraints. To overcome these challenges, we propose a Multi-Agent Conditional Diffusion Model Planner (MA-CDMP) for decentralized communication resource management. Following the Decentralized Training with Decentralized Execution (DTDE) paradigm, MA-CDMP models each communication node as an autonomous agent and employs Diffusion Models (DMs) to capture environment dynamics. An inverse dynamics model is further incorporated to guide action generation, improving sample efficiency and policy scalability. Moreover, to efficiently approximate large-scale agent interactions, we introduce a Mean Field (MF) mechanism to assist the classifier in conditional DMs. This mitigates inter-agent non-stationarity and enhances cooperation with minimal communication overhead. We further establish an upper bound on the distributional approximation error induced by MF-based diffusion generation, guaranteeing convergence stability and reliable modeling of multi-agent stochastic dynamics. Extensive experiments demonstrate that MA-CDMP consistently outperforms existing MARL baselines in terms of average reward and QoS metrics, showcasing its scalability and practicality for real-world wireless network optimization.

Figures & tables

Explore similar work

Oct 6, 2026

Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization

Random access (RA) is one of the most foundational medium access control (MAC) layer scheduling schemes for handling unpredictable data traffic from multiple terminals. While multi-agent reinforcement learning (MARL) has been explored to optimize RA-based wireless networks, its reliance on experience-driven, distributed policy learning incurs significant training overhead for each optimization task, limiting its feasibility in real-world applications. In this work, we propose to leverage a foundation model (FM) to improve MARL efficiency across diverse RA network optimization tasks. Specifically, we design an FM-aided actor-critic algorithm within a consensus-based decentralized MARL architecture and provide its convergence analysis under local reward exchanges and nonlinear value function approximations to show that our algorithm achieves the same convergence order as the conventional MARL with critic model exchanges and linear approximations. Our numerical results show that our FM-based approach significantly enhances MARL speed for RA network optimization.
May 28, 2026

Mean-Field Diffuser: Scaling Offline MARL to Thousands of Agents

Diffusion-based planning has achieved strong results in single-agent offline reinforcement learning, yet scaling to many-agent systems remains intractable due to the curse of dimensionality in the joint trajectory space. We introduce MF-Diffuser, a framework that lifts trajectory planning to the Wasserstein space of trajectory distributions, where the propagation of chaos ensures a small representative subset of agents captures the full population dynamics. Our approach features a value-weighted chaotic entropy objective that reconciles generative fidelity with return maximization, and a hierarchical coarse-to-fine strategy that progressively grows the agent population during denoising. We establish end-to-end suboptimality bounds with four interpretable terms, revealing that mean-field approximation error scales as O(H2/N)O(H^2/\sqrt{N}) while offline distribution shift provably does not grow with population size NN, and prove the generated policy is an approximate mean-field Nash equilibrium with explicit convergence guarantees. Experiments on three mean-field RL benchmarks -- spanning stage games, sequential dynamics, and adversarial team competition -- show MF-Diffuser achieves the best return in the majority of settings, with the largest gains on suboptimal offline data and at extreme scales (N≥103N \geq 10^3).
Aug 28, 2026

Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management

This paper develops a hybrid offline-online multi-agent reinforcement learning framework based on decision transformers. The policy is first pretrained offline via supervised sequence modeling of trajectories generated by existing policies, providing a safe and sample-efficient initialization. It is then fine-tuned online using a hybrid objective that incorporates critic-guided gradients, enabling performance improvements beyond the offline policy. To facilitate stable offline-to-online transfer and effective multi-agent coordination, the framework incorporates return-weighted sampling, a critic conditioned on neighbors' actions, and neighborhood-correlated exploration. The approach is fully distributed: both training and execution rely only on local observations and limited information exchange among neighboring agents. Evaluations with dynamic traffic arrivals in two settings: (i) joint scheduling and power allocation and (ii) coordinated beamforming, show that the proposed method achieves quality-of-service (QoS) performance comparable to centralized methods. Moreover, when pretrained on lower-quality datasets, online fine-tuning is also observed to surpass the initial offline policy. These results demonstrate a promising learning-based alternative for wireless resource management.