Organizations: BNU-HKBU United International College, Zhuhai 519087, China, and the Hong Kong Baptist University, Hong Kong. · Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai 519087, China, and the Guangdong Key Lab of AI and Multi-Modal Data Processing, BNU-HKBU United International College, Zhuhai 519087, China. · Department of Computing, Hong Kong Polytechnic University, Hong Kong. · Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai 519087, China.
Artificial Intelligence Generated Content (AIGC) has gained significant popularity for creating diverse content. However, current AIGC models primarily focus on content quality within a centralized framework, resulting in a high service delay and negative user experiences. The workload of an AIGC task is determined by model complexity and multi-layered encoder architectures, posing significant challenges for its modeling, deployment, and scheduling within resource-constrained edge networks. We model an offloading problem among edges for providing real AIGC services and propose LAD-TS, a novel Latent Action Diffusion-based Task Scheduling method that orchestrates multiple edge servers for expedited AIGC services by leveraging the diffusion model's conditional guidance capability and the reinforcement learning's environment interaction ability. We first design an innovative Latent Action Diffusion Network (LADN) model based on traditional diffusion models. We then incorporate the LADN model into deep reinforcement learning, forming an effective LAD-TS method for task scheduling in distributed edge networks. Meanwhile, we design a latent action diffusion strategy to achieve near-optimal decisions quickly by utilizing historical action probabilities instead of Gaussian noise. Furthermore, we implement a refined AIGC model and develop DEdgeAI, a prototype edge system with the refined AIGC model deployment to evaluate our LAD-TS method. DEdgeAI provides a real AIGC service for users, demonstrating up to 7.36%−18.11% and 34.07%−94.96% shorter service delays than the state-of-the-art methods and representative commercial platforms, respectively. We release our open-source code at https://github.com/ChangfuXu/DEdgeAI/
Figures & tables
Fig. 1: An illustration of existing AIGC applications. Users send their prompts to the cloud server that has deployed the corresponding AIGC models by backhaul network. Then, the corresponding content (e.g., image) are generated according to the prompts and are sent back to users.
Method
Cloud-enabled AIGC
Edge-enabled AIGC
Diffusion scheduling
Latent action diffusion
Offline-online decoupling
Midjourney [ 4 ]
✓
✕
✕
✕
✕
Xu et al. [ 26 ] , Wu et al. [ 27 ]
✕
✓
✕
✕
✕
Du et al. [ 28 ] , Tang et al. [ 29 ]
✕
✓
✓
✕
✕
Ours
✕
✓
✓
✓
✓
TABLE I: Comparison of ours and current representative methods
Symbol
Description
T
The set of time slots.
B
The set of ESs or BSs.
Nb,t
The task set that arrives to BS b∈B at time slot t∈T .
Δ
The length of each time slot t∈T .
dn
The data size of task n∈Nb,t .
zn
The generation quality demand of an AIGC task n .
TABLE II: Symbols frequently used in this paper.
Fig. 2: An illustration of the efficient distributed AIGC in this paper. For each AIGC task in each time slot, the scheduler will decide which ES to process the task by the proposed LAD-TS method.
Fig. 3: The overall architecture of our method. For each ES b , new arrival AIGC tasks are online offloaded to ESs for parallel processing by the actor. The actor utilizes the system state sb,n,t and historical action probability xb,n,t with edge-edge collaboration. The xb,n,t is stored in memory. The t-LADN model is periodically offline trained using history samples.
Fig. 4: The Actor structure with proposed LADN model. The actor input are the timestep I , latent action probability xb,n,t,I , and system state sb,n,t . The output is the action decision ab,n,t . The historical action probability xb,n,t,0 is stored (or updated) into the array Xb[n] .
Param.
Value
Param.
Value
B
20 [ 28 ]
∣T∣
60 [ 44 ]
Nb,t
[1, 50] [ 44 ]
Δ
1.0 second [ 33 ]
ρn
[100, 300] cycles/bit [ 15 ]
dn
[2, 5] Mbits [ 44 ]
vn,b′,t
[400, 500] Mbits/s [ 15 ]
fb′
[10, 50] GHz [ 15 ]
TABLE III: Default setting of environment parameter.
Parameters
Value
Hidden layers of actor and CN
2 full connection (20, 20)
Learning rate ηa
1e-4
Learning rate ηc
1e-3
Learning rate ηα
3e-4
Reward decay’s factor γ
0.95
Soft updating’s weight τ
0.005
TABLE IV: Default setting of model parameters.
Fig. 5: The average service delays of our LAD-TS method and baselines with increasing training episodes.
Fig. 6: The comparison results of our method and baselines by varying the number of tasks and the ESs’ capacities.
Fig. 7: The comparison results of our method and baselines by varying the AIGC qualities and the number of BSs.
Fig. 8: The service delay of our method by varying denoising step I and entropy temperature αb .
Method
Average service delay (Sec.)
Converged episode
SAC-TS
8.88 ± 1.43
200
SAC-TS-U
8.54 ± 1.21
150
LAD-TS-NU
8.13 ± 0.82
90
LAD-TS
7.67 ± 0.64
60
TABLE V: Performance and comparison results of the SAC-TS, SAC-TS-U, LAD-TS-NU, and LAD-TS methods.
Fig. 9: DEdgeAI is implemented on five Jetsons.
Fig. 10: An instance of DEdgeAI worker in each Jetson.
Fig. 11: The results of the total service delay of our method and baselines in our developed DEdgeAI system by varying the number of text-to-image tasks N .
Model
Task category
Hardware configuration
Method
Delay (Sec.)
Improved delay
SD3 med.
Text-to-image
Original HC 1
DQN-TS
161.45
20.45%
SAC-TS
151.12
15.01%
D2SAC-TS
139.96
8.24%
LAD-TS
128.43
NA
Image-to-image
Original HC 1
DQN-TS
190.67
19.86%
SAC-TS
178.24
14.27%
TABLE VI: The service delay results of the LAD-TS method and baselines in our developed DEdgeAI system under different AIGC models, AIGC tasks, and edge HCs.
Model
Running time (Sec.)
Memory (GB)
Image quality
FID ↓
CLIP ↑
SD3 med.
35.68
40
50.12
0.231
reSD3-m
18.32
16
56.38
0.226
TABLE VII: The results of the running time, memory occupation, and image quality of the implemented reSD3-m and SD3 medium models in our developed DEdgeAI system.
Fig. 12: The partial experimental results for AIGC tasks by the implemented reSD3-m and original SD3 medium models in our DEdgeAI system. Intuitively, these results align well with the task’s requirements. Importantly, the running time and memory usage of the reSD3-m model is significantly lower than that of original SD3 medium model, as shown in Table VII .
Platform
Model
Median generated delay (Sec.)
/System
∣N∣= 1
100
500
1000
Midjourney
Mid. v6
75.9
7590.0
37950.0
75900.0
OpenAI
DALL ⋅ E3
15.5
1550.0
7750.0
15500.0
Replicate
SD1.5
32.9
3290.0
16450.0
32900.0
Together.AI
SD2.1
8.3
830.0
4150.0
8300.0
Replicate
SD3 med.
5.8
580.0
2900.0
5800.0
TABLE VIII: The experimental results of the total generation delay of our DEdgeAI system and existing representative platforms under different numbers of AIGC tasks.
Edge artificial intelligence (Edge AI) runs and adapts AI models on devices or nearby networks to meet constraints on latency, energy, memory, and privacy.
Artificial Intelligence-Generated Content (AIGC) services employ Generative AI (GenAI) models to automatically generate diverse content. Mobile AIGC networks host GenAI models on edge-located AIGC Service Providers (ASPs) to deliver low-latency and personalized AIGC services for mobile users. However, AIGC inference tasks typically occupy GPU memory until task completion, causing GPU memory exhaustion at serving ASPs and triggering out-of-memory failures rather than merely increasing service latency. Existing studies on AIGC task orchestration have largely overlooked GPU memory feasibility constraints. To address this issue, we develop a cooperative multi-agent orchestration framework, in which each edge node is equipped with a scheduling agent to route tasks to local ASPs or neighboring edge nodes. Since scheduling agents make decisions based only on local observations, while peer offloading couples their resource states and long-term utilities, we formulate the orchestration process as a cooperative Decentralized Partially Observable Markov Decision Process (Dec-POMDP). To solve the Dec-POMDP, we propose an Attention-guided Multi-Agent deep reinforcement learning algorithm with Diffusion policies (A-MADiff) under the centralized training with a decentralized execution paradigm. A-MADiff employs diffusion-based decentralized actors to generate multi-modal preferences over feasible orchestration actions, and an attention-guided centralized critic to estimate per-agent values from cross-agent states under GPU memory heterogeneity. Numerical results demonstrate that A-MADiff significantly improves the cumulative reward over the state-of-the-art baseline.
Chongzhi Wu, Zhengtao Li, Jiawen Kang +4
School of Automation, Guangdong University of Technology, Guangzhou 510006, China · Department of Computer Science, City University of Hong Kong, Hong Kong · School of Information and Communication, Guilin University of Electronic Technology, Guilin 541004, China +3
Generative AI (GenAI) has transformed applications in natural language processing and content creation, yet centralized inference remains hindered by high latency, limited customizability, and privacy concerns. Deploying large models (LMs) in mobile edge networks emerges as a promising solution. However, it also poses new challenges, including heterogeneous multi-modal LMs with diverse resource demands and inference speeds, varied prompt/output modalities that complicate orchestration, and resource-limited infrastructure ill-suited for concurrent LM execution. In response, we propose a Multi-Agentic AI framework for latency- and fairness-aware multi-modal LM inference in mobile edge networks. Our solution includes a long-term Global Planning Agent, a short-term Prompt Scheduling Agent, and multiple on-node LM Deployment Agents, all powered by foundation language models. These agents cooperatively optimize prompt routing and LM deployment through natural language reasoning over runtime telemetry and historical experience. To evaluate its performance, we further develop a city-wide testbed that supports network monitoring, containerized LM deployment, intra-server resource management, and inter-server communications. Experiments demonstrate that our solution reduces average end-to-end latency by 42.9% compared with a state-of-the-art online-learning baseline and by 80.6% compared with the full-activation baseline, while maintaining a mean service-type Jain's index of 0.925. Moreover, our solution adapts quickly without fine-tuning, offering a generalizable solution for optimizing GenAI services in edge environments.
Haiyuan Li, Hari Madhukumar, Shuangyi Yan +2
Smart Internet Lab, Department of Electrical and Electronic Engineering, University of Bristol, BS8 1QU, U.K.
Large language model (LLM)-powered agentic AI services increasingly demand low-latency inference, motivating the deployment of LLMs across distributed edge servers. However, heterogeneous communication and computing capabilities, together with dynamically evolving inference states, make the edge server selection for each incoming request time-varying and tightly coupled across slots. In this paper, we investigate an online request scheduling framework for edge LLM inference that jointly minimizes long-term average end-to-end latency and regulates workload distribution across heterogeneous edge servers. Two main challenges arise in this context. First, conventional latency models cannot accurately capture the fine-grained dynamics of multi-stage LLM execution. Second, the latency consequence of a scheduling decision is observed only after request completion, making immediate decision evaluation difficult. To address these challenges, we develop a cross-slot inference model that captures transmission, prefill, iteration-level decoding, and key-value (KV) cache evolution for each diverse request, and characterize server workload through a KV cache memory-time consumption metric. We propose the LYREO approach that transforms the long-term load-balancing constraint via Lyapunov optimization and employs reward redistribution with sequencebased return prediction to convert delayed outcomes into timely learning signals for earlier decisions. Simulations under various configurations demonstrate that LYREO consistently achieves lower latency and more balanced load distribution than representative learning-based and heuristic baseline schemes.
Zhen Li, Jun Cai, Haoran Gao +2
Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, H3G 1M8, Canada · Department of Computer Science, Hang Seng University of Hong Kong, Hong Kong SAR