Multi-Agentic AI for Fairness-Aware and Accelerated Multi-modal Large Model Inference in Real-world Mobile Edge Networks

Haiyuan Li; Hari Madhukumar; Shuangyi Yan; Yulei Wu; Dimitra Simeonidou

eess.SYFeb 6, 2026

Multi-Agentic AI for Fairness-Aware and Accelerated Multi-modal Large Model Inference in Real-world Mobile Edge Networks

Authors: Haiyuan Li, Hari Madhukumar, Shuangyi Yan, Yulei Wu, Dimitra Simeonidou

Organizations: Smart Internet Lab, Department of Electrical and Electronic Engineering, University of Bristol, BS8 1QU, U.K.

Abstract

Generative AI (GenAI) has transformed applications in natural language processing and content creation, yet centralized inference remains hindered by high latency, limited customizability, and privacy concerns. Deploying large models (LMs) in mobile edge networks emerges as a promising solution. However, it also poses new challenges, including heterogeneous multi-modal LMs with diverse resource demands and inference speeds, varied prompt/output modalities that complicate orchestration, and resource-limited infrastructure ill-suited for concurrent LM execution. In response, we propose a Multi-Agentic AI framework for latency- and fairness-aware multi-modal LM inference in mobile edge networks. Our solution includes a long-term Global Planning Agent, a short-term Prompt Scheduling Agent, and multiple on-node LM Deployment Agents, all powered by foundation language models. These agents cooperatively optimize prompt routing and LM deployment through natural language reasoning over runtime telemetry and historical experience. To evaluate its performance, we further develop a city-wide testbed that supports network monitoring, containerized LM deployment, intra-server resource management, and inter-server communications. Experiments demonstrate that our solution reduces average end-to-end latency by 42.9% compared with a state-of-the-art online-learning baseline and by 80.6% compared with the full-activation baseline, while maintaining a mean service-type Jain's index of 0.925. Moreover, our solution adapts quickly without fine-tuning, offering a generalizable solution for optimizing GenAI services in edge environments.

Figures & tables

Figure 1: A mobile edge network example where MEC servers host LMs of different modalities and collaboratively handle diverse inference requests.

Figure 2: The architecture of the proposed Multi-Agentic AI solution for resource management of Edge LM inference.

Figure 3: Architecture of the testbed and the allocation of network elements among servers.

	CPU (model-cores)	RAM	GPU	vRAM
VM 1	Intel Xeon Icelake - 8	16GB	N/A	N/A
VM 2	AMD EPYC 7453 - 24	32GB	N/A	N/A
VM 3	AMD EPYC 7453 - 16	24GB	NVIDIA A5000 * 1	24GB
VM 4	AMD EPYC 7453 - 16	24GB	NVIDIA A5000 * 1	24GB
VM 5	AMD EPYC 7453 - 16	24GB	NVIDIA A5000 * 1	24GB
VM 6	AMD EPYC 7453 - 16	24GB	NVIDIA A5000 * 1	24GB

Table I: Configurations of VMs in the experiment network

Figure 4: Sequence diagram of the runtime operation. Communications between MEC servers are achieved by Fast API.

Panel of Figure 5 — Figure 5: Pre-benchmark calibration of the Lyapunov-based deployment algorithm. (a) Trajectory of the profiled service-rate weights and margin-accepted DPP coefficients. (b) Raw validation-window objective and objective of the active calibrated configuration.

Figure 6: Comparison of different scheduling and deployment strategies in terms of service-type fairness and latency. (a) Jain’s fairness index over LM service-type success ratios. (b) Latency over the same windows.

Figure 7: Latency and success ratio of each LM. (a) End-to-end latency for each LM and the overall mean. (b) Success ratio.

Method	Inactive-target ratio	Conflict queue delay	Non-conf. queue delay	Conflict timeout ratio
MAL	6.36%	679.9 s	127.1 s	44.46%
PerLLM	6.29%	480.9 s	216.7 s	37.45%
MA	7.25%	302.3 s	98.4 s	17.12%

Table II: Cross-node inactive-target conflict analysis.

LM type	Activation events per 100 node-slot decisions	Deactivation events per 100 node-slot decisions
LM1	$2.28$	$2.26$
LM2	$1.61$	$1.62$
LM3	$2.32$	$2.34$
LM4	$0.61$	$0.58$
All LMs	$6.82$	$6.81$

Table III: MA deployment reconfiguration frequency.

Panel of Figure 8 — Figure 8: Learning dynamics. Both panels report rolling-window objective values with bootstrap 95% confidence intervals. MA updates occur once per epoch.

Figure 9: Heterogeneous bursty-arrival stress test. (a) Request arrivals by LM type. (b) Rolling latency–fairness objective with bootstrap 95% confidence intervals.

Setting	Worker agents	Avg. load/slot	Lat. (s)	Jain index	Obj. value
Original testbed	$6$	$\approx 96$	$96.7$	$0.925$	$0.182$
Expanded testbed	$8$	$\approx 128$	$132.1$	$0.932$	$0.215$

Table IV: Expanded-testbed scalability under the same per-node traffic-generation pattern.

Efficient Multimodal Inference

Efficient multimodal inference reduces the compute, memory, latency, and energy needed to process image, video, audio, text, and sensor inputs.

See all 231 papers

$E^3$ -Agent: An Executable and Evolving Agent for Resource Management of Edge Generative Inference
May 21, 2026Rui Bao, Yaping Sun, Zhiyong Chen +4LLM Inference Efficiency Edge Inference
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai 200240, China · Department of Broadband Communication, Pengcheng Laboratory, Shenzhen 518000, China · China Mobile Research Institute, Beijing 100053, China
Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges
Sep 4, 2026Chenqi Li, Minghui Min, Dusit Niyato +1On-Device Language Model Inference Diffusion Language Model Inference
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, 221116, Jiangsu, China · College of Computing and Data Science, Nanyang Technological University, Singapore, 639798, Singapore · School of Engineering, Edith Cowan University, Perth, Western Australia, 6027, Australia
End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services
Sep 15, 2026Zhen Li, Jun Cai, Haoran Gao +2LLM Inference Scheduling LLM Request Scheduling
Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, H3G 1M8, Canada · Department of Computer Science, Hang Seng University of Hong Kong, Hong Kong SAR

May 21, 2026J/K move · Enter open · S save

$E^3$ -Agent: An Executable and Evolving Agent for Resource Management of Edge Generative Inference

Rui Bao, Yaping Sun, Zhiyong Chen, Feng Yang+3

Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai 200240, China · Department of Broadband Communication, Pengcheng Laboratory, Shenzhen 518000, China · China Mobile Research Institute, Beijing 100053, China

LLM Inference Efficiency Edge Inference Edge Computing LLM Agent Orchestration

Open paper AskPDF arXiv

Edge deployments of generative inference increasingly face two practical realities: per-device per-model performance is often unknown at deployment time, and it is non-stationary due to user-driven semantic events, background load, and device churn. Consequently, a resource manager that is tuned offline under a fixed regime can become brittle and expensive to maintain. This paper presents

E^3

-Agent, an executable and evolving agent for edge artificial intelligence generated content (AIGC) resource management.

E^3

-Agent separates a fast-path router that makes millisecond-level dispatch decisions from a slow-path, event-driven large language model (LLM) meta-controller that mitigates regime shifts through a small, explicit control surface exposed via a tool interface, including risk gating, router configuration, and rapid performance calibration. The agent learns online from execution feedback and continuously adapts to unknown and time-varying service-time mappings. We evaluate

E^3

-Agent in a discrete-event simulator driven by MLPerf-derived device-model measurement priors, covering cold-start warmup and three dynamic regimes: semantic dynamics, device churn, and hidden drift. Across the dynamic scenarios,

E^3

-Agent reduces average latency by 65%-73% compared to the best static baseline, stays within 7%-10% of an online full-information Oracle used for evaluation, and effectively suppresses stutter rate under semantic degradation.

Abstract

Efficient Multimodal Inference

Explore similar work

E3E^3E3-Agent: An Executable and Evolving Agent for Resource Management of Edge Generative Inference

Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges

End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services

$E^3$ -Agent: An Executable and Evolving Agent for Resource Management of Edge Generative Inference