Hybrid Orchestration of Edge AI and Microservices via Graph-based Self-Imitation Learning

Chen Yang; Jin Zheng; Yang Zhuolin; Lai Pan; Zhang Xiao; Hu Menglan; Yin Haiyan

cs.NIMar 3, 2026

Hybrid Orchestration of Edge AI and Microservices via Graph-based Self-Imitation Learning

Authors: Chen Yang, Jin Zheng, Yang Zhuolin, Lai Pan, Zhang Xiao, Hu Menglan, Yin Haiyan

Organizations: School of Computer Science, South-Central Minzu University, Wuhan, China, 430074, and Key Laboratory of Cyber-Physical Fusion Intelligent Computing, State Ethnic Affairs Commission · Hubei Key Laboratory of Smart Internet Technology, School of Electronic Information and Communication, Huazhong University of Science and Technology, Wuhan, China, 430074 · Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore

Abstract

Modern edge AI applications increasingly rely on microservice architectures that integrate both AI services and conventional microservices into complex request chains with stringent latency requirements. Effectively orchestrating these heterogeneous services is crucial for ensuring low-latency performance, yet remains challenging due to their diverse resource demands and strong operational interdependencies under resource-constrained edge environments. In particular, frequent interactions between services tightly couple deployment and routing decisions, yet existing approaches optimize them in isolation, leading to fundamentally inadequate system performance.In this paper, we propose SIL-GPO, a reinforcement learning framework that optimizes hybrid orchestration for edge AI microservice systems. SIL-GPO formulates the orchestration problem as a sequential decision-making task and leverages graph attention networks to encode service topologies and routing dependencies within the agent state representation. Moreover, SIL-GPO integrates a self-imitation learning strategy into proximal policy optimization, enabling the agent to prioritize and reuse high-reward trajectories. This guides policy updates towards globally promising solutions that standard RL often fails to discover under sparse rewards and large combinatorial action spaces. We conduct extensive experiments on trace-driven edge AI workloads, demonstrating that SIL-GPO significantly reduces end-to-end service latency and enhances resource utilization compared to state-of-the-art heuristic, metaheuristic, and deep RL baselines. Our framework offers a unified and scalable solution for efficient orchestration of AI services and microservices in the edge, paving the way for low-latency, high-performance edge AI deployments.

Figures & tables

Fig. 1: An example of image-generating intelligent application.

Notations	Definitions
$\mathcal{G}(N,E)$	The edge AI service network
$R_{n}^{res}$	The size of heterogeneous resources owned by server $n$
$B_{ij}$	The bandwidth between server $n_{i}$ and server $n_{j}$
$s,S$	The service instance and the set of service instances
$sc,SC$	The service request and The set of service requests
$r_{s}^{res}$	The size of heterogeneous resources consumed by service $s$

TABLE I: Notation and terminology

Fig. 2: Diagram of hybrid deployment and request routing for AI service and microservice.

Fig. 3: The schematic diagram of graph-based structures $G_{t}^{D}$ , $G_{t}^{R}$ , and $G_{t}^{S}$ .

Fig. 4: The architecture of the SIL-GPO algorithm.

Fig. 5: Reward curves under different learning rates.

Fig. 6: Delay curves under different learning rates.

Panel of Figure 8 — Fig. 7: Actor loss and critic loss of the SIL-GPO algorithm when the learning rate is 0.00005.

Hyperparameters	Value
Learning rate for actor	0.00005
Learning rate for critic	0.00001
Discount factor	0.9
GAE parameter	0.95
PPO clip parameter	0.2
The weight for entropy regularization	0.05 $\sim$ 0.1

TABLE II: Hyperparameter settings

Panel of Figure 10 — Fig. 8: Comparison of algorithm performance under different service request arrival rates

Panel of Figure 11 — Fig. 9: Comparison of algorithm performance under different service request chain lengths

Panel of Figure 12 — Fig. 10: Comparison of algorithm performance under different numbers of service request chains

Edge Computing

Edge computing places computation and AI near data sources to reduce latency, bandwidth use, and reliance on centralized clouds.

See all 192 papers

Learning to Remember: Attentive Reinforcement Learning for Edge Serverless Autoscaling
Mar 21, 2026Faraz Shaikh, Gianluca Reali, Mauro FemminellaRL Control Reinforcement Learning
Department of Engineering, University of Perugia, Perugia, Italy · Consorzio Nazionale Interuniversitario per le Telecomunicazioni (CNIT), 43124 Parma, Italy
End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services
Sep 15, 2026Zhen Li, Jun Cai, Haoran Gao +2LLM Inference Scheduling LLM Request Scheduling
Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, H3G 1M8, Canada · Department of Computer Science, Hang Seng University of Hong Kong, Hong Kong SAR
IMPACT: Intent-driven Multi-agent Policy with Attention for SLO-guaranteed Microservice Migration in Cloud-edge Systems
Sep 21, 2026Xinjin Li, Siru Tao, Shihan Yin +6Edge-Cloud Computing Cooperative MARL
Columbia University · Carnegie Mellon University · Brandeis University +5

Mar 21, 2026J/K move · Enter open · S save

Learning to Remember: Attentive Reinforcement Learning for Edge Serverless Autoscaling

Faraz Shaikh, Gianluca Reali, Mauro Femminella

Department of Engineering, University of Perugia, Perugia, Italy · Consorzio Nazionale Interuniversitario per le Telecomunicazioni (CNIT), 43124 Parma, Italy

RL Control Reinforcement Learning Edge Computing Memory-Augmented Agents

Open paper AskPDF arXiv

In edge computing, the stochastic and bursty nature of serverless workloads challenges autonomous resource orchestration. Traditional reactive controllers, such as the Kubernetes Horizontal Pod Autoscaler (HPA), suffer from reaction latency, leading to Service Level Objective (SLO) violations during traffic spikes and resource flapping during ramp-downs. While Deep Reinforcement Learning (DRL) offers a pathway toward proactive management, standard agents suffer from temporal blindness, an inability to exploit the recent temporal context in non-Markovian edge environments. To bridge this gap, we propose a stability-aware autoscaling framework unifying short-horizon temporal context and control via an Attention-Enhanced Double-Stacked LSTM architecture integrated within a Proximal Policy Optimization (PPO) agent. Unlike shallow recurrent models, our approach employs a learned attention mechanism that weights recent historical states non-uniformly, suppressing high-frequency jitter while preserving the trend that precedes demand shifts. We validate the framework on two independent Kubernetes clusters using real-world Azure Functions traces. Against the single-layer LSTM ablation and the static HPA baseline, our approach reduces P90 latency by

\approx

67%, and holds average latency within the 50ms hard SLO for 98.8% of the run against 49.6% and 43.5% respectively. Against Kubernetes Event-Driven Autoscaling (KEDA), it matches latency performance at 75% fewer replica-steps and 59% less churn, with P90 hard-SLO violation bursts of at most 5 consecutive intervals against up to 24 for KEDA. These results indicate that mitigating temporal blindness through deep attentive memory improves the reliability and stability of Kubernetes autoscaling under bursty edge workloads.

Abstract

Edge Computing

Explore similar work

Learning to Remember: Attentive Reinforcement Learning for Edge Serverless Autoscaling

End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services

IMPACT: Intent-driven Multi-agent Policy with Attention for SLO-guaranteed Microservice Migration in Cloud-edge Systems