Agentic-Kube: A Graph-Enhanced Multi-Agent Reinforcement Learning Framework for Multi-Objective Kubernetes Scheduling

Hamed Hamzeh

cs.DCMar 12, 2026

Agentic-Kube: A Graph-Enhanced Multi-Agent Reinforcement Learning Framework for Multi-Objective Kubernetes Scheduling

Authors: Hamed Hamzeh

Organizations: University of Westminster Computer Science and Engineering London, United Kingdom

Abstract

Cloud-native container orchestration requires resource schedulers capable of balancing infrastructure expenditure, fault resilience, and node utilisation. Conventional reinforcement learning approaches typically rely on monolithic single-agent models that suffer from gradient interference and reward dilution when mapping conflicting operational goals into a single scalar reward. We present Agentic-Kube, a cooperative multi-agent reinforcement learning framework designed for real-time Kubernetes pod placement. The architecture decomposes multi-objective scheduling into a tripartite optimisation space managed by dedicated sub-agents for cost minimisation, anti-affinity fault tolerance, and vector resource balancing. Agentic-Kube integrates a bipartite Graph Convolutional Network to capture dynamic host-pod dependencies, a two-stage monotonic QMIX value factorisation network to maintain joint action value coherence, and a plurality voting consensus mechanism with action feasibility masking against allocatable node predicates. We evaluate the framework across live heterogeneous Google Kubernetes Engine deployments and macro-scale cluster environments spanning 50 to 1,000 nodes under empirical Alibaba trace data, diurnal microservice variations, and flash-crowd bursts. Across physical and simulated evaluations, Agentic-Kube consistently achieves Pareto-efficient placements. In diurnal microservice workloads, it reduces anti-affinity service collisions to 7.11%, representing a 53.0% relative reduction compared to the default Kubernetes scheduler. Under Alibaba traces, the policy achieves a 65.15% spot instance allocation ratio, while macro-scale benchmarks demonstrate scaling up to 1,000 nodes with mean decision latencies under 17ms and 99th-percentile latencies under 31ms, executing without container restart failures and operating well within standard scheduling admission timeouts.

Figures & tables

Framework	Multi-Agent	Graph	Monotonic Value	Kubernetes	Spot Cost	Topological	Action	Sub-25 ms
	(CTDE)	Neural Nets	Factorisation	Native	Optimisation	Resilience	Masking	Latency
DRL4HFC ( 29 )	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\checkmark$
DMOS ( 30 )	$\times$	$\times$	$\times$	$\times$	$\times$	$\checkmark$	$\times$	$\times$
Game-Theoretic ( 32 )	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\checkmark$	$\times$	$\times$
HMAO ( 33 )	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\checkmark$
RLKube ( 21 )	$\times$	$\times$	$\times$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$

Table 1. Architectural and functional comparison of cloud and edge scheduling frameworks.

Figure 1. End-to-end architectural blueprint of the Agentic-Kube framework, illustrating the mapping of the Dec-POMDP cluster state into the GCN and QMIX multi-agent networks.

Figure 2. End-to-end system design and deployment architecture, detailing the offline artifact pipeline, the Kubernetes control plane webhook integration, the operational GKE environment, and the multi-objective telemetry system.

Workload Tier	CPU Request	Memory Request	Priority Level	Target Profile
frontend	0.5 vCPUs	1,024 MiB	3 (High)	User-facing API
gateway	0.2 vCPUs	512 MiB	3 (High)	Ingress Routing
worker	1.0 vCPUs	2,048 MiB	2 (Medium)	Asynchronous Queue
redis	0.25 vCPUs	512 MiB	1 (Low)	In-memory Cache
analytics	2.0 vCPUs	4,096 MiB	1 (Low)	Batch Data Processing

Table 2. Workload Archetypes for Scalability Benchmarks.

Workload Regime	Scheduler Architecture	Spot Ratio	Anti-Affinity	CPU Var.	Mean Latency	$P_{99}$ Latency	Makespan
		(%)	Collision (%)	( $\sigma^{2}$ )	(ms)	(ms)	(s)
Alibaba Trace	Default Kubernetes	50.00 $\pm$ 0.00	14.80 $\pm$ 1.25	0.284 $\pm$ 0.012	3.12 $\pm$ 0.45	8.40 $\pm$ 1.10	14.22 $\pm$ 0.85
	Single-Agent PPO	58.80 $\pm$ 2.10	31.20 $\pm$ 3.45	0.195 $\pm$ 0.018	12.45 $\pm$ 1.80	42.10 $\pm$ 5.20	18.60 $\pm$ 1.15
	MAPPO (SOTA MARL)	61.20 $\pm$ 1.85	22.40 $\pm$ 2.60	0.162 $\pm$ 0.015	18.90 $\pm$ 2.10	68.50 $\pm$ 7.40	21.40 $\pm$ 1.40
(Heterogeneous, 200 Tasks)	NSGA-II (Upper Bound)	68.40 $\pm$ 1.50	2.40 $\pm$ 0.80	1.420 $\pm$ 0.110	312.40 $\pm$ 24.50	1420.0 $\pm$ 115.0	94.50 $\pm$ 6.20
	Agentic-Kube (Ours)	65.15 $\pm$ 1.20	4.90 $\pm$ 0.95	0.148 $\pm$ 0.010	38.60 $\pm$ 3.20	512.40 $\pm$ 42.0	12.80 $\pm$ 0.65

Table 3. Comprehensive performance evaluation on the 8-node physical GKE cluster across empirical Alibaba traces, diurnal microservice variations, and Pareto flash-crowd bursts (mean

\pm

standard deviation across 3 runs per configuration).

Evaluation Dimension	Default Heuristic / Evolutionary	Flat DRL (PPO / MAPPO)	Agentic-Kube (GNN-QMIX)
Multi-Objective	Static, unweighted scoring or exponential	Linear scalar combination causing reward	Decoupled sub-agents with monotonic QMIX
Formulation	sorting complexity ( $\mathcal{O}(G\cdot M\cdot N^{2})$ ).	dilution and gradient conflict.	factorisation satisfying the IGM condition.
Topological	Node-local capacity evaluation; blind to	Vector flattening destroys container co-	Bipartite GCN captures host-pod relational
Representation	multi-tier microservice dependencies.	location and zone failure domains.	graphs in sparse $\mathcal{O}(\|V\|+\|\mathcal{E}\|)$ complexity.
Financial Spot	Fixed allocation ratio mirroring physical	Moderate Spot harvesting (54%–61%)	Active Spot exploitation (63%–66% on GKE;
Exploitation	cluster capacity (50.0%).	limited by scalar reward interference.	up to 91.53% in macro-scale benchmarks).

Table 4. Comparative architectural and performance advantages of Agentic-Kube against baseline scheduling paradigms.

Figure 3. Tri-objective Pareto trade-off between cost efficiency (Spot instance ratio), fault tolerance index ( $100-\text{Collision Rate}\,\%$ ), and workload makespan across physical GKE cluster runs.

Figure 4. Multi-objective architectural radar profile comparing schedulers across six operational dimensions normalised on a

[0,1]

scale.

Figure 5. Physical cluster load balancing and pool specialisation heatmap displaying mean task allocations per node across On-Demand and Spot lifecycle tiers.

Figure 7. Cluster resource reservation envelope comparing total allocated CPU capacity against total allocated memory capacity relative to physical allocatable boundaries.

Figure 8. Algorithm stability and anti-affinity collision variance across independent benchmark iterations, showing mean trajectories and shaded variance envelopes.

Figure 9. Master performance summary matrix across Alibaba trace, diurnal microservice, and flash-crowd burst workloads: (a) anti-affinity collision rate, (b) total scheduling makespan, (c) decision latency, and (d) Spot instance allocation ratio.

Figure 10. Macro-scale scalability benchmark under the offline static policy regime ( $\nabla_{\theta}=0$ ) across physical cluster sizes from 50 to 1,000 nodes, comparing (a) anti-affinity collision rate, (b) Spot instance allocation ratio, (c) scheduling decision latency on a logarithmic scale, and (d) inter-node CPU allocation variance.

Figure 11. Macro-scale scalability benchmark under the online adaptive streaming regime ( $\nabla_{\theta}\neq 0$ ) across physical cluster sizes from 50 to 1,000 nodes, comparing (a) anti-affinity collision rate, (b) Spot instance allocation ratio, (c) scheduling decision latency on a logarithmic scale, and (d) inter-node CPU allocation variance.

Multi-Objective Reinforcement Learning

Multi-objective reinforcement learning studies how agents can balance competing goals and learn policies for different trade-offs.

See all 144 papers

Aug 25, 2026

When Does Learning Beat Heuristics? A Case Study in Kubernetes Scheduler Score Plugins

Kubernetes scheduler plugins that score candidate nodes are, in production, hand-tuned heuristics. We ask whether a learned scoring function - trained on real placement decisions from a production cluster trace - can match or exceed these heuristics, and if not, why. We implement an external, HTTP-backed scoring plugin for a widely used scheduler simulator and evaluate two learned models (a Random Forest over engineered features, and a graph neural network encoder over per-job task-dependency graphs) trained on a large-scale production cluster trace. Under standard regression fit (R^2), both models improve modestly but monotonically across four feature-engineering iterations, reaching R^2 of about 0.042. However, on the metric that actually matters for scheduling - Top-1 ranking accuracy, whether the model scores the machine the production scheduler actually chose highest - both learned models are outperformed by a trivial single-feature heuristic (rank by free CPU: 74-84% vs. 65-66% for either model). We show this gap is best explained by objective mismatch: both models were trained with pointwise regression (MSE) rather than a ranking-specific objective, echoing a long-standing distinction in the learning-to-rank literature. This parallels prior evidence that RL-trained schedulers can outperform heuristics when the training objective is aligned with the deployment task, suggesting objective misalignment, not architecture, is the primary obstacle here. We further report an ablation of the occupancy reconstruction required to make offline trace data usable (naive features yield R^2 near 0), a controlled comparison isolating feature richness and data volume between the two model families, and a sensitivity analysis of inference latency and serving-container memory constraints. Code, data pipelines, and experiment scripts are released for reproducibility.

Wang Xuying, Zhibek Sarypbekova

I. Razzakov Kyrgyz State Technical University

Learning to Rank

Ask PDF

Sep 11, 2026

MCRL2: Multi-resource Cross-attention-based Representation Learning-augmented Reinforcement Learning for Cloud Microservice Scheduling

Efficient microservice scheduling is crucial for maintaining load balance across nodes in data centers and ensuring high quality of service. However, achieving this in practice remains challenging due to dynamic resource imbalance under fluctuating workloads, nonlinear coupling across multiple resource dimensions, and the heterogeneity of microservice resource demands. While reinforcement learning-based approaches have shown promise, they struggle to capture the complex interdependencies among heterogeneous resources and neglect the importance of learning informative system representations. To address these limitations, we propose MCRL2, a novel reinforcement learning approach augmented with multi-resource cross-attention-based representation learning for microservice scheduling. Specifically, we first propose MCRL, a novel representation learning approach that captures structured and informative interactions among nodes, resources, and microservices via a multi-resource cross-attention mechanism. Then, MCRL2 augments reinforcement learning through MCRL-enhanced actor-critic architecture combined with a maximum entropy objective, improving system state expressiveness and leading to more stable and effective scheduling decisions. Extensive experiments on real production cluster traces demonstrate that MCRL2 significantly outperforms existing baselines in load balancing, scheduling success rate and average completion time across diverse workload patterns.

Tiangang Li, Shi Ying, Xiangbo Tian +2

School of Computer Science, Wuhan University, Wuhan 430072, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

Cross-Attention Cloud Computing Representation Learning for RL

Ask PDF

Sep 14, 2026

Cloud Workflow Scheduling Based on Graph Attention-Driven Hierarchical Reinforcement Learning

Dynamic cloud workflow scheduling must balance deadline satisfaction, container utilization, and energy consumption while dealing with stochastic task-execution speeds, placement-dependent communication, and coupled task and container decisions. Workflows are naturally modeled as directed acyclic graphs (DAGs), but conventional vector- or matrix-based states do not fully capture their dependency topology. To better represent task urgency and structural relationships, we assign predicted sub-deadlines to tasks and use a multi-head graph attention network (GAT) to extract dependency information from the evolving DAGs. Based on these representations, we develop a Graph Attention-Driven Hierarchical Reinforcement Learning (GA-HRL) framework and model the scheduling process as an event-driven hierarchical semi-Markov decision process (SMDP). Workflow arrivals and task completions trigger scheduling events. At each scheduling event, the Task Scheduling (TS) agent first processes the currently ready tasks by assigning them to admissible existing containers or requesting new ones. The requested containers are then processed by the Container Scheduling (CS) agent for host placement before the environment advances. The two agents are trained alternately using separate Proximal Policy Optimization (PPO). Experiments on the 2018 Alibaba cluster trace show that GA-HRL maintains competitive workflow success rate and, in settings where success is comparable, generally achieves higher container utilization and lower energy consumption. Under the largest speed variation, it trades a small success-rate margin for substantially lower energy. Simulation code is available at: https://github.com/zongjin130/GA-HRL.

Zongjin Li, Shaohan Feng, Chunxi Yang +1

Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, China · School of Information and Electronic Engineering (Sussex Artificial Intelligence Institute), Zhejiang Gongshang University, Hangzhou 310018, China

Real-Time Scheduling Hierarchical RL

Ask PDF

Abstract

Multi-Objective Reinforcement Learning

Explore similar work

When Does Learning Beat Heuristics? A Case Study in Kubernetes Scheduler Score Plugins

MCRL2: Multi-resource Cross-attention-based Representation Learning-augmented Reinforcement Learning for Cloud Microservice Scheduling

Cloud Workflow Scheduling Based on Graph Attention-Driven Hierarchical Reinforcement Learning