Agentic-Kube: A Graph-Enhanced Multi-Agent Reinforcement Learning Framework for Multi-Objective Kubernetes Scheduling

Hamed Hamzeh

cs.DCMar 12, 2026

Agentic-Kube: A Graph-Enhanced Multi-Agent Reinforcement Learning Framework for Multi-Objective Kubernetes Scheduling

Authors: Hamed Hamzeh

Organizations: University of Westminster Computer Science and Engineering London, United Kingdom

Abstract

Cloud-native container orchestration requires resource schedulers capable of balancing infrastructure expenditure, fault resilience, and node utilisation. Conventional reinforcement learning approaches typically rely on monolithic single-agent models that suffer from gradient interference and reward dilution when mapping conflicting operational goals into a single scalar reward. We present Agentic-Kube, a cooperative multi-agent reinforcement learning framework designed for real-time Kubernetes pod placement. The architecture decomposes multi-objective scheduling into a tripartite optimisation space managed by dedicated sub-agents for cost minimisation, anti-affinity fault tolerance, and vector resource balancing. Agentic-Kube integrates a bipartite Graph Convolutional Network to capture dynamic host-pod dependencies, a two-stage monotonic QMIX value factorisation network to maintain joint action value coherence, and a plurality voting consensus mechanism with action feasibility masking against allocatable node predicates. We evaluate the framework across live heterogeneous Google Kubernetes Engine deployments and macro-scale cluster environments spanning 50 to 1,000 nodes under empirical Alibaba trace data, diurnal microservice variations, and flash-crowd bursts. Across physical and simulated evaluations, Agentic-Kube consistently achieves Pareto-efficient placements. In diurnal microservice workloads, it reduces anti-affinity service collisions to 7.11%, representing a 53.0% relative reduction compared to the default Kubernetes scheduler. Under Alibaba traces, the policy achieves a 65.15% spot instance allocation ratio, while macro-scale benchmarks demonstrate scaling up to 1,000 nodes with mean decision latencies under 17ms and 99th-percentile latencies under 31ms, executing without container restart failures and operating well within standard scheduling admission timeouts.

Figures & tables

Framework	Multi-Agent	Graph	Monotonic Value	Kubernetes	Spot Cost	Topological	Action	Sub-25 ms
	(CTDE)	Neural Nets	Factorisation	Native	Optimisation	Resilience	Masking	Latency
DRL4HFC ( 29 )	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\checkmark$
DMOS ( 30 )	$\times$	$\times$	$\times$	$\times$	$\times$	$\checkmark$	$\times$	$\times$
Game-Theoretic ( 32 )	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\checkmark$	$\times$	$\times$
HMAO ( 33 )	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$	$\checkmark$
RLKube ( 21 )	$\times$	$\times$	$\times$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$

Table 1. Architectural and functional comparison of cloud and edge scheduling frameworks.

Figure 1. End-to-end architectural blueprint of the Agentic-Kube framework, illustrating the mapping of the Dec-POMDP cluster state into the GCN and QMIX multi-agent networks.

Figure 2. End-to-end system design and deployment architecture, detailing the offline artifact pipeline, the Kubernetes control plane webhook integration, the operational GKE environment, and the multi-objective telemetry system.

Workload Tier	CPU Request	Memory Request	Priority Level	Target Profile
frontend	0.5 vCPUs	1,024 MiB	3 (High)	User-facing API
gateway	0.2 vCPUs	512 MiB	3 (High)	Ingress Routing
worker	1.0 vCPUs	2,048 MiB	2 (Medium)	Asynchronous Queue
redis	0.25 vCPUs	512 MiB	1 (Low)	In-memory Cache
analytics	2.0 vCPUs	4,096 MiB	1 (Low)	Batch Data Processing

Table 2. Workload Archetypes for Scalability Benchmarks.

Workload Regime	Scheduler Architecture	Spot Ratio	Anti-Affinity	CPU Var.	Mean Latency	$P_{99}$ Latency	Makespan
		(%)	Collision (%)	( $\sigma^{2}$ )	(ms)	(ms)	(s)
Alibaba Trace	Default Kubernetes	50.00 $\pm$ 0.00	14.80 $\pm$ 1.25	0.284 $\pm$ 0.012	3.12 $\pm$ 0.45	8.40 $\pm$ 1.10	14.22 $\pm$ 0.85
	Single-Agent PPO	58.80 $\pm$ 2.10	31.20 $\pm$ 3.45	0.195 $\pm$ 0.018	12.45 $\pm$ 1.80	42.10 $\pm$ 5.20	18.60 $\pm$ 1.15
	MAPPO (SOTA MARL)	61.20 $\pm$ 1.85	22.40 $\pm$ 2.60	0.162 $\pm$ 0.015	18.90 $\pm$ 2.10	68.50 $\pm$ 7.40	21.40 $\pm$ 1.40
(Heterogeneous, 200 Tasks)	NSGA-II (Upper Bound)	68.40 $\pm$ 1.50	2.40 $\pm$ 0.80	1.420 $\pm$ 0.110	312.40 $\pm$ 24.50	1420.0 $\pm$ 115.0	94.50 $\pm$ 6.20
	Agentic-Kube (Ours)	65.15 $\pm$ 1.20	4.90 $\pm$ 0.95	0.148 $\pm$ 0.010	38.60 $\pm$ 3.20	512.40 $\pm$ 42.0	12.80 $\pm$ 0.65

Table 3. Comprehensive performance evaluation on the 8-node physical GKE cluster across empirical Alibaba traces, diurnal microservice variations, and Pareto flash-crowd bursts (mean

\pm

standard deviation across 3 runs per configuration).

Evaluation Dimension	Default Heuristic / Evolutionary	Flat DRL (PPO / MAPPO)	Agentic-Kube (GNN-QMIX)
Multi-Objective	Static, unweighted scoring or exponential	Linear scalar combination causing reward	Decoupled sub-agents with monotonic QMIX
Formulation	sorting complexity ( $\mathcal{O}(G\cdot M\cdot N^{2})$ ).	dilution and gradient conflict.	factorisation satisfying the IGM condition.
Topological	Node-local capacity evaluation; blind to	Vector flattening destroys container co-	Bipartite GCN captures host-pod relational
Representation	multi-tier microservice dependencies.	location and zone failure domains.	graphs in sparse $\mathcal{O}(\|V\|+\|\mathcal{E}\|)$ complexity.
Financial Spot	Fixed allocation ratio mirroring physical	Moderate Spot harvesting (54%–61%)	Active Spot exploitation (63%–66% on GKE;
Exploitation	cluster capacity (50.0%).	limited by scalar reward interference.	up to 91.53% in macro-scale benchmarks).

Table 4. Comparative architectural and performance advantages of Agentic-Kube against baseline scheduling paradigms.

Figure 3. Tri-objective Pareto trade-off between cost efficiency (Spot instance ratio), fault tolerance index ( $100-\text{Collision Rate}\,\%$ ), and workload makespan across physical GKE cluster runs.

Figure 4. Multi-objective architectural radar profile comparing schedulers across six operational dimensions normalised on a

[0,1]

scale.

Figure 5. Physical cluster load balancing and pool specialisation heatmap displaying mean task allocations per node across On-Demand and Spot lifecycle tiers.

Figure 7. Cluster resource reservation envelope comparing total allocated CPU capacity against total allocated memory capacity relative to physical allocatable boundaries.

Figure 8. Algorithm stability and anti-affinity collision variance across independent benchmark iterations, showing mean trajectories and shaded variance envelopes.

Figure 9. Master performance summary matrix across Alibaba trace, diurnal microservice, and flash-crowd burst workloads: (a) anti-affinity collision rate, (b) total scheduling makespan, (c) decision latency, and (d) Spot instance allocation ratio.

Figure 10. Macro-scale scalability benchmark under the offline static policy regime ( $\nabla_{\theta}=0$ ) across physical cluster sizes from 50 to 1,000 nodes, comparing (a) anti-affinity collision rate, (b) Spot instance allocation ratio, (c) scheduling decision latency on a logarithmic scale, and (d) inter-node CPU allocation variance.

Figure 11. Macro-scale scalability benchmark under the online adaptive streaming regime ( $\nabla_{\theta}\neq 0$ ) across physical cluster sizes from 50 to 1,000 nodes, comparing (a) anti-affinity collision rate, (b) Spot instance allocation ratio, (c) scheduling decision latency on a logarithmic scale, and (d) inter-node CPU allocation variance.

Multi-Objective Reinforcement Learning

Multi-objective reinforcement learning studies how agents can balance competing goals and learn policies for different trade-offs.

See all 144 papers

When Does Learning Beat Heuristics? A Case Study in Kubernetes Scheduler Score Plugins
Aug 25, 2026Wang Xuying, Zhibek SarypbekovaLearning to Rank
I. Razzakov Kyrgyz State Technical University
MCRL2: Multi-resource Cross-attention-based Representation Learning-augmented Reinforcement Learning for Cloud Microservice Scheduling
Sep 11, 2026Tiangang Li, Shi Ying, Xiangbo Tian +2Cross-Attention Cloud Computing
School of Computer Science, Wuhan University, Wuhan 430072, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China
Cloud Workflow Scheduling Based on Graph Attention-Driven Hierarchical Reinforcement Learning
Sep 14, 2026Zongjin Li, Shaohan Feng, Chunxi Yang +1Real-Time Scheduling Hierarchical RL
Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, China · School of Information and Electronic Engineering (Sussex Artificial Intelligence Institute), Zhejiang Gongshang University, Hangzhou 310018, China

Aug 25, 2026J/K move · Enter open · S save

When Does Learning Beat Heuristics? A Case Study in Kubernetes Scheduler Score Plugins

Wang Xuying, Zhibek Sarypbekova

I. Razzakov Kyrgyz State Technical University

Learning to Rank

Open paper AskPDF arXiv

Kubernetes scheduler plugins that score candidate nodes are, in production, hand-tuned heuristics. We ask whether a learned scoring function - trained on real placement decisions from a production cluster trace - can match or exceed these heuristics, and if not, why. We implement an external, HTTP-backed scoring plugin for a widely used scheduler simulator and evaluate two learned models (a Random Forest over engineered features, and a graph neural network encoder over per-job task-dependency graphs) trained on a large-scale production cluster trace. Under standard regression fit (R^2), both models improve modestly but monotonically across four feature-engineering iterations, reaching R^2 of about 0.042. However, on the metric that actually matters for scheduling - Top-1 ranking accuracy, whether the model scores the machine the production scheduler actually chose highest - both learned models are outperformed by a trivial single-feature heuristic (rank by free CPU: 74-84% vs. 65-66% for either model). We show this gap is best explained by objective mismatch: both models were trained with pointwise regression (MSE) rather than a ranking-specific objective, echoing a long-standing distinction in the learning-to-rank literature. This parallels prior evidence that RL-trained schedulers can outperform heuristics when the training objective is aligned with the deployment task, suggesting objective misalignment, not architecture, is the primary obstacle here. We further report an ablation of the occupancy reconstruction required to make offline trace data usable (naive features yield R^2 near 0), a controlled comparison isolating feature richness and data volume between the two model families, and a sensitivity analysis of inference latency and serving-container memory constraints. Code, data pipelines, and experiment scripts are released for reproducibility.

Abstract

Multi-Objective Reinforcement Learning

Explore similar work

When Does Learning Beat Heuristics? A Case Study in Kubernetes Scheduler Score Plugins

MCRL2: Multi-resource Cross-attention-based Representation Learning-augmented Reinforcement Learning for Cloud Microservice Scheduling

Cloud Workflow Scheduling Based on Graph Attention-Driven Hierarchical Reinforcement Learning