When Does Learning Beat Heuristics? A Case Study in Kubernetes Scheduler Score Plugins

Cloud-native container orchestration requires resource schedulers capable of balancing infrastructure expenditure, fault resilience, and node utilisation. Conventional reinforcement learning approaches typically rely on monolithic single-agent models that suffer from gradient interference and reward dilution when mapping conflicting operational goals into a single scalar reward. We present Agentic-Kube, a cooperative multi-agent reinforcement learning framework designed for real-time Kubernetes pod placement. The architecture decomposes multi-objective scheduling into a tripartite optimisation space managed by dedicated sub-agents for cost minimisation, anti-affinity fault tolerance, and vector resource balancing. Agentic-Kube integrates a bipartite Graph Convolutional Network to capture dynamic host-pod dependencies, a two-stage monotonic QMIX value factorisation network to maintain joint action value coherence, and a plurality voting consensus mechanism with action feasibility masking against allocatable node predicates. We evaluate the framework across live heterogeneous Google Kubernetes Engine deployments and macro-scale cluster environments spanning 50 to 1,000 nodes under empirical Alibaba trace data, diurnal microservice variations, and flash-crowd bursts. Across physical and simulated evaluations, Agentic-Kube consistently achieves Pareto-efficient placements. In diurnal microservice workloads, it reduces anti-affinity service collisions to 7.11%, representing a 53.0% relative reduction compared to the default Kubernetes scheduler. Under Alibaba traces, the policy achieves a 65.15% spot instance allocation ratio, while macro-scale benchmarks demonstrate scaling up to 1,000 nodes with mean decision latencies under 17ms and 99th-percentile latencies under 31ms, executing without container restart failures and operating well within standard scheduling admission timeouts.

Efficiently allocating incoming jobs to nodes in large-scale clusters can lead to substantial improvements in both cluster utilization and job performance. In order to allocate incoming jobs, cluster schedulers usually rely on a set of scoring functions to rank feasible nodes. Results from individual scoring functions are usually weighted equally, which could lead to sub-optimal deployments as the one-size-fits-all solution does not take into account the characteristics of each workload. Tuning the weights of scoring functions, however, requires expert knowledge and is computationally expensive. This paper proposes a reinforcement learning approach for learning the weights in scheduler scoring algorithms with the overall objective of improving the end-to-end performance of jobs for a given cluster. Our approach is based on percentage improvement reward, frame-stacking, and limiting domain information. We propose a percentage improvement reward to address the objective of multi-step parameter tuning. The inclusion of frame-stacking allows for carrying information across an optimization experiment. Limiting domain information prevents overfitting and improves performance in unseen clusters and workloads. The policy is trained on different combinations of workloads and cluster setups. We demonstrate the proposed approach improves performance on average by 33% compared to fixed weights and 12% compared to the best-performing baseline in a lab-based serverless scenario.

Efficient task scheduling in heterogeneous computing systems remains a fundamental challenge in embedded and high-level synthesis flows. This paper presents MoSAIC (Motif-aware Scheduling with AI-driven Cognition), a learning-based framework that bridges the gap between exact and heuristic scheduling approaches for DAG-based applications. The system uses the reasoning capabilities of large language models (LLMs) to learn the relationships between DAG structures and scheduling priority functions. The framework first leverages the OR-Tools CP-SAT solver [4] to generate optimal schedules, which are then used to supervise the learning of compact priority-based scheduling functions through Bayesian Optimization. These learned heuristics capture relationships between scheduling decisions and structural graph features such as rank, depth, and fanout, enabling the construction of a reusable knowledge base across diverse DAG instances. Experimental results across synthetic graph families and real-world benchmarks, including PolyBench and MachSuite, show that the learned heuristics consistently improve over classical approaches such as HEFT [2], while maintaining low runtime overhead. The results indicate that combining optimal supervision with learning-based generalization provides a practical approach for scalable and adaptive task scheduling.