Autoscaling in cloud-native platforms like Kubernetes is reactive and metric-driven, leading to a strategic void problem. This comes from the decoupling of higher-level business policies from lower-level resource provisioning. The strategic void, coupled with a fragmented coordination of pod and node scaling, can lead to significant resource waste and performance degradation under dynamic workloads. In this paper, we present MAS-H2, a new hierarchical multi-agent system that addresses the challenges of autonomic cloud resource management with a complete end-to-end solution. MAS-H2 systematically decomposes the control problem into three layers: a Strategic Agent that formalises business policies (e.g., cost vs. performance) into a global utility function; Planning Agents that produce a joint, proactive scaling plan for pods and nodes with time-series forecasting; and Execution Agents that execute the scaling plan. We built and tested a MAS-H2 prototype as a Kubernetes Operator on Google Kubernetes Engine (GKE) to benchmark it against the native Horizontal Pod Autoscaler (HPA) and Cluster Autoscaler (CA) baselines under two realistic, spiky, and stress-inducing workload scenarios. The results show that the MAS-H2 system maintained application CPU usage under 40% for predictable Heartbeat workloads. This resulted in over 50% less sustained CPU stress than the native HPA baseline, which typically operated above 80%. The MAS-H2 system demonstrated proactive planning in a volatile Chaotic Flash Sale scenario by filtering transient noise and deploying more replicas compared to HPA. It reduced peak CPU load by 55% without under-provisioning. Beyond performance, MAS-H2 seamlessly performed a zero-downtime strategic migration between two cost- and performance-optimised infrastructures.
Figures & tables
Figure 1: High-level diagram of the MAS-H² architecture.
Figure 2: Prototype implementation pipeline
Phase
Dur. (min:sec)
VUs (Target)
Description
Heartbeat 1
Ramp Up
00:30
400
Rapidly ramp to peak load.
Hold Peak
02:00
400
Sustain peak load.
Ramp Down
00:30
10
Rapidly drop to trough.
Hold Trough
01:00
10
Maintain low baseline.
Heartbeat 2
Table 1: Detailed workload profile for the ’Heartbeat’ scenario, designed to test predictive accuracy and scale-down efficiency.
Phase
Dur. (min:sec)
VUs (Target)
Description
1: Pre-Sale Chatter
02:00
20
Low, fluctuating baseline to test noise filtering.
01:00
50
Minor interest spike.
01:00
20
Return to baseline.
2: Chaotic Ramp-Up
00:30
200
Initial surge of traffic.
Table 2: Detailed workload profile for the Chaotic Flash Sale scenario, designed to test forecasting robustness and strategic adaptation under stress.
Figure 3: Comprehensive performance dashboard comparing the MAS-H² agent (orange) against the HPA Baseline (blue), conducting Heartbeat scenario. The plots detail pod and infrastructure scaling, application performance, resource efficiency, and cost metrics over the duration of the experiment.
Figure 4: Comprehensive performance dashboard comparing the MAS-H² agent (orange) against the HPA Baseline (blue), conducting Flash Sale scenario. The plots detail pod and infrastructure scaling, application performance, resource efficiency, and cost metrics over the duration of the experiment.
Figure 5: Pairwise relationship of key metrics in Chaotic Flash Sale scenario. The diagonal shows the distribution for each metric, while the off-diagonal scatter plots reveal correlations. The MAS-H² agent (orange) explores a much wider operational space and navigates efficiency trade-offs more effectively than the static HPA baseline (blue).
Figure 6: Pairwise relationship of key metrics in Chaotic Flash Sale scenario. The diagonal shows the distribution for each metric, while the off-diagonal scatter plots reveal correlations. The MAS-H² agent (orange) explores a much wider operational space and navigates efficiency trade-offs more effectively than the static HPA baseline (blue).
Cloud native architecture is about building and running scalable microservice applications to take full advantage of the cloud environments. Managed Kubernetes is the powerhouse orchestrating cloud native applications with elastic scaling. However, traditional Kubernetes autoscalers are reactive, meaning the scaling controllers adjust resources only after they detect demand within the cluster and do not incorporate any predictive measures. This can lead to either over-provisioning and increased costs or under-provisioning and performance degradation. We propose NimbusGuard, an open-source, Kubernetes-based autoscaling system that leverages a deep reinforcement learning agent to provide proactive autoscaling. The agents perception is augmented by a Long Short-Term Memory model that forecasts future workload patterns. The evaluations were conducted by comparing NimbusGuard against the built-in scaling controllers, such as Horizontal Pod Autoscaler, and the event-driven autoscaler KEDA. The experimental results demonstrate how NimbusGuard's proactive framework translates into superior performance and cost efficiency compared to existing reactive methods.
Chamath Wanigasooriya, Indrajith Ekanayake
Department of Computer Science, Informatics Institute of Technology, Sri Lanka
In modern distributed cloud environments, efficient resource allocation is required as traditional scaling mechanisms are often subject to cloud thrashing due to network-induced latencies. In this paper, we propose C-SAS (Complex-Stability Aware Scaling), an intelligent autonomous orchestration framework that leverages complex analytic methods to achieve system-wide equilibrium. In contrast to heuristic-based models, C-SAS acts as a stability-aware agent, converting telemetry noise into a deterministic "Safety Envelope" on the s-plane using the Argument Principle and Rouché's Theorem. The algorithm smartly suppresses oscillatory scaling operations that would otherwise degrade performance, by computing a real-time Analytic Stability Index (ASI). The experimental results show that C-SAS reduces VM flapping by 94%, and achieves 96% resource efficiency, significantly outperforming standard PID and ML-based autonomous agents. Our results suggest that future resilient autonomous cloud infrastructures will require AI-driven orchestrators with built-in formal stability constraints.
Gopal Krishna Shyam, Priyanka Bharti
Department of Computer Science & Engineering Presidency University, Bengaluru, India · Department of Computer Science & Engineering REVA University, Bengaluru, India
In edge computing, the stochastic and bursty nature of serverless workloads challenges autonomous resource orchestration. Traditional reactive controllers, such as the Kubernetes Horizontal Pod Autoscaler (HPA), suffer from reaction latency, leading to Service Level Objective (SLO) violations during traffic spikes and resource flapping during ramp-downs. While Deep Reinforcement Learning (DRL) offers a pathway toward proactive management, standard agents suffer from temporal blindness, an inability to exploit the recent temporal context in non-Markovian edge environments. To bridge this gap, we propose a stability-aware autoscaling framework unifying short-horizon temporal context and control via an Attention-Enhanced Double-Stacked LSTM architecture integrated within a Proximal Policy Optimization (PPO) agent. Unlike shallow recurrent models, our approach employs a learned attention mechanism that weights recent historical states non-uniformly, suppressing high-frequency jitter while preserving the trend that precedes demand shifts. We validate the framework on two independent Kubernetes clusters using real-world Azure Functions traces. Against the single-layer LSTM ablation and the static HPA baseline, our approach reduces P90 latency by ≈67%, and holds average latency within the 50ms hard SLO for 98.8% of the run against 49.6% and 43.5% respectively. Against Kubernetes Event-Driven Autoscaling (KEDA), it matches latency performance at 75% fewer replica-steps and 59% less churn, with P90 hard-SLO violation bursts of at most 5 consecutive intervals against up to 24 for KEDA. These results indicate that mitigating temporal blindness through deep attentive memory improves the reliability and stability of Kubernetes autoscaling under bursty edge workloads.
Faraz Shaikh, Gianluca Reali, Mauro Femminella
Department of Engineering, University of Perugia, Perugia, Italy · Consorzio Nazionale Interuniversitario per le Telecomunicazioni (CNIT), 43124 Parma, Italy