Two-Layer Evolutionary Scheduling for Machine Learning Jobs in Multi-Cluster Environments
Organizations: Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering of Southern University of Science and Technology, Shenzhen, China. · School of AI and Liberal Arts, Beijing Normal-Hong Kong Baptist University, Zhuhai, China. · Huawei Technologies Company, Ltd., Shenzhen, China. · School of Data Science, Lingnan University, Hong Kong SAR, China.
Abstract
The rapid growth of large-scale machine learning (ML) workloads has led to unprecedented demands on GPU cluster scheduling systems. However, existing heuristic schedulers struggle to handle the complexity and dynamics of multi-cluster GPU environments, often leading to load imbalance, long queuing delays, and poor alignment to business objectives. This paper formulates the ML scheduling problem in multi-cluster GPU environments as online multi-objective optimization that minimizes the average and maximum weighted queuing times while maximizing GPU time utilization. To solve this optimization problem, we propose EVO-GP, a two-layer evolutionary online scheduling method that decouples global job-to-cluster allocation from intra-cluster GPU resource allocation. At the job-to-cluster allocation layer, an evolutionary algorithm assigns arriving jobs to feasible clusters to promote load balance across clusters. At the resource allocation layer, a priority function automatically learned offline via genetic programming is used to rank jobs and guide GPU allocation within each cluster. Extensive experiments on both real-world data and simulated workloads demonstrate that the proposed EVO-GP method consistently outperforms state-of-the-art baselines, while maintaining superior adaptability and scalability.
Figures & tables
| Terminal Set | Description |
|---|---|
| The remaining duration of job . | |
| The length of time that job j has been queued. | |
| The priority defined by the user for job . | |
| The total number of GPUs required by job . | |
| Function Set | (protected division) |
| Worker Num | GPU Num | Avg. Training Hours | Weight |
|---|---|---|---|
| 1 | 1 | 1.499136 | 2.44 10 -1 |
| 1 | 2 | 0.906435 | 1.23 10 -1 |
| 1 | 4 | 1.050502 | 8.20 10 -2 |
| 1 | 8 | 3.285060 | 4.29 10 -1 |
| 2 | 2 | 1.701724 | 3.00 10 -2 |
| 2 | 4 | 1.348965 | 1.00 10 -3 |
Online Resource Allocation
Online resource allocation studies how to assign scarce capacity to arriving requests under uncertainty while balancing reward, cost, fairness, and feasibility.
See all 100 papers