cs.NENov 6, 2026

Two-Layer Evolutionary Scheduling for Machine Learning Jobs in Multi-Cluster Environments

Authors: Jingyuan Xu, Changwu Huang, Lemeng Pan, Ke Tang, Xin Yao

Organizations: Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering of Southern University of Science and Technology, Shenzhen, China. · School of AI and Liberal Arts, Beijing Normal-Hong Kong Baptist University, Zhuhai, China. · Huawei Technologies Company, Ltd., Shenzhen, China. · School of Data Science, Lingnan University, Hong Kong SAR, China.

Abstract

The rapid growth of large-scale machine learning (ML) workloads has led to unprecedented demands on GPU cluster scheduling systems. However, existing heuristic schedulers struggle to handle the complexity and dynamics of multi-cluster GPU environments, often leading to load imbalance, long queuing delays, and poor alignment to business objectives. This paper formulates the ML scheduling problem in multi-cluster GPU environments as online multi-objective optimization that minimizes the average and maximum weighted queuing times while maximizing GPU time utilization. To solve this optimization problem, we propose EVO-GP, a two-layer evolutionary online scheduling method that decouples global job-to-cluster allocation from intra-cluster GPU resource allocation. At the job-to-cluster allocation layer, an evolutionary algorithm assigns arriving jobs to feasible clusters to promote load balance across clusters. At the resource allocation layer, a priority function automatically learned offline via genetic programming is used to rank jobs and guide GPU allocation within each cluster. Extensive experiments on both real-world data and simulated workloads demonstrate that the proposed EVO-GP method consistently outperforms state-of-the-art baselines, while maintaining superior adaptability and scalability.

Figures & tables

Explore similar work

CardsList
  1. Plenty of Room in the Middle: Revealing Opportunities on a GPU Cluster via Fine-Grained Monitoring

    Oct 26, 2026Andrew Sheinberg, Samyak Gupta, Kai LiHigh-Performance Computing

  2. Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads

    Jun 8, 2026Minyu Cui, Miquel PericasGPU Kernel OptimizationCommunication-Efficient Distributed Training

  3. AEGIS: Runtime-Guided GPU Collocation for Multi-Tenant Deep Learning Training

    Aug 26, 2025Ehsan Yousefzadeh-Asl-Miandoab, Büşra Karatay Demiray, Florina M. Ciorba +2Real-Time SchedulingOnline Resource Allocation