Solver-Free and Cost-Aware Online Scheduling of AI Data Center Tasks
Authors: Xin Qin, Dawei Qiu, Fei Teng
Organizations: School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Department of Electrical and Electronic Engineering, Imperial College London, London, UK
AI data centers receive computing tasks continuously, whereas solver-based optimization takes time to run and electricity is priced per settlement period. This paper bridges this timing gap through a solver-free online optimization rule inspired by dual decomposition. The rule assigns each task a start period upon arrival to optimize energy cost subject to operational constraints, shifting flexible tasks toward low-price periods while keeping high-priority tasks on schedule. Simulations on the A100 GPU trace (3,456 GPUs, 1,343 tasks) show that the proposed rule keeps GPU and CPU use within capacity while scheduling tasks at low cost. Compared with a multi-timescale optimization benchmark, it achieves a lower cost and, because it needs no solver, much less computation time.
Figures & tables
Symbol
Meaning
t∈T
Time period, T={1,…,T}
c∈{G,C}
Resource: GPU or vCPU
j∈J , JHP
Task submitted; set of all and high-priority (HP) tasks
Sj
Set of possible start periods of task j
Fj
Set of feasible start periods of task j (fit free capacity)
Δt
Operation period length
TABLE I: Nomenclature
Fig. 1: Workflow of the proposed solver-free online scheduling rule.
Fig. 2: Day-ahead and real-time prices of CAISO NP15 on August 21, 2026.
Fig. 3: GPU demand of the tasks if every task starts at its submission: tasks running from yesterday and the HP, Priority and Spot tasks of the day.
Parameter
Value
Parameter
Value
Horizon
26 h
Dj , Spot
120 (120 min)
Period length Δt
1 min
Benchmark 2 update
5 min
Dj , HP
0
Offline MILP gap
10−4
Dj , Priority
60 (60 min)
Offline time limit
600 s
TABLE II: Simulation Settings
Method
Flexible-task cost ($)
Cost above offline (%)
Late HP tasks (#)
Time per decision (ms)
Benchmark 1
378.02
12.9
0
0.032
Benchmark 2
341.27
1.9
7
41.4
Proposed
340.74
1.7
0
0.024
Perfect
334.97
0
0
52,800
TABLE III: Energy Cost, Scheduling Performance, and Computation Time
Fig. 4: GPU use per period of the four cases.
Fig. 5: vCPU use per period of the four cases.
Fig. 6: Hourly cost of the tasks of the day.
Fig. 7: Difference in hourly cost relative to Perfect.
Online resource allocation studies how to assign scarce capacity to arriving requests under uncertainty while balancing reward, cost, fairness, and feasibility.
The rise of Artificial Intelligence and Large Language Models is driving increased GPU usage in data centers for complex training and inference tasks, impacting operational costs, energy demands, and the environmental footprint of large-scale computing infrastructures. This work addresses the online scheduling problem in GPU datacenters, which involves scheduling tasks without knowledge of their future arrivals. We focus on two objectives: minimizing GPU fragmentation and reducing power consumption. GPU fragmentation occurs when partial GPU allocations hinder the efficient use of remaining resources, especially as the datacenter nears full capacity. A recent scheduling policy, Fragmentation Gradient Descent (FGD), leverages a fragmentation metric to address this issue. Reducing power consumption is also crucial due to the significant power demands of GPUs. To this end, we propose PWR, a novel scheduling policy to minimize power usage by selecting power-efficient GPU and CPU combinations. This involves a simplified model for measuring power consumption integrated into a Kubernetes score plugin. Through an extensive experimental evaluation in a simulated cluster, we show how PWR, when combined with FGD, achieves a balanced trade-off between reducing power consumption and minimizing GPU fragmentation.
Francesco Lettich, Emanuele Carlini, Franco Maria Nardini +2
Istituto di Scienza e Tecnologie dell’Informazione “Alessandro Faedo”, Consiglio Nazionale delle Ricerche, Pisa, Italy
Rapid AI data-center power fluctuations challenge grid integration. Sustained grid-import smoothing requires allocating limited battery power and energy under uncertain demand. We formulate risk-aware BESS scheduling that coordinates advance reserve commitments and demand-responsive dispatch, accounting for operating costs, grid-ramp penalties, and their conditional value-at-risk. We develop battery scheduling decision-focused learning (BS-DFL) with a constraint-aware neural optimization layer trained directly from this objective without optimal-solution labels. The trained layer is then frozen to guide demand-predictor learning through scheduling costs and ramping risk, enabling solver-free rolling-horizon scheduling. In a 10,000-minute simulation using loads derived from measured AI workloads, BS-DFL reduces mean absolute minute-to-minute grid-import changes by 71.70% and total scheduling cost by 18.01% relative to an MSE-trained predictor coupled with a mixed-integer linear programming solver, with an average decision time of 18.77 ms.
Xinda Zheng, Canchen Jiang, Frits de Nijs +2
Department of Data Science and AI, Faculty of Information Technology, Monash University, Australia
The rapid growth of AI training has dramatically increased datacenter traffic demand and energy consumption, which has motivated renewed interest in optical circuit switches (OCSes) as a high-bandwidth, energy-efficient alternative for AI fabrics. Deploying multiple parallel OCSes is a leading alternative. However, efficiently scheduling time-varying traffic matrices across parallel optical switches with non-negligible reconfiguration delays remains an open challenge. We consider the problem of scheduling a single AI traffic demand matrix D over s parallel OCSes while minimizing the makespan under reconfiguration delay δ. Our algorithm Spectra relies on a three-step approach: Decompose D into a minimal set of weighted permutations; Schedule these permutations across parallel switches using load-aware assignment; then Equalize the imbalanced loads on the switches via controlled permutation splitting. Evaluated on realistic AI training workloads (GPT model and Qwen MoE expert routing) as well as standard benchmarks, Spectra vastly outperforms a baseline based on state-of-the-art algorithms, reducing schedule makespan by an average factor of 1.4× on GPT AI workloads, 2.7× on MoE AI workloads, and 2.4× on standard benchmarks. Further, the makespans achieved by Spectra consistently approach newly derived lower bounds.