cs.LGAug 19, 2026

LLM-Powered Predictive Decision-Making for Sustainable Data Center Operations

Authors: Hanzhao Wang, Jingxuan Wu, Yumeng Li, Yu Pan, Guanting Chen

Organizations: The University of Sydney Business School, The University of Sydney · Department of Statistics and Operations Research, University of North Carolina at Chapel Hill · Department of Industrial Engineering, Purdue University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The growing demand for AI-driven workloads, particularly from Large Language Models (LLMs), has raised concerns about the significant energy and resource consumption in data centers. This work introduces a novel LLM-based predictive scheduling system designed to enhance operational efficiency while reducing the environmental impact of data centers. Our system utilizes an LLM to predict key metrics such as execution time and energy consumption from source code, and it has the potential to extend to other sustainability-focused metrics like water usage for cooling and carbon emissions, provided the data center can track such data. The predictive model is followed by a real-time scheduling algorithm that allocates GPU resources, aiming to improve sustainability by optimizing both energy consumption and queuing delays. With fast inference times, the ability to generalize across diverse task types, and minimal data requirements for training, our approach offers a practical solution for data center scheduling. This framework demonstrates strong potential for advancing sustainability objectives in AI-driven infrastructure. Through our collaboration with a data center, we achieved a 32% reduction in energy consumption and a 30% decrease in waiting time.

Figures & tables

Figure 1: Our Proposed Pipeline
Figure 2: Illustration of our unified predictive approach. Notably, previous methods were unable to handle tasks that had not been seen before, or composite tasks (e.g., training a CNN followed by an LSTM). However, the generalization capacity of LLMs allows our model to effectively manage such cases, making it adaptable to a wider variety of task types and combinations.
Figure 3: Illustration of our model architecture. Our streamlined design is easy and fast to implement, highly flexible, and generalizable across a variety of tasks. Notice that to estimate other quantities of interest, we simply need to add additional probes. In contrast, previous methods required different handcrafted features and separate prediction models for each task, making them far less flexible and scalable. The lower graph illustrates the performance difference, where points closer to the straight line indicate more accurate predictions. The performance is evaluated under test sets with additional adversarially generated data, and our method outperforms the previous approach ( [ 22 ] and [ 23 ] ) in both accuracy and robustness. Notably, the previous method shows consistent systematic bias, likely due to its reliance on “physical” features like the number of forward/backward passes and layer depth, which lack generalizability.
ModelTWT (s)TDTCRT(s)TEC (kWh)
Simple3,135,824.0129.7916,924,533.71470.69
Value-based2,186,166.4922.5016,565,627.33322.07
Improvement (%)30.28%24.47%2.12%31.58%
Table 1: Performance gain for our algorithm implemented in data centers. Here, TWT stands for total waiting time, TDT stands for total delayed tasks, the tasks that have to wait, CRT stands for cumulative running time, and TEC stands for the total energy cost.
Figure 4: (a) Task arrival pattern and (b), (c) performance comparison among the proposed predictive decision-making algorithms, Value-based and Greedy (see Appendix A.3.1 ), and the benchmark algorithm, Simple Rule. The Simple Rule assigns the available GPU type to the first task in the waiting queue, following a first-come-first-served policy when sufficient GPUs are available. If multiple GPU types are available, the most powerful one (e.g., A100) is selected.
Figure 5: Architecture of the extended pipeline to resolve out-of-distribution tasks
Figure 6: Figure (a) displays the heat map of embedding distances between two classes. Indexes 0–3 represent tasks training the ViT model, while the remaining indexes correspond to tasks training the GAN Model. In Figure (b), indexes 0 and 1 represent ViT model training tasks, indexes 2 and 4 represent out-of-distribution ViT model tasks, and indexes 3 and 6 are LLM-rewritten versions of the tasks of index 2 and 4. Figures (c) and (d) compare the prediction performance of the execution time for the out-of-distribution tasks before(c) and after(d) the LLM rewrite.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

 t^ij=fθ(xi,j),e^ij=gθ(xi,j).\hat{t}_{ij}=f_{\theta}(\bm{x}_{i},j),\qquad\hat{e}_{ij}=g_{\theta}(\bm{x}_{i},j). 
Appendix
Algorithm 1 Predictive Decision-Making: Greedy
 t^ij=fθ(xi,j),e^ij=gθ(xi,j).\hat{t}_{ij}=f_{\theta}(\bm{x}_{i},j),\qquad\hat{e}_{ij}=g_{\theta}(\bm{x}_{i},j). 
Appendix
Algorithm 2 Predictive Decision-Making: Value Based
MetricSimple ruleValue-based rule
Waiting timeRunning timeEnergy
Total Waiting Time (s)2,466,062.561,810,123.922,045,399.252,112,773.56
-Tasks with Wait Time27.3821.1622.1322.80
Total Running Time (s)15,333,868.4715,215,721.2115,001,371.1515,291,965.57
Total Energy Cost (kWh)440.98245.42243.97240.07
Appendix
Table 2: Comparison of testing total waiting time, tasks with wait time, cumulative running time, and energy cost for the benchmark algorithm (simple rule) and value-based models tuned under different emphasized objectives.
Figure 7: Performance comparison across different metrics for the benchmark algorithm (simple rule) and value-based models tuned under different emphasized objectives.
More in this topic

Data Center Energy Management

See all 15 papers

Explore similar work

CardsList

  1. InFactPlanner: Planning Sustainable Geo-Distributed LLM Data Centers

    Aug 13, 2026Nicoletta Tsiopani, Moysis Symeonides, George Pallis +1Energy-Efficient MLLLM Inference Serving

    Department of Computer Science University of Cyprus

  2. AI-driven Thermal-aware Data Center Capacity Planning

    Oct 1, 2026Yixing Li, Mark Fenton, Matthew Kaufeler +3Surrogate Modeling

    Cadence Design Systems

  3. Sustainable LLM Inference using Context-Aware Model Switching

    Feb 25, 2026Yuvarani, Akashdeep Singh, Zahra Fathanah +3LLM Inference EfficiencyLLM Inference

    Faculty of Computer Science and Information Technology, Universiti Malaya, 50603 Kuala Lumpur, Malaysia