eess.SYApr 8, 2026

Measurement of Generative AI Workload Power Profiles for Whole-Facility Data Center Infrastructure Planning

Authors: Roberto Vercellino, Jared Willard, Gustavo Campos, Weslley da Silva Pereira, Olivia Hull, Matthew Selensky, Juliane Mueller

Organizations: National Laboratory of the Rockies · National Laboratory of the Rockies (NLR), Golden, CO, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The rapid growth of generative artificial intelligence (AI) has introduced unprecedented computational demands, driving significant increases in the energy footprint of data centers. However, existing power consumption data is largely proprietary and reported at varying resolutions, creating challenges for estimating whole-facility energy use and planning infrastructure. In this work, we present a methodology that bridges this gap by linking high-resolution workload power measurements to whole-facility energy demand. Using NLR's high-performance computing data center equipped with NVIDIA H100 GPUs, we measure power consumption of AI workloads at 0.1-second resolution for AI training, fine-tuning and inference jobs. Workloads are characterized using MLCommons benchmarks for model training and fine-tuning, and vLLM benchmarks for inference, enabling reproducible and standardized workload profiling. The dataset of power consumption profiles is made publicly available. These power profiles are then scaled to the whole-facility-level using a bottom-up, event-driven, data center energy model. The resulting whole-facility energy profiles capture realistic temporal fluctuations driven by AI workloads and user-behavior, and can be used to inform infrastructure planning for grid connection, on-site energy generation, and distributed microgrids.

Figures & tables

AcronymMeaning
AIArtificial Intelligence
CPUCentral Processing Unit
GPUGraphics Processing Unit
GenAIGenerative Artificial Intelligence
HPCHigh-Performance Computing
ITInformation Technology
List of Acronyms
Figure 1: Diagram illustrating the workflow and scaling from data center compute node to facility level 2 2 2 Photos by Agata Bogucka (NLR 104051), Werner Slocum (NLR 63014, 67310), and Joe DelNero (NLR 101334). .
Figure 2: Data Center Infrastructure Planning, Load Optimization for Energy Efficiency (DIPLOEE) model diagram.
Figure 3: DIPLOEE model details for inference data center use case.
Figure 4: Llama-2 70B fine-tuning power profiles. Replicate experiments using the same node count are shown as superimposed transparent curves with different colors. Each power profile aggregates power consumption across all CPU and GPU devices utilized by the workload.
Figure 5: Llama-2 70B fine-tuning energy metrics as a function of number of nodes. Replicate experiments are shown as red dots, vertically distributed with a box-and-whisker plot for each node value. Fitted curves are also presented as dashed lines.
NodesMean Total Runtime (sec)Mean Epoch Runtime (sec)Mean Epoch SpeedupNumber of EpochsNumber of Evaluations
25,152 ± 2263,930 ± 610.000.91 ± 0.047.20 ± 0.45
43,171 ± 4031,353 ± 1782.901.18 ± 0.144.00 ± 0.71
82,090 ± 170795 ± 501.701.51 ± 0.126.17 ± 0.75
161,798 ± 15342 ± 32.322.64 ± 0.005.00 ± 0.00
Table 1: Summary of runtimes from six replicate Llama-2 70B fine-tuning workloads.
Figure 6: Stable Diffusion training power profiles. Replicate experiments using the same node count are shown as superimposed transparent curves with different colors. Each power profile aggregates power consumption across all CPU and GPU devices utilized by the workload.
Figure 7: Stable Diffusion training energy metrics as a function of number of nodes. Replicate experiments are shown as red dots, vertically distributed with a box-and-whisker plot for each node value. Fitted curves are also presented as dashed lines.
NodesMean Total Runtime (sec)Mean Epoch Runtime (sec)Mean Epoch SpeedupNumber of EpochsNumber of Evaluations
124,632 ± 12062,669 ± 3060.000.39 ± 0.005.00 ± 0.00
212,480 ± 1,57631,751 ± 521.970.39 ± 0.055.00 ± 0.63
45,784 ± 79212,262 ± 1,6792.590.47 ± 0.003.00 ± 0.00
83,202 ± 4215,091 ± 6692.410.63 ± 0.002.00 ± 0.00
161,432 ± 1492,277 ± 2382.240.63 ± 0.001.00 ± 0.00
Table 2: Summary of runtimes from six replicate Stable Diffusion training workloads.
Figure 8: Llama-3 70B offline inference power profiles as a function of number of prompts and output token length. Median values are shown as lines, and 10-90 th percentiles as shaded regions.
Figure 9: Llama-3 70B offline inference energy metrics as a function of number of prompts and output tokens. Median values across experiments are shown as dots, and 10-90 th percentiles as a shaded regions.
Figure 10: Power series of Llama-3 70B online inference benchmark as a function of dataset, request rate and output token length, assuming a constant batch size of 1,000 prompts. Median values are shown as lines, and 10-90 th percentiles as shaded regions.
Figure 11: Llama-3 70B offline inference performance metrics as a function of dataset, request-rate (in log-scale) and output token length. Median values are shown as lines, and 10-90 th percentiles as shaded regions.
Figure 12: Llama-3 70B online inference energy metrics as a function of dataset, request-rate (in log-scale) and output token length. Median values are shown as lines, and 10-90 th percentiles as shaded regions.
Figure 13: Power series of Llama-3 70B online inference benchmark as a function of dataset, request rate and output token length, assuming a sustained request rate for 3 minutes.
 Average Utilization
 20%40%60%80%
Power
Mean (MW)2.383.564.745.91
Std Dev (MW)0.851.591.911.78
Median (MW)2.223.274.697.00
90th Percentile (MW)3.536.407.087.12
Table 3: Colocation data center aggregate metrics across the whole year for different target utilization levels.
Figure 14: Facility-level power consumption profile distributions for a 10 MW colocation data center at varying utilization levels. 14(a) : by time of the day; 14(b) : by day of the week. Distributions were generated from year-long simulations at 1-minute timesteps. The solid lines represent the median value, and the shaded areas different percentiles.
 Average Utilization
 20%40%60%80%
Power
Mean (MW)0.260.390.530.66
Std Dev (MW)0.060.110.150.13
Median (MW)0.240.360.490.68
90th Percentile (MW)0.350.580.800.80
Table 4: Inference data center aggregate metrics across the whole year for different target utilization levels.
Figure 15: Facility-level power consumption profile distributions for a 1 MW inference data center at varying utilization levels. 15(a) : by time of the day; 15(b) : by day of the week. Distributions were generated from year-long simulations at 1-minute timesteps. The solid lines represent the median value, and the shaded areas different percentiles.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 16: GPU power when stressed using gpu-burn [ 51 ] , median value, and 99% confidence bound (z score: 2.58). Mean value of 668.2 W ( ±\pm 1.4 W std) after warm up of 150 seconds.
Figure 17: CPU power for a dense matrix multiplication kernel, median value, and 99% confidence bound (z score: 2.58). Mean value of 338.6 W ( ±\pm 1.0 W std) after warm up of 11 seconds.
Figure 18: CPU (left) and GPU (right) idle power on a Kestrel GPU node. We show the linear regression for the CPU data.
Figure 19: Power consumption of a single node, four device HPL-NVIDIA run with two matrix solves.

Explore similar work

CardsList
Jun 23, 2026

Power-Flexible AI Data Centers: A New Paradigm for Grid-Responsive Compute

The rapid expansion of artificial intelligence (AI) infrastructure is driving unprecedented growth in electricity demand from data centers. Traditional power-system planning treats large computing facilities as inflexible peak loads, leading to costly infrastructure upgrades and long delays in grid interconnection. Recent work has shown that AI clusters can reduce electricity consumption during peak demand through software-based workload orchestration. This article explores how modern GPU-based AI data centers can operate as grid-interactive assets that respond dynamically to power system conditions. We describe an architecture integrating grid signals, workload scheduling, and power telemetry for fine-grained cluster power control. Experimental results from a real-world deployment on a 130 kW GPU cluster demonstrate multiple forms of flexibility, including rapid load reduction, sustained curtailment, and carbon-aware operation while preserving service levels for priority jobs. We further demonstrate performance-aware load shifting across geographically distributed clusters, enabling workloads to migrate toward regions with lower grid stress. Together, these capabilities transform AI infrastructure from static electricity consumers into flexible resources that support grid reliability, accelerate interconnection, and improve computing sustainability.

Chris Williams, Philip Colangelo, Ayse Coskun +15

Emerald AI · Electric Power Research Institute (EPRI) · National Grid +2

Demand ResponseCarbon-Aware ComputingPower SystemsData Center Energy Management
AskPDF
Mar 13, 2026

Concentrated siting of AI data centers drives regional power-system stress under rising global compute demand

The rapid rise of generative artificial intelligence (AI) is driving unprecedented growth in global computational demand, placing increasing pressure on electricity systems. This study introduces an AI-energy coupling framework that combines large language models (LLMs)-based analysis of corporate, policy, and media data with quantitative energy-system modeling to forecast the electricity footprint of AI-driven data centers from 2025 to 2030. Results show that the new AI infrastructure is highly concentrated in North America, Western Europe, and the Asia-Pacific, which together account for more than 90% of projected compute capacity. Aggregate electricity consumption by the six leading firms is projected to increase from roughly 118 TWh in 2024 to between 239 TWh and 295 TWh by 2030, equivalent to about 1% of global power demand. Regions such as Oregon, Virginia, and Ireland may experience high Power Stress Index (PSI) values exceeding 0.25, indicating local grid vulnerability, whereas diversified systems such as those in Texas and Japan can absorb new loads more effectively. These findings demonstrate that AI infrastructure is evolving from a marginal digital service into a structural component of power-system dynamics, underscoring the need for anticipatory planning that aligns computational growth with renewable expansion and grid resilience.

Danbo Chen, Zijun Zhou, Yongyang Cai +3

The Ohio State University, Columbus, OH 43210, USA. · Meta Platforms Inc., Menlo Park, CA 94025, USA. · Zhejiang A&F University, Hangzhou, Zhejiang 311300, China.

Power SystemsElectricity Load Forecasting
AskPDF
May 15, 2026

Designing Datacenter Power Delivery Hierarchies for the AI Era

Demand for AI accelerators is rapidly increasing rack power density, with projections approaching 1MW per deployment by 2027. This poses a major challenge for datacenter power delivery designers. As power densities increase, a datacenter designed for a different target density may strand power, i.e., may be unable to use all the power that its delivery hierarchy has provisioned. Designs must remain efficient over long datacenter lifetimes and multiple hardware generations. Power utilization is particularly important as grid power capacity is a scarce resource in the AI era. Designing an efficient power delivery hierarchy for the long run is difficult because rack placement feasibility, workload impact, and cost depend jointly on electrical topology, deployment granularity, placement policy, power oversubscription, and workload mix. Moreover, each of these factors evolve over time, have inter-dependencies across multiple resource dimensions, and generally do not lend themselves to closed-form analysis. To address this challenge, we develop a framework for evaluating datacenter power delivery designs using throughput, power, and cost metrics over realistic arrival, oversubscription, and decommissioning sequences. The framework combines projection models for GPU, compute, and storage deployments with operational factors grounded in production data from Microsoft Azure. Our results show that multi-resource stranding materially changes deployable capacity, effective capital expenditure, and delivered performance, and quantify how rising density from rack- and pod-scale AI systems shapes these outcomes. For AI datacenter design, the relevant planning objective is not installed megawatts, but deployable capacity over time.

Grant Wilkins, Fiodar Kazhamiaka, Alok Gautam Kumbhare +2

Stanford University Stanford, CA, USA · Microsoft Azure Research Redmond, WA, USA

Power Systems
AskPDF