Organizations: Electrical and Computer Engineering, Boston University, Boston, MA · Application Performance Group (NERSC), Lawrence Berkeley National Laboratory, Berkeley, CA · Advanced Technologies Group (NERSC), Lawrence Berkeley National Laboratory, Berkeley, CA · Data & AI Sciences Group (NERSC), Lawrence Berkeley National Laboratory, Berkeley, CA
Efficient utilization of GPU resources and power has become critical with the growing demand for GPUs in high-performance computing (HPC). In this paper, we analyze GPU utilization and GPU memory utilization, as well as the power consumption of the Vienna ab initio Simulation Package (VASP), using the Slurm workload manager historical logs and GPU performance metrics collected by NVIDIA's Data Center GPU Manager (DCGM). VASP is a widely used materials science application on Perlmutter at NERSC, an HPE Cray EX system based on NVIDIA A100 GPUs. Using our insights from the resource utilization analysis of VASP applications, we propose a resource prediction framework to predict the average GPU power, maximum GPU utilization, and maximum GPU memory utilization values of heterogeneous HPC system applications to enable more efficient scheduling decisions and power-aware system operation. Our prediction framework consists of two stages: 1) using only the Slurm accounting logs as training data and 2) augmenting the training data with historical GPU profiling metrics collected with DCGM. The maximum GPU utilization predictions using only the Slurm submission features achieve up to 97% accuracy. Furthermore, features engineered from GPU-compute and memory activity metrics exhibit good correlations with average power utilization, and our runtime power usage prediction experiments result in up to 92% prediction accuracy. These findings demonstrate the effectiveness of DCGM metrics in capturing application characteristics and highlight their potential for developing predictive models to support dynamic power management in HPC systems.
Figures & tables
Feature
Description
User
Name of the user submitting the job.
JobName
User selected job name.
Account
User’s account to charge for computing.
Category
Scientific category the user belongs to.
ReqCPUs
Number of CPUs requested.
ReqNode
Number of GPU nodes requested.
TABLE I: Selected Slurm job accounting fields.
Metric
Description
gpu_utilization
GPU utilization activity (%).
fb_used
Used frame buffer memory (MB).
fb_free
Free frame buffer memory (MB).
sm_active
The ratio of cycles where an SM has at least 1 active warp.
sm_occupancy
The ratio of the number of warps resident on an SM, relative to the theoretical maximum number of active warps.
dram_active
The ratio of cycles with active device memory; sending or receiving data.
TABLE II: Selected GPU profiling metrics from NVIDIA DCGM.
Fig. 1: Distributions of maximum GPU utilization, maximum memory utilization, and average power for VASP GPU jobs on Perlmutter (March 2025).
Fig. 2: Overview of the proposed two-stage framework for GPU resource and power prediction.
Target Variable
Sym. Acc ( ↑ )
MAE ( ↓ )
R2 ( ↑ )
Max. GPU Utilization
0.97
2.39%
0.59
Max. Memory Utilization
0.88
2.58%
0.70
Average Power
0.94
8.57W
0.84
TABLE III: First-Stage Prediction Performance for VASP Jobs
Fig. 3: Before job execution prediction: Normalized feature importance scores from the LightGBM regression models for each target variable.
Fig. 4: During job execution power prediction: Normalized confusion matrices comparing baseline methods and the proposed framework.
Model
Acc ( ↑ )
F1 Score ( ↑ )
Max Baseline
0.63
0.62
Mean Baseline
0.63
0.62
Proposed Framework
0.82
0.80
TABLE IV: Runtime Power Prediction Performance for VASP Jobs
Fig. 5: Test samples from a 10-minute VASP job execution window, showing runtime predictions of average power consumption class.
Fig. 6: During job execution prediction: Normalized feature importance scores from the LightGBM classifier.
Estimating the energy use of virtual servers is difficult because physical power interfaces and privileged host counters are typically accessible only on the underlying physical host. We investigate a machine learning approach that predicts the CPU-package power attributable to an individual virtual server from system and performance metrics observable within that virtual server itself. During training, host-side RAPL measurements provide the power target used to learn this mapping, while deployed inference requires only measurements available within the virtual server. We evaluate the approach across heterogeneous CPU platforms, multiple workload families, unseen workloads and hardware, concurrent workload combinations, and varying resource allocations using temporally blocked evaluation. Prediction is generally strong when deployment conditions are represented during training, while zero-shot transfer to unseen workloads, hardware, and configurations is less reliable. A pooled multi-platform model retains good performance when target platforms are represented during training, and pooled multi-configuration modeling performs well across different processor and memory allocations. Feature-importance analysis reveals interpretable resource signatures consistent with workload behavior across CPU, memory, storage, and network activity. Monitoring and inference add little direct CPU-power overhead. The results support practical virtual server power estimation when training data cover the intended deployment domain.
Amandip Sangha
The Climate and Environmental Institute NILU , Postboks 100, 2027 Kjeller, Norway
Energy supply and heat dissipation are two of the main challenges with modern GPU deployments. While typically discussed in the context of new datacenter constructions, the same constraints also apply to small form-factor consumer devices, such as the DGX spark. In workloads characterized by alternating compute-intensive tasks such as matmuls with memory-bound operations such as norms or cross-entropy, the compute-intensive parts might hit power and/or thermal limits and start throttling. In this short paper, we show that chunking the workload into smaller parts that alternate compute and memory in higher frequencies, these power and temperature spikes can be smoothed out, preventing throttling and resulting in considerably faster wall-clock time and reduced total energy consumption. We present several scenarios in which this effect can be exploited on a DGX Spark with up to 2% performance and energy improvements, and demonstrate that the same phenomenon also happens on less constrained systems, such as a multi-GPU server, albeit at significantly reduced effect size of 1-2%.
Erik Schultheis, Maximilian Kleinegger, Dan Alistarh
Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption. Optimizing these deployments requires matching specific LLMs to the most efficient GPUs, but operators currently lack the tools to do so without exhaustively profiling each combination. While some predictive models exist, they still require profiling data and struggle to generalize to hardware unseen during training. To address this, we introduce WattGPU, featuring two predictive models for mean GPU power draw and Inter-Token Latency (ITL). Our approach leverages only publicly available LLM metadata and GPU specifications, eliminating the need for hardware access or profiling while enabling generalization to unseen NVIDIA server-grade GPUs and LLMs. We evaluate our models using rigorous leave-one-GPU-out and leave-one-LLM-out cross-validation on a dataset of 42 open-source LLMs (0.1B--27B parameters) and 8 GPUs under both offline and server scenarios. The mean power draw model achieves a median absolute percentage error of ≤3.4% for offline and ≤13.5% for server scenarios on unseen GPUs, while the latency model achieves ≤8.5% in server mode, both maintaining strong GPU ranking correlations for server scenarios (Kendall τ≥0.76). Compared to standard physically grounded baselines -- Load-Scaled Thermal Design Power (TDP) for power draw and roofline for latency -- our models reduce median absolute percentage error by approximately 4× on unseen LLM-GPU combinations for server scenarios or approximately 2× for completely unseen GPUs. WattGPU's data and code are publicly available at https://github.com/maufadel/wattgpu.
Mauricio Fadel Argerich, Jonathan Fürst, Marta Patiño-Martínez
Universidad Politécnica de Madrid, C. de los Ciruelos, 28660 Boadilla del Monte, Madrid, Spain · Zurich University of Applied Sciences, Gertrudstrasse 15, 8400 Winterthur, Switzerland