LLM-PowerTrace: A Dataset of Time-Resolved GPU Power Traces for AI Compute

Jiansheng Wang; Jaskeerat Singh; Haochen Xu; Xinyi Li; Mingyuan Yan; Shaohui Liu; Yuzhuo Li; Yize Chen

	L40S	A100	H100	H200
Architecture	Ada Lovelace	Ampere	Hopper	Hopper
Memory	48 GB GDDR6	80 GB HBM2e	80 GB HBM3	141 GB HBM3e
Memory bandwidth	864 GB/s	2.04 TB/s	3.35 TB/s	4.8 TB/s
BF16 Tensor Core	362 TFLOPS	312 TFLOPS	989 TFLOPS	989 TFLOPS
Max. power limit	350 W	400 W	700 W	700 W
Inter-GPU link	PCIe (no NVLink)	NVLink 600 GB/s	NVLink 900 GB/s	NVLink 900 GB/s

Appendix

Table 4: GPU platforms in LLM-PowerTrace .

Family	Generation	Model sizes (B)
Main fine-tuning collection
Gemma	Gemma	2, 7
	Gemma 2	2, 9, 27
	Gemma 3	0.27, 1, 4, 12, 27
	Gemma 4	E2, E4, 26(4), 31
Llama	Llama 2	7, 13, 70

Appendix

Table 5: Models included in the main fine-tuning collection and additional fine-tuning runs.

Parameter	Default	Parameter	Default
Node	1 node, 4 identical GPUs	Learning rate	$2\times 10^{-5}$
Framework	Transformers + Accelerate	Warmup	100 optimizer steps
Dataset	Alpaca, $\leq$ 52K samples	Gradient clipping	max norm 1.0
Sequence length	1,024 tokens	Attention	eager
Micro-batch size	8 per process/sharded run	LoRA rank/scale/drop.	64 / 16 / 0.05
Grad. accumulation	4 micro-steps	QLoRA quantization	4-bit NF4, double

Appendix

Table 6: Default configuration for the controlled LLM fine-tuning experiments.

Layer	Factor	Primary trace manifestation
Workload	Fine-tuning method	Overall power profile and phase characteristics
	Model scale	Power level and iteration duration
	Model architecture	Waveform morphology and temporal pattern
	Batch size	Micro-step duration and power level
	Sequence length	Forward/backward duration
	Numerical precision	Forward/backward power contrast and duration

Appendix

Table 7: Primary effects of the investigated factors on fine-tuning power traces.

Figure 8: Calibration of the energy scaling law for QLoRA fine-tuning on H200. (a) Optimizer-step energy versus analytical FLOPs, excluding checkpoint intervals. (b–d) Energy per token as a function of batch size, sequence length, and model size. Lines show the fitted relations used to motivate Eq. 20 .

Figure 9: Predicted versus measured energy per token for the held-out experiments in Fig. 5 . The dashed diagonal denotes perfect prediction, and the shaded region marks

\pm 5\%

relative error. Model-size points are subsampled for visualization only; all samples are retained for fitting and evaluation.

Relation	$x$	$y$	Fitted law	$R^{2}$
Compute–energy	$F_{\mathrm{step}}$	$E_{\mathrm{step}}$	$2176.3+4.5241x$	0.992
Batch size	$B_{\mu}$	$e_{\mathrm{tok}}$	$0.7555+1.2552/x$	0.997
Sequence length	$S$	$e_{\mathrm{tok}}$	$0.0770+1.51{\times}10^{-4}x+57.20/x$	1.000
Model size	$N$	$e_{\mathrm{tok}}$	$0.1147+0.0216x$	0.940

Appendix

Table 8: Calibration results for the energy scaling law for QLoRA fine-tuning on H200.

Parameter	Default	Parameter	Default
Node	1 node, 1 GPU (TP 1)	Request pattern	closed loop ( b32 )
Engine	vLLM 0.28.0	Max Batch Size	32
Batching	continuous	Output length	512 tokens (EOS ignored)
Chunked prefill	on	Maximum context	2,048 tokens (closed loop)
Prefix caching	on	Sampling	temperature 0.7, top- $p$ 0.9 ∗
Weights / KV cache	BF16 / BF16	Warm-up	$\sim$ 1 min before the window

Appendix

Table 9: Default configuration for the controlled inference experiments.

Layer	Factor	Primary trace manifestation
Workload	Arrival process	Idle–busy switching; ramp magnitude
	Concurrency limit	Power level until saturation; ramps peak mid-load
	Model scale	Power level and batch-turnover period
	Model architecture	Power level and cross-GPU balance
	Input length	Prefill share and proximity to the power limit
	Output length	Spacing of batch-turnover events

Appendix

Table 10: Primary effects of the investigated factors on inference power traces.

Candidate	Form	$k$	T1	T2	T3	T4	Primary
Step 1, saturation (162 runs). T1: leave one model out. T2: 1–64 $\to$ 128–256 requests.
Power law	$e^{\theta_{H}+\delta\ell_{N}+\gamma\ell_{n}}$	5	0.222	0.256			0.239
Power law, quadratic load	$e^{\theta_{H}+\delta\ell_{N}+\gamma_{1}\ell_{n}+\gamma_{2}\ell_{n}^{2}}$	6	0.216	0.301			0.259
Saturating	$S_{H}\,\sigma(\theta_{H}+\delta\ell_{N}+\gamma\ell_{n})$	8	0.164	0.228			0.196
Saturating, quadratic load	$S_{H}\,\sigma(\theta_{H}+\delta\ell_{N}+\gamma_{1}\ell_{n}+\gamma_{2}\ell_{n}^{2})$	9	0.107	0.114			0.110
Floor plus rise	$S_{H}\,[F+(1-F)\,R]$ , two logistics	12	0.110	0.120			0.115

Appendix

Table 11: Candidate forms and held-out error at each step. Errors are held-out RMS in

\log(\bar{P}-GP_{0,H})

. T1–T4 are the step’s tests, listed in its header row;

a\to b

means fit on

a

and predict

b

. The primary score averages the tests every candidate can be scored on; – marks a test on which a parameter is not identified.

k

is the number of free parameters, and the chosen form is in bold. Here

\ell_{N}=\log(N_{\mathrm{act}}/8)

,

\ell_{n}=\log(n/32)

and

\ell_{f}=\log(f/1980\,\mathrm{MHz})

.

Level	Pairs	No term	Inside $z$	On $S_{H}$	$p$	Term kept
KV cache FP8, all GPUs	10	0.037	0.022	0.017	0.021	$m_{\mathrm{KV}}=0.967$
FP8 weights, H200	4	0.197	0.032	0.082	0.012	$b_{w}=-0.81$
INT4 weights, H200	4	0.232	0.025	0.089	0.015	$b_{w}=-0.92$
FP8 weights, A100	4	0.070	0.089	0.056	0.96	none
INT4 weights, A100	4	0.097	0.074	0.068	0.60	none

Appendix

Table 12: Precision terms from paired runs: held-out RMS (leave one pair out) with no term, a shift inside

z

, or a factor on

S_{H}

.

p

is the paired

t

-test of the better placement against no term.

Symbol	Meaning	A100	H100	H200	H100, locked clock
$P_{0,H}$	idle power per GPU (W)	72.9	126.3	120.9	77.8
$S_{H}$	power scale per GPU (W)	329 [318, 346]	537 [509, 593]	604 [583, 639]	shared with H100
$\theta_{H}$	intercept of $z$	1.99 [1.55, 2.21]	1.32 [0.88, 1.50]	1.47 [1.20, 1.68]	5.53 [2.96, 238]
Shared across all platforms
$\delta$	model size per GPU	0.598 [0.465, 0.666]
$\gamma_{1}$	load, linear	0.749 [0.649, 0.823]

Appendix

Table 13: Fitted values of the inference scaling law (Eq. 5.2 ; inputs in Table 3 ) from the joint fit on 253 runs, with 95% bootstrap intervals in brackets.

P_{0,H}

is measured; the locked-clock host shares

S_{H100}

. TP enters through

\alpha=-\delta

.

b_{w}

shifts

z

(0 on A100) and

m_{\mathrm{KV}}

multiplies

S_{H}

(1 for BF16); both come from paired runs and are held fixed.

\theta

on the locked-clock host is weakly identified: there

\sigma\approx 1

, and

\phi(f)

carries the clock effect.

Set	$n$	Median $\|$ err $\|$ (%)	$\leq 5\%$	Bias (%)
Mean power, fitted runs	253	2.7	68%	$+0.1$
A100 FP8/INT4 weights (no term)	8	4.3	62%	$-2.9$
Free-length generation	9	4.6	78%	$-3.5$
Throttled clock (about 1830 MHz)	3	1.3	67%	$+1.6$
Llama-3.1-70B, closed loop	7	6.0	43%	$+10.4$
Poisson, 2-min segments, per-request log	120	2.3	100%	–

Appendix

Table 14: Accuracy of the frozen law: median absolute error, share within 5%, and bias (positive is overprediction). Only the first row was fitted, and the A100 runs informed Step 5; every other set was held out. “10-s count” is vLLM’s running count; “per-request log” rebuilds

n(t)

from request times on a 50-ms grid.

Workload	Capacity	Power
Fixed length	1.25	2.28
Mooncake	3.15	2.35
Azure	1.68	2.23

Appendix

Table 15: MAPE (%) across the 24 evaluated deployments of each workload under the shared calibration.

Figure	Analysis	Factor / comparison	Content
Fig. 8	Scaling-law calibration	FLOPs, batch size, sequence length, model size	Calibration of the fine-tuning-energy scaling law on H200 with QLoRA.
Fig. 9	Scaling-law validation	Held-out parameter levels	Predicted versus measured energy per token under level-held-out validation.
Fig. 10	Representative traces	MP vs. DP	Per-GPU and aggregate power under layer-wise model and data parallelism.
Fig. 11	Representative traces	Batch size; sequence length	Changes in fine-tuning waveform and power level as workload dimensions increase.
Fig. 12	Representative traces	Model size; dense vs. MoE	Effects of model scale and architecture on cycle duration and power profile.
Fig. 13	Representative traces	Full, LoRA, QLoRA	Power-profile differences across fine-tuning methods.

Appendix

Table 16: Guide to the figures in the appendix.

Figure 10: [Fine-tuning] Parallelism strategy. Comparison of layer-wise model parallelism (MP) and data parallelism (DP) for Llama 2 13B under QLoRA on four H100 GPUs. MP exhibits a staggered and cyclic cross-GPU pattern, with recurring forward- and backward-related peak trains distributed across GPUs. The overlap between adjacent GPU activities produces pronounced surges in aggregate power. In contrast, DP executes replicated computation across GPUs, resulting in closely aligned per-GPU traces with higher per-GPU and aggregate power. The aggregate DP trace therefore largely preserves the individual waveform, while the iteration periods remain comparable between MP and DP.

Figure 11: [Fine-tuning] Batch size and sequence length. Comparison of fine-tuning power traces across batch sizes and sequence lengths for Llama 2 13B under QLoRA on four H100 GPUs. Increasing either workload dimension increases the computation within each fine-tuning iteration, leading to longer execution intervals and generally higher power levels. These comparisons show that workload scale affects both the temporal structure and power level of fine-tuning.

Figure 12: [Fine-tuning] Model size. Comparison of Gemma 3 4B and 27B shows that increasing model size increases the computational demand of each fine-tuning iteration, resulting in longer execution intervals and generally higher power levels. Model architecture. Gemma 4 26B-A4B and Gemma 4 31B have comparable total parameter counts, but the MoE model activates substantially fewer parameters per token. The resulting lower effective computation leads to shorter fine-tuning cycles and lower sustained power, while its temporal pattern is more irregular than that of the dense model.

Figure 13: [Fine-tuning] Fine-tuning method. Comparison of per-GPU and aggregate power of full fine-tuning, LoRA, and QLoRA power traces for Gemma 3 4B on L40S GPUs. This figure illustrates the power profile under different fine-tuning methods. In our measured traces, the checkpoint-saving duration progressively decreases from full fine-tuning to LoRA and QLoRA. But as QLoRA further reduces the memory footprint, the resulting cross-GPU workload redistribution concentrates more computation on a single GPU in our experiments. As a result, the heavily utilized GPU exhibits a higher power level and longer execution intervals.

Figure 14: [Fine-tuning] GPU platform. Comparison of Gemma 3 27B QLoRA fine-tuning power traces across A100, H100, H200, and L40S GPUs. The L40S and A100 traces exhibit weaker temporal regularity, whereas H100 and H200 show more clearly repeated fine-tuning patterns. The fine-tuning intervals are longest on L40S, followed by A100, while H100 and H200 exhibit similar and substantially shorter durations. In contrast, the sustained power level increases progressively from L40S to A100, H100, and H200.

Figure 15: [Fine-tuning] Model size. Complete quantitative comparison across different model sizes for fine-tuning tasks. The panels report iteration duration, energy per iteration, energy per token, power distribution, ramping rate, and spectral entropy. It summarizes how the six fine-tuning metrics vary with active parameter count. Mean iteration duration and energy per iteration increase with model size, with a particularly pronounced increase for the largest models (Figs. 15(a) , 15(b) ). Energy per token exhibits a similar upward trend at larger model sizes, while remaining non-monotonic across intermediate configurations (Fig. 15(c) ). Mean and upper-tail power increase initially before approaching a broad plateau, whereas lower-tail power varies more modestly (Fig. 15(d) ). Ramping rate increases toward medium–large models but remains non-monotonic across configurations, while spectral entropy exhibits no consistent relationship with model size (Figs. 15(e) , 15(f) ).

Figure 16: [Fine-tuning] Batch size. Complete quantitative comparison across different batch sizes. The panels report iteration duration, energy per iteration, energy per token, power distribution, ramping rate, and spectral entropy. It summarizes the effects of batch size on the six fine-tuning metrics. Iteration duration and energy per iteration increase with batch size as more samples are processed per optimizer step (Figs. 16(a) , 16(b) ). In contrast, energy per token generally decreases, indicating that larger batches amortize fixed computational and synchronization overheads over more tokens (Fig. 16(c) ). Mean and 95th-percentile power increase only moderately, while 5th-percentile power rises sharply at small batch sizes before leveling off (Fig. 16(d) ). Ramping rate also increases with batch size, whereas spectral entropy varies non-monotonically and peaks at intermediate batch sizes (Figs. 16(e) , 16(f) ).

Figure 17: [Fine-tuning] Model architecture. Complete quantitative comparison between mixture-of-experts (MoE) and dense model architectures. The panels report iteration duration, energy per iteration, energy per token, power distribution, ramping rate, and spectral entropy. It compares dense and mixture-of-experts (MoE) architectures across the six metrics. Compared with dense models, MoE models have shorter iteration duration and lower energy per iteration and per token (Figs. 17(a) , 17(b) , 17(c) ). Mean and 95th-percentile power are also lower, while 5th-percentile power remains similar (Fig. 17(d) ). MoE models show slightly lower ramping rate and slightly higher spectral entropy, although both exhibit substantial overlap in the error bars (Figs. 17(e) , 17(f) ).

Figure 18: [Fine-tuning] Fine-tuning method. Complete quantitative comparison across different fine-tuning methods. The panels report iteration duration, energy per iteration, energy per token, power distribution, ramping rate, and spectral entropy. It compares full fine-tuning, LoRA, and QLoRA across the six metrics. Full fine-tuning and LoRA have similar mean iteration durations, while QLoRA is substantially slower and more variable (Fig. 18(a) ). The same pattern appears in energy per iteration and per token, with QLoRA showing the highest energy consumption (Figs. 18(b) , 18(c) ). In contrast, power distributions are broadly similar across the three methods, with only modest differences in mean and percentile values (Fig. 18(d) ). Full fine-tuning and LoRA also have similar ramping rates, while QLoRA is lower; spectral entropy varies only slightly across methods (Figs. 18(e) , 18(f) ).

Figure 19: [Fine-tuning] GPU platform. Complete quantitative comparison across different GPU platforms. The panels report iteration duration, energy per iteration, energy per token, power distribution, ramping rate, and spectral entropy. It compares A100, H100, H200, and L40S GPUs across the six metrics. L40S has the longest mean iteration duration, followed by A100, while H100 and H200 complete iterations substantially faster (Fig. 19(a) ). Energy per iteration and per token follow the same broad ordering, with H100 and H200 consuming less energy than A100 and L40S (Figs. 19(b) , 19(c) ). In contrast, H200 and H100 operate at higher mean and upper-tail power, with H200 showing the highest values (Fig. 19(d) ). H200 also has the highest ramping rate, whereas L40S has the lowest (Fig. 19(e) ). Spectral entropy is lower on H100 and H200 than on A100 and L40S (Fig. 19(f) ), indicating more regular power traces.

Figure 20: [Fine-tuning] Sequence length. Quantitative comparison across different sequence lengths. The panels report iteration duration, energy per iteration, energy per token, power distribution, ramping rate, and spectral entropy. It summarizes the effects of sequence length on the six fine-tuning metrics. Iteration duration and energy per iteration increase sharply beyond 256 tokens (Figs. 20(a) , 20(b) ), while energy per token decreases from 16 to 256 tokens before approaching a plateau (Fig. 20(c) ). Mean, 5th-percentile, and 95th-percentile power increase with sequence length, although upper-tail power begins to saturate at the longest sequences (Fig. 20(d) ). Ramping rate shows no clear monotonic trend, while spectral entropy decreases toward 256 tokens before increasing again at longer sequences (Figs. 20(e) , 20(f) ).

Figure	Analysis	Factor / comparison	Content
Fig. 21	Representative traces	Tensor-parallel degree	Per-GPU and node power of Qwen3-14B at TP 1, 2, and 4 on H100.
Fig. 22	Representative traces	GPU platform	Qwen3-14B at TP 4 on A100, H100, and H200.
Fig. 23	Representative traces	Arrival pattern	Steady and bursty BurstGPT replays of Qwen3-32B on two H200s.
Fig. 24	Representative traces	Concurrency; input/output length	Segments across concurrency limits 1–256 (A100) and input/output pairs (H200).
Fig. 25	Representative traces	Model size; dense vs. MoE	Qwen3-8B, 14B, 32B, and 30B-A3B on two H200s at TP 2.
Fig. 26	Representative traces	Weight and KV-cache precision	BF16, FP8, and INT4 weights and an FP8 KV cache for Qwen3-14B on H100.

Appendix

Table 17: Guide to the inference figures in the appendix.

Figure 21: [Inference] Tensor-parallel degree: Qwen3-14B on (a) one, (b) two, and (c) four H100 GPUs. Each row shows per-GPU power (left) and their sum (right) over 60 s taken 120 s into the measurement window. The TP 1 run ran on a host with the GPU power limit lowered to 575 W; TP 2 and TP 4 ran at 700 W.

Figure 22: [Inference] GPU platform: Qwen3-14B at TP 4 on (a) A100, (b) H100, and (c) H200. Each row shows per-GPU power (left) and their sum (right) over 60 s taken 120 s into the measurement window.

Figure 23: [Inference] Representative power traces of Qwen3-32B served on two H200 GPUs with tensor parallelism, replaying (a) a steady and (b) a bursty segment of the BurstGPT trace. Each row shows per-GPU power (left) and their sum (right) over the full measurement window. Model, hardware, and serving settings are identical; only the arrival pattern differs.

GPU	Model	TP1	TP2	TP4
A100	Qwen3-8B	7.4	5.3 †	–
A100	Qwen3-14B	12.9	8.2 †	5.3
A100	Qwen3-32B	26.4	16.2 †	9.7
H100	Qwen3-14B	6.9 ‡	4.2	2.8
H100	Qwen3-32B	14.3 ‡	8.3	5.0
H200	Qwen3-8B	3.3	2.6	–

Appendix

Table 18: Batch-turnover period (s) versus tensor-parallel degree, closed-loop load with 32 requests in flight. † Host with its application clock held at 1155 MHz (maximum 1410 MHz). ‡ Host with the GPU power limit lowered to 575 W. Both are excluded from the scaling-law fit and the pooled comparisons.

Figure 24: [Inference] Workload sweeps: (a) maximum concurrency from 1 to 256 for Qwen3-32B on one A100; (b) input/output length from 128/128 to 2048/2048 tokens for Qwen3-14B on one H200 with 32 requests in flight. Each setting is a separate run; 30 s of each run are shown side by side, separated by grey lines, so the segments are not contiguous in time.

Figure 25: [Inference] Model-size and architecture examples on two H200 GPUs at TP 2: (a) Qwen3-8B, (b) Qwen3-14B, (c) Qwen3-32B (dense), and (d) Qwen3-30B-A3B (MoE). Each row shows per-GPU power (left) and their sum (right) over 60 s taken 120 s into the measurement window. The dense/MoE pair differs in both total and active parameters.

Figure 26: [Inference] Numerical-precision examples for Qwen3-14B on one H100: (a) BF16, (b) FP8 weights, (c) GPTQ-INT4 weights, and (d) FP8 KV cache with BF16 weights, over 60 s taken 120 s into the measurement window. Runs (a)–(c) ran on a host with the GPU power limit lowered to 575 W.

Figure 27: [Inference] Complete quantitative comparison across different model sizes. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It summarizes how the six inference metrics vary with active parameter count (BF16, TP 1, closed loop, pooled over A100, H100, and H200). Request latency and energy per request rise from 5.2 s and 72 J for Qwen3-8B to 18.8 s and 269 J for Qwen3-32B, while the MoE Qwen3-30B-A3B (3B active) is slower and costlier than the dense 8B model (Figs. 27(a) , 27(b) ). Energy per token follows the same order, from 0.14 J for Qwen3-8B to 0.52 J for Qwen3-32B (Fig. 27(c) ). Mean and 95th-percentile power rise only modestly with size, from 493 to 516 W on average (Fig. 27(d) ). The ramping rate falls from 78 to 48 W/s between Qwen3-8B and Qwen3-32B, with the MoE model (76 W/s) close to Qwen3-8B, while spectral entropy is highest for the MoE model (0.74 against 0.58–0.60) (Figs. 27(e) , 27(f) ).

Figure 28: [Inference] Complete quantitative comparison across different model architectures. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It compares the dense Qwen3-32B and the MoE Qwen3-30B-A3B (BF16, TP 1, closed loop, pooled over A100, H100, and H200). The MoE model serves requests faster (11.5 versus 18.8 s) at lower energy per request (147 versus 269 J) and per token (0.29 versus 0.52 J) (Figs. 28(a) , 28(b) , 28(c) ). It also draws less power, 458 versus 516 W on average and 599 versus 674 W at the 95th percentile (Fig. 28(d) ). Its ramping rate is about 1.6 times that of the dense model (76 versus 48 W/s), and its spectral entropy is higher (0.74 versus 0.58) (Figs. 28(e) , 28(f) ).

Figure 29: [Inference] Complete quantitative comparison across different GPU platforms. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It compares A100, H100, and H200 (BF16, TP 1, closed loop, pooled over Qwen3-8B and 14B, the only models with an H100 run at the standard power limit; 6, 2, and 6 runs). Request latency falls from 10.1 s on A100 to 5.5 and 4.2 s on H100 and H200, and energy per request and per token fall in the same order (111, 99, and 83 J; 0.22, 0.19, and 0.16 J) (Figs. 29(a) , 29(b) , 29(c) ). Mean power rises from 353 W on A100 to 565 and 625 W on H100 and H200, each within a narrow band (Fig. 29(d) ). The ramping rate falls from 94 to 83 and 55 W/s, and spectral entropy from 0.79 to 0.53 and 0.41 (Figs. 29(e) , 29(f) ).

Figure 30: [Inference] Complete quantitative comparison across different maximum concurrency levels. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It summarizes the effects of the concurrency limit on the six inference metrics (six dense Qwen3 models from 0.6B to 32B on A100, H100, and H200). Request latency stays near 5–6 s up to 32 concurrent requests and then rises to 21 s at 256, while energy per request falls from 2.0 kJ to 45 J (Figs. 30(a) , 30(b) ). Energy per token falls about 45-fold, from 3.9 to 0.088 J, as more requests share each decode step (Fig. 30(c) ). Mean power rises from 373 to 573 W, and the pooled band spans the three GPUs: its 95th percentile (near 694 W from 64 requests on) reflects the H100 and H200 runs and its 5th percentile (198 to 396 W) the A100 runs, which reach their 400 W limit at high concurrency (Fig. 30(d) ). The ramping rate peaks at 32 requests (55 W/s) and falls once the GPUs saturate, while spectral entropy rises at high concurrency (Figs. 30(e) , 30(f) ).

Figure 31: [Inference] Complete quantitative comparison across different Poisson arrival rates. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It summarizes the effects of the arrival rate on the six inference metrics (Qwen3-8B, 14B, and 32B on H200, free-length generation; Qwen3-32B at TP 2). Request latency rises only slightly with rate, from 3.1 to 4.1 s, while energy per request falls from 1.9 kJ to 109 J (Figs. 31(a) , 31(b) ). Energy per token falls from 4.9 to 0.28 J as idle time is amortized over more requests (Fig. 31(c) ). The 5th-percentile power stays near idle up to 0.5 requests/s (117–128 W on one GPU and 261–270 W for the two-GPU Qwen3-32B node) and rises to 577–638 W on one GPU (1.28 kW for Qwen3-32B) at 8 requests/s, as the GPUs stop draining between requests (Fig. 31(d) ). The ramping rate stays at 410–490 W/s up to 1 request/s and drops to 42–48 W/s from 4 requests/s, while spectral entropy peaks at 1 request/s (Figs. 31(e) , 31(f) ).

Figure 32: [Inference] Complete quantitative comparison across different input/output lengths. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It summarizes the effects of the forced input/output length on the six inference metrics (Qwen3-8B, 14B, and 32B on one H200 with 32 requests in flight). Request latency and energy per request grow mainly with the output length, from 1.7 s and 33 J at 128/128 to 35 s and 753 J at 2048/2048 (Figs. 32(a) , 32(b) ). Energy per output token is highest for long prompts with short outputs (0.89 J at 2048/128) and 0.26–0.37 J otherwise (Fig. 32(c) ). The 2048-token prompts hold power within a few percent of the 700 W limit, with the 5th percentile at 681 W (Fig. 32(d) ). The same workloads have the smallest ramps (20 W/s against 39–56 W/s) and the highest spectral entropy (Figs. 32(e) , 32(f) ).

Figure 33: [Inference] Complete quantitative comparison across different weight precisions. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It compares BF16, FP8, and GPTQ-INT4 weights across the six metrics (five models on A100 and H200; all H100 weight-precision runs ran with a lowered power limit and are excluded). Quantized weights shorten request latency from 11.2 to 8.1 and 7.5 s and cut energy per request and per token by about 30% (Figs. 33(a) , 33(b) , 33(c) ). Mean power falls from 576 to 533 and 505 W, and 95th-percentile power falls more (Fig. 33(d) ). The ramping rate rises from 71 to 88 and 101 W/s, while spectral entropy rises slightly, from 0.59 to 0.65 and 0.64 (Figs. 33(e) , 33(f) ).

Figure 34: [Inference] Complete quantitative comparison across different KV-cache precisions. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It compares BF16 and FP8 (E4M3) KV caches across the six metrics (five models on A100, H100, and H200, BF16 weights). Request latency changes by less than 1%, and energy per request and per token fall by about 2% on average (a median 2.7% and at most 5% across the 13 same-host pairs) (Figs. 34(a) , 34(b) , 34(c) ). Mean power falls slightly, from 574 to 559 W (Fig. 34(d) ). The ramping rate rises from 80 to 89 W/s, less than its run-to-run spread, and spectral entropy falls from 0.60 to 0.58 (Figs. 34(e) , 34(f) ).

Figure 35: [Inference] Complete quantitative comparison across different tensor-parallel degrees. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It summarizes the effects of the tensor-parallel degree on the six inference metrics (five models on A100, H100, and H200; node power and energy summed over the GPUs of the group; Llama-3.1-70B runs only at TP 4; with capped hosts excluded, TP 1 pools A100 and H200 runs, TP 2 pools H100 and H200 runs, and TP 4 pools all three GPUs, so the pooled trend mixes the TP degree with the platform). Request latency falls from 11.1 s at TP 1 to 4.8 s at TP 2 and is 6.4 s at TP 4, whose group adds the A100 and Llama-3.1-70B, while energy per request and per token stay flat from TP 1 to 2 and double at TP 4, partly because the TP 4 group includes Llama-3.1-70B (Figs. 35(a) , 35(b) , 35(c) ). Node power rises from 489 to 1,013 and 1,594 W (Fig. 35(d) ). The ramping rate grows with the degree, from 67 to 119 and 316 W/s, as more GPUs ramp together, while spectral entropy falls from 0.65 to 0.54–0.57 (Figs. 35(e) , 35(f) ).

Figure 36: [Inference] Complete quantitative comparison across different SM clock frequencies. The panels report request latency, energy per request, energy per token, power distribution, ramping rate, and spectral entropy. It summarizes the effects of the locked SM clock on the six inference metrics (Qwen3-8B, Qwen3-30B-A3B, and Llama-3.1-8B on one H100, closed loop; the 1980 MHz setting throttles to a median of 1830 MHz). Request latency falls from 14.4 s at 345 MHz to 5.8 and 5.7 s at 1605 and 1980 MHz, while energy per request and per token are lowest at 1200 MHz (66 J and 0.13 J) and rise again at higher clocks (Figs. 36(a) , 36(b) , 36(c) ). Mean power rises from 184 to 535 W, with a narrow band at every clock (Fig. 36(d) ). The ramping rate grows at every clock step, from 8 to 68 W/s, while spectral entropy falls from 0.48 to 0.43–0.45 (Figs. 36(e) , 36(f) ).

Resource	Train	Infer	Power series	Per-GPU	Controlled	Resolution
Words to Watts ( Samsi et al., 2023 )	✗	✓	✗	✗	✓	aggregate
ML.ENERGY ( Chung et al., 2025 )	✗	✓	✗	✗	✓	aggregate
BurstGPT ( Wang et al., 2025 )	✗	✓	✗	✗	✗	request log
MIT Supercloud ( Samsi et al., 2021 )	✓	✓	✓	✓	✗	100 ms
LLM-PowerTrace (ours)	✓	✓	✓	✓	✓	100 ms

Dimension	Fine-tuning subset	Inference subset
Observation unit	A fine-tuning execution with per-GPU traces and aggregate GPU power.	A serving execution with device telemetry and available request records.
GPU deployment	L40S, A100, H100, H200; four homogeneous GPUs per main-collection run.	A100, H100, H200; TP 1, 2, and 4 in designated configurations.
Models	Gemma, Llama, Qwen, DeepSeek, Mistral/Mixtral; dense and MoE models. Model inventory in Table 5 .	Qwen3 dense models (0.6B–32B), Qwen3-30B-A3B MoE, and Llama-3.1-8B/70B.
Execution stack	Transformers and Accelerate; full fine-tuning, LoRA, QLoRA; DP or layer-wise MP.	vLLM 0.28.0; version 0.21.0 for the concurrency sweep.
Input workload	Alpaca instruction data, up to 52K examples, with common preprocessing in the main collection.	Controlled input/output lengths; chat, code, and math prompts; Poisson arrivals and BurstGPT replay.
Workload sweeps	Micro-batch $\{1,2,4,8,16,32\}$ ; sequence length $\{128,256,512,1024,2048\}$ ; model size and type.	Concurrency $\{1,2,4,\ldots,256\}$ ; input/output lengths 128/128, 512/512, 2048/128, 512/2048, 2048/2048; thinking on/off in free generation.

Type	Input	Enters inference scaling law Eq. 5.2 as
Discrete	platform $H$	$P_{0,H}$ (loaded idle power per GPU, measured), $S_{H}$ (power scale per GPU), $\theta_{H}$ (intercept)
	TP $G$	multiplies the per-GPU terms; divides $N_{\mathrm{act}}$ in $\ell_{N}$
	weight precision	shift $b_{w}$ in $z$ (FP8, INT4; 0 for BF16 and on A100)
	KV-cache precision	factor $m_{\mathrm{KV}}$ on $S_{H}$ (FP8; 1 for BF16)
Linear in $z$	model size	$\ell_{N}=\log(N_{\mathrm{act}}/(8G))$ , $N_{\mathrm{act}}$ in billions
	running requests	$\ell_{n}=\log(n(t)/32)$ , from the serving log

LLM-PowerTrace: A Dataset of Time-Resolved GPU Power Traces for AI Compute

Abstract

Figures & tables

Appendix

LLM Inference

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models

Abstract

LLM Inference

Explore similar work

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models