From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

Tina Vartziotis; Rodopi Kosteli; Elli Vartziotis; George Dasoulas; Michael Keckeisen; Konstantinos Skianis; Sotirios Kotsopoulos; Francesca Dominici

cs.LGJul 29, 2026

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

Authors: Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis, George Dasoulas, Michael Keckeisen, Konstantinos Skianis, Sotirios Kotsopoulos, Francesca Dominici

Organizations: 1National Technical University of Athens, Patission Complex 42, 10682 Athens, Greece · 2Harvard University, 1350 Massachusetts Avenue, 02138 Cambridge, MA, USA · 3TWT GmbH Science & Innovation, Industriestraße 6, 70565 Stuttgart, DE · NIKI Ltd Digital Engineering, 205 National Resistance Street, 45500 Ioannina, Greece · 6National and Kapodistrian University of Athens, Panepistimiou 30, 106 79 Athens, Greece · University of Ioannina, Campus, 451 10 Ioannina, Greece · 7Massachusetts Institute of Technology, 77 Massachusetts Avenue, 02139 Cambridge, MA, USA

Abstract

The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting. This report presents an analytically structured, empirically calibrated, GPU-level methodology for estimating LLM inference energy on NVIDIA H100-class accelerators without direct runtime measurement. The proposed estimator combines parameter-scaled transformer FLOP accounting, calibrated memory-traffic factors, and hardware-specific energy coefficients for FP16/BF16 tensor-core computation and high-bandwidth-memory movement. It explicitly separates prompt prefill from autoregressive decoding, enabling energy estimates for input tokens, output tokens, and complete inference requests. The methodology further decomposes total energy into compute, parameter-access, key-value-cache write, and attention-read components, allowing the scaling behavior with model size, context length, and generated-token count to be analyzed. The resulting estimates are not intended to replace physical power measurements; rather, they provide transparent, reproducible, and assumption-explicit approximations suitable for model comparison, green-coding analysis, and design-time evaluation of LLM inference workloads.

Explore similar work

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs
Jul 2, 2026Mauricio Fadel Argerich, Jonathan Fürst, Marta Patiño-MartínezLLM Inference Efficiency Inference Workloads
EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization
May 14, 2026Zhiye Song, Kyungmi Lee, Eun Kyung Lee +3LLM Inference Efficiency Multi-Gpu Systems
Multi-Level Modeling of Large Language Model Inference Latency and Energy via Hybrid Analytical--Machine-Learning Predictors
Aug 7, 2026Saeid Shokoufa, Mohammad Erfan Sadeghi, Mehdi Kamal +1LLM Inference Efficiency Nvidia Nemotron

Jul 2, 2026 · cs.DCJ/K move · Enter open · S save

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

Mauricio Fadel Argerich, Jonathan Fürst, Marta Patiño-Martínez

Universidad Politécnica de Madrid, C. de los Ciruelos, 28660 Boadilla del Monte, Madrid, Spain · Zurich University of Applied Sciences, Gertrudstrasse 15, 8400 Winterthur, Switzerland

Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption. Optimizing these deployments requires matching specific LLMs to the most efficient GPUs, but operators currently lack the tools to do so without exhaustively profiling each combination. While some predictive models exist, they still require profiling data and struggle to generalize to hardware unseen during training. To address this, we introduce \textit{WattGPU}, featuring two predictive models for mean GPU power draw and Inter-Token Latency (ITL). Our approach leverages only publicly available LLM metadata and GPU specifications, eliminating the need for hardware access or profiling while enabling generalization to unseen NVIDIA server-grade GPUs and LLMs. We evaluate our models using rigorous leave-one-GPU-out and leave-one-LLM-out cross-validation on a dataset of 42 open-source LLMs (0.1B--27B parameters) and 8 GPUs under both offline and server scenarios. The mean power draw model achieves a median absolute percentage error of

\leq3.4\%

for offline and

\leq13.5\%

for server scenarios on unseen GPUs, while the latency model achieves

\leq8.5\%

in server mode, both maintaining strong GPU ranking correlations for server scenarios (Kendall

τ\geq0.76

). Compared to standard physically grounded baselines -- Load-Scaled Thermal Design Power (TDP) for power draw and roofline for latency -- our models reduce median absolute percentage error by approximately 4

\times

on unseen LLM-GPU combinations for server scenarios or approximately 2

\times

for completely unseen GPUs. WattGPU's data and code are publicly available at https://github.com/maufadel/wattgpu.

LLM Inference Efficiency Inference Workloads Multi-Gpu Systems Nvidia Nemotron Per-Token Latency Low-Latency Systems Data Centers

Open paper Ask about it PDF

Abstract

Explore similar work

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization

Multi-Level Modeling of Large Language Model Inference Latency and Energy via Hybrid Analytical--Machine-Learning Predictors