This Is Taking Too Long -- Investigating Time as a Proxy for Energy Consumption of LLMs

Lars Krupp; Daniel Geißler; Francisco M. Calatrava-Nicolas; Vishal Banwari; Paul Lukowicz; Jakob Karolus

cs.PFMar 16, 2026

This Is Taking Too Long -- Investigating Time as a Proxy for Energy Consumption of LLMs

Authors: Lars Krupp, Daniel Geißler, Francisco M. Calatrava-Nicolas, Vishal Banwari, Paul Lukowicz, Jakob Karolus

Organizations: Embedded Intelligence DFKI and RPTU Kaiserslautern, Germany · Centre for Applied Autonomous Sensor Systems (AASS) Örebro University Örebro, Sweden

Abstract

The energy consumption of Large Language Models (LLMs) is raising growing concerns due to their adverse effects on environmental stability and resource use. Yet, these energy costs remain largely opaque to users, especially when models are accessed through an API -- a black box in which all information depends on what providers choose to disclose. In this work, we investigate inference time measurements as a proxy to approximate the associated energy costs of API-based LLMs. We ground our approach by comparing our estimations with actual energy measurements from locally hosted equivalents. Our results show that time measurements allow us to infer GPU models for API-based LLMs, grounding our energy cost estimations. Our work aims to create means for understanding the associated energy costs of API-based LLMs, especially for end users.

Figures & tables

GPU	VRAM	Thermal Design Power (TDP) [W]	Optimal TDP Load Range [W]
A100-40GB	40 GB	400	340–380
A100-80GB	80 GB	400	340–380
A100-PCI	40 GB	250	213–238
H100	80 GB	700	595–665
H100-PCI	94 GB	400	340–380
H200	140 GB	700	595–665

TABLE I: TDP and Optimal Work Load Range

Model	Type	GPU	mean $\pm$ sd
Mistral-7B	Local	A100-40GB	2447. $\pm$ 8.85
		A100-80GB	2308. $\pm$ 8.36
		A100-PCI	2525. $\pm$ 14.7
		H100	1662. $\pm$ 22.5
		H100-PCI	1820. $\pm$ 9.24
		H200	1559. $\pm$ 4.72

TABLE II: Table depicting mean and standard deviation of time (s) to compute the benchmark. GPUs are ordered chronologically by release date.

Fig. 1: Boxplot showing the computation time per token

\bar{T}_{token}

in seconds for running the same benchmark on both models across different local GPUs and API executions. Cluster A includes A100 GPUs, while Cluster H groups H100 and H200 GPUs.

Model	GPU	Total tokens	TDP (calculated)		Carbon Tracker (measured)		Difference (% of local) mean [%]
Model	GPU	Total tokens	Per token (mWh)	Total (Wh)	Per token (mWh)	Total (Wh)	Difference (% of local) mean [%]
Mistral-7B	A100-40GB	286219	0.86 $\pm$ 0.003	245. $\pm$ 0.885	1.06 $\pm$ 0.023	303. $\pm$ 6.54	19
	A100-80GB	286219	0.81 $\pm$ 0.003	231. $\pm$ 0.836	1.10 $\pm$ 0.024	314. $\pm$ 6.75	26
	A100-PCI	286219	0.55 $\pm$ 0.003	158. $\pm$ 0.921	0.949 $\pm$ 0.007	272. $\pm$ 1.95	42
	H100	327674	0.88 $\pm$ 0.012	291. $\pm$ 3.94	1.07 $\pm$ 0.016	350. $\pm$ 5.37	16
	H100-PCI	327674	0.56 $\pm$ 0.003	182. $\pm$ 0.924	0.904 $\pm$ 0.009	296. $\pm$ 2.89	39

TABLE III: Average sustained power and energy use estimated via TDP vs. locally measured (Carbon Tracker). Each super-level group per token and total energy.

Model	API cost		Number of Tokens	H100-PCI Total energy (Wh)
Mistral-7B	free	M	111272. $\pm$ 6351.	100.60 $\pm$ 6.00
Mistral-7B	free	C	111272. $\pm$ 6351.	62.31 $\pm$ 3.56
Mistral-7B	paid	M	110882. $\pm$ 5922.	100.23 $\pm$ 5.60
Mistral-7B	paid	C	110882. $\pm$ 5922.	62.09 $\pm$ 3.32
Mistral-NeMo	free	M	173743 $\pm$ 4967.	203.28 $\pm$ 5.81
Mistral-NeMo	free	C	173743 $\pm$ 4967.	123.36 $\pm$ 3.53

TABLE IV: Comparison between measured and calculated total energy costs for the API version (free and paid) of Mistral-7B and Mistral-NeMo.

Energy-Aware LLM Inference

See all 25 papers

The Language-Energy Divide: Measuring Energy Costs of Multilingual LLM Inference
Jun 20, 2026Naihao Deng, Alissa Shen, Yiming Feng +5LLM Inference Efficiency Multilingual Language Models
University of Michigan · University of Cambridge · University of Aberdeen
From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs
Jul 29, 2026Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis +5LLM Inference Efficiency Energy-Efficient ML
1National Technical University of Athens, Patission Complex 42, 10682 Athens, Greece · 2Harvard University, 1350 Massachusetts Avenue, 02138 Cambridge, MA, USA · 3TWT GmbH Science & Innovation, Industriestraße 6, 70565 Stuttgart, DE +4
Measuring the Inference Energy Efficiency of Open-Weight LLMs across Benchmarks, Reasoning Modes, and Hardware
Oct 9, 2026Zeyu Yang, Wes ArmourEnergy-Aware LLM Inference LLM Inference Efficiency
Department of Engineering Science, University of Oxford

Jun 20, 2026J/K move · Enter open · S save

The Language-Energy Divide: Measuring Energy Costs of Multilingual LLM Inference

Naihao Deng, Alissa Shen, Yiming Feng, Joan Nwatu+4

University of Michigan · University of Cambridge · University of Aberdeen

LLM Inference Efficiency Multilingual Language Models Multilingual Language Model Evaluation Low-Resource Language Processing

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly deployed in multilingual settings, yet the energy costs of serving these models across different languages remain poorly understood. We present a systematic study of inference energy consumption across languages with ML.Energy framework (Chung et al., 2026). We find striking disparities: energy consumption per output token varies by up to 8.3 times across languages, while total energy for a fixed set of requests varies by up to 179 times between the cheapest (English, 17.6 kJ) and the most expensive (Pashto, 3,147 kJ) languages. Our analysis shows that this disparity is driven by two compounding factors: (1) higher per-token energy costs for languages using complex or rare scripts, and (2) more tokens generated for low-resource languages. Moreover, we find a double cost + performance penalty: languages with the highest energy footprints also tend to achieve the lowest task accuracy. We reveal that the energy divide persists across models, hardware, and tasks, suggesting a systemic energy inequity in multilingual LLM deployment. Finally, we recommend that the community treat energy as a first-class evaluation axis, extend reporting checklists and model cards to include it, and adopt deployment-side mitigations for better energy efficiency.

Abstract

Energy-Aware LLM Inference

Explore similar work

The Language-Energy Divide: Measuring Energy Costs of Multilingual LLM Inference

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

Measuring the Inference Energy Efficiency of Open-Weight LLMs across Benchmarks, Reasoning Modes, and Hardware