Greenpixie's AI Token Methodology: Assessing the Energy, Water and CO2-eq Impact of AI Tokens for Open and Closed Weight Models

Joshua Horswill; Ross Hunter; Matt Clifford; James Hall

cs.CYSep 27, 2026

Greenpixie's AI Token Methodology: Assessing the Energy, Water and CO2-eq Impact of AI Tokens for Open and Closed Weight Models

Authors: Joshua Horswill, Ross Hunter, Matt Clifford, James Hall

Organizations: Greenpixie Ltd., London, UK.

Abstract

We describe a methodology for estimating the per-token energy cost of cloud-hosted large language model (LLM) inference, separating between input (prefill) and output (decode) tokens. Graphics processing unit (GPU) energy usage is measured during inference benchmarking with open-weights models on a wide range of text-based tasks. The remaining server energy contribution from non-GPU hardware is estimated from the inference wall time. Bayesian linear regression is used to model the relationship between energy per token and LLM size, request traffic, and hardware deployment configuration. Proprietary frontier LLMs of unknown size and deployment are binned into size buckets based on naming conventions and performance priors, and the space of possible LLM configurations is sampled with Monte-Carlo methods to give a representative average energy per token and uncertainty. We also describe how these energy measurements can be used to estimate the carbon-dioxide equivalent ( $\mathrm{CO_2\text{-}eq}$ ) emissions, both usage and embodied, and water consumed per token of AI inference. This methodology provides actionable data that enables reductions in cost, electricity usage, $\mathrm{CO_2\text{-}eq}$ emitted and water consumed in cloud and Software as a Service (SaaS).

Figures & tables

Setting	Value	Role
max_num_batched_tokens	65,536	Cap on the total number of tokens in any single batched forward pass.
enable_chunked_prefill	True	Splits long prefills into chunks so they dynamically interleave with active decodes, matching production serving.
max_tokens	8,192	Maximum tokens generated per request.
gpu_memory_utilization	0.90	Fraction of GPU memory allocated to the KV cache + weights.
distributed_executor_backend	"mp"	Multi-process executor (rather than Ray).
pipeline_parallel_size	1	No pipeline parallelism; tensor parallelism set to detected GPU count.

Table 1 : vLLM engine settings held constant across the campaign.

GPU count	vCPU	Memory (GiB)	p5 power ( $\mathrm{W}$ )	p6-b200 power ( $\mathrm{W}$ )
1	24	256	35.14	60.81
2	48	512	70.28	121.62
4	96	1024	140.56	243.23
8	192	2048	281.13	486.46
16	384	4096	562.25	972.93
32	768	8192	1124.50	1945.85

Table 2 : Assumed AI server specification used to estimate the non-GPU server contribution.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Provider	$N_{\mathrm{active}}$ (B)	$N_{\mathrm{total}}$ (B)
Qwen3-1.7B-FP8	Alibaba	1.7	1.7
Qwen3-4B-Instruct-2507-FP8	Alibaba	4.0	4.0
DeepSeek-R1-Distill-Qwen-7B-FP8	DeepSeek	7.0	7.0
Mistral-7B-Instruct-v0.3-FP8	Mistral AI	7.0	7.0
OpenHermes-2.5-Mistral-7B-FP8-Dynamic	Nous Research	7.0	7.0
Llama-3-8B-Instruct-FP8	Meta	8.0	8.0

Appendix

Table 3 : Benchmarked open-weights models on H100, FP8. Provider, active and total parameter counts are listed in ascending

N_{\mathrm{total}}

order.

Model	Provider	$N_{\mathrm{active}}$ (B)	$N_{\mathrm{total}}$ (B)
DBRX-Instruct-FP8-KV	Databricks	36.0	132.0
Mixtral-8x22B-Instruct-FP8	Mistral AI	39.0	141.0
GLM-4.7-FP8	Z.ai	32.0	358.0
Hermes-4-405B-FP8	Nous Research	406.0	406.0
Qwen3-Coder-480B-A35B-Instruct-FP8	Alibaba	35.0	480.0
Ling-1T-FP8	Ling-1t.ai	50.0	1000.0

Appendix

Table 4 : Benchmarked open-weights models on B200, FP8.

Model	Provider	$N_{\mathrm{active}}$ (B)	$N_{\mathrm{total}}$ (B)
Phi-4-reasoning-plus-NVFP4	Microsoft	14.0	14.0
Qwen3-30B-A3B-NVFP4	Alibaba	3.3	30.0
Qwen3-235B-A22B-NVFP4	Alibaba	22.0	235.0
DeepSeek-V3-0324-NVFP4	DeepSeek	37.0	671.0
Kimi-K2-Thinking-NVFP4	Kimi AI	32.0	1000.0

Appendix

Table 5 : Benchmarked open-weights models on B200, NVFP4.

Explore similar work

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs
Jul 29, 2026Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis +5LLM Inference Optimization Energy Consumption
1National Technical University of Athens, Patission Complex 42, 10682 Athens, Greece · 2Harvard University, 1350 Massachusetts Avenue, 02138 Cambridge, MA, USA · 3TWT GmbH Science & Innovation, Industriestraße 6, 70565 Stuttgart, DE +4
Token Arena: A Continuous Benchmark Unifying Energy and Cognition in AI Inference
May 1, 2026Yuxuan Gao, Megan Wang, Yi Ling YuArtificial Intelligence Benchmarks Artificial Intelligence Systems
University of Pennsylvania · OpenMesh AI · Columbia University
How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks
Apr 24, 2026Longju Bai, Zhemin Huang, Xingyao Wang +5Coding Agents Artificial Intelligence Agents
University of Michigan · Stanford University · Microsoft AI +3

Jul 29, 2026, cs.LGJ/K move · Enter open · S save

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis, George Dasoulas+4

1National Technical University of Athens, Patission Complex 42, 10682 Athens, Greece · 2Harvard University, 1350 Massachusetts Avenue, 02138 Cambridge, MA, USA · 3TWT GmbH Science & Innovation, Industriestraße 6, 70565 Stuttgart, DE+4

LLM Inference Optimization Energy Consumption Graphics Processing Unit Architectures Floating-Point

Open paper AskPDF arXiv

The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting. This report presents an analytically structured, empirically calibrated, GPU-level methodology for estimating LLM inference energy on NVIDIA H100-class accelerators without direct runtime measurement. The proposed estimator combines parameter-scaled transformer FLOP accounting, calibrated memory-traffic factors, and hardware-specific energy coefficients for FP16/BF16 tensor-core computation and high-bandwidth-memory movement. It explicitly separates prompt prefill from autoregressive decoding, enabling energy estimates for input tokens, output tokens, and complete inference requests. The methodology further decomposes total energy into compute, parameter-access, key-value-cache write, and attention-read components, allowing the scaling behavior with model size, context length, and generated-token count to be analyzed. The resulting estimates are not intended to replace physical power measurements; rather, they provide transparent, reproducible, and assumption-explicit approximations suitable for model comparison, green-coding analysis, and design-time evaluation of LLM inference workloads.

Abstract

Explore similar work

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

Token Arena: A Continuous Benchmark Unifying Energy and Cognition in AI Inference

How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks