Re-Reading Is Not Free: Token Traces and Cache-Aware Energy Estimation for LLM Agents in Software Engineering
Authors: Tina Vartziotis, Ketevan Peranidze, Rodopi Kosteli, Ezgi Sarikayak, George Dasoulas, Michael Keckeisen, Sotirios Kotsopoulos, Francesca Dominici
Organizations: TWT GmbH Science & Innovation, Stuttgart, Germany · National Technical University of Athens, Athens, Greece · Harvard University, USA · Massachusetts Institute of Technology, Cambridge, MA, USA · NIKI Digital Engineering · Technical University of Munich, Munich, Germany · Siemens Research, Germany
As large language models (LLMs) evolve from conversational assistants into autonomous agents embedded in software development workflows, estimating their environmental footprint requires moving beyond conventional token-based accounting. Existing energy estimators focus on isolated request-response interactions, overlooking the repeated processing of accumulated context, cache reuse, and work delegated to subagents. We analyze nearly one billion tokens collected across 42 sessions and 752 task episodes over ten weeks of real-world AI-assisted software development. Cached-context reads make up 94.8% of all API-reported tokens, while uncached input accounts for only 0.017%; for every generated token, approximately 260 cached tokens are read. Context grows 5.3-fold over a session compared with 1.4-fold within individual tasks. To quantify the energy implications, we extend an existing analytical inference-energy model to account for cached-context processing, yielding a cache-aware hybrid estimator. It estimates approximately 250 kWh for the workload, compared with approximately 10 kWh from the output-based estimator EcoLogits, a 25-fold difference largely driven by cached-context reads. Neither estimator has been validated on long cached contexts, but no modeling assumption we varied closes the gap. Starting each task from a fresh context would lower our estimate by about 60%. We also find that parent-session logs report only a subagent's final internal call, undercounting delegated tokens by a factor of ten; for this workload the correction changes the energy estimate by about 5%, but token-volume studies are fully exposed to it. Our findings expose a substantial gap between how current energy estimators and agentic workloads treat context, highlighting the need for workflow-level energy accounting that captures context accumulation, cache reuse, and delegated execution.
Figures & tables
Fig. 1: Study pipeline. Usage is recorded in the transcripts; API calls and delegated runs are reconstructed from them; energy is modeled per request by two estimators, the hybrid one in three cache scenarios. Tokens are reported for the main-thread trace, energy for the accounted and reconstructed workloads. Shaded: model-based.
Workstream
Transcr.
Turns
Ep.
Tokens
Viralis-Optimalis
9
1,117
188
474,726,571
Green-Code
12
916
209
332,447,525
Cross
9
646
139
85,182,813
Customer-Project
6
596
106
56,681,332
Artemis
7
405
89
40,935,105
Formalis
2
58
21
2,575,698
TABLE I: The ten-week trace by workstream. Transcr.: transcript files, three of which resume earlier sessions (42 sessions after merging). A turn is one API response, not one transcript record (Section II-B ).
Estimator
Tfresh
Tcached
Tout
Typical ( Wtrueh )
EcoLogits (reference)
—
—
D
0.71
Hybrid, cache omitted
P, A
—
D, A
0.66
Hybrid, cache-aware
P, A
A
D, A
11.5
Hybrid, cache-naive
P, A
P, A
D, A
30.1
TABLE II: How each estimator charges the token classes of a request, and the facility energy it assigns to a typical request: the main-thread call closest to the medians of context, fresh input and output (804 fresh, 153,355 cached and 389 output tokens on Sonnet 4.6; central values). P: dense prefill and KV write; D: decode (hybrid: including KV write); A: read by attention; —: not charged. All variants add server energy, PUE and embodied impact (Eqs. ( 5 )–( 6 )).
Class
Tokens
Share
Cache read
940,461,095
94.752%
Cache write
48,302,053
4.866%
Output
3,618,355
0.365%
Uncached input
167,541
0.017%
Total
992,549,044
100%
TABLE III: Token composition of the main-thread trace.
Fig. 2: Mean cache-read tokens per turn by decile of progress, at session scale (sessions of at least 20 turns) and episode scale (episodes of at least 10 turns). Means are pooled over all turns in a decile, so longer sessions contribute more turns. Shaded bands are 95% confidence intervals from a cluster bootstrap over sessions and episodes, respectively ( 2,000 resamples).
Fig. 3: Median total tokens (a) and median output tokens (b) per episode by declared task type, working episodes only, on logarithmic axes; faint bars span the interquartile range. The number of episodes is given in parentheses.
Energy ( kWtrueh )
Carbon ( kg CO 2 eq)
EcoLogits
9.89
3.97
[7.24–13.5]
[2.94–5.35]
Cache omitted
36.8
15.0
[18.5–73.4]
[7.82–29.0]
Cache-aware
248
96.2
[105–585]
[41.1–225]
TABLE IV: Modeled totals for the accounted workload of 3,829 API calls ( 3,723 main-thread, 106 delegated), US grid. Central value: geometric mean of the two active-parameter bounds, given in brackets as low–high. The hybrid rows are the three scenarios of Eq. ( 4 ): cache omitted, cache-aware, and cache-naive (the full context as fresh input).
EcoLogits
Hybrid
Per engineer
Electricity per working day
0.19kWtrueh
4.9kWtrueh
Electricity per engineer-year
43kWtrueh
1,070kWtrueh
share of a US household’s year
0.4%
10%
Ten weeks of emissions, in car travel
16km
390km
Scaled, per year
TABLE V: The workload in familiar terms, central values (hybrid at the low bound: 2.4 times smaller). Engineer-year: 220 working days. Household: 863kWtrueh a month, US 2024 [ 23 ] . Car: 400g tailpipe CO 2 per mile [ 24 ] . Chatbot prompt: median 0.24Wtrueh0.31Wtrueh [ 25 , 26 ] . Median request: median per-call energy ( 0.91 and 18.5Wtrueh ), not the typical request of Table II .
Fig. 4: Hybrid IT energy by physical term, low active-parameter bound. The dashed line marks the total EcoLogits estimate for the same trace on the same basis (IT energy, low bound, 6.6kWtrueh ).
Factor
Setting
Hybrid total
Ratio
(a) Changes the size of the gap
Cached context
omitted → naive
18.5 → 232 kWh
2.6 → 32
Attention reads
removed → modeled
10.1 → 105 kWh
1.4 → 14.5
Model tier
Haiku → Opus
16.9 → 172 kWh
73 → 14
KV dimension
dkv→dmodel
105 → 700 kWh
14.5 → 97
Active parameters
low → high
105 → 585 kWh
14.5 → 43
TABLE VI: Every factor examined, varied one at a time from the baseline at the low parameter bound ( 105.0kWtrueh , 41.1kg CO 2 eq, hybrid/EcoLogits ratio 14.5 , against 25 at the central value). Each row changes one assumption from the first setting to the second and gives, in the same order, the hybrid total and the hybrid/EcoLogits ratio at both settings; the two estimators would agree at a ratio of 1. Group (a) changes the size of the gap, group (b) only the absolute values; within each group, factors are ranked by effect. PUE 1.56: average reported in the Uptime Institute 2024 survey [ 27 ] ; —: not computed.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Per transcript
Per distinct
record
response
Units counted
7,706
3,738
Uncached input
556,594
167,541
Output
9,350,343
3,618,355
Cache write
110,574,136
48,302,053
Cache read
1,982,069,379
940,461,095
Appendix
TABLE VII: Usage counted once per transcript record and once per distinct response. Tool and delegation counts are read from content blocks, which do not repeat, so they are identical in both columns—the check that the collapse deduplicates usage and nothing else. The 3,738 responses include 15 synthetic records with zero usage.
TABLE IX: Constants of Layer 1 [ 11 ] . The energy coefficients are the total (control plus datapath) values measured on the Hopper GPU of an NVIDIA GH200 (Table 3 of [ 17 ] ): HBM energy per bit for emem and FP16 matrix-unit energy per FLOP for αTC ; the memory factors were fitted to the measurements of [ 18 ] .
Term
Original model
Layer 1
Dense prefill compute
KNTprompt
KNTfresh
Prefill attention compute
2nlayersdmodelTprompt2
2nlayersdmodelTfreshTctx
Dense decode compute
KNTout
KNTout
Decode attention compute
4nlayersdmodelA(Tprompt)
4nlayersdmodelA(Tctx)
Parameter traffic
bwNγ(N) , N all parameters
bwNγ(N) , N active parameters
Key–value writes
bKVorigTout
bKV(Tfresh+Tout)
Appendix
TABLE X: Terms of the original model [ 11 ] and of Layer 1, with bKVorig=2nlayersdmodelb and bKV=2nlayersdkvb . For Tcached=0 , dkv=dmodel and without the key–value writes of prefill, both columns coincide.
Symbol
Value
Meaning
αL
6.785×10−4s
latency per token and B param.
βL
3.119×10−4s
latency per token and batch slot
γL
1.947×10−2s
latency per token, offset
Pserver
1.2kW
non-GPU server power
nsrv
8
GPUs per server
tlife
3 years
hardware lifetime
Appendix
TABLE XI: Constants of Layer 2 and of the EcoLogits GPU term, from the EcoLogits v0.11.1 source [ 10 ] , together with the defaults of this study (Section II-D ).
Model
dkv
dmodel
Measured / predicted
with dkv
with dmodel
Llama 3.3 70B
1024
8192
3.0
0.37
gpt-oss-20b
512
2880
0.69
0.12
Qwen3-8B
1024
4096
0.97
0.24
Appendix
TABLE XII: Measured context slope of decode time divided by the slope predicted from the key–value bytes read per context token (1 = exact), for the two readings of the key–value width, settings as in the text; for gpt-oss-20b only the 12 full-attention layers grow with the context.
Fig. 5: Decode time per output token against context length for three open-weight models with at most 256 output tokens. (a) Medians over three repetitions, with linear fits; the medians of uncached and repeated runs coincide within 2% . (b) Increase over the fitted zero-context time; solid: measured fit, dashed: prediction from the ratio of key–value bytes per context token to weight bytes per decoding step, taken from each model’s configuration (for gpt-oss-20b only the full-attention layers grow with the context).
Fig. 6: Token composition: (a) share of each token class by workstream; (b) trace-wide absolute totals per class, log scale. Cache reads dominate every workstream.
Task type
n
%
Tok. k
Out. k
Turns
Feature
270
36.2
314
1.5
2
Understand
154
20.6
182
0.9
1
Integration
82
11.0
490
2.2
1
Docs
53
7.1
439
2.0
3
Bugfix
42
5.6
269
1.5
3
Other
39
5.2
166
0.9
1
Appendix
TABLE XIII: The 746 working episodes by declared type. Medians are per episode; k denotes thousands of tokens.
Fig. 7: Median tokens per episode by task type and primary model, as routed during the trace.
Workstream
Calls
Tokens
Energy ( Wtrueh )
Ratio
EcoLogits
Hybrid
Viralis-Optimalis
1,147
477,411,480
4,034.8
125,697.3
31.2 ×
Green-Code
928
333,632,151
2,078.4
78,976.4
38.0 ×
Cross
662
85,889,425
1,941.7
26,138.0
13.5 ×
Customer-Project
607
57,208,752
1,003.6
9,032.5
9.0 ×
Artemis
427
41,801,671
787.5
7,637.3
9.7 ×
Appendix
TABLE XIV: Per workstream, central values. The ratio column is hybrid energy divided by EcoLogits energy. Calls and tokens include the 106 delegated subagent calls, with their reported usage, and exclude the 15 synthetic records, so they exceed the turn and token counts of Table I ; for Green-Code, for example, the 916 turns less 3 synthetic records plus 15 delegations give 928 calls.
Fig. 8: Energy (a) and carbon (b) per workstream under both estimators, log scale, central values. Workstreams are ordered by hybrid energy. On the log axis the horizontal gap between each pair of bars is the hybrid-to-EcoLogits ratio of Table XIV .
Term
Wtrueh
Share
Memory: attention reads
79,040.6
90.36%
Compute: prefill (fresh)
5,050.6
5.77%
Non-GPU server
2,855.9
3.26%
Compute: decode
523.8
0.60%
Memory: parameter loads
4.8
0.01%
Memory: KV writes
1.0
<0.01\text{,}\mathrm{%}$$
Appendix
TABLE XV: Hybrid energy by term, low active-parameter bound. Facility energy is IT energy × PUE.
Fig. 9: Per-call facility energy against output tokens, both estimators at the low active-parameter bound, faceted by model tier, log–log. Lines are binned medians (dashed: EcoLogits). At fixed output count (5th–95th percentile, median over ten output bins) the EcoLogits points span a factor of 1.4 , whereas the hybrid estimate spans 14.3 (Sonnet) and 8.4 (Opus). Haiku is omitted ( 6 calls).
Fig. 10: Per-call hybrid facility energy at the low active-parameter bound against fresh prefill tokens (a) and full context (b), log–log, colored by model tier. The line is a binned median, not a fit. Raw uncached input is omitted: median 2 , Spearman ρ=0.34 .
Model
Vendor
Active (B)
kWtrueh
Haiku 4.5
Anthropic
1035
16.9
Sonnet 4.6
Anthropic
44132
92.5
GPT-5.4
OpenAI
45150
96.0
Opus 4.8
Anthropic
67200
172.1
Gemini 2.5 Pro
Google
200600
1042.2
as actually run (mixed)
105.0
Appendix
TABLE XVI: The accounted workload re-estimated on current models from three vendors, tokens held fixed. Energy is facility energy at the low active-parameter bound. The Gemini row lies outside the calibrated parameter range.
Workstream
Haiku
Sonnet
Opus
actual
O/H
Viralis-Optimalis
8.37
45.42
84.46
52.96
10.1 ×
Green-Code
6.10
33.33
62.07
33.65
10.2 ×
Cross
1.53
8.61
16.12
10.99
10.5 ×
Customer-Project
0.43
2.30
4.19
3.96
9.6 ×
Artemis
0.48
2.70
5.04
3.30
10.5 ×
Formalis
0.02
0.11
0.21
0.11
10.3 ×
Appendix
TABLE XVII: The same tokens re-estimated on each Claude tier, facility kWtrueh at the low active-parameter bound. Registry active-parameter ranges: Haiku 1035 B, Sonnet 44132 B, Opus 67200 B.
Fig. 11: Facility energy for one identical request across fifteen models, under a chat-shaped workload (a) and this trace’s median agentic workload (b), log scale, with usage-phase carbon (embodied excluded) on the upper axis. Open-weight models are points because their parameter counts are published; hosted models are bars spanning the registry’s estimated range, labeled low–high. Models are ordered by active parameter count.
Agent type
Calls
Int.
Rep.
Actual
Ratio
Sec.
Edits
Explore
57
891
3.28
37.84
11.5 ×
107
0
general-purpose
38
260
1.64
11.08
6.8 ×
56
41
Plan
11
183
1.05
10.85
10.3 ×
577
0
Total
106
1,334
5.97
59.77
10.0 ×
41
Appendix
TABLE XVIII: The 106 foreground delegations by agent type. Int.: internal API calls in the private transcripts; tokens (millions) as reported in the parent’s tool result and as summed over the internal calls; Sec.: median duration; Edits: file edits.
Fig. 12: Distribution of tokens (a), duration (b) and tool calls (c) per subagent call, by subagent type; (a) and (b) on logarithmic axes. Boxes span the interquartile range, the black line marks the median and whiskers extend to 1.5 times the interquartile range; points are individual calls, and the number of calls is given in parentheses. Six calls (two Plan, four general-purpose) lasted longer than one hour. File edits are given in Table XVIII .
Fig. 13: Share of tool calls by category and subagent type (n = calls). Other: planning, task-list and connector tools. Unlike the episode-level categories of Section II-B , search is counted separately from reading. Only the general-purpose agent edits files.