Re-Reading Is Not Free: Token Traces and Cache-Aware Energy Estimation for LLM Agents in Software Engineering
Organizations: TWT GmbH Science & Innovation, Stuttgart, Germany · National Technical University of Athens, Athens, Greece · Harvard University, USA · Massachusetts Institute of Technology, Cambridge, MA, USA ·
Abstract
As large language models (LLMs) evolve from conversational assistants into autonomous agents embedded in software development workflows, estimating their environmental footprint requires moving beyond conventional token-based accounting. Existing energy estimators focus on isolated request-response interactions, overlooking the repeated processing of accumulated context, cache reuse, and work delegated to subagents. We analyze nearly one billion tokens collected across 42 sessions and 752 task episodes over ten weeks of real-world AI-assisted software development. Cached-context reads make up 94.8% of all API-reported tokens, while uncached input accounts for only 0.017%; for every generated token, approximately 260 cached tokens are read. Context grows 5.3-fold over a session compared with 1.4-fold within individual tasks. To quantify the energy implications, we extend an existing analytical inference-energy model to account for cached-context processing, yielding a cache-aware hybrid estimator. It estimates approximately 250 kWh for the workload, compared with approximately 10 kWh from the output-based estimator EcoLogits, a 25-fold difference largely driven by cached-context reads. Neither estimator has been validated on long cached contexts, but no modeling assumption we varied closes the gap. Starting each task from a fresh context would lower our estimate by about 60%. We also find that parent-session logs report only a subagent's final internal call, undercounting delegated tokens by a factor of ten; for this workload the correction changes the energy estimate by about 5%, but token-volume studies are fully exposed to it. Our findings expose a substantial gap between how current energy estimators and agentic workloads treat context, highlighting the need for workflow-level energy accounting that captures context accumulation, cache reuse, and delegated execution.
Figures & tables
| Workstream | Transcr. | Turns | Ep. | Tokens |
|---|---|---|---|---|
| Viralis-Optimalis | 9 | |||
| Green-Code | 12 | |||
| Cross | 9 | |||
| Customer-Project | 6 | |||
| Artemis | 7 | |||
| Formalis | 2 |
| Estimator | Typical ( ) | |||
|---|---|---|---|---|
| EcoLogits (reference) | — | — | D | 0.71 |
| Hybrid, cache omitted | P, A | — | D, A | 0.66 |
| Hybrid, cache-aware | P, A | A | D, A | 11.5 |
| Hybrid, cache-naive | P, A | P, A | D, A | 30.1 |
| Class | Tokens | Share |
|---|---|---|
| Cache read | ||
| Cache write | ||
| Output | ||
| Uncached input | ||
| Total |
| Energy ( ) | Carbon ( CO 2 eq) | |
| EcoLogits | 9.89 | 3.97 |
| [7.24–13.5] | [2.94–5.35] | |
| Cache omitted | 36.8 | 15.0 |
| [18.5–73.4] | [7.82–29.0] | |
| Cache-aware | 248 | 96.2 |
| [105–585] | [41.1–225] |
| EcoLogits | Hybrid | |
| Per engineer | ||
| Electricity per working day | ||
| Electricity per engineer-year | ||
| share of a US household’s year | ||
| Ten weeks of emissions, in car travel | ||
| Scaled, per year | ||
| Factor | Setting | Hybrid total | Ratio |
|---|---|---|---|
| (a) Changes the size of the gap | |||
| Cached context | omitted naive | 18.5 232 kWh | 2.6 32 |
| Attention reads | removed modeled | 10.1 105 kWh | 1.4 14.5 |
| Model tier | Haiku Opus | 16.9 172 kWh | 73 14 |
| KV dimension | 105 700 kWh | 14.5 97 | |
| Active parameters | low high | 105 585 kWh | 14.5 43 |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Per transcript | Per distinct | |
| record | response | |
| Units counted | ||
| Uncached input | ||
| Output | ||
| Cache write | ||
| Cache read |
| Symbol | Meaning |
|---|---|
| uncached input tokens | |
| all input tokens (original model, no caching) | |
| uncached input plus cache-write tokens | |
| cache-read tokens | |
| context length, | |
| generated output tokens |
| Symbol | Value | Meaning |
|---|---|---|
| 6 | operations per parameter and token | |
| per op. | tensor-core energy | |
| HBM energy per bit | ||
| , | bits | weight and KV element width |
| parameter-access factor | ||
| , , | , , 24B |
| Term | Original model | Layer 1 |
|---|---|---|
| Dense prefill compute | ||
| Prefill attention compute | ||
| Dense decode compute | ||
| Decode attention compute | ||
| Parameter traffic | , all parameters | , active parameters |
| Key–value writes |
| Symbol | Value | Meaning |
| latency per token and B param. | ||
| latency per token and batch slot | ||
| latency per token, offset | ||
| non-GPU server power | ||
| GPUs per server | ||
| years | hardware lifetime |
| Model | Measured / predicted | |||
|---|---|---|---|---|
| with | with | |||
| Llama 3.3 70B | 1024 | 8192 | 3.0 | 0.37 |
| gpt-oss-20b | 512 | 2880 | 0.69 | 0.12 |
| Qwen3-8B | 1024 | 4096 | 0.97 | 0.24 |
| Task type | % | Tok. | Out. | Turns | |
|---|---|---|---|---|---|
| Feature | 270 | 36.2 | 1.5 | 2 | |
| Understand | 154 | 20.6 | 0.9 | 1 | |
| Integration | 82 | 11.0 | 2.2 | 1 | |
| Docs | 53 | 7.1 | 2.0 | 3 | |
| Bugfix | 42 | 5.6 | 1.5 | 3 | |
| Other | 39 | 5.2 | 0.9 | 1 |
| Workstream | Calls | Tokens | Energy ( ) | Ratio | |
|---|---|---|---|---|---|
| EcoLogits | Hybrid | ||||
| Viralis-Optimalis | 31.2 | ||||
| Green-Code | 38.0 | ||||
| Cross | 13.5 | ||||
| Customer-Project | 9.0 | ||||
| Artemis | 9.7 | ||||
| Term | Share | |
|---|---|---|
| Memory: attention reads | ||
| Compute: prefill (fresh) | ||
| Non-GPU server | ||
| Compute: decode | ||
| Memory: parameter loads | ||
| Memory: KV writes | 0.01\text{,}\mathrm{%}$$ |
| Model | Vendor | Active (B) | |
|---|---|---|---|
| Haiku 4.5 | Anthropic | 16.9 | |
| Sonnet 4.6 | Anthropic | 92.5 | |
| GPT-5.4 | OpenAI | 96.0 | |
| Opus 4.8 | Anthropic | 172.1 | |
| Gemini 2.5 Pro | 1042.2 | ||
| as actually run (mixed) | 105.0 | ||
| Workstream | Haiku | Sonnet | Opus | actual | O/H |
|---|---|---|---|---|---|
| Viralis-Optimalis | 8.37 | 45.42 | 84.46 | 52.96 | 10.1 |
| Green-Code | 6.10 | 33.33 | 62.07 | 33.65 | 10.2 |
| Cross | 1.53 | 8.61 | 16.12 | 10.99 | 10.5 |
| Customer-Project | 0.43 | 2.30 | 4.19 | 3.96 | 9.6 |
| Artemis | 0.48 | 2.70 | 5.04 | 3.30 | 10.5 |
| Formalis | 0.02 | 0.11 | 0.21 | 0.11 | 10.3 |
| Agent type | Calls | Int. | Rep. | Actual | Ratio | Sec. | Edits |
|---|---|---|---|---|---|---|---|
| Explore | 57 | 3.28 | 37.84 | 11.5 | 107 | 0 | |
| general-purpose | 38 | 1.64 | 11.08 | 6.8 | 56 | 41 | |
| Plan | 11 | 1.05 | 10.85 | 10.3 | 577 | 0 | |
| Total | 106 | 5.97 | 59.77 | 10.0 | 41 |