cs.CLNov 7, 2026

Think or Remember: The Exchange Rate of Agentic Inference

Authors: Juan Wisznia, Santiago Luis Groba Alonso, Matías Gómez Taddeo, Gonzalo Zabaleta, Joaquín Cullen, Luciano Del Corro

Organizations: ELIAS Lab, Escuela de Ingeniería, Universidad de San Andrés, Buenos Aires, Argentina · Escuela de Ingeniería, Universidad de San Andrés, Buenos Aires, Argentina

Abstract

Agents built on reasoning models resend their growing transcript at every call. Two transacting parties, not the model, decide what that costs: the client keeps or edits messages; the server decides whether earlier reasoning returns in the prompt and what its cache reuses. We define a cost model whose counts match an instrumented engine exactly; the architecture charges cc per token processed and ββ per pair attended. Their ratio P∗=c/βP^*=c/β is an exchange rate between thinking and remembering: per token, processing dominates below this context length, a thinking regime, and attention above it, a memory regime. A policy change has a mechanical cost, incurred even if the agent writes the same tokens, and a behavioral one: it writes different tokens, changing cost and outcome. Serving alone can flip which policy is cheaper: on recorded Qwen3-8B ScienceWorld runs, dropping earlier reasoning costs 1.74 times keeping it when the server reuses generated states and 0.34 times when it reuses nothing. In live reruns, dropping it cuts correct HotpotQA answers by 9-23 percentage points on four models, at lower point-estimate cost. Comparing context policies means comparing systems: architecture and serving set the cost of the history kept and the work it induces.

Figures & tables

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More

    Mar 25, 2026Lingjiao Chen, Chi Zhang, Yeye He +3Cost-Aware InferenceLarge Reasoning Models

  2. Total Recall at What Cost? Benchmarking the Serving Cost of Agentic Memory Systems

    Aug 12, 2026Natchanon Pollertlam, Witchayut KornsuwannawitLLM ServingLLM Agent Memory