Think or Remember: The Exchange Rate of Agentic Inference
Authors: Juan Wisznia, Santiago Luis Groba Alonso, Matías Gómez Taddeo, Gonzalo Zabaleta, Joaquín Cullen, Luciano Del Corro
Organizations: ELIAS Lab, Escuela de Ingeniería, Universidad de San Andrés, Buenos Aires, Argentina · Escuela de Ingeniería, Universidad de San Andrés, Buenos Aires, Argentina
Agents built on reasoning models resend their growing transcript at every call. Two transacting parties, not the model, decide what that costs: the client keeps or edits messages; the server decides whether earlier reasoning returns in the prompt and what its cache reuses. We define a cost model whose counts match an instrumented engine exactly; the architecture charges c per token processed and β per pair attended. Their ratio P∗=c/β is an exchange rate between thinking and remembering: per token, processing dominates below this context length, a thinking regime, and attention above it, a memory regime. A policy change has a mechanical cost, incurred even if the agent writes the same tokens, and a behavioral one: it writes different tokens, changing cost and outcome. Serving alone can flip which policy is cheaper: on recorded Qwen3-8B ScienceWorld runs, dropping earlier reasoning costs 1.74 times keeping it when the server reuses generated states and 0.34 times when it reuses nothing. In live reruns, dropping it cuts correct HotpotQA answers by 9-23 percentage points on four models, at lower point-estimate cost. Comparing context policies means comparing systems: architecture and serving set the cost of the history kept and the work it induces.
Figures & tables
Figure 1: Serving changes the price of the same history. A call adds 2,000 reasoning and 100 action tokens to a cached 16,000-token task; the next call adds 500 input and 100 generated tokens. Hatching marks rebuilding: Drop rebuilds the action under ρ=1 , and Keep also rebuilds reasoning under ρ=0 . Qwen3.6-27B prices, w=1 , token-granular reuse; widths are schematic.
Figure 2: The architectural crossover. Attention’s marginal share βP/(c+βP) reaches 50% at 124K tokens for Qwen3.6-27B and 30K for its synthetic full-attention counterpart, which replaces the recurrent layers by full attention and recomputes both coefficients.
Figure 3: Editing history and generating more work have different costs. Left: schedule/Keep cost on 132 fixed continuations ( w=1 , 16-token blocks) at the Qwen3-30B-A3B, Qwen3-8B and Qwen3.6-27B exchange rates. Right: cost of generating a token and returning it in the next prompt, relative to one input read, at 32K keys.
Figure 4: Prices reverse rankings on unchanged work. Total Drop/Keep cost of the same 103 Qwen3-8B ScienceWorld pairs under three cache rules, with pointwise 95% family-bootstrap bands; the dotted line marks the 8B exchange rate.
Figure 5: Lower cost comes with fewer completed tasks. Qwen3-8B on 32 matched five-task chains, 160 task executions per arm. Cost is replay cost at ρ=1 , w=1 , relative to Fresh under the same retention rule; bars are 95% chain-bootstrap intervals.
Figure 6: Client-policy rankings change across cohorts. CLM/Summary ratio of total API cost in dollars, compaction included; below one, CLM costs less. Five matched task streams yield 20 complete chains: four on the original six-task stream and eight on each expansion’s two five-task streams. Appendix F gives outcomes and configurations.
Model
n
Cost D/K
Solved D − K (pp)
SW
8B
103
1.74 [1.46, 2.08]
− 3.9 [ − 10.2, 2.0]
SW
4B
126
1.18 [0.93, 1.49]
0.8 [ − 8.8, 9.0]
SW
30B-A3B
119
1.73 [1.43, 2.09]
− 0.8 [ − 6.6, 5.4]
SW
32B
52
1.94 [1.65, 2.26]
0.0 [0.0, 0.0]
SW
gpt-oss
88
1.20 [0.96, 1.49]
19.3 [10.7, 29.3]
HP
8B
165
0.81 [0.64, 1.00]
− 13.3 [ − 20.6, − 6.1]
Table 1: Live Keep (K) and Drop (D) from the same forks, generated-state reuse, w=1 . Cost: total D over total K. Solved: paired difference in full-score rate. 95% cluster-bootstrap intervals. SW: ScienceWorld; HP: HotpotQA, titles hidden, answer-only grading. Qwen3 unless named. The 32B outcomes are identical.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
ρ
Policy
P
R
G
N
Q
C (T)
1
Keep
18,600
18,100
100
600
11,040,300
33.56
1
Drop
16,600
16,000
100
700
11,445,350
38.59
0
Keep
18,600
16,000
100
2,700
46,846,350
149.91
0
Drop
16,600
16,000
100
700
11,445,350
38.59
Appendix
Table 2: Next-call prices for the identical history under two reuse rules.
Figure 7: Architectures occupy different price regimes. A point specifies linear price c and full-attention price β ; diagonals have constant P∗=c/β . Filled points include the open models used in the live or replay studies and the hybrid references. The arrow replaces recurrent layers of the 27B hybrid with full attention and recomputes both coefficients. That synthetic counterpart is distinct from the live dense/MoE comparison linked by the dotted segment. Appendix B gives the coefficient scope and values.
Model or counterfactual
c (G/token)
β (M/pair)
P∗ (K tokens)
Qwen3-4B
7.260
0.589824
12.31
Qwen3-8B
13.900
0.589824
23.57
Qwen3-30B-A3B
5.462
0.786432
6.95
Qwen3-32B
62.400
2.097152
29.75
Qwen3-1.7B
2.819
0.229376
12.29
Qwen3.5-2B
2.745
0.049152
55.86
Appendix
Table 3: Declared reduced coefficients. G and M denote decimal billions and millions; K denotes thousands.
Reference
MLP
Attention projections
Recurrent projections
Qwen3-1.7B
2.113929216
0.704643072
0.000000000
Qwen3.5-2B
1.811939328
0.176160768
0.757334016
Qwen3.5-9B
9.663676416
0.939524096
3.233808384
Qwen3.6-27B
34.225520640
3.355443200
11.120148480
27B full-attention counterpart
34.225520640
13.421772800
0.000000000
Appendix
Table 4: Reference projection subtotals in billions of operations per evaluated token. Their sum is the reduced token price.
Figure 8: Serving changes the time price. Measured incremental decode time under three batch sizes. The account’s architectural coefficients remain fixed within each panel, while batching changes the time paid for an additional generation step. Each point uses the stated subtraction and minimum over three repetitions.
Figure 9: Predictions agree with GPU execution. Predictions fixed before execution versus operation counts from instrumented vLLM: 27 dense/hybrid schedules, 648 requests. Counts cover matrix multiplications, the output head, causal attention pairs and recurrent-state updates. Both axes show the Keep-minus-Drop difference.
Model
Block
Requests
Max. error
Qwen3-1.7B
16
264
0
Qwen3.5-2B
544
384
0
Appendix
Table 5: Predicted versus engine-recorded work. Block sizes are in tokens. The last column gives the largest absolute discrepancy in evaluated-token, attended-pair, or cached-token counts across all requests.
Model
Reuse
ΔMK
−ΔBK
ΔMD
−ΔBD
ΔC
8B
Prompt + generation
-1.5
73.3
14.0
88.8
-74.8
8B
Prompt only
40.5
42.8
95.8
98.1
-2.3
4B
Prompt + generation
38.6
56.7
95.8
113.9
-18.1
4B
Prompt only
67.3
30.1
154.6
117.4
37.2
30B-A3B
Prompt + generation
27.4
100.8
144.8
218.3
-73.5
30B-A3B
Prompt only
60.0
55.5
235.3
230.8
4.5
Appendix
Table 6: Changing the replay reference changes the allocation. Subscripts identify the Keep or Drop continuation used to price the mechanical term. Entries are aggregate percentages of Keep cost; ΔC=ΔM−(−ΔB) under either reference. The total and repayment side are invariant by construction. The panels contain 103, 126, 118, and 52 forks. Totals come from the aligned replay (Appendix C.2 ), which prices Keep 1.8% below the logged reconstruction; for 8B with generated-state reuse its ΔC implies Drop/Keep of 1.748, against 1.738 logged (Table 10 ).
Task
Reuse
Reasoning
ΔMW
−ΔBW
ΔMF
−ΔBF
ΔC
ScienceWorld
Prompt + generation
Keep
338.4
-14.6
301.9
-51.0
352.9
ScienceWorld
Prompt + generation
Drop
35.7
-9.7
38.6
-6.7
45.3
ScienceWorld
Prompt only
Keep
141.1
-13.9
126.3
-28.7
155.0
ScienceWorld
Prompt only
Drop
34.1
-9.4
36.9
-6.6
43.5
HotpotQA
Prompt + generation
Keep
146.2
0.3
140.3
-5.6
145.9
HotpotQA
Prompt + generation
Drop
36.2
5.1
47.4
16.4
31.0
Appendix
Table 7: Client-window repayment under two continuation references. W uses the window continuation and F the full-history continuation. Every entry is a percentage of aggregate full-history cost with the same reasoning-retention rule. Policy 1 is the client window and the baseline the full history; positive −ΔB is a continuation saving, and ΔC=ΔM−(−ΔB) . The mechanical term exceeds the magnitude of the behavioral term in every displayed configuration and under both references. The 114 ScienceWorld and 171 HotpotQA forks reproduce all 9,296 actual-arm prompt hashes and token counts.
Collection
Planned
Pre-fork
Terminal
Cost pairs
Missing
8B ScienceWorld, sequential repeat
40
6
34
34
0
4B ScienceWorld, thought truncation
149
28
121
116
0
8B ScienceWorld, thought truncation
149
38
111
111
0
8B ScienceWorld, fork at call 3
149
15
134
134
0
8B ScienceWorld, fork at call 8
149
59
90
90
0
gpt-oss ScienceWorld
100
12
88
88
0
Appendix
Table 8: Collection flow. Pre-fork ends have no continuation comparison. Terminal sets have every arm completed; cost pairs additionally require a priced call in every arm. Missing ends are incomplete episode records.
Figure 10: Re-reading history changes its cost. The same 134 Qwen3-8B ScienceWorld Keep continuations, from the fork-three collection rather than the primary 103-fork panel, replayed with earlier reasoning retained or removed at w=1 . Each bar sums the cost of the entire panel. Prompt-only reuse ( ρ=0 ), the standard-serving convention in Shao et al. (2026, Appendix C) , reprocesses returning output. Retention adds 45.8% of Keep’s cost under this rule, versus 4.2% with generated-state reuse.
Collection
Policy
All terminal
Ratio panel
Zero calls
4B ScienceWorld, thought truncation
Keep
11/121
11/116
5
First 150
11/121
11/116
0
First 40
16/121
16/116
0
First 10
9/121
9/116
0
Swap
7/121
7/116
5
Drop
16/121
15/116
0
Appendix
Table 9: Full-score outcomes with and without the positive-cost inclusion rule. Zero-call counts are arm-specific.
Collection
Reuse
Pairs
Median D/K
Total D/K
8B ScienceWorld, primary
full
103
1.871
1.738
8B ScienceWorld, primary
prompt
103
1.036
1.023
4B ScienceWorld, primary
full
126
1.704
1.181
4B ScienceWorld, primary
prompt
126
0.959
0.628
30B-A3B ScienceWorld
full
119
1.867
1.729
30B-A3B ScienceWorld
prompt
119
1.093
0.954
Appendix
Table 10: Median paired ratios and ratios of total costs over identical completed panels. Reuse includes generation (full) or only the prompt.
Reused prefix
Median D / K
Total D / K
Prompt + generation
1.87 [1.38, 2.29]
1.74 [1.46, 2.08]
Prompt only
1.04 [0.79, 1.40]
1.02 [0.86, 1.23]
None
0.40 [0.30, 0.49]
0.34 [0.28, 0.42]
Appendix
Table 11: Repricing the same 103 ScienceWorld 8B forks at w=1 . Both estimands retain paired family-bootstrap uncertainty. The prompt-only intervals span equal cost; the full-reuse and no-reuse intervals lie on opposite sides of one. Intervals use bootstrap seed 20261012.
Task
Model
Forks
Full-score K / D
Total cost D / K
Full-score D − K (pp)
SW
8B
103
16 / 12
1.74 [1.46, 2.08]
-3.9 [-10.2, 2.0]
SW
4B
126
17 / 18
1.18 [0.93, 1.49]
0.8 [-8.8, 9.0]
SW
30B-A3B
119
19 / 18
1.73 [1.43, 2.09]
-0.8 [-6.6, 5.4]
SW
32B
52
7 / 7
1.94 [1.65, 2.26]
0.0 [0.0, 0.0]
SW
gpt-oss-20b
88
3 / 20
1.20 [0.96, 1.49]
19.3 [10.7, 29.3]
HP
8B
165
64 / 42
0.81 [0.64, 1.00]
-13.3 [-20.6, -6.1]
Appendix
Table 12: Cost and outcome on the same eligible forks. SW: ScienceWorld; HP: title-hidden HotpotQA. Models are Qwen3 unless named otherwise. Cost ratios divide total Drop cost by total Keep cost at the recorded reduced price; bracketed values are paired 95% cluster-bootstrap intervals. A zero interval for the 32B outcome contrast records identical paired outcomes in this panel, not population equivalence.
Collection
Policy
Forks
Full-score
Cost / Keep
95% interval
8B ScienceWorld, sequential repeat
Keep
34
2
1.00
[1.00, 1.00]
Keep repeat
34
2
1.00
[0.98, 1.14]
Drop
34
2
1.27
[0.84, 2.96]
Rule
34
3
0.98
[0.84, 1.37]
4B ScienceWorld, thought truncation
Keep
116
11
1.00
[1.00, 1.00]
First 150
116
11
0.99
[0.81, 1.10]
Appendix
Table 13: Full cost-ratio inventory. Cost / Keep is the median within-pair ratio at w=1 under the logged generated-state-reuse reconstruction; intervals resample families or questions with bootstrap seed 20261011, so they can differ in the second decimal from Tables 11 and 12 , which use seed 20261012.
Collection
Policy
Forks
Full-score
Cost / Keep
95% interval
32B ScienceWorld
Keep
52
7
1.00
[1.00, 1.00]
Keep repeat
52
4
1.00
[1.00, 1.00]
Drop
52
7
2.24
[1.77, 2.70]
Rule
52
6
1.26
[0.99, 1.55]
4B ScienceWorld, primary
Keep
126
17
1.00
[1.00, 1.00]
Keep repeat
126
19
1.00
[1.00, 1.00]
Appendix
Table 13: Full cost-ratio inventory (continued).
Collection
Policy
Forks
Full-score
Cost / Keep
95% interval
8B HotpotQA, hidden titles repeat
Keep
164
66
1.00
[1.00, 1.00]
Keep repeat
164
62
1.00
[0.99, 1.00]
Drop
164
48
1.00
[0.92, 1.18]
Rule
164
56
1.04
[1.00, 1.09]
8B HotpotQA, input window
Keep
171
57
1.00
[1.00, 1.00]
Keep repeat
171
63
1.00
[1.00, 1.00]
Appendix
Table 13: Full cost-ratio inventory (continued).
Figure 11: Mechanical change and continuation saving under the declared cache reconstructions, normalized by the reference population’s cost. The retention panels contain 103/126/118/52 pairs for 8B/4B/MoE/32B; the window panels contain 114 ScienceWorld and 171 HotpotQA forks. Positive continuation savings offset mechanical expense. Intervals resample task families or questions.
Figure 12: Retention changes future work. One hosted Qwen3.7-Plus ScienceWorld pair: Keep repeats an identical 371-token block and action at calls 60 and 66; Drop restates a plan after earlier reasoning is removed (232 and 617 tokens). The Keep sentence is quoted; Drop text is paraphrased. The same retention intervention permits very different continuation patterns. Appendix E.2 identifies the public task and calls.
Condition
Passed
USD
Calls
Output tokens
Compactions
Rollbacks
Original: one six-task stream
Summary Keep
5/6
0.3168
80
77,609
4
0
Summary Drop
5/6
0.2386
77
58,080
2
0
CLM Keep
5/6
0.1959
55
32,649
2
0
CLM Drop
4/6
0.3713
89
81,119
3
0
Expansion 1: two five-task streams
Appendix
Table 14: Completed hosted cohorts, reported separately. Output includes native reasoning once. Costs include summary calls, edits and failed task attempts. Compactions count automatic summaries or applied substantive edits; rollbacks are automatic harness history resets.
Client
ρ
ΔM
−ΔB
ΔC
Summary
0
+21.73
-19.47
+41.19
Summary
1
+3.60
-23.98
+27.58
CLM
0
+11.95
+62.99
-51.04
CLM
1
-8.82
+77.76
-86.58
Appendix
Table 15: Declared compaction replay at P∗=123,850 , w=1 . Entries are percentages of replayed Keep cost, not API expenditure. Policy 1 is Keep and the baseline Drop, so positive ΔC means Keep costs more; positive −ΔB is a continuation saving, and ΔC=ΔM−(−ΔB) . Rounding accounts for differences in displayed sums.
Policy
Calls
Solved
Cost / fresh
New tasks
Clear actions
Fresh / Keep
1,094
47/160
1.00 [1.00, 1.00]
30/96
0
Carry / Keep
1,139
34/160
0.60 [0.41, 0.87]
17/96
0
Clear option / Keep
1,207
42/160
0.98 [0.53, 1.67]
25/96
45
Fresh / Drop
1,199
33/160
1.00 [1.00, 1.00]
26/96
0
Carry / Drop
1,222
19/160
0.43 [0.36, 0.52]
13/96
0
Appendix
Table 16: All five policies on the same 32 chains. Cost is the ratio of summed generated-state-reuse replay cost to Fresh under the same reasoning-retention rule. Intervals resample whole matched chains. “New tasks” counts positions two through four; the first and repeated task are excluded from that column.
Reasoning policy
Prompt volume
Full reuse
Prompt-only
No reuse
Keep all
1.000
1.000
1.000
1.000
Drop immediately
0.976
0.983
0.955
0.946
Keep latest 1
0.978
1.861
1.809
0.949
Keep latest 5
0.984
4.542
4.416
0.960
Clear every 5
0.978
1.598
1.560
0.948
Appendix
Table 17: Policy totals relative to Keep all on the 132 confirmation continuations. Prompt volume counts all submitted tokens. The three cost columns use the reference prices, w=1 and 16-token cache blocks. Across P∗∈[1K,2M] and w∈[1,5] with generated-state reuse, keeping the latest five blocks costs 3.97 to 4.56 times Keep and immediate deletion stays within 2% of Keep.