cs.SEOct 26, 2026

The Hidden Costs of Forgetting: A Study of Trajectory Deletion in Software Engineering Agents

Authors: Fernando Vallecillos-Ruiz, Islem Bouzenia, Michael Pradel, Leon Moonen

Organizations: Simula Research Laboratory, Norway · CISPA Helmholtz Center for Information Security, Germany

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Software engineering agents resend their entire trajectory to an LLM at every turn, motivating online trajectory reduction. Prior work shows that randomly deleting 75% of tokens in old turns may preserve agent effectiveness, but it remains unclear why, how agents compensate for missing content, and whether it actually reduces costs. This paper presents an empirical study of 42 trajectory-reduction configurations, focusing on effectiveness, agent behavior, and cost. We systematically explore deleting content from old turns by varying the scope, granularity, amount, and timing of deletion across 9,400 runs with two state-of-the-art agents on SWE-bench Verified. We find deleting large amounts of trajectory content (e.g., up to 100% of reasoning text) can preserve overall effectiveness. Importantly, such deletion must be applied with a delay, as immediate deletion tends to significantly degrade effectiveness. Agents often compensate for deleted content by re-fetching it, increasing both the number of turns taken and the input/output tokens. Whether trajectory reduction reduces costs depends on the pricing model: Under a flat pricing model, estimated costs fall in 26 of 42 evaluations. Surprisingly, under a caching-aware pricing model, zero evaluations yield lower costs, since reducing the trajectory breaks the cached prefix billed at a lower rate. Our findings have three implications: First, token counts alone are misleading; trajectory reduction techniques must consider overall costs under a realistic pricing model. Second, analyzing trajectories is crucial to understand agent behavior, as end-to-end effectiveness hides differences such as excessive re-fetching. Finally, these results highlight the need for future work on memory management to reduce costs without sacrificing effectiveness.

Figures & tables

Fig. 1: The longest prefix identical to the previous prompt is billed at the cached price. Reducing T2 at Call 4 rewrites it to T2 ′ , i.e., T2 ′ and every later turn are re-billed. At Call 5, the prefix matches again and only T5 is new.
Fig. 2: The first three turns (T1–T3, left) of a Trae agent trajectory and the prompts at the next call under two configurations (right). The first configuration deletes T1 and T3, while T2 remains intact. The second configuration modifies parts of T1.
   Mini-SWE-agentTrae agent
   DelayedImmediateDelayedImmediate
ScopeGranularity% DeletedTasks solvedTurnsTasks solvedTurnsTasks solvedTurnsTasks solvedTurns
Uncompressed72.6 (71–76)31.5  74.6 (74–76)42.5  
ReasoningTurn25710%75+11%70−9%76+1%
  5069+5%73+13%72−3%69−5%
  7571+3%71+13%79−11%72−6%
TABLE I: RQ1: Tasks solved (of 100) and change in median run length. Bold marks statistically significant changes from the uncompressed baseline. –: the median run reaches the 200-turn budget.
   Mini-SWE-agent Trae agent
   Re-fetch Cost Re-fetch Cost
ScopeGranul.Del%DelayImm. FlatCache DelayImm. FlatCache
ReasoningTurn251.01.0 0.921.12 0.91.0 0.901.05
  501.01.1 0.931.06 1.00.9 0.851.11
  751.21.1 0.841.23 1.01.2 0.801.11
  1001.11.9 0.991.44 0.91.9 0.821.22
TABLE II: RQ2 and RQ3: Re-fetch rates under delayed and immediate deletion, and costs under delayed deletion with flat and caching-aware pricing, as ratios to the uncompressed runs.
  Mini-SWE-agentTrae agent
Scope% DeletedDelayedImmediateDelayedImmediate
Uncompressed13% 9% 
Tool output2511% / 22%14% / 30%9% / 28%9% / 45%
 5012% / 18%10% / 25%9% / 27%7% / 36%
 7511% / 17%8% / 16%7% / 21%7% / 28%
Both2512% / 23%12% / 28%9% / 37%10% / 50%
TABLE III: RQ2: Percentages of retained and deleted file-read outputs subsequently re-fetched under turn-level deletion. Each cell lists the retained-output percentage followed by the deleted-output percentage.
More in this topic

Context Compaction for LLM Agents

See all 38 papers

Explore similar work

CardsList
Oct 10, 2026

TurnCost: Turns, Not Observations: Cache-Aware Loop Engineering for Coding Agents

AI coding agents operate in iterative loops: they modify code, run tools, inspect the results, and decide what to do next. We call the design of this process loop engineering, which includes context management, model selection, compaction, and termination. Existing efficiency work mainly compresses tool observations or routes selected turns to cheaper models, but the practical opportunity for these optimizations remains unclear. We analyze 600 publicly released coding-agent trajectories spanning two agent scaffolds and two language models. Median observations contain only 118-286 tokens, while the large build logs commonly used to motivate compression occur near the ninety-ninth percentile. Moreover, 98.6% of input tokens are served from a prefix cache, so flat input pricing overestimates loop cost by roughly 7x on average under near-ideal caching. We derive and empirically validate a closed-form break-even condition for model delegation. Delegation increases cost in half of the evaluated settings, near the predicted boundary. More broadly, turn count explains 84% of the variance in log trajectory cost, compared with 4% for mean observation size; the most expensive trajectories have the most turns despite having the smallest observations. These associations suggest that observation-focused methods target a secondary cost factor, and identify unnecessary turns as the larger and largely unexamined opportunity, though we do not establish that turns can be removed while holding task success fixed. We release TurnCost, a tool for re-scoring trajectory corpora under alternative caching, pricing, compression, and routing policies.

Pramod Kumar, Raj Kumar Pandey

The Pennsylvania State University, University Park, USA

LLM Agent Workflow OptimizationCoding AgentsCost-Aware InferenceLLM Routing
AskPDF
Sep 1, 2026

TRIAGE: Three-level Routing and Intelligent Agent Guidance for Efficient Execution

Large Language Model (LLM) agents based on the ReAct paradigm have demonstrated remarkable capabilities in tool use and task execution. However, ReAct suffers from a fundamental efficiency problem: every query triggers a complete reasoning loop from scratch, and similar queries repeat identical steps without leveraging historical experience. We propose TRIAGE,a three-level routing framework that reduces token consumption by reusing historical execution trajectories. Its core innovation is TaaS (Trajectory-as-a-Skill), which abstracts historical execution trajectories into reusable skills, realizing 'experience as a service'. TRIAGE classifies queries into three levels: (1) Direct Reuse-identical queries, 0 tokens; (2) Skill Substitution-similar queries, 0 tokens via deterministic parameter substitution; (3) Full ReAct-novel queries, automatically stored for future reuse. In large-scale experiments on 1,007 security monitoring queries, TRIAGE achieves 62.3% token savings, with 56.0% of queries at Level 2 and 5.5% at Level 1, both executing at zero cost. Cross-domain validation on ToolBench (15 domains, 345 queries) achieves 76.3% token reduction, confirming the generalizability of semantic routing. An online learning experiment demonstrates cold-start-to-mature evolution: the L2 hit rate rises from 0% to 57% within the first 100 queries, and the average token cost drops from 198 to 74.7. We also propose an automatic Skill extraction mechanism that distills high-frequency trajectory patterns into deterministic Skills, creating a positive feedback loop of 'the more you use it, the more efficient it becomes'.

Ruocan Wei

China Telecom Cloud, Beijing, China

LLM Inference EfficiencyLLM Agent Skill LearningLLM AgentsLLM Agent Routing
AskPDF
Oct 6, 2026

Agent-Controlled Forgetting for Tool-Using Agents: Reversible Context Curation in Practice

Tool-using agents repeatedly carry observations whose useful content can be much smaller than their original payload. We study agent-controlled forgetting: the acting model selects previously observed tool results, replaces each with a short note at its original position, and retains the exact original in a recoverable archive. A Python harness exposes batch archival and explicit recovery without task-specific model training, while protecting user instructions and assistant messages from these operations. In an exploratory OpenTelemetry debugging case followed by an unrelated implementation task, the method ended with 231,951 provider-reported prompt tokens versus 912,492 under retained history, used 50% fewer cumulative input tokens, and had an estimated API cost of USD 1.28-1.44 versus approximately USD 4.38. Both arms passed the two-case primary behavioral oracle; neither fully satisfied the follow-up evaluation. The method made more requests and took 17% longer. A contrasting application-development pair produced no context or cost saving, and an earlier continuation exhibited lower manually assessed quality despite reduced context. These observations demonstrate substantial resource savings in noisy tool-use trajectories and identify workload dependence as a central consideration for reversible context management.

Jan-Peter Franke

Münster, Germany

LLM Context ManagementAgent Memory ManagementTool-Using Agents
AskPDF