StateHarness: Maintaining Evolving Project State for Long-Horizon Coding Agents
Organizations: University of Pennsylvania · University of Notre Dame
Abstract
Long-horizon coding requires agents to maintain an evolving view of the project they are changing: what problem is currently being solved, what has already been learned or modified, which other artifacts may be affected, and what work remains unresolved. When this information is scattered across a growing interaction history, local edits can become disconnected from their later consequences and progress can be difficult to carry forward. We introduce StateHarness, a runtime framework that maintains evolving project state without updating model parameters. The working agent records its current problem and hypothesis, important findings and decisions, discovered repository relationships, and unfinished work. Runtime support prompts record updates, searches for callers, tests, and documentation that may be affected by a change, and maintains a compact working brief for subsequent decisions. When behavioral signals accumulate, a separate diagnostic session revisits the recorded history and surfaces overlooked evidence. Across three repository-level evaluations, StateHarness improves task completion and several measures of change consistency. On RefactorBench, full-chain completion increases from 29.4% to 35.3% and cleanup-check accuracy from 87.0% to 95.4%. On five LHTB software-engineering tasks, mean reward improves from 0.427 to 0.576, exceeding LongHorizon-Harness at substantially lower recorded execution cost. On ChainSWE, bug-fix rate increases from 35.1% to 39.5% and residual debt decreases from 60 to 43 items, although pass-to-pass regressions increase. These results show that maintaining project state can improve long-horizon coding while introducing measurable execution and regression trade-offs.
Figures & tables
| RefactorBench | |||||||
|---|---|---|---|---|---|---|---|
| Completion | Checks | Cleanup Breakdown | |||||
| Method | Subtasks | Chains | Refactor | Cleanup | Imports | Calls | Docs |
| Claude Code | 77.9 | 29.4 | 94.6 | 87.0 | 82.7 | 93.3 | 90.9 |
| StateHarness | 81.4 | 35.3 | 95.9 | 95.4 | 94.4 | 93.3 | 100.0 |
| Long-Horizon Terminal-Bench (LHTB) | |||||||
| Setting | Method | Recorded tokens (K) | Tool calls | Time (s) |
| RefactorBench | Claude Code | 25.6 | 81 | 383 |
| StateHarness | 33.1 | 110 | 529 | |
| ChainSWE | Claude Code | 24.9 | 75 | 300 |
| StateHarness | 35.3 | 96 | 383 | |
| LongHorizon-Harness | 208.0 | 272 | 2214 |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Record | What changes during the task | Stored representation |
|---|---|---|
| Task state | Record the initial observation and environment; replace the current focus when the active subproblem or hypothesis changes. | task_state.md |
| Event log | Append important changes, tests, findings, decisions, and rejected assumptions together with their reasons. | breakthrough_log.md |
| Relation notes | Record discovered relationships to callers, tests, documentation, configuration, or schemas. | file_relation_graph.md |
| Procedure guidance | Maintain a named procedure’s invocation history, latest observed failure, usage policy, and rationale. | skill_states.json |
| Working brief | Refresh the current focus, relevant procedure guidance, recent events, and diagnostic alerts returned to the working agent. | context_brief.md |
| Observed condition | Response and state change |
|---|---|
| Missing command-completion events | At the next shell call, count a missing normal completion callback. Three such events produce a signal and reset the streak; a normal shell completion clears the streak. |
| Repeated reads of one path | Three counted reads produce a signal and reset that path’s count. A proposed edit to the path clears the count before the edit outcome is known. |
| Working brief neglected for 25 calls | Deliver the brief and restart the window. Add diagnostic score only when no non-state edit or recognized test was proposed during that window. Recognized brief access restarts the window. |
| Recognized test completion | Clear diagnostic score and diagnostic alerts on the normal post-event, without parsing whether the test passed. A proposed test alone does not perform this reset. |
| Recognized progress tag | Synchronization clears the diagnostic score and alerts when the latest event indicates recognized progress, such as a passed test or an established root cause. |
| Successful nonblank diagnosis | Replace diagnostic alerts, clear diagnostic score and the brief-neglect count, start a 15-call cooldown, and return the updated brief. |
| Function | Recorded project state | Subsequent use |
|---|---|---|
| Preserve understanding | Retrieval identifies the service, CLI, test, and README as potentially affected by the loader signature change. The event log records that the loader and service have been updated, while relation notes preserve the remaining caller and documentation relationships. | The discovered scope remains available after the immediate edit, including the still-unchecked CLI caller. |
| Return current state | The current focus becomes “Checking loader.py ; I suspect encoding is still handled incorrectly.” Recent modification and decision events are copied into the working brief together with relevant procedure guidance. | Subsequent model requests receive the active hypothesis and recent progress without rereading the full project history. |
| Update after changes | Further edits push the earlier CLI decision outside the recent-event window, but the decision remains in the full event log. Later state records show that the loader and service have been updated while the CLI still requires review. | The compact working view follows recent work, while older project evidence remains recoverable from persistent state. |
| Recover overlooked evidence | Repeated reads of loader.py raise diagnostic score to the diagnostic threshold. Diagnosis revisits the full event log and surfaces the deferred CLI inspection as overlooked evidence. | The alert returns the CLI to the current working view. After the CLI is updated and the relevant regression test passes, the new result is recorded and ordinary execution continues. |
| Component | Claude Code | StateHarness |
|---|---|---|
| Persistent project records | No | Yes |
| Working-brief synchronization | No | Yes |
| Recording reminders and behavioral signals | No | Yes |
| Repository change-impact retrieval | No | Yes |
| On-demand diagnostic session | No | Yes |
| Record-structure and history checks | No | Yes |
| Material | Delivery | Role |
|---|---|---|
| Working-agent startup instructions | Included in the initial request | Provide environment information, identify the project-state directory, request initialization and completion checks, and specify when the working agent should update project records. |
| Initialization and completion protocols | Explicitly requested reads | Establish the initial task observation and current focus, and later request review of recorded relationships, unresolved warnings, and relevant tests before termination. |
| Procedure-guidance lifecycle | Available as protocol files | Specifies how repeated procedure failures can produce use , verify , avoid , or freeze guidance. These files are not independently injected on every procedure failure. |
| Record templates and maintenance protocols | Available as supporting files | Define the persistent record formats and additional maintenance behavior. The working records are initialized from these templates rather than requiring the agent to reread the templates during ordinary execution. |
| Diagnostic instructions | Supplied directly to a fresh diagnostic session | Request evidence-backed identification of overlooked information or unproductive execution patterns using the triggering signal, task state, procedure guidance, full event log, and current working brief. |
Software Engineering Agents
Software Engineering Agents are AI systems that use code, tools, and feedback to perform software development tasks.
See all 337 papers