RepoTrace: Improving Requirement Tracing in Multi-Round Agent Interactions

Huanyu Liu; Yezheng Cheng; Jinliang You; Tianxiao Huang; Linyu Li; Zheng Fang; Peng Cao; Ge Li

cs.SEOct 26, 2026

RepoTrace: Improving Requirement Tracing in Multi-Round Agent Interactions

Authors: Huanyu Liu, Yezheng Cheng, Jinliang You, Tianxiao Huang, Linyu Li, Zheng Fang, Peng Cao, Ge Li

Organizations: Key Laboratory of High Confidence Software Technologies, Ministry of Education; School of Computer Science, Peking University · Xidian University · School of Cyber Science and Engineering, Wuhan University

Abstract

Code agents have become increasingly capable of performing software engineering tasks through interaction with software environments. However, agents tend to degrade as the number of interaction rounds increases, making it difficult to maintain existing software during iterative modification. We analyze 2,656 agent trajectories in EvoCode-Bench and identify requirement focus misalignment as a recurring failure pattern, where agents struggle to distinguish the primary modification targets of a requirement from unrelated code. We propose Repotrace a requirement-driven repository understanding harness that combines repository structure with requirement semantics to construct a requirement-code correspondence and injects it into the agent context. This correspondence enables code agents to follow the repository architecture, trace requirements across multi-round interactions, and identify the core code elements corresponding to the current requirement. Experiments show that Repotrace improves the mean per-round reward (MPR) across different models (e.g., +19.99 on GLM-5.2) and agent harnesses (e.g., +10.22 on Claude Code). Across rounds, Repotrace increases the average per-round reward from 0.163 to 0.371 on Terminus-2 with DeepSeek-V4-Pro, while reducing average regression errors by 56.78%. We release the source code and agents' trajectories to support future research.

Figures & tables

Figure 1: Two examples of requirement focus misalignment . (a) Unrelated code modification: the agent rewrites a shared dispatcher and omits the existing validate() command. (b) Missing update: the agent adds the output field but omits the code that collects its values.

Figure 2: Overview of RepoTrace in multi-round interactions. Left: an illustrative sequence of requirements for the same code repository. Right: before each code modification, RepoTrace analyzes repository structure, extracts requirement semantics, and constructs a requirement-code correspondence message

M_{t}

for the code agent.

	MPR $\uparrow$	FCR $\uparrow$	PTR $\uparrow$	HTR $\downarrow$	TCS $\uparrow$	IT(M) $\downarrow$	OT(M) $\downarrow$
DeepSeek-V4-Pro	14.14	11.54	3.85	92.31	61.74	441.33	12.51
w/ RepoTrace	36.39	34.62	15.38	61.54	73.63	319.48	16.92
GLM-5.2	16.19	3.85	3.85	88.46	47.52	98.26	7.62
w/ RepoTrace	36.18	30.77	7.69	61.54	77.33	175.62	12.30
MiniMax-M3	23.42	23.08	7.69	73.08	61.46	726.66	5.89
w/ RepoTrace	28.33	15.38	7.69	73.08	76.76	1445.57	7.99

Table 1: Main results and token usage on EvoCode-Bench.

Agent	MPR $\uparrow$	FCR $\uparrow$	PTR $\uparrow$	HTR $\downarrow$	TCS $\uparrow$	IO $\downarrow$	OT $\downarrow$
Terminus-2	14.14	11.54	3.85	92.31	61.74	441.33	12.51
w/ RepoTrace	36.39	34.62	15.38	61.54	73.63	319.48	16.92
Claude Code	44.65	42.31	19.23	57.69	91.14	842.06	7.99
w/ RepoTrace	54.87	46.15	26.92	50.00	93.02	675.36	9.44

Table 2: RepoTrace across agent harnesses on EvoCode-Bench.

Panel of Figure 3 — Figure 3: Results of DeepSeek-V4-Pro on EvoCode-Bench using Terminus-2 and Claude Code, with and without RepoTrace . (a) Mean reward (left axis) and mean pass rate (right axis) over 10 rounds: the average reward and the average fraction of test cases passed per round, each averaged over tasks. (b) Regression test accuracy in multi-round interactions: for each round $i\in\{2,\ldots,11\}$ , we report the mean pass rate over all test cases belonging to rounds $1,\ldots,i-1$ , averaged over 26 tasks. Higher values indicate fewer regression errors.

Figure 4: Heatmap of the per-round mean reward for each harness- RepoTrace combination.

Figure 5: Case study of RepoTrace on JobForge. In the Vanilla setting, adding the critical subcommand breaks the existing schedule dispatch. In the RepoTrace setting, the requirement-code correspondence guides the agent to add an independent critical branch while preserving the schedule function, satisfying both the current and prior requirements.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Repository structure and selected requirement entities for JobForge Round 7. Panel (a) shows the source-file set

V_{7}

, and panel (b) shows candidate targets and potentially affected prior code elements from

Q_{7}

. The bottom row shows the added-file entry in

\Delta_{7}

. These records distinguish the existing schedule.go file from the candidate new file critical.go .

Figure 7: Reference prompt for requirement semantics extraction in Section 4.2. The upper box organizes the extraction inputs and specifies the fields of

Q_{t}

. The lower box expands {prev_block} with prior requirement records and structural changes. The template wording is retained; line wrapping and Python brace escaping are adjusted for display.

Figure 8: Excerpts from the requirement-code correspondence message

M_{7}

supplied to the code agent. The message includes prior requirements, repository structure, candidate targets, existing/new-file mappings, and verification guidance. Ellipses mark omitted lines. The empty impact list reflects

E_{7}=\emptyset

; the original phrase leaf components does not establish the absence of source-code dependencies.

Panel of Figure 9 — Figure 9: Cropped screenshots of three consecutive records in the supplementary HTML report. The report combines extraction fields with function-call graphs re-collected from a reconstructed agent workspace. Its call-edge counts therefore describe the supplementary reconstruction, not the saved file-relation sets in Appendix A.1 .

Figure 10: The schedule.go region of the Round 7 HTML diagram. The screenshot shows scheduleCmd and functions for parsing worker settings, computing the schedule, and formatting output. This file corresponds to the added-file entry in Figure 9 (b).

Figure 11: The critical.go region of the Round 8 HTML diagram. This file corresponds to the added-file entry in Figure 9 (c). The full diagram, including connections to other files, is available in the supplementary HTML report.

Figure 12: The overall repository structure and function-call graph for Round 7 in the supplementary HTML report.

Software Engineering Agents

Software Engineering Agents are AI systems that use code, tools, and feedback to perform software development tasks.

See all 349 papers

RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations
May 25, 2026Hanyu Li, Yichi Zhang, Speed Zhu +3AI Coding Agents Software Engineering Benchmarks
Beijing University of Posts and Telecommunications · Tsinghua University · Tencent
CodeTracer: Towards Traceable Agent States
Apr 13, 2026Han Li, Yifan Yao, Letian Zhu +13AI Coding Agents Agent Trajectory Analysis
Nanjing University · Alibaba Group · Renmin University of China +3
A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents
Aug 10, 2026Xin Zhou, Chun Yong Chong, Kisub Kim +11Requirements Engineering AI Coding Agents
Singapore Management University, Singapore · Monash University Malaysia, Malaysia · Daegu Gyeongbuk Institute of Science and Technology (DGIST), Republic of Korea +5

May 25, 2026J/K move · Enter open · S save

RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations

Hanyu Li, Yichi Zhang, Speed Zhu, Hang Su+2

Beijing University of Posts and Telecommunications · Tsinghua University · Tencent

AI Coding Agents Software Engineering Benchmarks Automated Evaluation LLM Reasoning

Open paper AskPDF arXiv

Code agents are currently having skillful performance on repository-level software engineering benchmarks, but it remains unclear whether success on end-to-end tasks such as issue resolution truly reflects repository context reasoning, the ability to identify the task-relevant information across multiple files and reason over the relations among them. To investigate this question, we introduce RepoMirage, a two-stage evaluation suite built on SWE-Bench Verified that adopts perturbation as a diagnostic tool to increase the demand for context reasoning by transforming how the repository is exposed. First, RepoMirage-Perturb applies three types of semantics-preserving repository-level perturbations, revealing a clear performance drop when correct solving requires broader context access. RepoMirage-Extend further turns perturbation-targeted structural bottlenecks into explicit tasks beyond issue resolution, where the average performance declines from 66.8% in the original setting to 25.3%, indicating a significant deficiency in repository context reasoning. Further trajectory analysis reveals an exploration drift, where agents access broader repository context but fail to turn it into effective structure information. Motivated by this observation, we propose RepoAnchor, a structure-first prototype workflow that separates repository exploration from downstream problem solving, and show that explicit structural scaffolding yields notable gains. These results uncover an previously overlooked gap in repository context reasoning for code agents and suggest that stronger structure-aware methods are potential to improve them.

Abstract

Software Engineering Agents

Explore similar work

RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations

CodeTracer: Towards Traceable Agent States

A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents