cs.SENov 17, 2026

Agent Trajectories as Evidence for Software Quality Assurance

Authors: Jonathan Ullrich, Julien Siebert, Michael Kläs, Adam Trendowicz, Andreas Jedlitschka

Organizations: Fraunhofer IESE, Kaiserslautern, Germany

Abstract

Autonomous software agents increasingly plan, implement, and test software with limited human oversight. Their decisions may introduce implicit assumptions that final code and passing tests do not adequately expose. We propose analyzing observable agent trajectories alongside software and agent specifications and AI generated artifacts as evidence for targeted quality assurance. The aim is to prioritize consequential decisions for scrutiny, investigate their possible contribution to software problems, and inform improvements to specifications and development practices. Industrial experience motivates the idea; available execution records and preliminary analysis methods make it plausible. However, trajectories alone do not establish that decisions are justified, outcomes acceptable, or corrective changes effective. We ask how this evidence can support review, diagnosis, and continuous quality improvement, and when simpler QA approaches suffice. Perspectives from requirements engineering, verification and validation, agent systems, causal inference, empirical software engineering, and human factors could help develop and evaluate the idea.

Explore similar work

CardsList
  1. A Case Study in Assuring AI-Written Software

    Oct 6, 2026Lindsey Ferris, Sierra BonillaAI Agent ReliabilityAI Assurance

  2. AgentLens: Revealing The Lucky Pass Problem in SWE-Agent Evaluation

    May 13, 2026Priyam Sahoo, Gaurav Mittal, Xiaomin Li +4Agent Failure AnalysisProcess-Based Agent Evaluation

  3. Agents as Software: A Programming Languages Agenda for Agent Reliability

    Sep 26, 2026Shraddha Barke, Adithya MuraliAI Agent ReliabilityAI Agent Monitoring