Automated Software Testing

Ask about this topic

Overview

Automated software testing aims to check whether programs meet their requirements, behave correctly across inputs and environments, and remain reliable as they change. Approaches include model-based and specification-based testing, fuzzing, property-based and metamorphic testing, mutation analysis, search-based prioritization, and AI agents that generate tests or diagnose failures. Work increasingly applies these methods to complex settings such as web applications, APIs, autonomous systems, machine-learning software, and distributed programs, where simulations, browser traces, and runtime evidence can help expose failures. A central challenge is establishing trustworthy test oracles: coverage and passing tests do not necessarily show that a test detects real faults, while generated tests and agent judgments can be brittle, incomplete, or misleading.

Latest papers

CardsList

136 papers18 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

  1. Risk-Aware Batch Testing for Performance Regression Detection

    Mar 31, 2026Ali Sayedsalehi, Peter C. Rigby, Gregory MierzwinskiSoftware EngineeringAutomated Software Testing

    Department of Computer Science and Software Engineering, Concordia University, Montreal, Quebec, Canada. · Mozilla, Potton, Quebec, Canada.

  2. The Kitchen Loop: User-Spec-Driven Development for a Self-Evolving Codebase

    Mar 26, 2026Yannick RoySoftware EngineeringAutomated Software Engineering

  3. WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing

    Mar 26, 2026Fanheng Kong, Jingyuan Zhang, Yang Yue +10Automated Software TestingComputer-Use Agent Benchmarks

    Northeastern University · Kuaishou Technology

  4. From Untestable to Testable: Metamorphic Testing in the Age of LLMs

    Mar 25, 2026Valerio TerragniAutomated Software TestingMetamorphic Testing

    University of Auckland

  5. Evaluating LLM-Based Test Generation Under Software Evolution

    Mar 24, 2026Sabaat Haroon, Mohammad Taha Khan, Muhammad Ali GulzarAutomated Test GenerationAutomated Software Testing

    Virginia Tech, USA · Carnegie Mellon University, USA

  6. TDAD: Test-Driven Agentic Development - Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis

    Mar 18, 2026Pepe Alonso, Sergio Yovine, Victor A. BrabermanAI Coding AgentsAI Agent Reliability

    Universidad ORT Uruguay · DC, UBA, Argentina

  7. STADA: Specification-based Testing for Autonomous Driving Agents

    Mar 11, 2026Joy Saha, Trey Woodlief, Sebastian Elbaum +1Specification-Based TestingAutomated Test Generation

    University of Virginia, Virginia, USA · William & Mary, Virginia, USA

  8. Automating Detection and Root-Cause Analysis of Flaky Tests in Quantum Software

    Mar 9, 2026Janakan Sivaloganathan, Ainaz Jamshidi, Andriy Miranskyy +1Root Cause AnalysisAutomated Software Testing

    Department of Computer Science, Toronto Metropolitan University, Toronto, Canada · Department of Information Systems, University of Maryland, Baltimore County, Baltimore County, United States of America

  9. An explainable hybrid deep learning-enabled intelligent fault detection and diagnosis approach for automotive software systems validation

    Mar 9, 2026Mohammad Abboush, Ehab Ghannoum, Andreas RauschFault DiagnosisFault Localization

    Technische Universität Clausthal, Institute for Software and Systems Engineering, Arnold-Sommerfeld-Straße 1, Clausthal-Zellerfeld, 38678, Niedersachsen, Germany

  10. From Code to Road: A Vehicle-in-the-Loop and Digital Twin-Based Framework for Central Car Server Testing in Autonomous Driving

    Mar 5, 2026Chengdong Wu, Sven Kirchner, Nils Purschke +9Automated Software TestingAutonomous Driving Simulation

    Technical University of Munich, Garching, Bayern, Germany · CARIAD SE, Germany

  11. Test Case Prioritization: A Snowballing Literature Review and TCPFramework with Approach Combinators

    Feb 26, 2026Tomasz Chojnacki, Lech MadeyskiAutomated Software Testing

    Wroclaw University of Science and Technology, Wroclaw, Poland

  12. UAMTERS: Uncertainty-Aware Mutation Analysis for DL-enabled Robotic Software

    Feb 23, 2026Chengjie Lu, Jiahui Wu, Shaukat Ali +5Mutation TestingAutomated Software Testing

    Simula Research Laboratory and University of Oslo · Simula Research Laboratory · Danish Technological Institute +1

  13. Automated structural testing of LLM-based agents: methods, framework, and case studies

    Jan 25, 2026Jens Kohl, Otto Kruse, Youssef Mostafa +9Automated Software TestingLLM Agent Evaluation

    BMW Group, Munich, Germany · Amazon Web Services

  14. MASTEST: A LLM-Based Multi-Agent System For Testing RESTful APIs

    Nov 22, 2025Xiaoke Han, Hong ZhuMulti-Agent LLM SystemsAutomated Software Testing

    School of Engineering, Computing and Mathematics, Oxford Brookes University, Oxford, UK

  15. Simulator Ensembles for Trustworthy Autonomous Driving Systems Testing

    Mar 11, 2025Lev Sorokin, Matteo Biagiola, Andrea StoccoAutomated Software TestingAutonomous Driving Simulation

    Technical University of Munich, Munich, Germany. · University of St. Gallen and Universit`a della Svizzera italiana, St. Gallen / Lugano, Switzerland. · fortiss GmbH, Munich, Germany.

  16. Requirement-Based Testing: Enhancing Reinforcement Learning with Game Theory

    Jul 26, 2024Ocan Sankur, Thierry Jéron, Nicolas Markey +2Automated Software TestingAutomated Test Generation

    DEVINE · MERCE-France · Mitsubishi Electric Corporation, Tokyo, Japan