Automated Software Testing

Ask about this topic

Overview

Automated software testing aims to check whether programs meet their requirements, behave correctly across inputs and environments, and remain reliable as they change. Approaches include model-based and specification-based testing, fuzzing, property-based and metamorphic testing, mutation analysis, search-based prioritization, and AI agents that generate tests or diagnose failures. Work increasingly applies these methods to complex settings such as web applications, APIs, autonomous systems, machine-learning software, and distributed programs, where simulations, browser traces, and runtime evidence can help expose failures. A central challenge is establishing trustworthy test oracles: coverage and passing tests do not necessarily show that a test detects real faults, while generated tests and agent judgments can be brittle, incomplete, or misleading.

Latest papers

CardsList

136 papers18 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

  1. Can AI Guide Testing Without Generating Tests? A Comparative Study of AI-Assisted Unit Testing

    Oct 26, 2026Zihan Fang, Ron Asoulin-Handelman, Helen Wu +2Computer Science EducationGenerative AI in Education

    Vanderbilt University, Nashville, TN, USA

  2. Build or Adopt? Custom-Built vs. General-Purpose LLM Agent Harnesses for Automotive GUI Testing

    Oct 24, 2026Arsham Gholamzadeh Khoee, Ruyu Fu, Jiaqi Wei +4LLM Agent HarnessesAutomated Software Testing

    Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden · Volvo Group, Gothenburg, Sweden

  3. ProductWebBench: Evaluating Website Continuity Beyond Task Completion

    Oct 22, 2026Jiajun Wu, Jian Yang, Yaxin Du +9Software Engineering BenchmarksCode Generation Evaluation

    Beihang University · IQuest Research · Huawei Technologies Ltd.

  4. From Prose to Properties: Testing Research Claims with ARATest for Agent-Native Science

    Oct 21, 2026Yifeng He, Jiachen Liu, Hao ChenProperty-Based TestingScientific Claim Verification

    University of California, Davis · ARA Lab · The University of Hong Kong

  5. Who Verifies the Builder? Toward Independent Assurance for Agent-Produced Software

    Oct 16, 2026Sion Israel Sion, Alain AprilProperty-Based TestingAI Agent Reliability

    École de technologie supérieure Montréal, Canada

  6. Stateful Testing for Deep Learning Compilers

    Oct 15, 2026Jiatao Ge, Ruixiang Qian, Yibo Hu +5Automated Software TestingDifferential Testing

    State Key Laboratory for Novel Software Technology, Nanjing University, China · UNIST, Republic of Korea

  7. Robotic-supported Data Loss Detection in Android Applications

    Oct 14, 2026Davi Freitas, Breno Miranda, Juliano IyodaAutomated Software Testing

    Federal University of Pernambuco, Recife, Brazil

  8. SolBehave: Evaluating the Behavioural Correctness of LLM-Generated Smart Contracts

    Oct 13, 2026Giovanni Pio Delvecchio, Stefano Fantazzini, Giacomo Frisoni +4Code Generation EvaluationLLM Code Generation

    University of Bologna, Bologna, Italy · University of Copenhagen, Copenhagen, Denmark

  9. Grounding Deep Learning Framework Testing in Data Flow

    Oct 11, 2026Jiatao Ge, Yanzhou Mu, Yinglong Zou +3Automated Software TestingDNN Testing

    State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · UNIST, Ulsan, Republic of Korea

  10. Targeting the Right Situations: Mutation Testing of Fuzzy-Powered Situation Hyperspace

    Oct 10, 2026Nawshin Mannan PromaAutomated Software TestingRobot Safety

    University of York, Heslington, United Kingdom

  11. QuSema: Detecting Silent Bugs in Quantum Libraries via Quantum-knowledge-enhanced Agents

    Oct 7, 2026Yujin Song, Kaining Zhang, Qixin Zhang +3Fault DetectionAutomated Software Testing

    Nanyang Technological University, Singapore · Hong Kong University of Science and Technology, Hong Kong, China · Zhejiang University of Technology, Hangzhou, China

  12. TestGRAD: Evolving Test Suites via Failure Pattern Momentum for SWE-Agent Ensemble

    Oct 7, 2026Pengfei He, Jiayuan Zhou, Shaowei Wang +1Automated Test GenerationAutomated Software Testing

    University of Manitoba, Winnipeg, Canada · Huawei Canada, Markham, Canada · Huawei Technologies, Hangzhou, China

  13. Multi-Label Perceptual Bug Detection in Video Games using Deep Learning on Gameplay Footage

    Oct 6, 2026Nahian Rifaat, Felix Morosov, Loutfouz ZamanAutomated Software TestingMulti-Label Classification

    Ontario Tech University Oshawa, Ontario, Canada · Universität Osnabrück Osnabrück, Niedersachsen, Germany

  14. DreamTest: World-Model Surrogates for Search-Based Testing of Deep Reinforcement Learning Agents

    Oct 3, 2026Qinghua Xu, Guancheng Wang, Boxi Yu +2Automated Software TestingNeural Surrogate Modeling

    Lero Research Centre, University of Limerick, Ireland · University of Ottawa, Canada

  15. GTDD: Generative Test-Driven Development for AI Coding Agents with Adversarial Testing

    Oct 2, 2026Masahiro KatoAutomated Software TestingAI Coding Agents

    Mizuho-DL Financial Technology Co., Ltd.

  16. Discriminating Fixture Coverage in Agent-Infrastructure Verification Suites

    Oct 2, 2026Xin Xu, Siru TaoAutomated Software TestingAI Agent Monitoring

    Carnegie Mellon University Eragon

  17. WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness

    Oct 2, 2026Yun-Yun Tsai, Yuning Mao, Shiqi Wang +2Code GenerationCode Generation Evaluation

    Columbia University · Meta SuperIntelligence Labs

  18. FastCI: Efficient GPU-Intensive CI for LLM Training Frameworks

    Oct 1, 2026Tianshuo Qiao, Naiqian Zheng, Xiaopeng Liu +4Automated Software Testing

    Peking University · ByteDance Seed

  19. Do Agent Benchmarks Do What They Say? An Executable-Contract Audit of Tool-Using Agent Environments

    Sep 29, 2026Rohith Reddy Bellibatlu, Zichong Wang, Wenbin ZhangTool-Use EvaluationAutomated Software Testing

    Florida International University, Miami, FL, USA

  20. Beyond Rule-Based Mutation Testing: Test-Aware Mutant Generation Using Large Language Models

    Sep 24, 2026Nils Kiele, Zainab Saad, Zirui Wang +2Automated Software TestingMutation Testing

    Department of Electrical and Software Engineering, University of Calgary, AB, Canada

  21. Scenario-Driven Neuroevolution: Using Models to Guide Test Generation for Games

    Sep 23, 2026Gijs van Cuyck, Patric Feldmeier, Jan Tretmans +1Automated Software TestingAutomated Test Generation

    Radboud University, Institute iCIS, Nijmegen, The Netherlands · University of Passau, Germany · TNO-ESI, Eindhoven, The Netherlands

  22. FairTest: Search-Based Fairness Testing for Multi-Agent Reinforcement Learning Systems

    Sep 23, 2026Xiaotong Wang, Xuan XieAutomated Software TestingAlgorithmic Fairness

    School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR, China

  23. WebCraftBench: Evaluating Web Application Generation from a Software Testing Perspective

    Sep 14, 2026Chenxu Liu, Zilu Zou, Peizhong Gao +9Automated Software TestingWeb Application Generation

    Hunyuan Team, Tencent · Tsinghua University · Peking University

  24. When LLM Decompilers Recompile More and Preserve Less

    Sep 4, 2026Chang Liu, Edward Raff, Kristopher MicinskiSoftware Reverse EngineeringDifferential Testing

    Syracuse University · CrowdStrike

  25. ARIA - An Agentic Framework for Autonomous Testing of Infotainment Systems

    Sep 4, 2026António Azevedo, Bruno Lima, João Pascoal FariaAutomated Software TestingGUI Automation

    INESC TEC, Faculty of Engineering, University of Porto, Portugal · LIACC, Faculty of Engineering, University of Porto, Portugal

  26. The reach of a verification tool decides its value: A controlled study of verification surface, artifact quality, and cost in AI coding agents

    Aug 28, 2026Achint MehtaAutomated Software TestingAI Coding Agents

  27. RiskBlend: A Multi-Signal Framework for Test Input Prioritization in Machine Learning Regression Testing

    Aug 27, 2026Madhusudan Srinivasan, Namith Nishal RaphaeAutomated Software Testing

    Department of Computer Science East Carolina University Greenville, USA

  28. FaultLens: Learning Compact Behavioral Test Suites for Generated Operational Programs

    Aug 27, 2026Zeming Liu, Hang Lyu, Jingtao ZhangAutomated Software TestingMutation Testing

    Independent Researchers

  29. FuzzingBrain-Bench V1: Evaluating Open-Ended Bug Discovery by LLMs

    Aug 25, 2026Ze Sheng, Aleksandar Kezic, Zhicheng Chen +1Software Engineering BenchmarksAutomated Software Testing

    Texas A&M University · University of Novi Sad

  30. ExploreAI: Agentic Exploration Knowledge Bases for Reproducible Observable-Regression Testing of Black-Box VR and 3D Applications

    Aug 21, 2026Jiajie Wang, Kebin Peng, Wei Wang +3Automated Software TestingLLM Agents

    The University of Arizona · East Carolina University · The University of Texas at San Antonio +1