Overview

Automated test generation studies how to create inputs and checks that reveal defects in software, APIs, hardware, and simulated systems. Methods include search-based testing and fuzzing, symbolic and constraint-based generation, property-based and metamorphic testing, and increasingly LLM agents guided by execution, coverage, or mutation feedback. Evaluation commonly measures coverage, mutation scores, bug detection, and whether tests distinguish correct behavior from faulty implementations. A central challenge is that executable tests and high coverage do not guarantee semantic correctness: weak oracles, incomplete specifications, and prompts based on buggy code can produce tests that miss faults or reinforce them.

Latest papers

CardsList

139 papers9 papers in the last four weeks, down 47% on the four weeks before. 0.1% of all new papers.

  1. Can Coding Agents Write and Transpile Unit Tests

    Nov 16, 2026Gil Pasternak, AJ Langley, Tom Yotam +1AI Coding AgentsAutomated Test Generation

  2. ADAPT: Agentic Diff-Aware Patch Testing for Pull Requests

    Nov 11, 2026Vahid Haratian, Atakan Akar, Eray TüzünRegression TestingAutomated Software Testing

  3. Benchmarking LLM Agents for Behaviorally Adequate Test Generation from Software Requirements

    Oct 27, 2026Binhang Qi, Yiming Liu, Yun Lin +3Automated Test GenerationLLM Agent Evaluation

  4. From Prose to Properties: Testing Research Claims with ARATest for Agent-Native Science

    Oct 21, 2026Yifeng He, Jiachen Liu, Hao ChenProperty-Based TestingScientific Claim Verification

  5. FaultGene: LLM-Assisted Fault-Augmented Test Generation for Hardware-in-the-Loop Validation of Autonomous Driving Systems

    Oct 21, 2026Chaoran Yuan, Fadi KurdahiAutomated Test GenerationAutonomous Driving Safety Evaluation

  6. SANE: Automated Scorer-Specific Metamorphic Testing for LLM Judges

    Oct 20, 2026Ethan Ludwick, Yu-Chung HsiaoLLM-as-a-JudgeAutomated Test Generation

  7. TEX: Test-Time Scaling Testing Agents via Execution-based Cross-Validation

    Oct 19, 2026Akash Gokul, Srijan Bansal, Yingbo Zhou +1Test-Time ScalingSoftware Engineering Agents

  8. Certifying Repeatable Entry-Replay Witnesses for COBOL-to-Java Migration

    Oct 19, 2026Andras L. Ferenczi, Mariya Bessonov, Allwyn Ponnudurai +3Software EngineeringAutomated Test Generation

  9. FuzzInsight: Navigating the Efficiency-Effectiveness Trade-off in Coverage-Guided DNN Fuzzing

    Oct 16, 2026Vivek Vekariya, Fathiyeh Faghih, Alexander PretschnerAutomated Test GenerationNeural Network Robustness

  10. SWEPatchTest: Evaluating Software Tests Beyond Reproduction

    Oct 13, 2026Hua Lu, Pengcheng Wang, Han Li +4Software Engineering BenchmarksAutomated Test Generation

  11. SpecGuardBench: Can Coding Agents Construct Reliable Validation Harnesses from Natural Language Specifications?

    Oct 12, 2026Zhiqing Zhong, Mingyu Chen, Aoyang Fang +5AI Coding AgentsSpecification-Based Testing

  12. Small but Mighty: Test Suites as Improvement Signals for Self-Evolving Agents

    Oct 11, 2026Yutong Cheng, Wei Cheng, Wenchao Yu +2Automated Test GenerationLLM Agent Self-Improvement

  13. Grounding Deep Learning Framework Testing in Data Flow

    Oct 11, 2026Jiatao Ge, Yanzhou Mu, Yinglong Zou +3Automated Software TestingDNN Testing

  14. Passing Tests, Missing Semantics: Evidence of Interpretive Lock-In in Plan-Based DataFrame Code Generation

    Oct 11, 2026Jingjing Liu, Dishu Yang, Hongbo Qin +1Code GenerationCode Generation Evaluation

  15. QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code

    Oct 9, 2026Meriem Terki, Nouhaila Innan, Minghao Shao +3Automated Test GenerationLLM Evaluation

  16. QuSema: Detecting Silent Bugs in Quantum Libraries via Quantum-knowledge-enhanced Agents

    Oct 7, 2026Yujin Song, Kaining Zhang, Qixin Zhang +3Fault DetectionAutomated Software Testing

  17. TestGRAD: Evolving Test Suites via Failure Pattern Momentum for SWE-Agent Ensemble

    Oct 7, 2026Pengfei He, Jiayuan Zhou, Shaowei Wang +1Automated Test GenerationAutomated Software Testing

  18. TestJack: Should you trust the results in coding benchmarks? Agentic Coding Benchmarks Auditing via Evaluator Evolution

    Oct 7, 2026Shuangjie Yao, Hao Wang, Koushik Sen +3LLM Agent EvaluationAI Agent Benchmarks

  19. Complex Agents, Shallow Tests: Demystifying and Enhancing Test Adequacy of Agent Harness in the Wild

    Oct 4, 2026Yifan Xiong, Jingyi Ge, Zhenpeng Chen +1LLM Agent HarnessesAutomated Test Generation

  20. AuraForge: Scaling Security Supervision for Training Coding Agents

    Oct 1, 2026Danqing Wang, Songwen Zhao, Harsh Sharma +4AI Coding AgentsSoftware Security

  21. Improving Large Language Models for Code through Runtime Program-State Reasoning

    Sep 28, 2026Hongwei Li, Spandan Garg, Yufan HuangAutomated Program RepairAutomated Test Generation

  22. Scenario-Driven Neuroevolution: Using Models to Guide Test Generation for Games

    Sep 23, 2026Gijs van Cuyck, Patric Feldmeier, Jan Tretmans +1Automated Software TestingAutomated Test Generation

  23. Probabilistic Modelling of Operational Design Domains, A New Approach for Testing AI Systems

    Sep 21, 2026Hans-Werner WiesbrockProbabilistic Graphical ModelsKnowledge Representation

  24. Information-Gain Rewards over Diversity-Pruned Tests: GT-Anchored Verifier Co-Training for Reliable Code Generation

    Sep 18, 2026Ana Nunez, Peyman NajafiradRL for Code GenerationCode Generation

  25. CovR: Coverage-Aware Hardware Verification via Reasoning-Guided Reinforcement Learning

    Sep 15, 2026Manar Abdelatty, Maryam Nouh, Sherief RedaAutomated Test GenerationReinforcement Learning with Verifiable Rewards