Test Generation

Ask about this topic

Momentum

9 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 82

All topics
CardsList

  1. Verification Trap: Understanding Test-Time Selection Failures under False Premises in Code Generation

    Oct 4, 2026Feng He, Hejia Wang, Linghao Meng +2Test GenerationVerification Framework

    University of Science and Technology of China, China · Beijing University of Posts and Telecommunications, China · National University of Singapore, Singapore +1

  2. A2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications?

    Sep 30, 2026Seonho Lee, Wonryeol Jeong, Alberto Cereser +4Test GenerationFeedback

    KRAFTON · KAIST · Korea National University of Arts

  3. LatentSift: Policy-State Filtering for Token-Efficient Verification of Software Engineering Agents

    Sep 28, 2026Yuning Han, Yangchenchen Jin, Tyler Jandreau +1Verification FrameworkSwe-Bench Verified

    University of Florida · Aeronix

  4. Scenario-Driven Neuroevolution: Using Models to Guide Test Generation for Games

    Sep 23, 2026Gijs van Cuyck, Patric Feldmeier, Jan Tretmans +1Test GenerationNeuroevolution

    Radboud University, Institute iCIS, Nijmegen, The Netherlands · University of Passau, Germany · TNO-ESI, Eindhoven, The Netherlands

  5. Teach-to-Crash: A Closed-Loop Student-Teacher LLM Framework for Collision-Inducing Test Scenario Generation

    Sep 23, 2026Zaid Ghazal, Khouloud Gaaloul, Bruce MaximAutonomous Driving SimulationCrashes

    University of Michigan-Dearborn Dearborn, MI, USA

  6. EquivSVA: A Formally Verified Dataset of Behavioral Assertions Across Equivalent RTL Implementations

    Sep 22, 2026FNU AditiSystemverilog AssertionsHierarchical Register Transfer Level Generation

  7. A Study of the Reliability of Agentic AI-Generated Programs

    Sep 16, 2026Ayesha Shafique, Barton P. MIller, Elisa R. HeymannTest GenerationCode Generation

  8. Confidence-Gated Transductive Test Generation for Code Reranking

    Sep 14, 2026Sungjae Lee, Youngsik Yoon, Seockbean Song +3Test GenerationTest Time

    Department of Computer Science and Engineering, POSTECH, South Korea · Graduate School of Artificial Intelligence, POSTECH, South Korea · Microsoft Research Asia, China

  9. ExecCritic: Learn to Test, Test to Improve for Coding Agents

    Sep 8, 2026Leitian Tao, Baolin Peng, Haorui Wang +7Coding AgentsTest Generation

    University of Wisconsin–Madison · Microsoft Research · Georgia Tech

  10. Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs

    Sep 3, 2026Jiacheng Xu, Wentao Zhang, Zhiyi Lyu +4Test GenerationLarge Language Model Generation

    Nanyang Technological University, Singapore · Skywork AI

  11. Answer Probing-Guided Search for Diverse Solution Exploration of LLMs

    Aug 31, 2026Yi Fang, Que Shen, Chengpeng Li +6LLM Reasoning StrategiesReasoning Paths

    University of Science and Technology of China · Zhongguancun Academy · Alibaba Group +2

  12. Can LLMs Test Terminal User Interfaces?

    Aug 4, 2026Chao Peng, Ruida Hu, Ajitha Rajan +3Generative User InterfaceTest Generation

    University of Edinburgh · United Kingdom · Harbin Institute of Technology +3

  13. Coding Agents as Test-Suite Auditors: Finding What Official Suites Miss While Approaching What They Catch

    Aug 3, 2026Shuyang Xie, Shuxiao Xie, Feng Zhu +2Raw Judge OutputsTest Generation

    Harbin Institute of Technology · Fudan University · Sun Yat-sen University

  14. Agentic Method for Deterministic Validation of Legacy Code Migration

    Jul 30, 2026Andras Ferenczi, Jordan Docherty, Mariya Bessonov +2Test GenerationAgentic

    American Express

  15. Adversarial Test-Hardening for AI-Written Code: An Instrument Autopsy and a Pre-Registered Causal Estimate of the Critic Loop

    Jul 25, 2026Jeff OttersonCode QualityAdversarial Evaluation

    W. P. Carey School of Business, Arizona State University

  16. Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests

    Jul 24, 2026Junda Zhao, Shurui Zhou, Eldan CohenTest GenerationLarge Language Models Fail

    University of Toronto, Department of Mechanical and Industrial Engineering, Toronto, Canada · University of Toronto, Department of Electrical and Computer Engineering, Toronto, Canada

  17. Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness? (Replicability Study)

    Jul 24, 2026Junda Zhao, Shurui Zhou, Eldan CohenTest GenerationCode Quality

    University of Toronto, Department of Mechanical and Industrial Engineering, Toronto, Canada · University of Toronto, Department of Electrical and Computer Engineering, Toronto, Canada

  18. From Resource Flow to Executable Tests: Petri-Net-Guided LLM Test Generation for Concurrent Stateful Rust APIs

    Jul 23, 2026Kaiwen Zhang, Guanjun LiuTest GenerationLlm-Driven Code Synthesis

    Tongji University Shanghai,China

  19. Code Monitor Red Teaming for Public-Test-Passing Code

    Jul 23, 2026Junchi Liao, Jiawen Deng, Fuji RenCode QualityRed-Teaming

    University of Electronic Science and Technology of China Chengdu, China

  20. Beyond Fail-to-Pass: Iterative Hardening of Co-Generated Bug Reproduction Tests and Fixes

    Jul 22, 2026Yuhao Tan, Zhibang Yang, Fangkai Yang +9Automated Program RepairTest Generation

    1Nanjing University · 2Peking University · 3Microsoft

  21. Integrating High-Level Requirements to Low-Level Tests with Machine-Readable V&V Specifications

    Jul 20, 2026Mansur Arief, Nur Ahmad Khatim, Ali Akarma +1Test GenerationHigh-Level Source Code

    Industrial and Systems Engineering, King Fahd University of Petroleum and Minerals (KFUPM), Saudi Arabia · Division of Information Science, Nara Institute of Science and Technology (NAIST), Japan · Faculty of Computer and Information Systems Islamic University of Madinah Madinah, Saudi Arabia +1

  22. Project Kaleidoscope: Contextual, Human-Aligned Evaluation for Real-World AI Applications

    Jul 16, 2026Leanne Tan, Rohan Jaggi, Shaun Khoo +1Artificial Intelligence EvaluationRubrics

    GovTech, Singapore · National University of Singapore · University of British Columbia

  23. Faithful Autoformalization of Natural Language Assertions

    Jul 14, 2026Hongyi Liu, Madhusudan Parthasarathy, Adithya MuraliAutoformalizationSystemverilog Assertions

    University of Wisconsin-Madison · University of Illinois at Urbana-Champaign

  24. Fail-Aware and Explainable Test Oracle Prediction

    Jul 13, 2026Yue Zhao, Binish Tanveer, Jelena ZdravkovicTest GenerationOracles

    Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden

  25. Generative Testing of Automated Speech Recognition Systems

    Jul 10, 2026Yanis Xabier Wilbrand Peña, Oliver Weißl, Andrea StoccoAutomatic Speech RecognitionGrapheme-To-Phoneme

    Technical University of Munich Germany · Technical University of Munich & fortiss GmbH Germany

  26. SCATE: Learning to Supervise Coding Agents for Cost-Effective Test Generation

    Jul 9, 2026Sijia Gu, Noor Nashid, Ali MesbahTest GenerationCoding Agents

    University of British Columbia Vancouver, Canada

  27. Specification Grounding Drives Test Effectiveness for LLM Code

    Jul 7, 2026Amin Haeri, Mahdi GhelichiTest GenerationGrounding

    Model Development Innovation TD Bank, Toronto, Canada

  28. An Exploration of Agentic Information Fusion for Test Maintenance Prediction

    Jul 6, 2026Jingxiong Liu, Nasser Mohammadiha, Gregory GayTest GenerationRepository-Level Code Understanding

    Chalmers University of Technology and University of Gothenburg · Ericsson AB

  29. EvoOtter: Evolutionary Reproduction Test Generator

    Jul 3, 2026Toufique Ahmed, Jatin Ganhotra, Avraham Shinnar +1Test GenerationEvolutionary Optimization

    IBM

  30. TestEvo-Bench: An Executable and Live Benchmark for Test and Code Co-Evolution

    Jul 2, 2026Jiale Amber Wang, Kaiyuan Wang, Pengyu NieTest GenerationCo-Evolution

    University of Waterloo · 2Google

  31. Prompt Coverage Adequacy

    Jul 2, 2026Florian Tambon, Michael Konstantinou, Cedric Richter +3Test GenerationPrompt Engineering

    SnT, University of Luxembourg · University College London

  32. TAG: A Lightweight Framework for Test-Driven Agentic Artifact Generation

    Jul 1, 2026Yaniv Melamed, Yoni Zukerman, Michal Shechter +3Test GenerationAgentic Framework

    1Microsoft

  33. SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests

    Jul 1, 2026Yaoqi Guo, Yang Liu, Jie M. Zhang +3Swe-Bench VerifiedTest Generation

    Nanyang Technology University · King’s College London · Peking University +2

  34. Building to the Test: Coding Agents Deliver What You Check, Not What You Requested

    Jun 26, 2026Yanuo Ma, Ben Kereopa-Yorke, Ben SchultzTest GenerationCoding Agents

    Microsoft

  35. Test-Input Generation for Tensor Programs: What Actually Finds Kernel Bugs

    Jun 23, 2026Dipankar SarkarFuzzingTest Generation

    Arizona State University, USA

  36. MANGO: Automated Multi-Agent Test Oracle Generation for Vision-Language-Action Models

    Jun 23, 2026Pablo Valle, Shaukat Ali, Aitor Arrieta +1Action GenerationTest Generation

    Mondragon University, Guipuzcoa, Spain · Simula Research Laboratory, Oslo, Norway · University of Ottawa, Ottawa, ON K1N 6N5, Canada +1

  37. Context-Aware Generative AI for Automated Telecom Test Script Generation

    Jun 19, 2026Gautam Prasad, Chandramohan T. N., Joy BoseTest GenerationMobile Networks

    Ericsson R&D Bangalore

  38. Library-Aware Doubles and Iterative Repair for Large Language Model-Generated Unit Tests in OpenSIL Firmware

    Jun 18, 2026Ma Toan Bach, Yuchi Zheng, Haingo Razafindranto +4Test GenerationChip Design

    School of Software Design and Data Science Seneca Polytechnic Canada · Advanced Micro Devices Canada

  39. All Smoke, No Alarm: Oracle Signals in Agent-Authored Test Code

    Jun 16, 2026Dipayan Banik, Kowshik Chowdhury, Shazibul Islam ShamimCode QualityTest Generation

    Danovo Energy Solutions · Raleigh, North Carolina, USA · Kennesaw State University +1

  40. Cluster-Aware Dual-Level Test Specification Generation for Large-Scale Automotive Software Requirements

    Jun 15, 2026Hazem Ayman, Menna Sedik, Kareem Mostafa +3Test GenerationSummarization

    CairoMotive Cairo, Egypt

  41. Auditing Reward Hackability in Code RL Training Environments

    Jun 14, 2026Shreshth RajanCode QualityVulnerable Code

  42. Structured Testbench Generation for LLM-Driven HDL Design and Verification-Oriented Data Curation

    Jun 11, 2026En-Ming Huang, Yu-Hung Kao, Ren-Hao Deng +10Hierarchical Register Transfer Level GenerationTest Generation

    National Taiwan University · Academia Sinica · Harvard University

  43. PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections

    Jun 10, 2026Pengfei He, Lesly Miculicich, Vishesh Sharma +5Prompt-Injection DetectorsAttacker Large Language Model

    Google Cloud AI Research · Google · Michigan State University

  44. AI-Driven Test Case Generation from Natural Language Requirements: A Survey of Techniques and Research Gaps

    Jun 4, 2026Orimoloye Folorunsho, Hassan RezaTest GenerationNatural Language

    School of Electrical Engineering and Computer Science University of North Dakota, Grand Forks, ND 58201, USA

  45. Latent Anchor-Driven Test Generation for Deep Neural Networks

    Jun 3, 2026Bin Duan, Matthew B. Dwyer, Guowei YangTest GenerationLatent Space

    School of Electrical Engineering and Computer Science, The University of Queensland, Australia · Department of Computer Science, University of Virginia, United States

  46. FATE-VLA:Failue-aware test generation for vision-language-action models

    Jun 1, 2026Arusa Kanwal, Pablo Valle, Shaukat Ali +1Vision-Language-Action FrameworkRobot Policies

    Mondragon University Mondragon, Spain · Simula Research Laboratory Oslo, Norway

  47. FVSpec: Real-World Property-Based Tests as Lean Challenges

    May 31, 2026Quinn Dougherty, Max von Hippel, Simon Henniger +2Formal VerificationTest Generation

    Forall R&D, work completed at Galois Inc · work completed at Galois Inc · Some work completed at Benchify +2

  48. GenPT: Beyond Self-Report for Reliable LLM Psychometrics via Generative Projective Testing

    May 30, 2026Ming Wang, Shuang Wu, Bixuan Wang +7Psychometric PropertiesPsychology

    School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China · Mental Health Education Center, Northeastern University, Shenyang 110819, China · School of Psychology, Northeast Normal University, Changchun 130024, China +3

  49. GUITestScape: Towards Open-set Evaluation on Exploratory GUI Testing

    May 28, 2026Xiaoyi Chen, Yifei Gao, Yang Xu +3Graphical User InterfaceTest Generation

    Beijing Jiaotong University · Independent Researcher

  50. STAB: Specification-driven Testing for Algorithmic Bottlenecks

    May 27, 2026Soohan Lim, Joonghyuk Hahn, Hyundong Jin +1Test GenerationBottlenecks

    Yonsei University, Seoul, Republic of Korea

  51. AssertLLM2: A Comprehensive LLM Benchmark for Assertion Generation from Design Specifications

    May 26, 2026Yuchao Wu, Wenji Fang, Jing Wang +3Systemverilog AssertionsHigh-Level Synthesis

    Hong Kong University of Science and Technology (HKUST)

  52. Inverting the Shield: Systematically Generating Safety Tests from Policy Specifications

    May 24, 2026Xiaoyue Lu, Xianglin Yang, Haijun Liu +4Artificial Intelligence SafetyTest Generation

    Shenzhen Campus of Sun Yat-sen University, Shenzhen, China · National University of Singapore, Singapore · Independent Researcher

  53. SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering?

    May 21, 2026Yuxuan Sun, Yuze Zhao, Yufeng Wang +6Test GenerationSwe-Bench Verified

    1State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China · 2Independent Researcher. · 3Beihang University. +3

  54. Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generation

    May 15, 2026Jinuk Kim, Junsoo Byun, Donghwi Hwang +2Test GenerationChip Design

    Department of Computer Science and Engineering, Seoul National University · Neural Processing Research Center · Samsung Electronics Co., Ltd.

  55. Autonomous Intelligent Agents for Natural-Language-Driven Web Execution with Integrated Security Assurance

    May 14, 2026Vinil Pasupuleti, Siva Rama Krishna Varma Bayyavarapu, Shrey TyagiPenetration TestingBrowser Agent

    International Business Machines (IBM) · South Carolina, United States · Docusign +4

  56. Given, When, Then, Again: Mining Subscenario Refactoring Candidates in Behaviour-Driven Test Suites with ML Classifiers and LLM-Judge Baselines

    May 14, 2026Ali Hassaan Mughal, Noor Fatima, Muhammad BilalCode RefactoringTest Generation

    Independent Researcher; Applied MBA (Data Analytics), Texas Wesleyan University, Fort Worth, TX, USA. · Independent Researcher; B.E. Computer Engineering, National University of Sciences and Technology (NUST), Pakistan. · Independent Researcher; M.Sc. Management, Technical University of Munich, Munich, Germany.

  57. PBT-Bench: Benchmarking AI Agents on Property-Based Testing

    May 13, 2026Lucas Jing, Xinqi Wang, Liao Zhang +1Agentic BenchmarksTest Generation

    Tsinghua University · University of Washington · Beneficial AI Foundation +1

  58. CA2: Code-Aware Agent for Automated Game Testing

    May 13, 2026Valliappan Chidambaram Adaikkappan, Vincent Martineau, Joshua Romoff +1Test GenerationCoding Agents

    † McGill University · ⋄Mila Quebec AI Institute · ‡ Ubisoft

  59. Cochise: A Reference Harness for Autonomous Penetration Testing

    May 12, 2026Andreas Happe, Jürgen CitoPenetration TestingTest Generation

    TU Wien · Vienna, Austria

  60. Arcane: An Assertion Reduction Framework through Semantic Clustering and MCTS-Guided Rule Exploring

    May 11, 2026Hongqin Lyu, Yonghao Wang, Zhiteng Chao +2Systemverilog AssertionsHigh-Level Synthesis

    State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, China · University of Chinese Academy of Sciences, Beijing, China