cs.SENov 11, 2026

ADAPT: Agentic Diff-Aware Patch Testing for Pull Requests

Authors: Vahid Haratian, Atakan Akar, Eray Tüzün

Organizations: Bilkent University, Ankara, Turkey

Abstract

Modern software development relies on pull requests (PRs) that introduce small, targeted code changes. Once merged, these changes should be protected by focused tests that exercise the behavior the PR introduces, yet developers often submit PRs without such tests, leaving the new behavior unguarded against future regressions. Most test generation techniques target entire software units and optimize objectives such as code coverage rather than the behavior a change introduces, and prior diff-aware approaches have had limited success in capturing it. We propose ADAPT (Agentic Diff-Aware Patch Testing), a regression test generation approach that uses an autonomous coding agent to produce change-focused tests for PRs. Given a PR, the agent analyzes the diff between the base and head versions to identify the modified functions and their context, then plans and generates candidate unit tests. A dedicated fail-to-pass (F2P) verification tool runs each candidate on both versions and, after F2P success, reports which executable changed lines remain uncovered; guided by this feedback, the agent iteratively refines its tests. We evaluate ADAPT on the same 426 real-world PRs used by the prior state-of-the-art baseline, with GPT-4o and GPT-5-mini. ADAPT+GPT-4o reaches an F2P rate of 62.0% (264/426), compared with BLAST's reported 35.4% (151/426), while ADAPT+GPT-5-mini reaches 89.9% (383/426). On 137 PRs shared with Change And Cover (ChaCo), ADAPT achieves full patch coverage on 83.9% of PRs, compared with 79.6% for ChaCo, at considerably lower time and cost. A manual evaluation confirms that the F2P-valid tests target each PR's intended change, with mean scores of up to 4.54/5 for importance, 4.35/5 for assertion strength, and 4.49/5 for readability, often covering behavior beyond the developer-written tests.

Figures & tables

Explore similar work

CardsList
  1. All Smoke, No Alarm: Oracle Signals in Agent-Authored Test Code

    Jun 16, 2026Dipayan Banik, Kowshik Chowdhury, Shazibul Islam ShamimAutomated Software TestingAI Coding Agents

  2. Evaluating LLM-Based Regression Test Generation

    Jan 19, 2025Jing Liu, Seongmin Lee, Eleonora Losiouk +1Automated Test GenerationLLM Code Generation

  3. SWEPatchTest: Evaluating Software Tests Beyond Reproduction

    Oct 13, 2026Hua Lu, Pengcheng Wang, Han Li +4Software Engineering BenchmarksAutomated Test Generation