cs.SEAug 14, 2026

Rethinking Automated Program Repair: The Impact of Bug Complexity, Fault Localization, and LLM Cost-efficiency

Authors: Junchi Liu, Ali Bigdeli, Roya Daneshi, Atu Ambala, Sudipto Ghosh, Fabio Santos

Organizations: Colorado State University, United States

Abstract

Background: Software bugs remain a critical challenge in development, necessitating effective Automated Program Repair (APR) techniques. While Large Language Model (LLM)-based APR systems have shown promise, prior studies primarily focus on overall repair effectiveness. The effects of bug complexity, fault localization, reasoning settings, and repair cost-effectiveness remain insufficiently explored. Aims: This study presents a comprehensive empirical analysis of LLM-based APR, focusing on how repair performance is shaped by bug complexity, fault localization, reasoning settings, and costs. Method: We evaluate two APR techniques (ChatRepair and CodeCorrector) using three LLMs (DeepSeek, GPT, and Llama), and examine their performance across diverse levels of bug complexity and localization strategies through a multi-dimensional empirical framework and statistical analysis. Results: Although structurally complex bugs and imprecise fault localization make repair more challenging, LLM-based APR techniques still achieve competitive repair effectiveness. Imprecise fault localization can substantially enlarge the performance gap between APR techniques. Furthermore, higher-cost LLMs and stronger reasoning settings do not consistently yield better cost-efficiency, revealing a nontrivial trade-off between repair effectiveness and computational cost. Conclusions: Over 50% of moderately complex bugs can be repaired by low-cost LLM-based APR techniques. The repair effectiveness gap between APR techniques becomes larger as fault localization becomes less precise. GPT-5 repairs 7 and 39 more complex bugs than DeepSeek-V4-pro and DeepSeek-V3.2, respectively; whereas the total repair cost of DeepSeek-V3.2 shows the best cost-efficiency performance.

Figures & tables

Explore similar work

CardsList
  1. On the Role of Fault Localization Context for LLM-Based Program Repair

    Apr 7, 2026Melika Sepidband, Hung Viet Pham, Hadi HemmatiFault LocalizationAutomated Program Repair

  2. How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair

    Jul 28, 2026Ramtin Ehsani, Irene Manotas, Saurabh Pujar +2Automated Program RepairLLM Interpretability

  3. From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs

    Jun 16, 2025Anh Ho, Thanh Le-Cong, Bach Le +1Automated Program RepairSoftware Engineering Benchmarks