Search-Based Software Engineering and AI Foundation Models: Current Landscape and Future Roadmap
Authors: Hassan Sartaj, Shaukat Ali, Paolo Arcaini, Andrea Arcuri
Organizations: Simula Research Laboratory Oslo, Norway · National Institute of Informatics Tokyo, Japan · Kristiania University of Applied Sciences and Oslo Metropolitan University Oslo, Norway
Search-based software engineering (SBSE), which integrates metaheuristic search techniques with software engineering, has been an active area of research for about 25 years. It has been applied to solve numerous problems across the entire software engineering lifecycle and has demonstrated its versatility in multiple domains. With recent advances in Artificial Intelligence (AI), particularly the emergence of foundation models (FMs) such as large language models (LLMs), the evolution of SBSE alongside these models remains undetermined. In this window of opportunity, we present a research roadmap that articulates the current landscape of SBSE in relation to FMs, identifies open challenges, and outlines potential research directions to advance SBSE through its synergy with FMs. Specifically, we analyze three core aspects: utilizing FMs to enhance SBSE, applying SBSE to advance FMs, and exploring the integration of SBSE and FMs. Furthermore, we present a forward-thinking perspective that envisions the future of SBSE in the era of FMs, highlighting promising research opportunities to address challenges in emerging domains.
Figures & tables
Figure 1: Roadmap overview showing the discussion flow: strengths and weaknesses ( Section 3 ), SBSE–FM synergies ( Sections 4 , 5 and 6 ), empirical evaluation challenges ( Section 7 ), and the 2030 research horizon ( Section 8 ).
Figure 2: Overview of the methodology used to develop the proposed roadmap.
Figure 3: Key aspects of the potential synergy between SBSE and FMs. The abbreviations used are: FMs (Foundation Models), SBSE (Search-Based Software Engineering), and SE (Software Engineering).
Figure 4: Key aspects of employing FMs to enhance SBSE. The abbreviations used are: FMs (Foundation Models), LLMs (Large Language Models), VLMs (Vision-Language Models), MMs (Multimodal Models), SBSE (Search-Based Software Engineering), and SE (Software Engineering).
FMs for SBSE Design
✦Automating the design and implementation of fitness functions for SBSE problems.
✦Designing search operators for complex domain problems involving multimodal data.
✦Automating solution encoding for the varying nature of problems.
✦Defining appropriate search spaces for SBSE to balance efficiency and solution quality.
✦Generating partial or complete SBSE implementations to enhance automation and efficiency.
✦Automating the testing, debugging, and repair of SBSE implementations.
Table 1: Key challenges – FMs for SBSE
Figure 5: Key aspects of applying SBSE to enhance FMs. The abbreviations used are: FMs (Foundation Models), SBSE (Search-Based Software Engineering), and SE (Software Engineering).
SBSE for FM Limitations
✦Managing non-determinism in FMs to ensure consistent and reliable outputs.
✦Effectively handling data-related uncertainties in FMs to improve model performance.
✦Detecting and mitigating hallucinations in FMs.
✦Optimizing the computationally expensive fine-tuning process of FMs, particularly for evolving SE requirements.
✦Balancing trade-offs in automated prompt optimization.
SBSE for SE Artifacts Generated with FMs
Table 2: Key challenges – SBSE for FMs
Figure 6: Integration between FMs and SBSE (two-way interactions). The abbreviations used are: FMs (Foundation Models), SBSE (Search-Based Software Engineering), SE (Software Engineering), SDLC (Software Development Lifecycle), and ADS (Autonomous Driving Systems).
Integrating SBSE and FMs
✦Developing FM-inspired search algorithms to enhance initialization, fitness calculation, and solution repair during the search process.
✦Using FMs to support SBSE in traditional SE problems, such as requirement prioritization, fault identification, and repair for SE tasks, including GUI testing.
✦Effectively integrating FMs with SBSE to tackle complex problems in emerging domains.
Table 3: Key challenges – Integration of SBSE and FMs
Empirical Evaluations involving both SBSE and FMs
✦Ensuring fair comparisons between FM-based and SBSE techniques, considering differences in computational resources and energy consumption.
✦Determining appropriate scaling factors for resource allocation to achieve fairness in experiments.
✦Addressing the replicability of experiments with remote, commercial FMs, which can change unpredictably over time.
✦Balancing the use of commercial FMs for best results with local, open-source LLMs for replicability in scientific research.
✦Managing non-determinism in FMs and SBSE techniques through repeated experiments and proper statistical analysis.
✦Establishing detailed guidelines for fair and reliable comparisons of hybrid techniques involving both FMs and SBSE.
Table 4: Key challenges – Empirical Evaluations
Figure 7: An overview of the tetrad illustrating the disruptive effects of FMs on SBSE.