Taking a Pulse on How Generative AI is Reshaping the Software Engineering Research Landscape
Authors: Bianca Trinkenreich, Fabio Calefato, Kelly Blincoe, Viggo Tellefsen Wivestad, Antonio Pedro Santos Alves, Júlia Condé Araújo, Marina Condé Araújo, Paolo Tell, +3 more
Organizations: Colorado State University, Fort Collins, CO, USA · University of Bari, Bari, Italy · University of Auckland, Auckland, New Zealand · SINTEF Digital, Trondheim, Norway · PUC-Rio, Rio de Janeiro, Brazil · IT University of Copenhagen, Copenhagen, Denmark · University of California Irvine, Irvine, CA, USA · University of Victoria, Victoria, Canada
Context: Software engineering (SE) researchers increasingly study Generative AI (GenAI) while also incorporating it into their own research practices. Despite rapid adoption, there is limited empirical evidence on how GenAI is used in SE research and its implications for research practices and governance. Aims: We conduct a large-scale survey of 457 SE researchers publishing in top venues between 2023 and 2025. Method: Using quantitative and qualitative analyses, we examine who uses GenAI and why, where it is used across research activities, and how researchers perceive its benefits, opportunities, challenges, risks, and governance. Results: GenAI use is widespread, with many researchers reporting pressure to adopt and align their work with it. Usage is concentrated in writing and early-stage activities, while methodological and analytical tasks remain largely human-driven. Although productivity gains are widely perceived, concerns about trust, correctness, and regulatory uncertainty persist. Researchers highlight risks such as inaccuracies and bias, emphasize mitigation through human oversight and verification, and call for clearer governance, including guidance on responsible use and peer review. Conclusion: We provide a fine-grained, SE-specific characterization of GenAI use across research activities, along with taxonomies of GenAI use cases for research and peer review, opportunities, risks, mitigation strategies, and governance needs. These findings establish an empirical baseline for the responsible integration of GenAI into academic practice.
Figures & tables
Research Questions
Questionnaire Item. [Analysis]
RQ1. Who is using GenAI in SE research and what is their motivation?
RQ1.1. Who (which researchers in SE) is using GenAI for their research?
Demographics [Quantitative]
RQ1.2. What motivates researchers to use GenAI?
How much of an impact do you think GenAI has or will have on SE Research? [Quantitative]
Table 1. Research questions with corresponding questionnaire items, analysis method [ Quantitative | Qualitative ], and reference to the source for adapted items.
Figure 1. GenAI Activity Distribution
Demographics
Category
n
Sample (%)
Use GenAI for Research (%)
Continental Origin
Europe
111
47
73
North America
64
27
84
Asia
41
17
93
South America
17
7
65
Oceania
5
2
60
Gender
Man
183
72
80
Table 2. Demographics of survey respondents. n is the number of respondents in each category. Sample (%) is the percentage of respondents in that category relative to all valid responses for that demographic variable (i.e., within each demographic variable, the percentages sum to 100%). Use GenAI for Research (%) is the percentage of respondents within that category who indicated they used Generative AI for software engineering research (Q1). For Organization, respondents could select multiple options; therefore, the summed counts across its categories can exceed the totals for single-choice demographics.
Figure 2. Perceived impact (past and future) of GenAI on SE research
Figure 3. Pressures related to GenAI
Figure 4. Where GenAI is being used. Research strategies and methods in the Y axis and research pipeline stages in the X axis.
Figure 5. GenAI use cases categorized using Andersen et al.’s framework ( andersen2025generative ) . We mark with (*) the categories and use cases that emerged from our data and are not part of ( andersen2025generative ) .
Category
Representative examples
# mentions
% (n=243)
Idea Generation
“ask ChatGPT about my research ideas and let it help me analyse them" (R287) "generate summaries of papers already published" (R13)
52
21%
Research Design
“Checking my memory on research methods." (R13)
9
4%
Data Collection
“as a subject of my research" (R39) “produce pilot datasets and scenarios to evaluate research protocols" (R202)
2
1%
Data Analysis
“code and analyze interview transcripts and survey responses" (R6), “making annotations on data […] and open coding" (R47) “help in writing scripts for evaluation purposes" (R75)
59
24%
Writing and Reporting
“review of research papers that I did; like, possible gaps and inconsistencies" (R71), “when I am writing a report or paper, I seek feedback on the clarity and accuracy of my language." (R217) “refine my writing (e.g., grammar checks, clarity of the sentences)" (R317)
94
39%
Peer Reviewing (**)
“revising difficult texts like rejection emails for careful tone" (R71), “improving text when writing [..] reviews" (R167)
4
2%
Table 3. Representative examples of answers to how GenAI is being used in SE research, number and percentage of use cases whose answer was coded for each category. We use (**) to represent new categories that are not part of Andersen et al.’s framework ( andersen2025generative ) .
Figure 6. The Use Cases of GenAI for Peer Review.
Figure 7. Perceived benefits of using SE research
Figure 8. Challenges faced when using GenAI
Figure 9. GenAI opportunities categorized using Andersen et al.’s framework ( andersen2025generative ) . We marked as (*) the categories and opportunities that emerged from our data and were not part of ( andersen2025generative ) and bold boxes as the opportunities that were not mentioned as current use cases in Section 4.2.2 .
Category
Representative examples
# mentions
% (n=266)
Idea Generation
“can be great to kickstart a literature search, summarize papers to see if they are relevant and warrant a deeper look, and similar tasks" (R63)
37
14%
Research Design
“Faster experimentation with techniques that a researcher is not familiar with by specifying the intended research goal as a prompt" (R201)
8
3%
Data Collection
“generating synthetic data" (R187)
7
3%
Data Analysis
“generate quick tools and scripts for analyzing data" (R82)
22
8%
Writing and Reporting
“Help the researcher to go through multiple documents in less time and express the ideas in better English" (R340)
19
7%
Peer Reviewing (**)
“GenAI can help with a fairer allocation of paper review tasks for SE conferences based on reviewers’ prior publications" (R17)
1
0%
Table 4. Representative examples of opportunities GenAI bring to SE research, number and percentage of use cases whose answer was coded for each category. We use (**) to represent new categories that were not part of Andersen et al.’s framework ( andersen2025generative )
Figure 10. The Different Dimensions of Trust in GenAI for SE Research (based on ( Choudhuri et al., 2025 ) )
Figure 11. Trust in GenAI per Stage in the SE Research Pipeline (based on ( Trinkenreich et al., 2025 ) )
Figure 12. Perceived risks of using GenAI for SE research
Figure 13. The mitigation strategies reported by SE researchers who participated in our study.
Mitigation Strategy
Representative examples
#
% (n=138)
Human in the Loop
“AI does not substitute the human, who should inspect everything" (R62) “The researcher has the final word. They have to review what the GenAI generates truly. We cannot blindly trust it." (R135)
90
65.2%
Usage Boundaries
“Researchers should use GenAI as a tool to help with specific, well-defined tasks [..] They should not attempt to "offload at the edge" of knowledge tasks to a text completion engine." (R42)
11
8.0%
Technical Safeguards
“RAG-based usage," (R164), “agentic systems to assist in verifying research," (R138) “use separate conversation thread and modify the question to see if the answer changes" (R172)
13
9.4%
Transparency
“Disclose the usage of AI in research in more detail," (R427), “mak[ing] it explicit which parts of the research were supported by them (e.g., Threats to Validity section)" (R198)
17
5.9%
Education
“Tell and demonstrate [researchers] what happens if [they] use GenAI incorrectly, (R107) “Promote ethical behaviour. Encourage the detection (and public report) of published materials corresponding to illustrative cases of misinformation, falsification, and plagiarism." (R159)
14
10.1%
Governance Controls
“Open science and replication requirements, (R74) “design new trustworthy assurance and evaluation methodolygy" (R15)
4
2.9%
Table 5. Representative examples of answers to the risk mitigation open question, number and percentage of respondents whose answer was coded for each category.
Generative Artificial Intelligence (GenAI) rapidly transforms software engineering, yet existing research remains fragmented across individual tasks in the Software Development Lifecycle. This study integrates a systematic literature review with a survey of 65 software developers. The results show that GenAI exerts its highest impact in design, implementation, testing, and documentation, where over 70 % of developers report at least halving the time for boilerplate and documentation tasks. 79 % of survey respondents use GenAI daily, preferring browser-based Large Language Models over alternatives integrated directly in their development environment. Governance is maturing, with two-thirds of organizations maintaining formal or informal guidelines. In contrast, early SDLC phases such as planning and requirements analysis show markedly lower reported benefits. In a nutshell, GenAI shifts value creation from routine coding toward specification quality, architectural reasoning, and oversight, while risks such as uncritical adoption, skill erosion, and technical debt require robust governance and human-in-the-loop mechanisms.
Vincent Gurgul, Robin Gubela, Stefan Lessmann
Chair of Information Systems Humboldt-Universität zu Berlin Unter den Linden 6, 10117 Berlin · Chair of Information Systems Hochschule für Technik und Wirtschaft Berlin Treskowallee 8, 10318 Berlin · Bucharest University of Economic Studies 6 Piata Romana, 010374, Romania
Generative AI (GenAI) is reshaping software engineering by shifting development from manual coding toward agent-driven implementation. While vibe coding promises rapid prototyping, it often suffers from architectural drift, limited traceability, and reduced maintainability. Applying the design science research (DSR) methodology, this paper proposes Shift-Up, a framework that reinterprets established software engineering practices, like executable requirements (BDD), architectural modeling (C4), and architecture decision records (ADRs), as structural guardrails for GenAI-native development. Preliminary findings from our exploratory evaluation compare unstructured vibe coding, structured prompt engineering, and the Shift-Up approach in the development of a web application. These findings indicate that embedding machine-readable requirements and architectural artifacts stabilizes agent behavior, reduces implementation drift, and shifts human effort toward higher-level design and validation activities. The results suggest that traditional software engineering artifacts can serve as effective control mechanisms in AI-assisted development.
Petrus Lipsanen, Liisa Rannikko, François Christophe +3
The post-ChatGPT surge has rapidly reframed IS research and practice. As organizations and society grapple with GenAI adoption, a body of secondary studies and research agendas has emerged to synthesize early evidence and chart directions for future inquiry. This study reviews secondary and roadmap papers to synthesize the state of knowledge on GenAI's benefits and challenges in IS, and to identify future research directions. We performed a systematic search across Scopus, WoS, and eAIS for publications from 2023 onwards. Following a rigorous, multi-stage screening process, we selected a final set of 28 papers for analysis using bibliometric mapping and thematic analysis. We also conducted a quality assessment of all sources to gauge confidence in each source's contribution to the findings. GenAI offers transformative potential to drive productivity, accelerate innovation, personalize services, and democratize access to expertise. However, its adoption is constrained by interrelated challenges: technical unreliability, societal-ethical risks, and a governance vacuum. Interpreted through a socio-technical lens, our findings reveal a persistent misalignment between GenAI's fast-evolving technical subsystem and the slower-adapting social subsystem, positioning IS research as critical for achieving joint optimization. To bridge this gap, we propose a research agenda that reorients IS scholarship from analyzing impacts toward actively shaping the co-evolution of technical capabilities with organizational routines, societal values, and regulatory institutions: emphasizing hybrid human-AI ensembles, situated validation, design principles for probabilistic systems, and adaptive governance. For practitioners and policymakers, responsible adoption requires balancing automation with human augmentation alongside transparent governance and adaptive regulations to ensure broadly shared benefits.
Aleksander Jarzębowicz, Adam Przybyłek, Jacinto Estima +14