Taking a Pulse on How Generative AI is Reshaping the Software Engineering Research Landscape
Authors: Bianca Trinkenreich, Fabio Calefato, Kelly Blincoe, Viggo Tellefsen Wivestad, Antonio Pedro Santos Alves, Júlia Condé Araújo, Marina Condé Araújo, Paolo Tell, +3 more
Organizations: Colorado State University, Fort Collins, CO, USA · University of Bari, Bari, Italy · University of Auckland, Auckland, New Zealand · SINTEF Digital, Trondheim, Norway · PUC-Rio, Rio de Janeiro, Brazil · IT University of Copenhagen, Copenhagen, Denmark · University of California Irvine, Irvine, CA, USA · University of Victoria, Victoria, Canada
Context: Software engineering (SE) researchers increasingly study Generative AI (GenAI) while also incorporating it into their own research practices. Despite rapid adoption, there is limited empirical evidence on how GenAI is used in SE research and its implications for research practices and governance. Aims: We conduct a large-scale survey of 457 SE researchers publishing in top venues between 2023 and 2025. Method: Using quantitative and qualitative analyses, we examine who uses GenAI and why, where it is used across research activities, and how researchers perceive its benefits, opportunities, challenges, risks, and governance. Results: GenAI use is widespread, with many researchers reporting pressure to adopt and align their work with it. Usage is concentrated in writing and early-stage activities, while methodological and analytical tasks remain largely human-driven. Although productivity gains are widely perceived, concerns about trust, correctness, and regulatory uncertainty persist. Researchers highlight risks such as inaccuracies and bias, emphasize mitigation through human oversight and verification, and call for clearer governance, including guidance on responsible use and peer review. Conclusion: We provide a fine-grained, SE-specific characterization of GenAI use across research activities, along with taxonomies of GenAI use cases for research and peer review, opportunities, risks, mitigation strategies, and governance needs. These findings establish an empirical baseline for the responsible integration of GenAI into academic practice.
Figures & tables
Research Questions
Questionnaire Item. [Analysis]
RQ1. Who is using GenAI in SE research and what is their motivation?
RQ1.1. Who (which researchers in SE) is using GenAI for their research?
Demographics [Quantitative]
RQ1.2. What motivates researchers to use GenAI?
How much of an impact do you think GenAI has or will have on SE Research? [Quantitative]
Table 1. Research questions with corresponding questionnaire items, analysis method [ Quantitative | Qualitative ], and reference to the source for adapted items.
Figure 1. GenAI Activity Distribution
Demographics
Category
n
Sample (%)
Use GenAI for Research (%)
Continental Origin
Europe
111
47
73
North America
64
27
84
Asia
41
17
93
South America
17
7
65
Oceania
5
2
60
Gender
Man
183
72
80
Table 2. Demographics of survey respondents. n is the number of respondents in each category. Sample (%) is the percentage of respondents in that category relative to all valid responses for that demographic variable (i.e., within each demographic variable, the percentages sum to 100%). Use GenAI for Research (%) is the percentage of respondents within that category who indicated they used Generative AI for software engineering research (Q1). For Organization, respondents could select multiple options; therefore, the summed counts across its categories can exceed the totals for single-choice demographics.
Figure 2. Perceived impact (past and future) of GenAI on SE research
Figure 3. Pressures related to GenAI
Figure 4. Where GenAI is being used. Research strategies and methods in the Y axis and research pipeline stages in the X axis.
Figure 5. GenAI use cases categorized using Andersen et al.’s framework ( andersen2025generative ) . We mark with (*) the categories and use cases that emerged from our data and are not part of ( andersen2025generative ) .
Category
Representative examples
# mentions
% (n=243)
Idea Generation
“ask ChatGPT about my research ideas and let it help me analyse them" (R287) "generate summaries of papers already published" (R13)
52
21%
Research Design
“Checking my memory on research methods." (R13)
9
4%
Data Collection
“as a subject of my research" (R39) “produce pilot datasets and scenarios to evaluate research protocols" (R202)
2
1%
Data Analysis
“code and analyze interview transcripts and survey responses" (R6), “making annotations on data […] and open coding" (R47) “help in writing scripts for evaluation purposes" (R75)
59
24%
Writing and Reporting
“review of research papers that I did; like, possible gaps and inconsistencies" (R71), “when I am writing a report or paper, I seek feedback on the clarity and accuracy of my language." (R217) “refine my writing (e.g., grammar checks, clarity of the sentences)" (R317)
94
39%
Peer Reviewing (**)
“revising difficult texts like rejection emails for careful tone" (R71), “improving text when writing [..] reviews" (R167)
4
2%
Table 3. Representative examples of answers to how GenAI is being used in SE research, number and percentage of use cases whose answer was coded for each category. We use (**) to represent new categories that are not part of Andersen et al.’s framework ( andersen2025generative ) .
Figure 6. The Use Cases of GenAI for Peer Review.
Figure 7. Perceived benefits of using SE research
Figure 8. Challenges faced when using GenAI
Figure 9. GenAI opportunities categorized using Andersen et al.’s framework ( andersen2025generative ) . We marked as (*) the categories and opportunities that emerged from our data and were not part of ( andersen2025generative ) and bold boxes as the opportunities that were not mentioned as current use cases in Section 4.2.2 .
Category
Representative examples
# mentions
% (n=266)
Idea Generation
“can be great to kickstart a literature search, summarize papers to see if they are relevant and warrant a deeper look, and similar tasks" (R63)
37
14%
Research Design
“Faster experimentation with techniques that a researcher is not familiar with by specifying the intended research goal as a prompt" (R201)
8
3%
Data Collection
“generating synthetic data" (R187)
7
3%
Data Analysis
“generate quick tools and scripts for analyzing data" (R82)
22
8%
Writing and Reporting
“Help the researcher to go through multiple documents in less time and express the ideas in better English" (R340)
19
7%
Peer Reviewing (**)
“GenAI can help with a fairer allocation of paper review tasks for SE conferences based on reviewers’ prior publications" (R17)
1
0%
Table 4. Representative examples of opportunities GenAI bring to SE research, number and percentage of use cases whose answer was coded for each category. We use (**) to represent new categories that were not part of Andersen et al.’s framework ( andersen2025generative )
Figure 10. The Different Dimensions of Trust in GenAI for SE Research (based on ( Choudhuri et al., 2025 ) )
Figure 11. Trust in GenAI per Stage in the SE Research Pipeline (based on ( Trinkenreich et al., 2025 ) )
Figure 12. Perceived risks of using GenAI for SE research
Figure 13. The mitigation strategies reported by SE researchers who participated in our study.
Mitigation Strategy
Representative examples
#
% (n=138)
Human in the Loop
“AI does not substitute the human, who should inspect everything" (R62) “The researcher has the final word. They have to review what the GenAI generates truly. We cannot blindly trust it." (R135)
90
65.2%
Usage Boundaries
“Researchers should use GenAI as a tool to help with specific, well-defined tasks [..] They should not attempt to "offload at the edge" of knowledge tasks to a text completion engine." (R42)
11
8.0%
Technical Safeguards
“RAG-based usage," (R164), “agentic systems to assist in verifying research," (R138) “use separate conversation thread and modify the question to see if the answer changes" (R172)
13
9.4%
Transparency
“Disclose the usage of AI in research in more detail," (R427), “mak[ing] it explicit which parts of the research were supported by them (e.g., Threats to Validity section)" (R198)
17
5.9%
Education
“Tell and demonstrate [researchers] what happens if [they] use GenAI incorrectly, (R107) “Promote ethical behaviour. Encourage the detection (and public report) of published materials corresponding to illustrative cases of misinformation, falsification, and plagiarism." (R159)
14
10.1%
Governance Controls
“Open science and replication requirements, (R74) “design new trustworthy assurance and evaluation methodolygy" (R15)
4
2.9%
Table 5. Representative examples of answers to the risk mitigation open question, number and percentage of respondents whose answer was coded for each category.
Chair of Information Systems Humboldt-Universität zu Berlin Unter den Linden 6, 10117 Berlin · Chair of Information Systems Hochschule für Technik und Wirtschaft Berlin Treskowallee 8, 10318 Berlin · Bucharest University of Economic Studies 6 Piata Romana, 010374, Romania