cs.SEApr 12, 2026

DynamicsLLM: a Dynamic Analysis-based Tool for Generating Intelligent Execution Traces Using LLMs to Detect Android Behavioural Code Smells

Authors: Houcine Abdelkader Cherief, Florent Avellaneda, Naouel Moha

Organizations: École de technologie supérieure Montréal, Québec, Canada · Université du Québec à Montréal Montréal, Québec, Canada

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Mobile apps have become essential of our daily lives, making code quality a critical concern for developers. Behavioural code smells are characteristics in the source code that induce inappropriate code behaviour during execution, which negatively impact software quality in terms of performance, energy consumption, and memory. Dynamics, the latest state-of-the-art tool-based method, is highly effective at detecting Android behavioural code smells. While it outperforms static analysis tools, it suffers from a high false negative rate, with multiple code smell instances remaining undetected. Large Language Models (LLMs) have achieved notable advances across numerous research domains and offer significant potential for generating intelligent execution traces, particularly for detecting behavioural code smells in Android mobile applications. By intelligent execution trace, we mean a sequence of events generated by specific actions in a way that triggers the identification of a given behaviour. We propose the following three main contributions in this paper: (1) DynamicsLLM, an enhanced implementation of the Dynamics method that leverages LLMs to intelligently generate execution traces. (2) A novel hybrid approach designed to improve the coverage of code smell-related events in applications with a small number of activities. (3) A comprehensive validation of DynamicsLLM on 333 mobile applications from F-DROID, including a comparison with the Dynamics tool. Our results show that, under a limited number of actions, DynamicsLLM configured with 100% LLM covers three times more code smell-related events than Dynamics. The hybrid approach improves LLM coverage by 25.9% for apps containing few activities. Moreover, 12.7% of the code smell-related events that cannot be triggered by Dynamics are successfully triggered by our tool.

Figures & tables

Figure 1. DroidAgent process DroidAgent process. The workflow starts with the Planning step, where a task is selected. In the Acting step, the agent performs UI actions. The Observer provides feedback to the Actor, which performs additional UI actions until the task is terminated. The Reflector then provides a task review to the Planner. This process repeats iteratively.
Figure 2. The different steps of the Dynamics Method and the DynamicsLLM Tool. Boxes represent steps, arrows connect the inputs and outputs of each step, described by dotted boxes. Green dotted boxes highlight new or modified elements compared to the Dynamics tool. DynamicsLLM has four steps: Specification, Processing, Execution, and Detection. The Specification step defines code smells and generates LTL properties along with the events associated with each code smell. The Processing step takes the APK file of the app and the code-smell events as input. It produces two outputs: the code smell-related events and the instrumented APK. The Execution step uses the outputs from Processing along with an input generator to produce execution traces. During execution, DynamicsLLM provides the input generator with information about triggered and newly triggered code smell-related events. The final step, Detection, takes the execution traces as input and verifies the LTL properties to determine whether a code smell has been detected.
Figure 7. Number of code smell-related events covered by each model. The dotted line indicates that the timeout was reached for certain executions. Comparison of large language models showing that GPT-4o triggers the highest number of code smell-related events (250 events), followed by Mistral-Small-3.1-24B with approximately 200 events. Other models trigger fewer events.
Code SmellHybrid100% LLM
DW1 / 8 (12%)2 / 8 (25%)
HP8 / 13 (61%)8 / 13 (61%)
IOD2 / 7 (28%)3 / 7 (42%)
NLMR1033 / 1057 (97%)1033 / 1057 (97%)
HMU154 / 210 (73%)155 / 210 (73%)
Table 1. Recall for code smell detection
Figure 8. Code smell-related events covered by each tool. The dotted line indicates that certain executions ended before. Coverage results grouped into three categories based on the number of activities per app: apps with at most four activities, apps with five to ten activities, and apps with eleven or more activities. For the first category, Hybrid triggers the highest number of events (over 326), followed by MonkeyRunner (around 285) and 100% LLM (259). In the second category, 100% LLM triggers the most events (253), followed by Hybrid (227) and MonkeyRunner (217). In the third category, the three configurations trigger a similar number of events, with MonkeyRunner (270), 100% LLM (271), and Hybrid (249).
Figure 9. Number of code smell-related events covered by each tool by the number of actions. The dotted line indicates that the timeout was reached for certain executions. Total number of triggered events for each configuration: 100% LLM with 868 events, Hybrid with 802 events, and MonkeyRunner with 772 events.
 In MonkeyRunnerNot in MonkeyRunner
In 100% LLM559309
Not in 100% LLM2133356
In Hybrid578224
Not in Hybrid1943441
Table 2. Code Smell-related Events contingency table between DynamicsLLM configurations and MonkeyRunner
More in this topic

Automated Software Testing

Automated software testing uses tools and algorithms to generate, run, and evaluate tests that find defects and prevent regressions.

See all 135 papers

Explore similar work

CardsList
May 21, 2026

LLM Code Smells: A Taxonomy and Detection Approach

Large Language Models (LLMs) are increasingly integrated into software systems for diverse purposes, due to their versatility, flexibility, and ability to simulate human reasoning to some extent. However, poor integration of LLM inference in source code can undermine software system quality. Therefore, inadequate LLM integration coding practices must be documented to help developers mitigate such issues. Following our earlier work on LLM code smells, this paper consolidates and refines the concept by presenting a self-contained taxonomy and a catalog of nine LLM code smells. We also create SpecDetect4LLM, a static source code analysis tool for their detection, and conduct extensive empirical evaluations of its detection effectiveness (precision and recall) as well as the prevalence of LLM code smells across 692 open-source software projects (171,194 source files). Our results show that LLM code smells affect 73.5% of the analyzed systems, with a detection precision of 91.3% and a recall of 71.8%.

Zacharie Chenail-Larcher, Brahim Mahmoudi, Naouel Moha +2

´Ecole de technologie sup´erieure, Montr´eal, Qu´ebec, Canada. · Universit´e du Qu´ebec `a Montr´eal, Montr´eal, Qu´ebec, Canada.

Static Code Analysis
AskPDF
Oct 3, 2025

Investigating The Smells of LLM Generated Code

Context: Large Language Models (LLMs) are increasingly being used to generate program code. Much research has been reported on the functional correctness of generated code, but there is far less on code quality. Objectives: In this study, we propose a scenario-based method of evaluating the quality of LLM-generated code to identify the weakest scenarios, for which the quality of LLM-generated code should be improved. Methods: The method measures code smells, an important indicator of code quality, and compares them with a baseline formed from reference solutions of professionally written code. The test dataset is divided into various subsets according to the topics of the code and complexity of the coding tasks to represent different scenarios of using LLMs for code generation. We will also present an automated test system for this purpose and report experiments with the Java programs generated in response to prompts given to four state-of-the-art LLMs: Gemini Pro, ChatGPT, Codex, and Falcon. Results: We find that LLM-generated code has a higher incidence of code smells compared to reference solutions. Falcon performed the least badly, with a smell increase of 42.28%, followed by Gemini Pro (62.07%), ChatGPT (65.05%) and finally Codex (84.97%). The average smell increase across all LLMs was 63.34%, comprising 73.35% for implementation smells and 21.42% for design smells. We also found that the increase in code smells is greater for more complex coding tasks and for more advanced topics, such as those involving object-orientated concepts. Conclusion: In terms of code smells, LLMs' performances on various coding task complexities and topics are highly correlated to the quality of human-written code in the corresponding scenarios. However, the quality of LLM-generated code is noticeably poorer than human-written code.

Debalina Ghosh Paul, Hong Zhu, Ian Bayley

School of Engineering, Computing and Mathematics, Oxford Brookes University, Oxford, OX3 0BP, UK

Software EngineeringCode Generation EvaluationLLM Code Generation
AskPDF
Mar 26, 2026

MobileDev-Bench: A Benchmark for Issue Resolution in Mobile Application Development

Large language models (LLMs) have shown strong performance on automated software engineering tasks, yet existing benchmarks focus primarily on library-style repositories, leaving mobile application development largely unexplored despite its framework-specific build systems, heterogeneous artifact types, and coordinated multi-file fix requirements. We introduce MobileDev-Bench, a benchmark comprising 407 real-world issue-resolution tasks collected from 19 production mobile applications spanning Android Native (Java/Kotlin), React Native (TypeScript), and Flutter (Dart). Each task pairs a verified developer-reported issue with executable test patches, enabling fully automated validation of model-generated fixes within mobile build environments. The benchmark exhibits substantially greater patch complexity than prior benchmarks: fixes modify 12.9 files and 334.6 lines on average, and 41% of instances require coordinated changes across multiple artifact types, such as source, build configuration, and resource files. Evaluation of four frontier LLMs (Claude Sonnet 4.5, Qwen3-Coder, GPT-5.2, and Gemini 2.5 Flash) yields end-to-end resolution rates of only 3.23% - 4.23% under automated retrieval and at most 5.69% under oracle retrieval, well below resolution rates reported on existing benchmarks. We release MobileDev-Bench with task instances, an evaluation harness, and containerized environments to support reproducible research on AI-assisted mobile application development.

Moshood A. Fakorede, Krishna Upadhyay, A. B. Siddique +1

Louisiana State University · University of Kentucky

Software EngineeringSoftware Engineering BenchmarksAutomated Software EngineeringBenchmark Design
AskPDF