Exploring LLMs for User Story Extraction from Mockups
Authors: Diego Firmenich, Leandro Antonelli, Bruno Pazos, Fabricio Lozada, Leonardo Morales
Organizations: Departamento de Informática, Facultad de Ingeniería, Universidad Nacional de la Patagonia, Argentina · LIFIA, Facultad de Informática, Universidad Nacional de La Plata, Argentina · CAETI, Facultad de Tecnología Informática - Universidad Abierta Interamericana · Laboratorio de Ciencias de la Imágenes, DIEC, UNS · UNIANDES, Universidad Regional Autónoma de los Andes, Ecuador · Instituto Patagónico de Ciencias Sociales y Humanas, CONICET-CENPAT
User stories are one of the most widely used artifacts in the software industry to define functional requirements. In parallel, the use of high-fidelity mockups facilitates end-user participation in defining their needs. In this work, we explore how combining these techniques with large language models (LLMs) enables agile and automated generation of user stories from mockups. To this end, we present a case study that analyzes the ability of LLMs to extract user stories from high-fidelity mockups, both with and without the inclusion of a glossary of the Language Extended Lexicon (LEL) in the prompts. Our results demonstrate that incorporating the LEL significantly enhances the accuracy and suitability of the generated user stories. This approach represents a step forward in the integration of AI into requirements engineering, with the potential to improve communication between users and developers.
Figures & tables
Figure 1: Proposed Collaborative Workflow
Role
Action
Tool
Product
Format
End-user
Defines requirements
High-fidelity mockup tool
WA based Mockup
WA Mockup, Images
Requirements Engineer
Defines the application domain language
LEL edition Tool
LEL
Text
Product Owner
Manages and prioritizes requirements
Kanban, Prompt
Mockup+US
Text, Image
Developer
Implements requirements
Kanban, IDEs
Artifact
Software
Table 1: Roles, Actions, Tools, Products, and Formats in the Proposed Approach
Figure 2: User Creating a High-Fidelity Mockup
Figure 3: User Creating a High-Fidelity Mockup
Figure 4: LeafLab web application displaying floristic species list, ordered by system #id.
Figure 5: Biologist’s mockup creation process for new LeafLab feature.
Figure 6: User story for the biologist mockpup, comparison without LEL (left) and with LEL (right) in the prompt.
Subject
Notion
Behavioral responses
Biologist
User of the application licensed or doctorate in biological sciences and specialized in botany.
The biologist orders the species discovered in the transects by family. The biologist orders the species discovered in the transects by form. The biologist orders the species discovered in the transects by type. The biologist orders the species discovered in the transects by points. The biologist edits the properties of a species.
Table 2: LEL portion for Subject, Notion, Behavioral responses
Object
Notion
Transect
Imaginary straight line of approximately 300 meters, on which biologists sample specimens of biological species. Each transect is composed of 300 points.
Point
At each point of the 300 points of a transect, the biologist takes a sample of a specimen of a biological species found.
Biological Species
Species found in the field campaigns carried out by biologists. Each species may have been found in 0 or more points.
Visit
A traverse of a particular transect. Each traverse of a transect involves walking approximately 300 meters, recording the species of a particular floristic specimen every 3 meters, resulting in 100 points per transect.
Table 3: LEL portion for Objects and Notions
Type of Requirement
Score without LEL
Score with LEL
A
3
5
B
3
5
C
4
4
D
3
5
E
3
5
Table 4: Type of Requirement Scores With and Without LEL
App store reviews provide a constant flow of real user feedback that can help improve software requirements. However, these reviews are often messy, informal, and difficult to analyze manually at scale. Although automated techniques exist, many do not perform well when replicated and often fail to produce clean, backlog-ready user stories for agile projects. In this study, we evaluate how well large language models (LLMs) such as GPT-3.5 Turbo, Gemini 2.0 Flash, and Mistral 7B Instruct can generate usable user stories directly from raw app reviews. Using the Mini-BAR dataset of 1,000+ health app reviews, we tested zero-shot, one-shot, and two-shot prompting methods. We evaluated the generated user stories using both human judgment (via the RUST framework) and a RoBERTa classifier fine-tuned on UStAI to assess their overall quality. Our results show that LLMs can match or even outperform humans in writing fluent, well-formatted user stories, especially when few-shot prompts are used. However, they still struggle to produce independent and unique user stories, which are essential for building a strong agile backlog. Overall, our findings show how LLMs can reliably turn unstructured app reviews into actionable software requirements, providing developers with clear guidance to turn user feedback into meaningful improvements.
Creating sprint backlogs requires considerable effort, as items such as epics, user stories, and tasks can be missed or inconsistently specified. We propose a multimodal approach to support backlog generation from visual app mockups, an artifact available at early project stages. We evaluate three prompting strategies on GPT-4o: a zero-shot baseline, Compositional Chain-of-Thought (CCoT) for vision-language reasoning, and a persona-driven prompt. We study seven app development projects across two countries and interview developers about the results. Overall, we observed that the baseline prompt favours recall over precision, whereas CCoT is more balanced, achieving average F1 scores of 52-66% for epics and user stories. Tasks were more challenging to generate accurately. Precision gains were most consistent when adding architectural context, particularly for backend tasks (precision gains up to 35%). Interviews with developers revealed that up to 26% of false positives were still considered useful, reflecting the creative and open-ended nature of backlog creation. To capture this, we propose a new measure called Revised Recall, which complements ground-truth evaluation with developer assessments. Our findings suggest that hybrid prompting with architectural context can assist backlog generation from early mockups, though results vary by item type and developer oversight remains necessary.
Andrea Lezcano Airaldi, Lourdes Romera, Walid Maalej
Department of Informatics · Hasso Plattner Institute
Large language models (LLMs) have changed the way people engage with stories. Drawing on public chatbot logs, we can see that when users generate stories, they iteratively edit their prompts to explore narrative possibilities, adjusting characters, redirecting plots, and swapping fictional universes. As aggregated data, these prompts represent rich traces of creative preference at scale. Yet story generation evaluation benchmarks rely on static, one-shot prompts that cannot capture this exploratory behavior. In this work, we study how users revise consecutive story prompts in the wild. Using a dataset of naturally occurring user-chatbot conversations, we construct WildStories, a sample of 275,635 story generation prompts (labeled with story format, prompt components, and explicitness), and WildEdits, a collection of 24,291 edit trees that model how users iteratively edit base story prompts and explore branching story possibilities. From these trees we develop a framework of edit types crossing four directions (adding, removing, changing, and extending) with fourteen targets (e.g., plot, character, genre). We then use our datasets and this framework to analyze user behavior in navigating narrative space via LLMs. Finally, we show how automated permutations based on the framework can be used for story generation benchmarking. Content Warning: This paper works with "wild" chatbot logs, which often include toxic and sexually explicit themes.
Advait Deshmukh, Nora Benedict, Melanie Walsh +1
University of Colorado Boulder · University of Georgia · University of Washington