User feedback is crucial for the evolution of mobile apps. However, research suggests that users tend to submit uninformative, vague, or destructive feedback. Unlike recent AI4SE approaches that focus on generating code and other development artifacts, our work aims at empowering users to submit better and more constructive UI feedback with concrete suggestions on how to improve the app. We propose LikeThis!, a GenAI-based approach that takes a user comment with the corresponding screenshot to immediately generate multiple improvement alternatives, from which the user can easily choose their preferred option. To evaluate LikeThis!, we first conducted a model benchmarking study based on a public dataset of carefully critiqued UI designs. The results show that GPT-Image-1 significantly outperformed three other state-of-the-art image generation models in improving the designs to address UI issues while keeping the fidelity and without introducing new issues. An intermediate step in LikeThis! is to generate a solution specification before sketching the design as a key to achieving effective improvement. Second, we conducted a user study with 10 production apps, where 15 users used LikeThis! to submit their feedback on encountered issues. Later, the developers of the apps assessed the understandability and actionability of the feedback with and without generated improvements. The results show that our approach helps generate better feedback from both user and developer perspectives, paving the way for AI-assisted user-developer collaboration.
Figures & tables
Figure 1. Overview of LikeThis! . A user reports an app issue along with a screenshot (1). The AI pipeline generates n solution specifications (2), which are used to create improvement suggestions (3). The user selects their preferred suggestion (4) and submits it along with the issue to developers (5).
Figure 2. LikeThis! iOS app enables users to report issues by capturing a screenshot, describing their issue, and optionally marking an area (Screens 1–2). Here, the user reports accidentally tapping the call button while entering a number. LikeThis! generates improvement suggestions (Screens 3–4). Users can then select a suggestion to submit, edit, or reject all. In this case, the user chooses to change the call button to a Slide-To-Call control and submits it with an optional comment (Screen 5).
Figure 3. Data sampling from UICrit dataset for RQ1.
Figure 4. Evaluation tool to assess generated UIs.
F#1
Resolution
Fidelity
Robustness
Frequency
Avg
Frequency
Avg
Frequency
Avg
1
2
3
1
2
3
1
2
3
GPT
214
22
18
200
2.74
4
33
203
2.83
1
27
212
2.88
Flux
14
153
29
58
1.60
4
30
206
2.84
11
70
159
2.62
Gemini
8
114
53
73
1.83
18
43
179
2.67
35
115
90
2.23
Bagel
4
181
27
32
1.38
9
19
212
2.85
100
35
105
2.02
Table 1. Results of 240 annotations (by 2 annotators for 120 tasks) comparing four models.
F#1
Resolution
Fidelity
Robustness
Frequency
Avg
Frequency
Avg
Frequency
Avg
1
2
3
1
2
3
1
2
3
S
Mask
28
6
4
30
2.60
0
0
40
3.00
1
10
29
2.70
No mask
12
8
2
30
2.55
0
4
36
2.90
0
5
35
2.88
M
Mask
21
0
2
38
2.95
5
2
33
2.70
1
7
32
2.77
No mask
19
1
2
37
2.90
2
6
32
2.75
0
3
37
2.92
Table 2. Results of 120 annotations (by 2 annotators for 60 tasks) comparing GPT-Image-1 with and without mask.
F#1
Resolution
Fidelity
Robustness
Frequency
Avg
Frequency
Avg
Frequency
Avg
1
2
3
1
2
3
1
2
3
Mask
SG
108
10
10
100
2.75
5
26
89
2.70
2
29
89
2.73
N-SG
12
78
7
35
1.64
66
11
43
1.81
61
22
37
1.80
No mask
SG
80
10
11
99
2.74
1
12
107
2.88
0
11
109
2.91
N-SG
40
25
13
82
2.48
3
6
111
2.90
3
16
101
2.82
Table 3. Results of 120 annotations (by 2 annotators for 60 tasks) comparing GPT-Image-1-mask with and without “Suggestion Generation”, and 120 annotations (by 2 annotators for 60 tasks) comparing GPT-Image-1 with and without “Suggestion Generation”.
Users
Id
Gender
Age
Occupation
U1
M
28
Dentist
U2
M
19
Pediatric caregiver
U3
M
27
Marketing consultant
U4
F
26
Teacher
U5
F
28
Kindergarten teacher
Table 4. Participants in user and developer studies.
Rating
Count
Percentage
Very inaccurate
0
0%
Rather inaccurate
0
0%
Somewhat accurate
5
6.9%
Mostly accurate
22
30.6%
Very accurate
41
56.9%
Canceled submission
4
5.6%
Table 5. “How accurate is your selected suggestion in addressing the issue you wanted to report?” (User testing)
Response
Count
Percentage
Strongly disagree
0
0%
Disagree
1
7.1%
Undecided
1
7.1%
Agree
5
35.7%
Strongly agree
8
57.1%
Total
15
100%
Table 6. “When encountering issues with the apps you regularly use, do you think this feedback approach (with suggestions to select from) will help you to submit your complaints and feedback?” (User testing)
Before seeing the improvement suggestion
Count (%)
Not Understandable
5 (7.4%)
Somewhat Understandable
15 (22.1%)
Clearly Understandable
48 (70.6%)
Total
68 (100%)
Table 7. Understandability of user feedback before and after seeing improvement suggestions. (Developer survey)
Response
A
B
Yes, I could directly implement a change the suggestion
40 (58.8%)
48 (70.6%)
Possibly, but I would need more information
15 (22.1%)
17 (25.0%)
No, I don’t know what I should do
13 (19.1%)
3 (4.4%)
Total
68 (100%)
Table 8. Actionability of the textual feedback (A) and the associated improvement suggestions (B). (Developer survey)
User feedback is essential for the success of mobile apps, yet what users report and what developers need often diverge. Research shows that users often submit vague feedback and omit essential contextual details. This leads to incomplete reports and time-consuming clarification discussions. To overcome this challenge, we propose FeedAIde, a context-aware, interactive feedback approach that supports users during the reporting process by leveraging the reasoning capabilities of Multimodal Large Language Models. FeedAIde captures contextual information, such as the screenshot where the issue emerges, and uses it for adaptive follow-up questions to collaboratively refine with the user a rich feedback report that contains information relevant to developers. We implemented an iOS framework of FeedAIde and evaluated it on a gym's app with its users. Compared to the app's simple feedback form, participants rated FeedAIde as easier and more helpful for reporting feedback. An assessment by two industry experts of the resulting 54 reports showed that FeedAIde improved the quality of both bug reports and feature requests, particularly in terms of completeness. The findings of our study demonstrate the potential of context-aware, GenAI-powered feedback reporting to enhance the experience for users and increase the information value for developers.
Ali Ebrahimi Pourasad, Meyssam Saghiri, Walid Maalej
User interface (UI) and user experience (UX) evaluation is central to product development, yet reliable feedback still relies on recruiting human participants or running online A/B tests, making early-stage iteration slow and costly. In light of this, recent work has explored Multimodal Large Language Models as proxy evaluators. However, existing approaches either produce surface-level critiques or a judgment that reflects the model's own biases rather than the genuine response of a particular user. We introduce PerceptUI, a framework for persona-conditioned UI/UX evaluation that predicts how a specific user would answer interface-related questions and produces natural-language rationales. PerceptUI is trained in two stages: (i) contrastive reflection fine-tuning distills teacher-generated rationales by extracting lessons from human decisions, and (ii) a reflective prompt-evolution step from the model's own failure traces. Across multiple domains and datasets, PerceptUI achieves human-level realism, generalizes to unseen questions and personas, and yields population-level response distributions.
Nicolas Bougie, Xiaotong Ye, Gian Maria Marconi +1
AI models excel at creating content, but typically render it with static, predefined interfaces. Specifically, the output of LLMs is often a markdown "wall of text". Generative UI is a long standing promise, where the model generates not just the content, but the interface itself. Until now, Generative UI was not possible in a robust fashion. We demonstrate that when properly prompted and equipped with the right set of tools, a modern LLM can robustly produce high quality custom UIs for virtually any prompt. When ignoring generation speed, results generated by our implementation are overwhelmingly preferred by humans over the standard LLM markdown output. In fact, while the results generated by our implementation are worse than those crafted by human experts, they are at least comparable in 50% of cases. We show that this ability for robust Generative UI is emergent, with substantial improvements from previous models. We also create and release PAGEN, a novel dataset of expert-crafted results to aid in evaluating Generative UI implementations, as well as the results of our system for future comparisons. Interactive examples can be seen at https://generativeui.github.io