cs.AIOct 2, 2026
SaveDesigning the Future of User Feedback for Generative AI
Organizations: International Computer Science Institute, Berkeley, CA, USA · University of California, Berkeley, Berkeley, CA, USA · eBay Inc., San Jose, CA, USA
Abstract
Post-deployment feedback from users can be a cost-effective, scalable, and representative means to monitor and improve generative AI systems and features. When implemented effectively, giving such feedback can increase users' engagement with and trust in GenAI systems. Government regulations and industry guidelines call for post-deployment user engagement, but there is little guidance on designing mechanisms that are usable for consumers and provide actionable input for product teams. We conducted a multi-phase study as a collaboration between academic researchers and eBay. Our benchmark evaluation of current industry approaches identified common issues including lack of discoverability, unclear terminology, and inattention to user value. Based on these findings, we developed best-practice recommendations and designed and tested a prototype feedback-collection tool. The tool aimed to provide users with an efficient, flexible, and positive feedback-giving experience, and provide product teams with rich data on performance and potential problems in a usable format.
Figures & tables
Figure 1: Example feedback forms for eBay’s AI Description Generator, Claude, Google Search’s AI Overview, and Amazon’s Rufus. Screenshots are from March 2025.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Heuristic/Criterion | Guiding Questions | Rating | Issues | Positive aspects | Recommen-dations |
|---|---|---|---|---|---|
| Entry Point: The entry point should be obvious and intuitive. Users should easily understand where to start if they want to leave feedback regarding the AI-generated response. | • Is it immediately obvious that the user can leave feedback about the AI-generated response? • How easy or difficult is it to discover where to start to leave feedback about the AI-generated response? | [Very well; Somewhat well; Somewhat poorly; Very poorly] | |||
| Navigation: It should be easy to navigate the system to leave feedback about the AI-generated response. Information should appear in a natural and logical order. | • Does it take a lot of effort or time to leave feedback about the AI-generated response? • Is information given at the points in the flow when it is most relevant to the user’s decision-making? • Is it obvious which form fields are optional and which ones are required? • Is the interface visually appealing and easy to navigate? | [Very well; Somewhat well; Somewhat poorly; Very poorly] | |||
| End of table | |||||
Table 1: Template for heuristic evaluation, with heuristic, guiding questions, and space to record rating and written evaluation. Only the first 2 heuristics are included in this format for illustrative purposes. The full list of heuristics is in Table 2 .
| Heuristic/Criterion | Guiding Questions |
|---|---|
| Entry Point: The entry point should be obvious and intuitive. Users should easily understand where to start if they want to leave feedback regarding the AI-generated response. | • Is it immediately obvious that the user can leave feedback about the AI-generated response? • How easy or difficult is it to discover where to start to leave feedback about the AI-generated response? |
| Navigation: It should be easy to navigate the system to leave feedback about the AI-generated response. Information should appear in a natural and logical order. | • Does it take a lot of effort or time to leave feedback about the AI-generated response? • Is information given at the points in the flow when it is most relevant to the user’s decision-making? • Is it obvious which form fields are optional and which ones are required? • Is the interface visually appealing and easy to navigate? |
| Clarity of the Wording/Information: The design should speak the users’ language. Use words, phrases, and concepts familiar to the user, rather than internal jargon. | • Will users be familiar with the terminology used by the feedback feature? (If not, what words may be confusing?) • Is the information given at each step sufficient? • If there are quick-answer response options (pills, checkboxes, etc.), are they clear, relevant, and helpful to have? |
| Clarity of the Process: Users should understand the consequences of their actions when they decide to provide feedback about AI-generated responses. | • Is it clear what information will be sent alongside the user’s responses in the feedback form? (e.g. user ID, personal info such as email address or name, entire chat, only the AI-generated response that the user is providing feedback for, query + the response, metadata (such as time stamp, browser version, device model), etc.) • Is it clear who will have access to the provided feedback? For example, would other users, buyers, or sellers be notified about the feedback? Will employees of the company that deployed the AI feature or the company that trained the model get access to the feedback? Will law enforcement be notified about any types of negative experiences reported? |
| User Value and Motivation: The benefit of providing feedback should be clear and specific. If it is not clear and specific, users may not understand why they should give feedback, they might avoid giving feedback, or if they can’t avoid it, they could give meaningless responses, or feedback that ends up being harmful to the product or community. | • Is it clear what will happen if the user submits the feedback? For example, will future AI-generated responses change for the specific user (i.e. they will not see similar responses anymore or recommendations will be more customized)? Will they change for all users (e.g. it will be used in re-training or fine-tuning of the model in general)? Will anyone (e.g. other users, buyers, sellers) be penalized if users report negative experiences with the AI-generated responses? • Is the tone of the communication friendly, respectful, and inviting? Will the user feel encouraged to submit feedback? • Does the flow include a thank you message to make users feel that their feedback is appreciated? |
| Help and Additional Information: It’s best if the system doesn’t need any additional explanation. However, it may be necessary to provide documentation to help users understand how to complete their tasks. | • Is help or additional information easy to find? • Is help or additional information provided in context right at the moment when the user requires it? (For example, via hover-over text, in-situ explanations, ""Learn more"" links.) |
Table 2: Description of heuristics and related guiding questions used in the heuristic evaluation template.
Figure 2: GenAI Feedback Guidelines (Part 1): Purpose of guidelines and skeleton for feedback form.
Figure 3: GenAI Feedback Guidelines (Part 2): Core principles and introduction to entry points.
Figure 4: GenAI Feedback Guidelines (Part 3): Entry points for simple features.
Figure 5: GenAI Feedback Guidelines (Part 4): Entry points for complex features.
Figure 6: GenAI Feedback Guidelines (Part 5): Structure pointers for feedback form and for negative and positive feedback flows.
Figure 7: GenAI Feedback Guidelines (Part 6): Negative feedback forms for different types of features.
Figure 8: GenAI Feedback Guidelines (Part 7): Positive feedback forms for different types of features.
Figure 9: GenAI Feedback Guidelines (Part 8): Detailed pointers for preset options, free-text responses, and privacy disclaimers.
Figure 10: GenAI Feedback Guidelines (Part 9): Thank-you screens and post-feedback signals.