cs.HCMar 7, 2026
SaveUser Review Writing via Interview with Dialogue Systems
Organizations: The University of Electro-Communications
Abstract
User reviews on e-commerce and review sites are crucial for making purchase decisions, although creating detailed reviews is time-consuming and labor-intensive. In this study, we propose a novel use of dialogue systems to facilitate user review creation by generating reviews from information gathered during interview dialogues with users. To validate our approach, we implemented our system using GPT-4 and conducted comparative experiments from the perspectives of system users and review readers. The results indicate that participants who used our system rated their interactions positively. Additionally, reviews generated by our system required less editing to achieve user satisfaction compared to those by the baseline. We also evaluated the reviews from the reader' perspective and found that our system-generated reviews are more helpful than those written by humans. Despite challenges with the fluency of the generated reviews, our method offers a promising new approach to review writing.
Figures & tables
Figure 1: Example of review creation supported by the proposed method. In the dialogue and review, the contents of the same-colored text correspond to each other.
Figure 2: Overview of our system. First, the interview dialogue system interviews the user to elicit their impressions and requests about the product they used. Next, the review text generator uses the dialogue history as input to generate a review text. Finally, the rating predictor predicts a rating consistent with the sentiment of the generated review text.
Figure 3: Participant responses to “ In the past year, how often have you posted reviews? ”
| Dimension | Labels in Figure 4 | Statements |
|---|---|---|
| Interview | Enjoyable | How fun was your interaction with the chatbot interviewer? |
| Skillful | The interviewer skillfully elicited your impressions or opinions. | |
| In-depth | The chatbot interviewer attempted to elicit your impressions or opinions in depth. | |
| Review | Faithful | The system-generated review faithfully reflects what you said during your interviews. |
| Concise | The system-generated review offers a concise summary of the points you mentioned during the interview. | |
| System | Quality | Please rate the overall quality of the system. |
Table 1: Likert Items in Post-interview Survey
Figure 4: Participant responses to questions on a Likert scale from 1 (Strongly disagree) to 5 (Strongly agree) in a post-interview survey. For each question, the upper bar shows the results from our system and the lower bar shows the baseline results.
Figure 5: Participant responses to “ If you had to edit and post a system-generated review to your satisfaction, how much of it would you need to rewrite? ”
| S | (…) To start off, could you tell me about your overall satisfaction with this product? |
| U | i would say well satisfied but with few minor issues. |
| S | I see, you’re generally satisfied but there are a few minor issues. Could you elaborate on what you particularly like about the Braun Series 9 9370cc Electric Shaver? |
| U | some times small hair from the beard gets stucks in particular spot on the blade area in the machine. |
| S | I understand that small hairs getting stuck in the blade area is one of the issues you’ve encountered. But before we delve into that, could you first share what you find satisfying or impressive about the product? What are its positive points in your opinion? |
| U | the postivie sides is smooth shaving experience and battery capacity which is good |
| S | It’s great to hear that you appreciate the smooth shaving experience and the battery capacity. Now, let’s go back to the issue you mentioned earlier about small hairs getting stuck. Could you tell me more about this problem? How often does it happen and how does it affect your shaving experience? |
Table 2: Example of a dialogue between our system and the user (top, S: system, U: user) and generated review text (bottom). In this example, our rating predictor predicted a rating of 4.
| Labels in Table 4 | Questions |
|---|---|
| Helpfulness | Which review would be more helpful for making a purchase decision? |
| Fluency | Which review exhibits a more fluent and human-like writing style? |
| Conciseness | Which review is more concise and to the point? |
| Experience | Which review provides more information based on the actual usage experience of the product? |
| Balance | Which review presents a more balanced view of the product’s pros and cons? |
| Depth | Which review provides more in-depth information about any specific aspect of the product? |
Table 3: Questions in comparative evaluation
| Reviews | Helpfulness | Fluency | Conciseness | Experience | Balance | Depth | Coverage | Overall |
|---|---|---|---|---|---|---|---|---|
| Human | 38.0 | 47.0 | 37.0 | 57.0 | 37.0 | 43.0 | 40.0 | 41.0 |
| Tie | 6.0 | 15.0 | 6.0 | 9.0 | 15.0 | 10.0 | 5.0 | 7.0 |
| Ours | 56.0 | 38.0 | 57.0 | 34.0 | 48.0 | 47.0 | 55.0 | 52.0 |
| Baseline | 38.5 | 28.1 | 45.8 | 21.9 | 34.4 | 35.4 | 35.4 | 37.5 |
| Tie | 12.5 | 33.3 | 6.2 | 16.7 | 15.6 | 12.5 | 10.4 | 17.7 |
| Ours | 49.0 | 38.5 | 47.9 | 61.5 | 50.0 | 52.1 | 54.2 | 44.8 |
Table 4: Results of third-party evaluation. The values represent the percentage of votes each received.
| Annotator/Source | Human-written | System-generated |
|---|---|---|
| Turkers | 0.59 | 0.12 |
| Participants in Section 4.1 | - | 0.57 |
Table 5: Average absolute difference in ratings between Amazon customers and Turkers (top-left), between system-predicted ratings and Turkers’ ratings for system-generated reviews (top-right), and between system-predicted ratings and participants’ ratings (bottom-right, see Section 4.1.4 ).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Your role is “interviewer” and my role is “interviewee”. |
| About the product I am going to present, please elicit my impressions and opinions from me when I have touched it. |
| Note the following statements. |
| 1. The interviewer elicits the interviewee’s satisfaction and dissatisfaction (the positive and negative points) with the product in a well-balanced and detailed. |
| 2. In response to the interviewee’s response, the interviewer asks more in-depth questions about the aspect or elicits feedback about other aspects of the product. |
| 3. Be sure to attach the name of your role at the beginning of your utterance. Since your role is “interviewer”, your generation should begin with “Interviewer:”. |
| 4. Don’t generate interviewee’s utterances. |
Table 6: Prompt template for interviewing.
| [DIALOGUE] |
| The above is a dialogue about “[PRODUCT_NAME]” between the interviewer and the interviewee who has touched on this product. |
| Write a customer review about the product as if written by the interviewee, by briefly summarizing the important information mentioned in the above interview, such as the good and bad points of the product and the interviewee’s experience with it. |
| Do not output the review’s title. |
| The following is a body of the product review of the product written by the interviewee: |
Table 7: Prompt template for review generation.
| Q-1 | First, could you tell me about the features and functions of this product? What kind of product is this? |
|---|---|
| Q-2 | What made you decide to purchase this product? |
| Q-3 | If you have any points that you like or are satisfied with this product, please tell me in detail. |
| Q-4 | What are the advantages of this product compared to other products? |
| Q-5 | If you have any dissatisfaction with this product or areas for improvement for this product, please tell me in detail. |
| Q-6 | What are the disadvantages of this product compared to other products? |
| Q-7 | Who would this product be suitable for? |
Table 8: Questions asked by the baseline system
More in this topic
Text Generation
Text generation studies methods for producing coherent, useful language from prompts, data, and dialogue context.
See all 422 papers