Investigating In-Context Privacy Learning by Integrating User-Facing Privacy Tools into Conversational Agents
Organizations: University of Massachusetts Amherst · New York University
Abstract
Supporting users in protecting sensitive information when using conversational agents (CAs) is crucial, as users may undervalue privacy protection due to outdated, partial, or inaccurate knowledge about privacy in CAs. Although privacy knowledge can be developed through standalone resources, it may not readily translate into practice and may remain detached from real-time contexts of use. In this study, we investigate in-context, experiential learning by examining how interactions with privacy tools during chatbot use enhance users' privacy learning. We also explore interface design features that facilitate engagement with these tools and learning about privacy by simulating ChatGPT's interface which we integrated with a just-in-time privacy notice panel. The panel intercepts messages containing sensitive information, warns users about potential sensitivity, offers protective actions, and provides FAQs about privacy in CAs. Participants used versions of the chatbot with and without the privacy panel across two task sessions designed to approximate realistic chatbot use. We qualitatively analyzed participants' pre- and post-test survey responses and think-aloud transcripts and describe findings related to (a) participants' perceptions of privacy before and after the task sessions and (b) interface design features that supported or hindered user-led protection of sensitive information. Finally, we discuss future directions for designing user-facing privacy tools in CAs that promote privacy learning and user engagement in protecting privacy in CAs.
Figures & tables
| ID | Degree - Year | Gender | ChatGPT Use | Task Assignment Version | ||
| Duration | Frequency | Session 1 | Session 2 | |||
| P1 | Master - 1st year | Female | ~2.5 year | Daily | A | B |
| P2 | Undergrad - 4th year | Female | ~2.5 years | Daily | B | A |
| P3 | Master - 1st year | Female | ~2.5 years | Several times a week | A | B |
| P4 | Master - 2nd year | Male | ~2.5 years | Daily | B | A |
| P5 | Undergrad - 3rd year | Male | ~1.5 years | Daily | A | B |
| RQ | Themes & Sub-Themes | Before Study | After Study |
| RQ1.1. How students conceptualize sensitive information. | • Potentials for harm if leaked ( 4.1.1 ) | P1, P2, P3, P5, P6, P7, P8 | P1, P2, P3, P7, P10 |
| • Restricted access ( 4.1.2 ) | P1, P4, P5, P6, P7, P9, P10 | P3 , P4, P5, P6, P7, P9, P10 | |
| • Linkable to identity ( 4.1.3 ) | P2, P3, P4, P8, P10 | P2, P3, P8 | |
| • Context-based sensitivity ( 4.1.4 ) | P5, P9 | P6 , P9, P10 | |
| • Security measures ( 4.1.5 ) | P5 | P6 | |
| RQ1.2. Views on the importance of protecting sensitive info during chatbot use | • Unclear data lifecycle ( 4.2.1 ) | P1, P4, P5, P6, P8, P9, P10 | P3 , P4, P5, P6, P7 , P8, P9 |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| V | Task Description | Sensitive Info | F | Info Subject |
| A | Task 1: Email Analysis • Step 1: Classifying seven emails 1 1 1 Drawn from the Enron Email Corpus and modified as needed. by their sentiments • Step 2: Summarizing two separate email threads | Name Email address Phone number Physical address | 61 23 6 4 | Others (e.g., employees) |
| A | Task 2: Searching Housing Documents • Step 1: Finding party restrictions from their housing contract (9-page doc) • Step 2: Finding maintenance information from the same housing contract and a welcome letter (5-page doc) | Name Email address Phone number Physical address SSN Date of birth | 38 9 16 16 4 4 | Self, Others (e.g., friend, home owner) |
| A | Task 3: Planning a Trip • Step 1: Finding timing conflicts between their travel itinerary (2-page doc) and semester schedule (1-page doc) • Step 2: Drafting an email to resolve the conflicts and provide their contact details for reservation purposes | Name Email address Phone number Physical address Date of birth | 5 1 3 7 2 | Self, Others (e.g., friend) |
| B | Task 1: Customer Complaint Analysis • Step 1: Classifying seven customer complaints 2 2 2 Drawn from a Ticket Classification Dataset and modified as needed. by relevant departments • Step 2: Summarizing recommendations for the company from two separate groups of customer complaints | Name Email address Phone number Physical address Date of birth | 18 6 4 4 2 | Others (e.g., customers) |
| B | Task 2: Searching Car & Housing Documents • Step 1: Finding costs and fees related to purchasing a leased car from their car lease contract (15-page doc) • Step 2: Finding information about utilities and responsibilities from their housing contract (9-page doc) | Name Email address Phone number Physical address SSN Date of birth | 23 4 10 12 4 4 | Self, Others (e.g., friend, home owner) |
| B | Task 3: Planning Teeth Removal • Step 1: Finding out-of-pocket expenses from their insurance document (6 pages) and an email from their doctor. • Step 2: Drafting an email to their doctor providing insurance details and asking if the insurance is accepted | Name Email address Phone number Physical address Date of birth | 8 2 2 1 5 | Self, Others (e.g., doctor) |