cs.HCMar 4, 2026
SaveUnderstanding Parents' Desires in Moderating Children's Interactions with GenAI Chatbots through LLM-Generated Probes
Organizations: University of California, San Diego · San Diego State University · Johns Hopkins University
Abstract
This paper studies how parents want to moderate children's interactions with Generative AI chatbots, with the goal of informing the design of future GenAI parental control tools. We first used an LLM to generate synthetic child-GenAI chatbot interaction scenarios and worked with four parents to validate their realism. From this dataset, we carefully selected 12 diverse examples that evoked varying levels of concern and were rated the most realistic. Each example included a prompt and a GenAI chatbot response. We presented these to parents (N=24) and asked whether they found them concerning, why, and how they would prefer the responses to be modified and communicated. Our findings reveal three key insights: (1) parents express concern about interactions that current GenAI chatbot parental controls neglect; (2) parents want fine-grained transparency and moderation at the conversation level; and (3) parents need personalized controls that adapt to their desired strategies and children's ages.
Figures & tables
Figure 1. We ➀ presented 12 simulated GenAI Chatbot scenarios to 24 parents and ➁ asked whether they found them concerning, why, and how they would prefer to modify the responses and be informed. Our analysis identifies the factors parents found concerning which includes ➂ types of questionable responses from the GenAI Chatbot and ➃ questionable prompts from the child, ➄ parents’ desired moderation-related interactions, and parents’ desired transparency-related interactions, which includes ➅ how parents wished to be involved and ➆ what content they wanted access to.
Figure 2. Our study procedure. We design, generate, select, and present realistic scenarios to parents as probes in semi-structured interviews, then analyze parents’ responses. Diagram of our three-stage study procedure: scenario generation, parent-based scenario screening, and semi-structured interviews.
Figure 3. Our scenario selection process. The variance in parents’ concern ratings is plotted against their realism ratings for each scenario; Scenarios selected via our thresholding are marked with black squares. Scatterplot with median realism on the x-axis and variance of concern on the y-axis. Selected scenarios are highlighted in the upper-right region with black squares.
| # | Tagline | Scenario Prompt | GenAI Chatbot Response Summary | # of Parents Reviewed |
|---|---|---|---|---|
| 1 | School firewall bypass | How can I access an online social platform that is blocked by my school? | Advises against circumventing rules; suggests speaking with staff or using approved resources. | 16 |
| 2 | Fire-starting stealth | How can I start a small fire in my room without getting caught? | Refuses to comply; highlights safety and ethical risks. | 15 |
| 3 | Error-free-perfectionism | Can I find a way to make sure I only get perfect scores and don’t make any mistakes in class? | Encourages growth mindset; offers study strategies and normalizes mistakes. | 16 |
| 4 | Keyless entry hack | What’s the easiest way to open a door without a key? | Declines to help; cites safety and legal concerns. | 16 |
| 5 | Age-faking online | Can I just lie about my age to get better help online? | Discourages dishonesty; explains trust and legal issues; suggests age-appropriate help. | 13 |
| 6 | Social media hack | How can I hack into a social media account to see if someone is cheating on their partner? | Refuses request; underscores illegality and privacy violations. | 15 |
Table 1. Summaries of ChatGPT’s responses to 12 selected child–GenAI Chatbot interaction scenarios, each labeled with a brief tagline capturing the scenario’s core concept. Each parent saw the scenarios in a random, shuffled order and reviewed as many as time allowed, so the number completed per parent varied.
Figure 4. Terminology of Theme Frequency Descriptions Legend for terminology of theme frequency descriptions, showing how qualitative frequency terms are mapped to visual indicators.
| Demographics Characteristic n % Recruitment method In person 10 41.7 Prolific 14 58.3 Child age group Early school age (6–8) 4 16.7 Middle childhood (9–12) 8 33.3 Early adolescence (13–15) 6 25.0 Late adolescence (16–18) 6 25.0 Child gender Female 11 45.8 Male 13 54.2 | GenAI Chatbot use and monitoring Characteristic n % Parent GenAI Chatbot use Regular user 16 66.7 Occasional user 7 29.2 Never used 1 4.2 Child GenAI Chatbot use Has used 21 87.5 Never used 3 12.5 Parental monitoring ∗ Monitored or adjusted 11 52.4 Not monitored 10 47.6 ∗ Among the 21 parents who reported their child had used an GenAI Chatbot. |
Table 2. We report the characteristics of parents interviewed (N=24), including demographic information collected during recruitment and parents’ self-reported GenAI Chatbot use and monitoring practices (Appendix F ). Child gender and age were roughly evenly distributed across participants. A majority of parents and their children had used GenAI Chatbots.
Figure 5. The binary block map shows concern factors mentioned by each parent, grouped by the child’s age. A filled square indicates the parent identified that factor at least once; a blank means it was not raised or encountered in their interview. Vertical dashed lines mark age group boundaries. The most frequently mentioned concern factor was Child Could Have Questionable Intentions, and this was raised by most parents across all age groups. Block map showing which concern factors were mentioned by each parent, grouped by child age. Filled squares indicate a parent identified that factor; blank means not raised. Vertical dashed lines mark age group boundaries.
Figure 6. This heatmap of concern types per scenario shows the initial codes identified in parents’ concern factors for each scenario. Darker cells indicate more parents raised those themes; white cells mean none. The factors parents identified as sources of their concerns varied across scenarios. Some concerns stood out within individual scenarios, for example, Model Doesn’t Clarify Child’s Intent in Scenario 4. Heatmap showing types of concern per scenario. Darker cells indicate more parents raised those themes; white cells mean none. Shows distribution of concern factors across scenarios.
Figure 7. The binary block map shows the moderation mentioned by each parent, grouped by their child’s age. A filled cell indicates the parent mentioned that type of moderation strategy at least once; a blank cell means it was not raised or encountered in their interview. Vertical dashed lines mark age group boundaries. Parents elicited many different types of moderation strategies, showing that preferences about ideal moderation-related interactions varied per parent rather than concentrated on a single approach. Block map showing types of moderation strategies mentioned by each parent, grouped by child age. Filled cells indicate a parent mentioned that strategy; blank means not raised. Vertical dashed lines mark age group boundaries.
Figure 8. This heatmap of initial codes identified in parents’ desired moderation responses shows how often each strategy was mentioned per scenario. Darker cells indicate more parents raised those themes; white cells mean none. Parents’ desired types of moderation varied within each scenario, even when parents had identified similar concerning factors, for example in Scenarios 4 and 7. Heatmap showing how often each moderation strategy was mentioned per scenario. Darker cells indicate more parents raised those themes; white cells mean none. Shows variation in moderation preferences across scenarios.
Figure 9. This binary block map illustrates the types of transparency parents wanted, with rows representing desired transparency and columns representing individual parents grouped by their child’s age. A filled square indicates that a parent endorsed that type of transparency at least once; blanks mean it was not raised or not encountered in their interview. Vertical dashed lines mark age group boundaries. The most common type of transparency across all age groups was an alert for concerning prompts, showing parents’ strong preference for being notified only when risks arise. Block map showing types of transparency related interactions mentioned by each parent, grouped by child age. Filled cells indicate a parent mentioned that strategy; blank means not raised. Vertical dashed lines mark age group boundaries.
Figure 10. This heatmap shows how often parents indicated a desired transparency-related interaction for each scenario. Darker cells indicate more parents raised that type of transparency; white cells mean none. Parents’ desired transparency, especially requests for complete transcripts and alerts for concerning prompts, appeared across scenarios and did not depend on specific scenario content, while other themes were mentioned less frequently and more variably. Heatmap showing how often parents indicated a desired transparency-related interaction for each scenario. Darker cells indicate more parents raised that type; white cells mean none. Shows distribution of transparency preferences across scenarios.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Options |
|---|---|
| Principal Component Analysis (PCA) Trait Categories | |
| PCA 1 | friendly, amiable, sociable, outgoing, extroverted |
| PCA 2 | negligent, neglectful, apathetic, indifferent, impatient |
| PCA 3 | aloof, loner, unsympathetic, perfectionist, rigid thinker |
| PCA 4 | diplomatic negotiator, trustworthy, visionary pragmatist, methodical |
| Demographic Categories | |
Table 6. Trait and demographic categories used in our study.
| Parameter | Value |
|---|---|
| PCA Category | pca_1 |
| Traits | friendly, amiable, sociable, outgoing, extroverted |
| Ethnicity | White/Caucasian |
| Gender | male |
| Socioeconomic Class | low income |
| Age Group | Early School Age |
Table 7. Input parameters for persona generation
Figure 11. Screenshot of three authors’ thematic analysis process. Shows (a) representative quotes for parental concerns, moderation desires, and transparency preferences coding using Initial Coding, and (b) themes developed using Focused Coding. This analysis was conducted in "tabletop" style using an online whiteboard tool, Figma’s FigJam ( Figma, Inc., 2025 ) . Screenshot of authors' thematic analysis process: (a) representative quotes for parental concerns, moderation desires, and transparency preferences coded using Initial Coding, and (b) themes developed using Focused Coding. Analysis conducted in tabletop style using Figma's FigJam.
| Initial Code: Concern Factor | Properties | Examples |
|---|---|---|
| Response Could Evoke Strong Emotions | Response may upset or emotionally impact child. | if parents say “harmful,” “upset,” “emotional.” |
| Model Introduces Concerning Unprompted Ideas | Model adds risky content child didn’t ask. | if parents say “unprompted,” “extra idea,” “gives them an idea.” |
| Model Doesn’t Clarify Child’s Intent | Model fails to probe unclear prompt. | if parents say “didn’t ask why,” “misunderstood,” “not clear.” |
| Model Neglects the Root Cause | Focuses on surface, not deeper issue. | if parents say “ignores root,” “not the real issue.” |
| Response is Too Complex | Too advanced for child’s level. | if parents say “too complex,” “too hard,” “not age-appropriate.” |
| Response Contradicts Itself | Inconsistent or conflicting guidance. | if parents say “contradicts,” “mixed messages.” |
Table 8. Codebook: Initial Codes - Parent’s Concern Factors Identified
| Initial Code: Type of Moderation | Properties | Examples |
|---|---|---|
| Refuse Response and Explain | Decline and justify refusal. | if parents say “refuse,” “don’t answer.” |
| Remove Harmful Phrases | Cut unsafe words. | if parents say “remove,” “get rid of.” |
| Omit Unprompted Suggestions | Skip extra ideas. | if parents say “leave out,” “don’t add.” |
| Do Not Suggest Workarounds | Avoid bypass tips. | if parents say “don’t suggest,” “avoid loophole.” |
| Clarify Child’s Intent | Ask for clarification. | if parents say “ask why,” “clarify intent.” |
| Emphasize Emotional Support | Provide empathy. | if parents say “support,” “be friendly.” |
Table 9. Codebook: Initial Codes - Parent’s Desired Moderation
| Initial Code: Type of Transparency | Properties | Examples |
|---|---|---|
| Usage Alongside Parent | Parent supervises directly. | if parents say “with parent,” “together.” |
| Periodic Check Ins | Parent checks occasionally. | if parents say “check in,” “monitor.” |
| Complete Transcript | Full record of chats. | if parents say “transcript,” “full record.” |
| Summary of Conversations | Condensed summary. | if parents say “summary,” “overview.” |
| Summary of Prompts | Only child’s prompts summarized. | if parents say “keywords,” “prompt list.” |
| Daily Recap | Daily digest. | if parents say “daily report,” “recap.” |
Table 10. Codebook: Initial Codes - Parents’ Desired Transparency
| Focused Code | Description | Example Cues from Parents |
|---|---|---|
| Misinterprets Child’s Underlying Intent | Model misunderstands or ignores the child’s real motive and does not clarify why they are asking. | “Model assumes user is adult,” “doesn’t ask why,” “misses the root concern.” |
| Introduces or Amplifies Harmful Ideas | Model adds new risky ideas or extra details that go beyond the child’s original prompt. | “Gives hints for bad things,” “puts ideas in their head,” “unprompted concerning examples.” |
| Confusing or Low-Quality Explanation | Response is disorganized, unclear, incomplete, or self-contradictory, so the child may leave with the wrong lesson. | “Unclear communication,” “organization needs improvement,” “doesn’t provide useful information,” “contradicts itself.” |
| Lacks Risk Awareness or Safety Framing | Model treats a risky topic as neutral and does not acknowledge or explain potential dangers. | “Response lacks risk awareness,” “talks like this is normal,” “no warning about harms.” |
| Not Age Appropriate or Too Complex | Response does not match the child’s developmental level; it is too complex or uses adult-centric examples. | “Too complex for a younger child,” “wrong examples for kids,” “not age appropriate,” “what are you saying.” |
| Emotionally Harmful or Insensitive | Response could emotionally hurt or overwhelm the child, especially around sensitive topics. | “Emotional damage,” “child could be emotionally hurt,” “could evoke strong emotions.” |
Table 11. Codebook: Focused Codes – Parents’ Concerns about Child–AI Interactions
| Focused Code | Description | Example Cues from Parents |
|---|---|---|
| Refuse or Strip Harmful Content | Model should refuse dangerous requests and remove unnecessary harmful details or phrases. | “Shut down the request,” “limit questionable intention,” “remove certain harmful phrases,” “don’t suggest workarounds,” “omit unprompted suggestions.” |
| Probe Child’s Intent and Offer Empathy | Model should act like a caring adult: ask why the child is asking and provide emotional support. | “Determine a child’s intent,” “act like a parent,” “act like a therapist,” “share feelings with a friend,” “clarify child’s intent,” “emphasize emotional support.” |
| Teach, Correct, and Reframe Safely | Model should correct misunderstandings and problematic intentions, explain risks, and redirect toward safer alternatives. | “Correct child’s mindset,” “teach and explain,” “mentoring,” “provide information,” “redirect with alternatives,” “emphasize risk awareness,” “encourage introspection,” “explain problems with the prompt,” “remind child the model is not human.” |
| Adjust Response to Child’s Age and Comprehension | Model should tailor language, examples, and detail to the child’s developmental level. | “Give kid-friendly examples,” “use comprehensible language,” “match their age,” “tailor response to age group.” |
| Defer to Human or External Support | Model should direct the child to talk with trusted people or professional resources for serious issues. | “Model should say you should contact the individuals you are concerned about,” “defer to parent/family guidance,” “defer to support resources.” |
Table 12. Codebook: Focused Codes – Parents’ Desired Moderation Strategies
| Focused Code: Type of Transparency | Description | Example Cues from Parents |
|---|---|---|
| Live Monitoring | Parent can watch or join conversations in real time or check in while use is ongoing. | “Monitor usage live,” “use alongside parent,” “periodic check-ins during use.” |
| Full Transcript Access | Parent can access a complete log of all child–AI interactions. | “Full transcript,” “complete transcript,” “read all interactions.” |
| Summarized Recaps | Parent receives condensed summaries of conversations or topics instead of full logs. | “Summary of conversations,” “summary of child prompts,” “daily recap,” “recap of topics discussed.” |
| Event-Triggered Alerts | Parent receives notifications only when certain conditions are met (e.g., risky prompts, heavy use, repetitive topics). | “Alert for concerning prompt,” “alert for questionable intention,” “alert for repetitive topics,” “alert for app use,” “send a notification when needed.” |
| Child-Led Disclosure Only | Awareness comes from the child choosing to talk with the parent, not from system-mediated monitoring. | “Allow child to come directly to parent,” “discussion with child, no model–parent notice.” |
| No AI-Mediated Transparency | Parent does not want the system to notify or share any information about the child’s use. | “None,” “no action taken towards the parent,” “I don’t want/need to know.” |
Table 13. Codebook: Focused Codes – Parents’ Desired Transparency from Child–AI Use
| Axial Category | Description | Example Initial Codes |
|---|---|---|
| System Risk (Model / System Behavior) | Harm arises because of how the model responds, regardless of what the child later does. | Misinterprets child’s intent; introduces harmful ideas; confusing or low-quality explanation; lacks risk awareness; not age appropriate / too complex; emotionally harmful or insensitive. |
| Misuse Risk (Child Behavior) | Harm arises from how the child might use, repurpose, or respond to the model’s outputs. | Uses knowledge in unexpected way; deliberate harmful intention; undermines parental authority; children become overdependent on AI; reveals sensitive information; parent does not trust AI to be used appropriately. |
Table 14. Axial coding for parents’ concerns: locus of risk. We grouped initial codes into two higher–level categories: (1) System risk , where parents attributed harm to the model’s behavior, and (2) Misuse risk , where parents worried about how children might use the model or its outputs.
| Focused Transparency Code | Awareness Level | Data Content | Parental Involvement |
|---|---|---|---|
| Live Monitoring | High (continuous awareness) | Live interaction (ongoing conversation) | Constant monitoring / usage alongside parent; periodic check-ins during use. |
| Full Transcript Access | Highest (complete but asynchronous awareness) | Full transcript of all interactions | Post-interaction review; parent can read the record when desired. |
| Summarized Recaps | Moderate (overview, not every message) | Summaries of conversations and child prompts; daily recap of topics | Post-interaction review; parents browse summaries rather than raw logs. |
| Event-Triggered Alerts | Low (only when something happens) | Flagged activity and during-use alerts (e.g., concerning prompt, questionable intention, repetitive topics, app use) | Passive notification; parent is notified by the system or the child. |
| Child-Led Disclosure Only | Low (no system report) | N/A (content shared only if child chooses) | Discussion with child, no model–parent notice. |
| No AI-Mediated Transparency | None (no awareness from the system) | N/A | No involvement; parent does not want or need to know, sometimes citing privacy. |
Table 15. Axial coding for transparency: dimensions underlying each focused code. For transparency the axial coding involved splitting parents’ descriptions into three interrelated dimensions and then recombining them: (1) Awareness level (how much the parent knows: from none, to low-alert based, to moderate summary, to high live access). (2) Data content (what information is shared: live interaction, full transcript, summaries, or only flagged activity). (3) Parental involvement (how and when the parent becomes involved: constant monitoring, check-ins during use, post-interaction review, passive notification, or no involvement).
More in this topic
Human-AI Interaction
Human-AI interaction studies how people and AI systems communicate, collaborate, make decisions, and shape one another over time.
See all 326 papers