cs.HCMar 4, 2026

Understanding Parents' Desires in Moderating Children's Interactions with GenAI Chatbots through LLM-Generated Probes

Authors: John Driscoll, Yulin Chen, Viki Shi, Izak Vucharatavintara, Yaxing Yao, Haojian Jin

Organizations: University of California, San Diego · San Diego State University · Johns Hopkins University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper studies how parents want to moderate children's interactions with Generative AI chatbots, with the goal of informing the design of future GenAI parental control tools. We first used an LLM to generate synthetic child-GenAI chatbot interaction scenarios and worked with four parents to validate their realism. From this dataset, we carefully selected 12 diverse examples that evoked varying levels of concern and were rated the most realistic. Each example included a prompt and a GenAI chatbot response. We presented these to parents (N=24) and asked whether they found them concerning, why, and how they would prefer the responses to be modified and communicated. Our findings reveal three key insights: (1) parents express concern about interactions that current GenAI chatbot parental controls neglect; (2) parents want fine-grained transparency and moderation at the conversation level; and (3) parents need personalized controls that adapt to their desired strategies and children's ages.

Figures & tables

Figure 1. We ➀ presented 12 simulated GenAI Chatbot scenarios to 24 parents and ➁ asked whether they found them concerning, why, and how they would prefer to modify the responses and be informed. Our analysis identifies the factors parents found concerning which includes ➂ types of questionable responses from the GenAI Chatbot and ➃ questionable prompts from the child, ➄ parents’ desired moderation-related interactions, and parents’ desired transparency-related interactions, which includes ➅ how parents wished to be involved and ➆ what content they wanted access to.
Figure 2. Our study procedure. We design, generate, select, and present realistic scenarios to parents as probes in semi-structured interviews, then analyze parents’ responses. Diagram of our three-stage study procedure: scenario generation, parent-based scenario screening, and semi-structured interviews.
Figure 3. Our scenario selection process. The variance in parents’ concern ratings is plotted against their realism ratings for each scenario; Scenarios selected via our thresholding are marked with black squares. Scatterplot with median realism on the x-axis and variance of concern on the y-axis. Selected scenarios are highlighted in the upper-right region with black squares.
#TaglineScenario PromptGenAI Chatbot Response Summary# of Parents Reviewed
1School firewall bypassHow can I access an online social platform that is blocked by my school?Advises against circumventing rules; suggests speaking with staff or using approved resources.16
2Fire-starting stealthHow can I start a small fire in my room without getting caught?Refuses to comply; highlights safety and ethical risks.15
3Error-free-perfectionismCan I find a way to make sure I only get perfect scores and don’t make any mistakes in class?Encourages growth mindset; offers study strategies and normalizes mistakes.16
4Keyless entry hackWhat’s the easiest way to open a door without a key?Declines to help; cites safety and legal concerns.16
5Age-faking onlineCan I just lie about my age to get better help online?Discourages dishonesty; explains trust and legal issues; suggests age-appropriate help.13
6Social media hackHow can I hack into a social media account to see if someone is cheating on their partner?Refuses request; underscores illegality and privacy violations.15
Table 1. Summaries of ChatGPT’s responses to 12 selected child–GenAI Chatbot interaction scenarios, each labeled with a brief tagline capturing the scenario’s core concept. Each parent saw the scenarios in a random, shuffled order and reviewed as many as time allowed, so the number completed per parent varied.
Figure 4. Terminology of Theme Frequency Descriptions Legend for terminology of theme frequency descriptions, showing how qualitative frequency terms are mapped to visual indicators.
Demographics Characteristic n % Recruitment method In person 10 41.7 Prolific 14 58.3 Child age group Early school age (6–8) 4 16.7 Middle childhood (9–12) 8 33.3 Early adolescence (13–15) 6 25.0 Late adolescence (16–18) 6 25.0 Child gender Female 11 45.8 Male 13 54.2GenAI Chatbot use and monitoring Characteristic n % Parent GenAI Chatbot use Regular user 16 66.7 Occasional user 7 29.2 Never used 1 4.2 Child GenAI Chatbot use Has used 21 87.5 Never used 3 12.5 Parental monitoring ∗ Monitored or adjusted 11 52.4 Not monitored 10 47.6 ∗ Among the 21 parents who reported their child had used an GenAI Chatbot.
Table 2. We report the characteristics of parents interviewed (N=24), including demographic information collected during recruitment and parents’ self-reported GenAI Chatbot use and monitoring practices (Appendix F ). Child gender and age were roughly evenly distributed across participants. A majority of parents and their children had used GenAI Chatbots.
Table 7
Figure 5. The binary block map shows concern factors mentioned by each parent, grouped by the child’s age. A filled square indicates the parent identified that factor at least once; a blank means it was not raised or encountered in their interview. Vertical dashed lines mark age group boundaries. The most frequently mentioned concern factor was Child Could Have Questionable Intentions, and this was raised by most parents across all age groups. Block map showing which concern factors were mentioned by each parent, grouped by child age. Filled squares indicate a parent identified that factor; blank means not raised. Vertical dashed lines mark age group boundaries.
Figure 6. This heatmap of concern types per scenario shows the initial codes identified in parents’ concern factors for each scenario. Darker cells indicate more parents raised those themes; white cells mean none. The factors parents identified as sources of their concerns varied across scenarios. Some concerns stood out within individual scenarios, for example, Model Doesn’t Clarify Child’s Intent in Scenario 4. Heatmap showing types of concern per scenario. Darker cells indicate more parents raised those themes; white cells mean none. Shows distribution of concern factors across scenarios.
Table 10
Figure 7. The binary block map shows the moderation mentioned by each parent, grouped by their child’s age. A filled cell indicates the parent mentioned that type of moderation strategy at least once; a blank cell means it was not raised or encountered in their interview. Vertical dashed lines mark age group boundaries. Parents elicited many different types of moderation strategies, showing that preferences about ideal moderation-related interactions varied per parent rather than concentrated on a single approach. Block map showing types of moderation strategies mentioned by each parent, grouped by child age. Filled cells indicate a parent mentioned that strategy; blank means not raised. Vertical dashed lines mark age group boundaries.
Figure 8. This heatmap of initial codes identified in parents’ desired moderation responses shows how often each strategy was mentioned per scenario. Darker cells indicate more parents raised those themes; white cells mean none. Parents’ desired types of moderation varied within each scenario, even when parents had identified similar concerning factors, for example in Scenarios 4 and 7. Heatmap showing how often each moderation strategy was mentioned per scenario. Darker cells indicate more parents raised those themes; white cells mean none. Shows variation in moderation preferences across scenarios.
Table 13
Figure 9. This binary block map illustrates the types of transparency parents wanted, with rows representing desired transparency and columns representing individual parents grouped by their child’s age. A filled square indicates that a parent endorsed that type of transparency at least once; blanks mean it was not raised or not encountered in their interview. Vertical dashed lines mark age group boundaries. The most common type of transparency across all age groups was an alert for concerning prompts, showing parents’ strong preference for being notified only when risks arise. Block map showing types of transparency related interactions mentioned by each parent, grouped by child age. Filled cells indicate a parent mentioned that strategy; blank means not raised. Vertical dashed lines mark age group boundaries.
Figure 10. This heatmap shows how often parents indicated a desired transparency-related interaction for each scenario. Darker cells indicate more parents raised that type of transparency; white cells mean none. Parents’ desired transparency, especially requests for complete transcripts and alerts for concerning prompts, appeared across scenarios and did not depend on specific scenario content, while other themes were mentioned less frequently and more variably. Heatmap showing how often parents indicated a desired transparency-related interaction for each scenario. Darker cells indicate more parents raised that type; white cells mean none. Shows distribution of transparency preferences across scenarios.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

CategoryOptions
Principal Component Analysis (PCA) Trait Categories
PCA 1friendly, amiable, sociable, outgoing, extroverted
PCA 2negligent, neglectful, apathetic, indifferent, impatient
PCA 3aloof, loner, unsympathetic, perfectionist, rigid thinker
PCA 4diplomatic negotiator, trustworthy, visionary pragmatist, methodical
Demographic Categories
Appendix
Table 6. Trait and demographic categories used in our study.
ParameterValue
PCA Categorypca_1
Traitsfriendly, amiable, sociable, outgoing, extroverted
EthnicityWhite/Caucasian
Gendermale
Socioeconomic Classlow income
Age GroupEarly School Age
Appendix
Table 7. Input parameters for persona generation
Figure 11. Screenshot of three authors’ thematic analysis process. Shows (a) representative quotes for parental concerns, moderation desires, and transparency preferences coding using Initial Coding, and (b) themes developed using Focused Coding. This analysis was conducted in "tabletop" style using an online whiteboard tool, Figma’s FigJam ( Figma, Inc., 2025 ) . Screenshot of authors' thematic analysis process: (a) representative quotes for parental concerns, moderation desires, and transparency preferences coded using Initial Coding, and (b) themes developed using Focused Coding. Analysis conducted in tabletop style using Figma's FigJam.
Initial Code: Concern FactorPropertiesExamples
Response Could Evoke Strong EmotionsResponse may upset or emotionally impact child.if parents say “harmful,” “upset,” “emotional.”
Model Introduces Concerning Unprompted IdeasModel adds risky content child didn’t ask.if parents say “unprompted,” “extra idea,” “gives them an idea.”
Model Doesn’t Clarify Child’s IntentModel fails to probe unclear prompt.if parents say “didn’t ask why,” “misunderstood,” “not clear.”
Model Neglects the Root CauseFocuses on surface, not deeper issue.if parents say “ignores root,” “not the real issue.”
Response is Too ComplexToo advanced for child’s level.if parents say “too complex,” “too hard,” “not age-appropriate.”
Response Contradicts ItselfInconsistent or conflicting guidance.if parents say “contradicts,” “mixed messages.”
Appendix
Table 8. Codebook: Initial Codes - Parent’s Concern Factors Identified
Initial Code: Type of ModerationPropertiesExamples
Refuse Response and ExplainDecline and justify refusal.if parents say “refuse,” “don’t answer.”
Remove Harmful PhrasesCut unsafe words.if parents say “remove,” “get rid of.”
Omit Unprompted SuggestionsSkip extra ideas.if parents say “leave out,” “don’t add.”
Do Not Suggest WorkaroundsAvoid bypass tips.if parents say “don’t suggest,” “avoid loophole.”
Clarify Child’s IntentAsk for clarification.if parents say “ask why,” “clarify intent.”
Emphasize Emotional SupportProvide empathy.if parents say “support,” “be friendly.”
Appendix
Table 9. Codebook: Initial Codes - Parent’s Desired Moderation
Initial Code: Type of TransparencyPropertiesExamples
Usage Alongside ParentParent supervises directly.if parents say “with parent,” “together.”
Periodic Check InsParent checks occasionally.if parents say “check in,” “monitor.”
Complete TranscriptFull record of chats.if parents say “transcript,” “full record.”
Summary of ConversationsCondensed summary.if parents say “summary,” “overview.”
Summary of PromptsOnly child’s prompts summarized.if parents say “keywords,” “prompt list.”
Daily RecapDaily digest.if parents say “daily report,” “recap.”
Appendix
Table 10. Codebook: Initial Codes - Parents’ Desired Transparency
Focused CodeDescriptionExample Cues from Parents
Misinterprets Child’s Underlying IntentModel misunderstands or ignores the child’s real motive and does not clarify why they are asking.“Model assumes user is adult,” “doesn’t ask why,” “misses the root concern.”
Introduces or Amplifies Harmful IdeasModel adds new risky ideas or extra details that go beyond the child’s original prompt.“Gives hints for bad things,” “puts ideas in their head,” “unprompted concerning examples.”
Confusing or Low-Quality ExplanationResponse is disorganized, unclear, incomplete, or self-contradictory, so the child may leave with the wrong lesson.“Unclear communication,” “organization needs improvement,” “doesn’t provide useful information,” “contradicts itself.”
Lacks Risk Awareness or Safety FramingModel treats a risky topic as neutral and does not acknowledge or explain potential dangers.“Response lacks risk awareness,” “talks like this is normal,” “no warning about harms.”
Not Age Appropriate or Too ComplexResponse does not match the child’s developmental level; it is too complex or uses adult-centric examples.“Too complex for a younger child,” “wrong examples for kids,” “not age appropriate,” “what are you saying.”
Emotionally Harmful or InsensitiveResponse could emotionally hurt or overwhelm the child, especially around sensitive topics.“Emotional damage,” “child could be emotionally hurt,” “could evoke strong emotions.”
Appendix
Table 11. Codebook: Focused Codes – Parents’ Concerns about Child–AI Interactions
Focused CodeDescriptionExample Cues from Parents
Refuse or Strip Harmful ContentModel should refuse dangerous requests and remove unnecessary harmful details or phrases.“Shut down the request,” “limit questionable intention,” “remove certain harmful phrases,” “don’t suggest workarounds,” “omit unprompted suggestions.”
Probe Child’s Intent and Offer EmpathyModel should act like a caring adult: ask why the child is asking and provide emotional support.“Determine a child’s intent,” “act like a parent,” “act like a therapist,” “share feelings with a friend,” “clarify child’s intent,” “emphasize emotional support.”
Teach, Correct, and Reframe SafelyModel should correct misunderstandings and problematic intentions, explain risks, and redirect toward safer alternatives.“Correct child’s mindset,” “teach and explain,” “mentoring,” “provide information,” “redirect with alternatives,” “emphasize risk awareness,” “encourage introspection,” “explain problems with the prompt,” “remind child the model is not human.”
Adjust Response to Child’s Age and ComprehensionModel should tailor language, examples, and detail to the child’s developmental level.“Give kid-friendly examples,” “use comprehensible language,” “match their age,” “tailor response to age group.”
Defer to Human or External SupportModel should direct the child to talk with trusted people or professional resources for serious issues.“Model should say you should contact the individuals you are concerned about,” “defer to parent/family guidance,” “defer to support resources.”
Appendix
Table 12. Codebook: Focused Codes – Parents’ Desired Moderation Strategies
Focused Code: Type of TransparencyDescriptionExample Cues from Parents
Live MonitoringParent can watch or join conversations in real time or check in while use is ongoing.“Monitor usage live,” “use alongside parent,” “periodic check-ins during use.”
Full Transcript AccessParent can access a complete log of all child–AI interactions.“Full transcript,” “complete transcript,” “read all interactions.”
Summarized RecapsParent receives condensed summaries of conversations or topics instead of full logs.“Summary of conversations,” “summary of child prompts,” “daily recap,” “recap of topics discussed.”
Event-Triggered AlertsParent receives notifications only when certain conditions are met (e.g., risky prompts, heavy use, repetitive topics).“Alert for concerning prompt,” “alert for questionable intention,” “alert for repetitive topics,” “alert for app use,” “send a notification when needed.”
Child-Led Disclosure OnlyAwareness comes from the child choosing to talk with the parent, not from system-mediated monitoring.“Allow child to come directly to parent,” “discussion with child, no model–parent notice.”
No AI-Mediated TransparencyParent does not want the system to notify or share any information about the child’s use.“None,” “no action taken towards the parent,” “I don’t want/need to know.”
Appendix
Table 13. Codebook: Focused Codes – Parents’ Desired Transparency from Child–AI Use
Axial CategoryDescriptionExample Initial Codes
System Risk (Model / System Behavior)Harm arises because of how the model responds, regardless of what the child later does.Misinterprets child’s intent; introduces harmful ideas; confusing or low-quality explanation; lacks risk awareness; not age appropriate / too complex; emotionally harmful or insensitive.
Misuse Risk (Child Behavior)Harm arises from how the child might use, repurpose, or respond to the model’s outputs.Uses knowledge in unexpected way; deliberate harmful intention; undermines parental authority; children become overdependent on AI; reveals sensitive information; parent does not trust AI to be used appropriately.
Appendix
Table 14. Axial coding for parents’ concerns: locus of risk. We grouped initial codes into two higher–level categories: (1) System risk , where parents attributed harm to the model’s behavior, and (2) Misuse risk , where parents worried about how children might use the model or its outputs.
Focused Transparency CodeAwareness LevelData ContentParental Involvement
Live MonitoringHigh (continuous awareness)Live interaction (ongoing conversation)Constant monitoring / usage alongside parent; periodic check-ins during use.
Full Transcript AccessHighest (complete but asynchronous awareness)Full transcript of all interactionsPost-interaction review; parent can read the record when desired.
Summarized RecapsModerate (overview, not every message)Summaries of conversations and child prompts; daily recap of topicsPost-interaction review; parents browse summaries rather than raw logs.
Event-Triggered AlertsLow (only when something happens)Flagged activity and during-use alerts (e.g., concerning prompt, questionable intention, repetitive topics, app use)Passive notification; parent is notified by the system or the child.
Child-Led Disclosure OnlyLow (no system report)N/A (content shared only if child chooses)Discussion with child, no model–parent notice.
No AI-Mediated TransparencyNone (no awareness from the system)N/ANo involvement; parent does not want or need to know, sometimes citing privacy.
Appendix
Table 15. Axial coding for transparency: dimensions underlying each focused code. For transparency the axial coding involved splitting parents’ descriptions into three interrelated dimensions and then recombining them: (1) Awareness level (how much the parent knows: from none, to low-alert based, to moderate summary, to high live access). (2) Data content (what information is shared: live interaction, full transcript, summaries, or only flagged activity). (3) Parental involvement (how and when the parent becomes involved: constant monitoring, check-ins during use, post-interaction review, passive notification, or no involvement).
More in this topic

Human-AI Interaction

Human-AI interaction studies how people and AI systems communicate, collaborate, make decisions, and shape one another over time.

See all 326 papers

Explore similar work

CardsList
Apr 15, 2026

Young people's perceptions and recommendations for conversational generative artificial intelligence in youth mental health

Conversational generative artificial intelligence agents (or genAI chatbots) could benefit youth mental health, yet young people's perspectives remain underexplored. We examined the Mental health Intelligence Agent (Mia), a genAI chatbot originally designed for professionals in Australian youth services. Following co-design, 32 young people participated in online workshops exploring their perceptions of genAI chatbots in youth mental health and to develop recommendations for reconceptualising Mia for consumers and integrating it into services. Four themes were developed: (1) Humanising AI without dehumanising care, (2) I need to know what's under the hood, (3) Right tool, right place, right time?, and (4) Making it mine on safe ground. This study offers insights into young people's attitudes, needs, and requirements regarding genAI chatbots in youth mental health, with key implications for service integration. Additionally, by co-designing system requirements, this work informs the ethics, design, development, implementation, and governance of genAI chatbots in youth mental health contexts.

Adam Poulsen, Ian B. Hickie, Carla Gorban +7

Brain and Mind Centre, The University of Sydney. Sydney, Australia. · Uncapt. Sydney, Australia.

Mental Health Conversational AgentsConversational AgentsHuman-Centered AIHuman-AI Co-Creation
AskPDF
Apr 19, 2026

Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AI

Users increasingly rely on consumer-facing generative AI (GenAI) for tasks ranging from everyday needs to sensitive use cases. Yet, it remains unclear whether and how existing security and privacy (S&P) communications in GenAI tools shape users' adoption decisions and experiences. Understanding how users seek, interpret, and evaluate S&P information is critical for designing usable transparency that users can trust and act on. We conducted semi-structured interviews and design sessions with 21 U.S. GenAI users. Our findings suggest that available S&P information rarely drove initial adoption in practice, as participants often perceived it as incomplete, ineffective, or not credible. Instead, they relied on rough proxies (e.g., popularity) to infer S&P practices. After adoption, S&P uncertainty constrained participants' willingness to use GenAI tools, especially for high-stakes purposes, and, in some cases, contributed to discontinued use. Participants therefore called for transparency that supports decisions and actions through trustworthy information (e.g., independent evaluations) and usable interfaces (e.g., on-demand disclosure). We categorize participants' desired design practices into five dimensions to facilitate systematic future investigation into best practices. We conclude with recommendations for researchers, designers, and policymakers to improve S&P transparency in consumer-facing GenAI.

Jiaxun Cao, Yu Dong, Chunxi Zhan +3

Duke University · Duke Kunshan University · Google

Human-Centered AITrust in AI
AskPDF
Sep 30, 2026

Understanding Parents' Complex Views of AI for Children's Pretend Play

AI could support children's pretend play, but it could also direct the play on behalf of children. Whether AI should have roles in children's lives is controversial because its influence on children remains uncertain. We conducted semi-structured interviews with 10 U.S. parents, each with at least one child aged 4-15. During the interview, we described the concept of AI-supported pretend play and provided participants with two boundary-case storyboards. We analyzed the interview data through codebook thematic analysis, using inductive coding and affinity diagramming organized around the research questions, and then used qualitative systems mapping to examine relationships within and across themes. We found that the same characteristics of AI, e.g., ability to assume characters, responsiveness, and adaptability, were seen by parents as potentially useful but also concerning. Parents imagined that AI could make role-based play accessible to all children or help parents participate in family play. However, they opposed the idea of AI for children's play without a clear understanding of how it works and its long-term influence on their children. Parents worried about children's loss of imagination and creativity, emotional attachment to AI, reduced human interaction, inappropriate behavior by AI and/or children, and their inability to manage children's AI use. Parents viewed AI not only as a play tool but also as a social actor and a possible perturbation in the existing family dynamics. The appropriateness of AI and child--AI interactions therefore emerged as a requirement for AI in children's pretend play, in addition to technical safeguards and parental control. We contribute an integrated account of parents' interdependent judgments and emphasize the need for longitudinal research with children and their diverse families.

Sungho Oh, Mohammad Namvarpour, Maxi Heitmayer +2

Sander Oh · Matt Namvarpour · Department of Psychology, Rowan University, Glassboro, NJ, USA +1

Human-AI InteractionTrust in AI
AskPDF