cs.HCMar 12, 2026
SaveELLA: Generative AI-Powered Social Robots for Early Language Development at Home
Organizations: Johns Hopkins University, USA
Abstract
Early language development shapes children's later literacy and learning, yet many families have limited access to scalable, high-quality support at home. Recent advances in generative AI make it possible for social robots to move beyond scripted interactions and engage children in adaptive, conversational activities, but it remains unclear how to design such systems for pre-schoolers and how children engage with them over time in the home. We present ELLA (Early Language Learning Agent), an autonomous, generative AI-powered social robot that supports early language development through interactive storytelling, parent-selected language targets, and scaffolded dialogue. Using a multi-phased, human-centered process, we interviewed parents (n=7) and educators (n=5) and iteratively refined ELLA through twelve in-home design workshops. We then deployed ELLA with ten children for eight days. We report design insights from in-home workshops, characterize children's engagement and behaviors during deployment, and distill design implications for generative AI-powered social robots supporting early language learning at home.
Figures & tables
Figure 1. We present ELLA (Early Language Learning Agent), an interactive storytelling robot designed with educators and families to support early language development in children ages 4–6. The robot tells at most four stories per day upon request with a goal of teaching four target words per day determined by parents. The robot incorporates the target words into stories generated using LLMs based on the children’s interests and engages the child through targeted and scaffolded interactions. This figure illustrates one sample story session from our deployment study. The children’s name was replaced with pseudonyms. The target vocabulary word that the robot is trying to teach is “Sympathy” and the theme of this story is the TV show “Fancy Nancy”. Greeting/farewell from different story sessions were presented to showcase different possibilities. This figure presents a flowchart illustrating the structure of an interactive storytelling session with ELLA, the Early Language Learning Agent. The target vocabulary word (sympathy) and the story theme (Fancy Nancy, a television show) are specified at the start of the session. The interaction begins with a Greeting stage, where the robot introduces itself and asks the child whether they would like to hear a story. If the child agrees, the robot proceeds to Storytelling, narrating a story that embeds the target word in context and provides a child-friendly definition. The session then moves through Story Perception, Story Recall, and Target Word Practice stages, during which the robot checks the child’s understanding and encourages reflection on the target word through guided questions. At the end of the session, the robot asks whether the child would like to hear another story. If the child chooses “yes,” the flow returns to storytelling for an additional story. If the child chooses “no,” or if the maximum number of stories has been reached, the interaction ends with a Farewell stage.
Figure 2. To design a social robot to support language development in children ages 4- 6, we engaged in a three-stage design process involving: 1) online interviews; 2) in-home workshops; and 3) in-home deployment study. This figure outlines a three-stage design process used to develop a social robot to support language development in children aged 4–6. Stage 1 consists of online interviews with parents (n=7) and educators (n=5) to understand needs and inform the initial system design. Stage 2 consists of in-home workshops (n=12) organized into multiple iterative phases: Phase 1 (n=3) focuses on improving interaction quality; Phase 2 (n=4) focuses on improving story content; Phase 3 (n=3) focuses on improving interaction questions; and Phase 4 (n=2) focuses on improving onboarding. Stage 3 is an eight-day in-home deployment study (n=10) with four steps: (1) an introduction letter from ELLA; (2) onboarding with a vocabulary test and setup; (3) eight days of interactive storytelling with ELLA; and (4) offboarding with a vocabulary test, interview, and survey.
| Pseudo- nyms | Age | Gender | Ethnicity | Behavioral & learning differences | Household size | Household income | 50% AMI | 80% AMI | Days/week reading (Eng.) | Minutes/day reading (Eng.) | Phases involved |
| P1-Grace* | 4 | female | Asian | typical | 6 | 125k | above | below | 5 days | 10–20 min | 1, 2, 3 |
| P2-Andrew | 4 | male | Black and White | speech delay, speech therapy, undergoing diagnosis for behavioral & neurological differences | 6 | 75k | below | below | 4 days | 10–20 min | 1, 2, 3 |
| P3-Sarah | 4 | female | White | typical | 3 | $200k | above | above | 7 days | 10–20 min | 1, 2, 3 |
| P4-Sam | 4 | male | Asian | speech therapy | 4 | 125k | above | above | 2 days | 10–20 min | 1 |
| P5-Tom | 4 | male | White | typical | 4 | prefer not to answer | above | above | 7 days | 30 min | 2 |
| P6-George | 4 | male | Multiethnic | typical | 3 | 200k | above | above | 7 days | 0–10 min | 3 |
Table 1. Family Demographics. (*) indicate siblings. ASD stands for autism spectrum disorder. ADHD stands for Attention-deficit/hyperactivity disorder. 50% of area median income (AMI) is a standard benchmark used by local government programs to define very low-income households, while 80 percent of AMI commonly defines low-income households ( of Housing and Development, ; Program, ; Aid, ) . Age indicates age at the start of the study.
| P | Gender | Race | Education | Experience | School Type | Student Age | Student Socio-Economic Status |
| 1 | F | Hispanic | Master’s | 7 yrs | Public (GE, SE, ESL) | 4–6 | Mixed |
| 2 | F | White | Bachelor’s | 20 yrs | Public (GE, SE, ESL) | 3–5 | Mixed |
| 3 | F | White | Master’s | 40 yrs | Private (GE) | 4–11 | High |
| 4 | F | Asian | Bachelor’s | 3.5 yrs | Private (GE, ESL) | 3–4; 14–18 | High |
| 5 | M | Black | Master’s | 7 yrs | Public Charter (GE, SE, ESL) | 7–8; 10–14 | Mixed |
Table 2. Teacher Demographics. GE stands for General Education. SE stands for Special Education. ESL stands for English as a Second Language.
Figure 3. Pipeline for generating personalized and targeted content for interactive storytelling with ELLA. Design evolution ( Phase 1 , Phase 2 , Phase 3 , Phase 4 ) of components indicated by color. This figure presents the ELLA Interaction Content Generation Pipeline, with color-coded components indicating design evolution across Phases 1–4. The pipeline begins with inputs including the child’s name, target vocabulary word, and story theme, which are used to generate a story and associated interactions. The pipeline then branches, with one path converting the story to audio and producing a timestamped transcript, and the other passing the story and target word directly forward. Both paths feed into a high-level analysis stage that identifies interaction elements such as palettes, motifs, cues, and story segments. These elements are used to generate facial and bodily behaviors, which are converted into executable code and stored, along with the story and audio, in a cloud-based story database. Each stage in the diagram includes example artifacts illustrating the information passed between components.
Figure 4. Example story and behaviors. S denotes behaviors generated with the storytelling behavior generation pipeline. The rest of the behaviors are generated using the interaction behavior generation pipeline. Child name is replaced with pseudonym. This figure shows an example storytelling session with ELLA, including both spoken content and the robot’s accompanying behaviors. The child’s name is replaced with a pseudonym. The session begins with a greeting and proceeds through storytelling, during which the robot performs synchronized gestures (e.g., arm, head, and base movements) and changes facial expressions to emphasize the narrative and target vocabulary. The interaction also includes comprehension questions and feedback, and concludes with a farewell and sleep animation. Behaviors marked with S are generated by the storytelling behavior generation pipeline, while the remaining behaviors are generated by the interaction behavior generation pipeline. Each interaction state is paired with a corresponding image of the robot.
Figure 5. ELLA System Diagram. Design evolution ( Phase 1 , Phase 2 , Phase 3 , Phase 4 ) of components indicated by color. This figure presents a system diagram of ELLA, illustrating how speech input, story content, and interaction logic are integrated to generate robot behaviors. Color-coded components indicate design evolution across Phases 1–4. Considered as a whole, the system takes audio input from the child, processes it to determine turn-taking and interaction state, and uses language models to manage storytelling flow, moderation, and dialogue decisions. Story content is retrieved or generated and combined with high-level interaction handling to determine whether to continue storytelling, scaffold responses, or conclude the session. Based on these decisions, the system generates coordinated facial expressions, gestures, and animations, which are executed by the robot. The diagram highlights the end-to-end flow from user speech and story content to robot behaviors, emphasizing modular design and interaction safety.
Figure 6. ELLA Usage over time. This figure shows a bar chart of ELLA usage over time, measured as the number of stories told per day for each child across an eight-day deployment period. The horizontal axis represents the day of deployment, and the vertical axis represents the number of stories per day. Each color of bars corresponds to an individual child, illustrating day-to-day variation in usage. Bars with diagonal hatching indicate weekend days, during which usage patterns differ from weekdays. Across children, usage ranges from zero to four stories per day, reflecting differences in engagement frequency over time. The chart highlights both periods of consistent daily use and days with no usage, providing an overview of how storytelling activity varied across participants and across the deployment duration.
| Who Initiated | n | Who Were Present | n | How Child Felt About Storytelling Session | n |
| Parent prompted | 34 | Parent | 42 | Neutral | 23 |
| Scheduled routine | 19 | Sibling | 27 | Excited | 20 |
| Child initiated | 16 | Child alone | 5 | Frustrated | 7 |
| Sibling encouraged | 7 | Other caregivers | 2 | Hesitant | 6 |
| Friend initiated | 1 | Friend | 1 | Want to continue beyond session | 3 |
| Happy | 1 |
Table 3. Context based on parent diaries.
Figure 7. Child survey results. The blue line marks the median. This figure presents results from a child survey evaluating interactions with ELLA. The survey includes items assessing enjoyment and perceived understanding, such as liking ELLA, finding the stories fun, feeling understood when talking to ELLA, and enjoying talking to ELLA. For each item, responses are summarized using descriptive statistics, with mean scores ranging from 3.9 to 4.2 and standard deviations between 1.1 and 1.32. The blue line indicates the median response for each survey item, providing a visual reference for central tendency across participants.
| Pseudonym | Age | Target Words | Example Theme | Days used ELLA | Stories listened to | |||
| Grace | 4 | Imitate | Ordinary | Clumsy | Massive | Rapunzel (disney princess) | 5 | 18 |
| Andrew | 4 | Imagine | Confident | Permission | Self-control | Tom and Jerry (TV show) | 4 | 13 |
| Sarah | 4 | Perseverance | Compassion | Gumption | Awestruck | Pony | 7 | 18 |
| George | 4 | Permission | Consequences | Orbit | Chirp | Ironman | 2 | 4 |
| Jason | 5 | Imitate | Somersault | Clumsy | Frisky | Science | 8 | 11 |
| Natalie | 5 | Apartment | Advocate | Justice | Bait | Dance | 7 | 12 |
Table 4. Target words and themes for each child during deployment. Green highlight represents words that the child learned by the end of the deployment study.
| Pseudonym | Age | Stories listened to | Target words used | Target word usage during story sessions | Total target word used | Avg words/ turn |
| Grace | 4 | 18 | 3/4 | Massive (14), Ordinary (16), Clumsy (13), Imitate (0) | 43 | 3.78 |
| Andrew | 4 | 13 | 3/4 | Permission (3), Self-Control (2), Imagine (1), Confident (0) | 6 | 3.48 |
| Sarah | 4 | 18 | 4/4 | Compassion (2), Awestruck (2), Perseverance (0), Gumption (0) | 4 | 3.48 |
| George | 4 | 4 | 1/4 | Chirp (2), Permission (0), Consequences (0), Orbit (0) | 2 | 2.46 |
| Jason | 5 | 11 | 2/4 | Clumsy (8), Imitate (1), Somersault (0), Frisky (0) | 9 | 4.91 |
| Natalie | 5 | 12 | 2/4 | Advocate (2), Bait (1), Justice (1), Apartment (0) | 4 | 3.02 |
Table 5. Child Language usage during deployment. Green highlight represents words that the child learned by the end of the deployment study.
| Lesson: Inviting Caregiver Partnership Could Enrich Language Development | |
| Design Opportunity 1 | Make learning gains visible through caregiver-facing reflective summaries. |
| Design Opportunity 2 | Extend language learning beyond session through robot-caregiver collaboration. |
| Lesson: Stories Need Freshness to Retain Playfulness Over Instructional Tone | |
| Design Opportunity 3 | Support low-effort, ongoing co-authoring to keep content aligned with shifting interests and goals. |
| Design Opportunity 4 | Balance meaningful repetition and novelty through narrative variation. |
| Lesson: Making Talk Feel Safe Needs Legible, Low-Pressure Interaction | |
Table 6. Lessons Learned and Design Opportunities for ELLA
Figure 8. Our deployment data highlighted interesting interaction patterns that inspire new design opportunities. We highlight instances of object-based engagement, sibling dynamics, and sharing of personal experiences here. Pseudonyms are used, and the target vocabulary for the given interaction is highlighted in red. This figure presents qualitative examples from the deployment that illustrate recurring interaction patterns with ELLA. The examples highlight three types of engagement: object-based engagement, sharing of personal experiences, and sibling dynamics. In the object-based engagement example, a child temporarily leaves the interaction to retrieve a personal toy related to the story theme and target vocabulary. In the personal experience example, the child connects the story to a real-life adventure and expands on their own experiences during the interaction. In the sibling dynamics example, an older sibling provides whispered assistance while the child responds to the robot’s questions. Pseudonyms are used for all children, and target vocabulary words referenced in the interactions are highlighted in red.
Figure 9. Examples demonstrating additional interaction patterns. We highlight instances of parent prompting, playful rambling, and story requests here. Pseudonyms are used, and the target word for the given interaction is highlighted in red. Overlapping speech is shown in brackets. This figure presents examples from the deployment that illustrate recurring interaction patterns with ELLA. The examples highlight three types of engagement: parent prompting, playful rambling, and story request. In the parent prompting example, the parent helps model for the child how to say their one-word response in a full sentence. In the playful rambling example, the child responds with blah blah blah and unintelligible halfway through their sentence. In the story request example, a child interrupts Ella to ask for a specific story.
More in this topic
Human-Robot Interaction
Human-robot interaction studies how people and robots communicate, coordinate, and safely work or live together.
See all 441 papers