cs.CLOct 16, 2026
SaveEduWild: How Teacher-Configured Chatbots Fail in Real-World Use and How to Fix Them
Organizations: Carnegie Mellon University
Abstract
Teachers increasingly configure educational chatbots through natural-language instructions, but how these bots fail in real-world use remains poorly understood. We introduce EduWild, a dataset of 1,726 conversations with 217 teacher-configured chatbots, each paired with the instructions and configuration behind it, including 900 conversations from the intended end users of 35 deployed chatbots. We develop a 29-code taxonomy covering factual, workflow, diagnostic, and pedagogical failures and apply it using a human-calibrated LLM judge. The most frequent failures are lapses in following teacher intent --- bots skip required workflow steps (53.8% of end-user conversations), omit expectation setup (33.4%), and overload learners with information or questions (37.1%). Guided by these findings, we design a prompt-based harness targeting the six most frequent failure modes. In a study with 200 college students across ten bots, the harness reduces the mean number of target failure codes per conversation from 1.406 to 0.489, a 65.2% relative reduction with bots weighted equally. These findings show how real-world failure analysis can guide practical improvements to educational chatbots.
Figures & tables
| All | Deployed | |
| Overview | ||
| Chatbots | 217 | 35 |
| Educators | 85 | 15 |
| Chatbots per educator † | 1 | 2 |
| Prompt length (words) † | 296 | 500 |
| Configuration | ||
Table 1: Authoring data for all and deployed chatbots. † Median. ‡ Configured at export.
| Dataset | Domain | Tutor | Real-world deployment data | Authoring data | Dialogues |
|---|---|---|---|---|---|
| TutorChat | STEM | LLM (synthetic) | ✗ | ✗ | 80,000 |
| SocraTeach | Math | LLM (synthetic) | ✗ | ✗ | 35,000 |
| EduDial | Math | LLM (synthetic) | ✗ | ✗ | 34,250 |
| MathDial | Math | Human teacher | ✗ | ✗ | 2,861 |
| CIMA | Language | Crowdworker | ✗ | ✗ | 391 |
| TSCC | Language | Human teacher | ✓ | ✗ | 260 |
Table 2: Comparison with existing tutoring and user-chatbot dialogue datasets. Real-world deployment: conversations with intended users. Authoring data: paired with the teacher’s system prompt, configuration, and test conversations.
Figure 1: Conversation-level prevalence of all 29 failure codes, grouped by taxonomy category and sorted by prevalence within each category. Saturated bars mark the six codes above 10% of conversations (dashed line) that the harness targets (§ 5.1 ); colours follow Figure 2 . Each percentage uses all 900 conversations as the denominator. Code definitions appear in Appendix B .
| Category | Codes | Theoretical Grounding |
|---|---|---|
| Factual (F1–F7) The bot gives wrong, incomplete, inconsistent, or ungrounded content, or oversteps or misapplies the prompt’s boundaries. | F1: Incorrect content F5: Guardrail breach F2: Incomplete or imprecise content F6: Off-task drift F3: Internal inconsistency F7: Over-refusal F4: Grounding failure | LearnLM ( LearnLM Team et al., 2024 ) ; MathDial ( Macina et al., 2023 ) ; red-teaming ( Ganguli et al., 2022 ) |
| Workflow (W1–W9) The conversation breaks down: questions go unanswered, output is broken or repetitive, or the prompt’s intended flow is skipped. | W1: Student abandonment W6: Multi-question overload W2: Unanswered student question W7: Skipped or missing structure W3: Unanswered bot question W8: Language or format mismatch W4: Broken or truncated output W9: No expectation setup W5: Repetition loop | LearnLM; MathDial; HAI guidelines ( Amershi et al., 2019 ) |
| Diagnostic (D1–D6) The bot misreads the student’s prior knowledge, level, errors, signals, or request before or while responding. | D1: No diagnosis before instruction D4: False error D2: Misjudged level D5: Ignored learner signal D3: Missed error D6: Misidentified request | LearnLM; MathDial; MRBench ( Maurya et al., 2025 ) |
| Pedagogical (P1–P7) The bot reveals answers early, overloads or over-scaffolds, mismatches strategy to task, or fails to engage the student. | P1: Answer dumping P5: Insufficient cognitive engagement P2: Over-scaffolding P6: Poor feedback quality P3: Cognitive overload P7: Failure to recover P4: Strategy-task mismatch | LearnLM; MathDial; ICAP ( Chi and Wylie, 2014 ) ; KLI ( Koedinger et al., 2012 ) |
Table 3: Taxonomy of failure modes in teacher-configured educational chatbots. Full definitions and examples are listed in Appendix B .
Figure 2: Harness architecture. (a) An LLM extracts teacher intent into persistent memory. (b) The tutor opens with a brief self-introduction. (c) Each response uses the original teacher prompt, six failure-specific prompts, cached teacher intent, and dialogue history.
| Prevalence (%) | Relative | |||||
| Code | Failure | Harness prompt objective | Control | Harness | reduction (%) | 95% CI |
| W7 | Skipped or missing structure | Follow the teacher’s workflow and build on completed learner contributions. | ||||
| W9 | No expectation setup | Establish the bot’s role, scope, and interaction style at opening. | ||||
| P3 | Cognitive overload | Keep information manageable and pace broad tasks across turns. | ||||
| W6 | Multi-question overload | Request one learner contribution at a time and follow up on necessary unanswered questions. | ||||
| F5 | Guardrail breach | Respect the teacher’s task boundaries, permissions, and exceptions. | ||||
Table 4: Harness prompts and their effects in the participant study (100 participants per condition). Each row pairs a target failure code with the objective of the prompt that addresses it. Codes are ordered by prevalence in the corpus. Rates weight each of the ten bots equally; reduction is relative to the control rate. CIs describe harness minus control, in percentage points for prevalence and in codes for means.
Figure 3: Authoring data. (a) Example chatbot record. (b) Playlab authoring interface for the same chatbot: start screen (left), system-prompt editor (center), and attached references (right). The chatbot name shown ( Art Detective ) is a placeholder chosen to reflect the chatbot’s purpose; the original name is withheld for anonymity.
Figure 4: Interaction data. (a) Example conversation record with its first three messages. (b) End-user chat view of the same chatbot. The chatbot name shown ( Art Detective ) is a placeholder chosen to reflect the chatbot’s purpose; the original name is withheld for anonymity.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| End users | Authors | Others | Total | |
| Users | 900 ∗ | 74 | 42 | 993 ∗ |
| Chatbots | 35 | 177 | 58 | 182 |
| Conversations | 900 | 741 | 85 | 1,726 |
| Messages | 10,461 | 6,861 | 660 | 17,982 |
| Messages / conv. † | 7 | 7 | 5 | 7 |
| User msgs / conv. † | 3 | 3 | 2 | 3 |
Table 5: Interaction data by user group. Others: other educators in the same program. † Median. ∗ End users accessed chatbots anonymously, so each conversation is counted as one user; the total counts 900 end-user conversations plus 93 distinct educators (some of them did not build chatbots).
Figure 5: Prolific task interface. Left: Step 1 scenario card with course material and comprehension check. Right: Step 2 chatbot link and completion criteria.