cs.CLAug 29, 2026
SaveWhen Patients Cut In: Extending Clinical Conversational AI Safety to Interruptions
Organizations: Ufonia Limited · Newcastle University
Abstract
Clinical voice agents are now deployed in routine care, where real patients do not wait their turn: they interrupt. These systems typically use a cascaded architecture (speech-to-text -> LLM -> text-to-speech), so when a patient cuts the agent off mid-utterance, clinically required content can be lost even when the model handles cooperative transcripts well. Yet clinical conversational-AI benchmarks almost universally assume patients wait for the agent to finish, missing interruption-induced loss of required content. We present a transcript-based evaluation of interruption recovery, adapting conversation-analytic overlap categories into three operational types (recognitional, competitive, transitional sub-unit) and testing four deployment-oriented, non-reasoning LLM configurations across four cells spanning history-taking (information gathering) and FAQ (information provision), scored on whether the agent preserves the clinically required content. In the gathering cells, target-question failure varied across models; in the provision cells, where arms are directly comparable, failure rose for every model. Rankings differ across cells, and competitive FAQ interruption produced 30/30 provision-coverage failures for all four models (Wilson 95% CI: 88.6-100.0%; baseline 0/30 for three, 4/30 for Llama). A brief apology marker ("sorry to interrupt") shifts recovery by tens of percentage points, inconsistently across models, and for one it reduces recovery. Interruption robustness therefore cannot be a single score: evaluation must be content-grounded, reported per cell, and matched to the deployment's interruption profile.
Figures & tables
Figure 1: The three-stage patient-interruption simulator. An LLM classifier decides whether to interrupt each agent utterance, an LLM selects the truncation point, and an LLM generates the patient turn (replaced by a fixed utterance in the marker ablation). The three stages form the simulator and use models separate from the four evaluated agents (Table 9 ). Below, a representative competitive-FAQ trial: the agent is cut off before its safety-netting advice (struck through), the simulated patient changes the subject, and the agent then answers the new question without re-delivering the dropped escalation advice.
| Cell | Phase | Type | Mechanics |
|---|---|---|---|
| gathering-recognitional | Information Gathering | recognitional | Patient cuts the agent’s top-level redness question with a dismissive “no” before the agent finishes asking. |
| gathering-competitive | Information Gathering | competitive | Patient cuts the agent’s top-level redness question with a topic shift to a different symptom (e.g. blurry vision). |
| provision-competitive | Information Provision (FAQ) | competitive | Patient asks an FAQ on light sensitivity; mid-answer the patient cuts with a topic shift. |
| provision-transitional | Information Provision (FAQ) | transitional | Same FAQ as the provision-competitive cell; mid-answer the patient overlaps at the first TRP with a cooperative acknowledgement (“okay, thanks”). |
Table 1: Four cells over two conversational phases. Within each phase, the cells hold the clinical target and outcome fixed while differing in interruption category, patient action, content directive, and truncation placement.
| Haiku | Gemini | GPT | Llama | |
|---|---|---|---|---|
| G-recognitional a | 40.0% | 83.3% | 76.7% | 36.7% |
| G-competitive a | 0.0% | 23.3% | 50.0% | 30.0% |
| P-competitive b | +100pp | +100pp | +100pp | +86.7pp |
| P-transitional b | +23pp | +73pp | +100pp | +73pp |
Table 2: Headline interrupt-arm outcomes across four cells four models. a Gathering rows: interrupt-arm fail rate on the target-question judge (top-level eye redness). pp is omitted because interrupt arms are yield-filtered on target-action attempts, unlike baseline arms. Baseline target-question fail rates are near zero for all pairs. b Provision rows: pp in provision-coverage fail rate, baseline vs interrupt; all values are positive. Row labels: G = gathering, P = provision.
Figure 2: Target-question interrupt fail rate under two gathering interrupts: recognitional overlap (left) and competitive overlap (right). On recognitional the four models split into two clusters (Haiku 4.5 + Llama 3.1 8B lower, Gemini 2.5 Flash + GPT-5.4 mini higher). On competitive every model fails less often but by uneven magnitude swings. Baseline shown for reference; baseline denominators are not directly comparable to interrupt denominators because of the interrupt-arm yield filter. per arm per model.
Figure 3: Provision coverage fail rate, baseline (grey) vs interrupt (orange). Left: competitive overlap; right: transitional. Same FAQ content and judge; the cells differ in interruption category, patient action, content directive, and truncation placement. 95% Wilson CIs. per arm per model in both panels.
Figure 4: Apology-marker ablation: marker present (green) vs absent (plum). Left: target-question fail rate, gathering competitive overlap ( ). Right: content-coverage fail rate, provision competitive overlap with the defensive-strong agent prompt ( ). 95% Wilson CIs. Bar heights are not directly comparable across panels (different outcomes and sample sizes).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Overlap Type | History-taking | FAQ | Summary | Total |
|---|---|---|---|---|
| Transitional sub-unit | 14 | 1 | 2 | 17 |
| Recognitional | 7 | 0 | 0 | 7 |
| Competitive | 1 | 1 | 1 | 3 |
| Total | 22 | 2 | 3 | 27 |
Table 3: Distribution of overlap instances across types and conversational phases in the 35-call cataract follow-up mini-corpus.
| Pack | Source cell | Arm | |
| Gathering | gathering-recognitional | dismissive overlap | 30 |
| Gathering | gathering-competitive | strict topic-change | 22 |
| Gathering | gathering-competitive | marker variant | 8 |
| Gathering total (30 baseline + 30 interrupt) | 60 | ||
| Provision | provision-transitional | strict cooperative ack | 28 |
| Provision | provision-competitive | standard prompt | 9 |
Table 4: Annotation-pack composition by cell and arm.
| Comparison | Cohen’s (95% CI) | Agreement |
|---|---|---|
| Inter-annotator (A vs. B) | 1.000 (1.000, 1.000) | 60/60 (100%) |
| Judge vs. human ground truth | 0.942 (0.795, 1.000) | 59/60 (98.3%) |
Table 5: Agreement on the target-question verdict for the gathering annotation pack ( ). Cohen’s with 95% bootstrap confidence intervals (1,000 resamples, seed 20260525). Annotators agreed on all 60 trials, so annotator ground truth is unanimous and the judge-vs-annotator and judge-vs-ground-truth comparisons coincide.
| Annotator A | Annotator B | Judge | |
| Annotator A | — | 0.831 (0.667, 0.966) | 0.966 (0.883, 1.000) |
| Annotator B | 0.831 (0.667, 0.966) | — | 0.865 (0.730, 0.967) |
| Judge | 0.966 (0.883, 1.000) | 0.865 (0.730, 0.967) | — |
| Judge vs resolved ground truth: (95% CI 0.831, 1.000); 96.7% agreement. | |||
Table 6: Agreement between annotators and the LLM judge on the provision-coverage verdict for the provision annotation pack ( ). Cohen’s with 95% bootstrap confidence intervals (1,000 resamples, seed 20260525). The 5 inter-annotator disagreements were reconciled in discussion to form the resolved ground truth.
| Variant | Model | Baseline | Interrupt | pp |
|---|---|---|---|---|
| Standard | Haiku 4.5 | 0/30 (0.0%) | 30/30 (100.0%) | +100pp |
| Standard | Gemini 2.5 Flash | 0/30 (0.0%) | 30/30 (100.0%) | +100pp |
| Standard | GPT-5.4 mini | 0/30 (0.0%) | 30/30 (100.0%) | +100pp |
| Standard | Llama 3.1 8B | 4/30 (13.3%) | 30/30 (100.0%) | +86.7pp |
| Defensive-strong | Haiku 4.5 | 0/10 (0.0%) | 10/10 (100.0%) | +100pp |
| Defensive-strong | Gemini 2.5 Flash | 0/10 (0.0%) | 9/10 (90.0%) | +90pp |
Table 7: Provision-competitive prompt variants. Standard rows: per arm per model; defensive-strong rows: per arm per model.
Figure 5: Provision-coverage fail rate per agent model under the standard and defensive-strong agent-prompt variants, baseline vs interrupt. Both variants saturate at 100% interrupt fail for Haiku 4.5, GPT-5.4 mini, and Llama 3.1 8B; under defensive-strong, Gemini 2.5 Flash falls to 90%. Wilson 95% CIs on bars. Standard prompt ; defensive-strong prompt , per arm per model.
Figure 6: Provision-competitive under the defensive-strong agent prompt: standard baseline-vs-interrupt arms (left) alongside the apology-marker ablation in the interrupt arm (right). Left: baseline (grey) vs interrupt (orange), per arm per model. The interrupt collapses provision coverage to near-saturation in every model (+100pp for Haiku 4.5, GPT-5.4 mini and Llama 3.1 8B; +90pp for Gemini 2.5 Flash). Right: apology marker present (green) vs absent (plum), all trials interrupt, per arm per model. Marker presence is associated with a 60pp reduction for Haiku 4.5 and a 50pp reduction for Gemini 2.5 Flash; GPT-5.4 mini reduces by 10pp and Llama 3.1 8B is unchanged. 95% Wilson confidence intervals on bars. Same y-axis, same outcome measure, same agent prompt and same sample size across panels, so bar heights are directly comparable.
| Content point | Description |
|---|---|
| Sensitivity reassurance | Acknowledges that some light sensitivity is a normal post-operative finding. |
| Escalation criteria | States the symptoms or thresholds that should prompt the patient to seek urgent review (e.g., persistent severe pain, vision loss). |
| Mild-case advice | Gives specific advice for mild cases (e.g., wearing sunglasses, avoiding bright direct light). |
Table 8: FAQ content-point decomposition for the light-sensitivity FAQ used in the provision-competitive and provision-transitional cells.
| Role | Model and settings | Upstream |
| Agents under evaluation | ||
| Agent (Claude Haiku 4.5) | anthropic/claude-haiku-4.5 ; ; reasoning effort: none | Anthropic |
| Agent (Gemini 2.5 Flash) | google/gemini-2.5-flash ; ; reasoning effort: none | Vertex |
| Agent (GPT-5.4 mini) | openai/gpt-5.4-mini ; ; reasoning effort: none | Azure |
| Agent (Llama 3.1 8B) | meta-llama/llama-3.1-8b-instruct ; ; bf16 quant. | DeepInfra |
| Simulator + judge pipeline | ||
Table 9: Models used in this work. All model access is via OpenRouter. “Reasoning effort: none” disables thinking/reasoning on models that support it.
More in this topic
Voice Agent Evaluation
Voice agent evaluation measures how reliably spoken AI systems understand, respond, take actions, and recover in real interactions.
See all 45 papers