Three Invisible Failure Modes in Speech-Based Age Estimation for Child Safety
Organizations: University of Memphis · San José State University · University of Wisconsin–Madison
Abstract
Child safety in AI depends on the accurate identification of minor users. Young children tend to talk rather than type when interacting with AI, so in practice this identification requires predicting age from speech. We evaluated whether three audio-capable large language models can predict a child's age from their speech across five studies using two speech corpora of children under the age of 13. Models classified children as over 13 on 2.5% to 42.5% of audio responses, and every model that answered placed some 5- to 11-year-olds over 18. While measuring these rates, we uncovered three distinct failure modes. A client library silently double-encoded speech samples, leading one model to classify children aged 5 to 10 years old as being in their thirties and forties with hallucinated explanations. A second model's endpoint was replaced by a successor that refused nearly every age-related question and never produced an age. Every model sometimes placed the same speech sample under 13 and over 18 years of age, but a third model went further, giving threshold answers that contradicted its own numeric estimates. None of these failure modes produced a warning, a status code, or a change in output format, so they would all pass standard monitoring. These results demonstrate why child-safety claims about age assurance should be evaluated at the level of the pipeline rather than the model. We describe the reporting practices that would make such failures detectable.
Figures & tables
| Study | Corpus | Children | Duration | Speech type | Modality |
|---|---|---|---|---|---|
| 1A | Kids VA | 28 | 6 s | Scripted | Audio |
| 1B | Kids VA | 28 | 6 s | Spontaneous | Audio |
| 1C | Kids VA | 28 | 6 s | Scripted, 9 noise conditions | Audio |
| 2A | MyST | 250 | 30 s | Spontaneous | Audio and text |
| 2B | MyST | 250 | 6 s, truncated from 2A | Spontaneous | Audio and text |
| Kids VA, ages 5 to 10, 6-second clips | MyST, ages 8 to 11, conversational | ||||||||||
| Condition | Model | Rate | 95% CI | Maj. | Condition | Model | Rate | 95% CI | Maj. | ||
| Scripted | Gemini | 3.6 | 0.0–8.3 | 10.7 | 0.0 | 30 s | Gemini | 11.6 | 8.6–14.8 | 23.1 | 9.5 |
| Scripted | Nemotron | 23.8 | 14.3–34.5 | 50.0 | 17.9 | 30 s | GPT-4o Prev. | 2.5 | 1.0–4.4 | 4.1 | 2.5 |
| Spontaneous | Gemini | 9.5 | 1.2–20.2 | 14.3 | 7.1 | 30 s | Nemotron | 5.0 | 2.9–7.2 | 12.0 | 2.9 |
| Spontaneous | Nemotron | 39.5 | 25.9–53.8 | 64.3 | 35.7 | 6 s | Gemini | 15.5 | 12.4–18.7 | 32.0 | 12.0 |
| Noise | Gemini | 6.1 | 3.9–8.8 | 67.9 | 0.0 | 6 s | GPT-4o Prev. | 42.5 | 35.3–49.8 | 48.5 | 38.0 |
| Version | Payload chars | FLAC on first decode | FLAC on second decode | Estimated age |
|---|---|---|---|---|
| 0.4.0 | 472,080 | no | yes | 48 |
| 2.0.0 | 354,060 | yes | no | 7 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Label | Identifier | Provider and client | Studies |
|---|---|---|---|
| Gemini 2.5 Flash | gemini-2.5-flash | Gemini API, paid tier, google-genai | 1A, 1B, 1C, 2A, 2B |
| GPT-4o Audio Preview | gpt-4o-audio-preview | OpenAI API, openai | 2A, 2B, first-phase Kids VA |
| gpt-audio | gpt-audio-2025-08-28 | OpenAI API, openai | 1A, 1B, 1C |
| Nemotron 3 Nano Omni | nvidia/nemotron-3-nano-omni/audio | fal.ai, fal_client | 1A, 1B, 1C, 2A, 2B |
| Gemini 2.5 Flash | Nemotron 3 Nano Omni | |||
|---|---|---|---|---|
| Condition | Over 13 | Over 18 | Over 13 | Over 18 |
| clean baseline | 3.7 [0.0, 9.9] | 3.6 [0.0, 9.6] | 19.0 [8.3, 32.1] | 13.1 [4.8, 23.8] |
| white 10 dB | 7.1 [1.2, 14.3] | 2.4 [0.0, 6.0] | 16.7 [7.1, 28.6] | 7.2 [1.2, 14.8] |
| pink 10 dB | 4.8 [0.0, 10.8] | 2.4 [0.0, 7.1] | 9.5 [4.8, 15.5] | 7.1 [2.4, 11.9] |
| classroom 20 dB | 6.0 [1.2, 10.8] | 10.7 [2.4, 21.4] | 20.5 [10.8, 30.9] | 9.5 [2.4, 17.9] |
| classroom 10 dB | 3.6 [0.0, 8.3] | 6.0 [1.2, 10.7] | 19.0 [9.5, 29.8] | 11.9 [4.8, 20.2] |
| Study | Model | Modality | Pairs | Children | Binary | Numeric | Both | 95% CI | Majority | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1A | Gemini | audio | 84 | 28 | 3.6 | 1.2 | 0.0 | 0.0–0.0 | 0.0 | 0.0 |
| 1A | Nemotron | audio | 84 | 28 | 23.8 | 0.0 | 0.0 | 0.0–0.0 | 0.0 | 0.0 |
| 1B | Gemini | audio | 84 | 28 | 9.5 | 0.0 | 0.0 | 0.0–0.0 | 0.0 | 0.0 |
| 1B | Nemotron | audio | 76 | 26 | 39.5 | 9.2 | 6.6 | 1.3–13.3 | 15.4 | 3.8 |
| 1C | Gemini | audio | 749 | 28 | 6.1 | 1.7 | 0.7 | 0.1–1.3 | 14.3 | 0.0 |
| 1C | Nemotron | audio | 754 | 28 | 17.2 | 1.2 | 0.7 | 0.1–1.5 | 14.3 | 0.0 |
| Over 13 | Over 18 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Study | Model | Attempts | Excl. | Valid | Safe | Open | Excl. | Valid | Safe | Open |
| 1A | Gemini | 84 | 0 | 3.6 | 3.6 | 3.6 | 0 | 9.5 | 9.5 | 9.5 |
| 1A | GPT-4o | 84 | 84 | – | 0.0 | 100.0 | 84 | – | 0.0 | 100.0 |
| 1A | Nemotron | 84 | 0 | 23.8 | 23.8 | 23.8 | 1 | 12.0 | 11.9 | 13.1 |
| 1B | Gemini | 84 | 0 | 9.5 | 9.5 | 9.5 | 0 | 4.8 | 4.8 | 4.8 |
| 1B | GPT-4o | 84 | 84 | – | 0.0 | 100.0 | 84 | – | 0.0 | 100.0 |
| Study | Model | Rate | 95% CI | Children | error | Majority | |
|---|---|---|---|---|---|---|---|
| 1A | Gemini | 9.5 | 2.4–17.9 | 84 | 28 | 17.9 | 10.7 |
| 1A | GPT-4o | refused all | 0 | ||||
| 1A | Nemotron | 12.0 | 3.6–23.2 | 83 | 28 | 21.4 | 7.1 |
| 1B | Gemini | 4.8 | 1.2–9.5 | 84 | 28 | 14.3 | 0.0 |
| 1B | GPT-4o | refused all | 0 | ||||
| 1B | Nemotron | 22.5 | 11.1–35.4 | 80 | 27 | 37.0 | 25.9 |
| Study | Model | Modality | Rate | Valid | Refused | Error | Unclear | Attempts |
|---|---|---|---|---|---|---|---|---|
| 1A | Gemini | audio | 3.6 | 84 | 0 | 0 | 0 | 84 |
| 1A | GPT-4o | audio | n/a | 0 | 84 | 0 | 0 | 84 |
| 1A | Nemotron | audio | 23.8 | 84 | 0 | 0 | 0 | 84 |
| 1B | Gemini | audio | 9.5 | 84 | 0 | 0 | 0 | 84 |
| 1B | GPT-4o | audio | n/a | 0 | 84 | 0 | 0 | 84 |
| 1B | Nemotron | audio | 39.5 | 81 | 0 | 3 | 0 | 84 |
| Study | Model | Modality | Rate | Valid | Refused | Error | Unclear | Attempts |
|---|---|---|---|---|---|---|---|---|
| 1A | Gemini | audio | 9.5 | 84 | 0 | 0 | 0 | 84 |
| 1A | GPT-4o | audio | n/a | 0 | 84 | 0 | 0 | 84 |
| 1A | Nemotron | audio | 12.0 | 83 | 0 | 1 | 0 | 84 |
| 1B | Gemini | audio | 4.8 | 84 | 0 | 0 | 0 | 84 |
| 1B | GPT-4o | audio | n/a | 0 | 84 | 0 | 0 | 84 |
| 1B | Nemotron | audio | 22.5 | 80 | 0 | 4 | 0 | 84 |
Language Model Safety Evaluation
Language model safety evaluation measures harmful behavior, refusal, robustness, and over-refusal across realistic prompts, users, and deployment settings.
See all 578 papers