The Importance of Speech Genre for End-of-Turn Detector Training
Organizations: ⋆Uniphore
Abstract
End-of-turn (EoT) detection has been studied mostly through features, models, and modalities, while the speech genre of the training data has drawn far less attention. We isolate this factor. Using one reproducible annotation pipeline, we build a matched pair of corpora, multi-turn human-human conversational and single-turn human-machine, and train the same streaming ConvLSTM EoT detector on each. Across four test sets the conversational model reaches up to +0.45 higher F1, driven by fewer false alarms on the mid-turn pauses and hesitations that are rare in single-turn corpora. This conversational-training effect is not specific to frame-level decoding. Under a matched complete-or-not test it also appears at the utterance level, across two encoders, but only on conversational test data. Testing only on single-turn speech, as most prior work does, therefore hides the benefit of conversational training data. Against open-source baselines, our 1.8M-parameter model attains the highest F1 on every test set.
Figures & tables
| Conv-10K | ST-10K | Int-5K | CASPER | ST-C | |
|---|---|---|---|---|---|
| Mean dur. (s) | 11.85 | 7.51 | 11.79 | 4.99 | 9.12 |
| Median dur. (s) | 10.90 | 7.04 | 10.80 | 3.33 | 9.76 |
| Pauses / utt | 8.37 | 5.13 | 8.47 | 6.60 | 6.00 |
| Long pauses / utt | 2.10 | 1.22 | 2.12 | 1.34 | 1.45 |
| Silence ratio | 0.687 | 0.333 | 0.688 | 0.600 | 0.433 |
| Test | Train | F1 | Miss% | FA% | FA/utt | Med. Lat. |
|---|---|---|---|---|---|---|
| Int-5K | Conv-10K | 0.894 | 0.1 | 19.1 | 1.22 | 190ms |
| ST-10K | 0.444 | 0.0 | 71.5 | 5.72 | 20ms | |
| SLURP | Conv-10K | 0.760 | 9.6 | 29.1 | 0.49 | 140ms |
| ST-10K | 0.558 | 20.2 | 41.1 | 1.04 | 20ms | |
| CASPER | Conv-10K | 0.959 | 1.3 | 6.6 | 0.07 | 400ms |
| ST-10K | 0.694 | 2.2 | 44.7 | 1.88 | 260ms |
| Int-5K | CASPER | ST-Complete | ||||||||||
| Model | F1 | FA% | Miss% | Med. Lat. | F1 | FA% | Miss% | Med. Lat. | F1 | FA% | Miss% | Med. Lat. |
| ConvLSTM (ours) | .894 | 19.1 | 0.1 | 190 (4040) | .959 | 6.6 | 1.3 | 400 (2301) | .911 | 15.4 | 0.9 | 250 (2092) |
| Silero VAD | .781 | 33.2 | 2.7 | 50 (3205) | .734 | 29.6 | 12.4 | 12 (1448) | .673 | 13.2 | 36.1 | 110 (1268) |
| WebRTC agg3 | .476 | 66.7 | 2.1 | 20 (1560) | .512 | 52.8 | 12.8 | 0 (861) | .420 | 41.9 | 31.5 | 90 (665) |
| SmartTurn v3 | .523 | 28.9 | 35.7 | 48 (1770) | .390 | 22.0 | 53.7 | 8 (606) | .652 | 50.6 | 12.3 | 150 (928) |
| SmartTurn FT | .525 | 64.0 | 0.4 | 50 (1780) | .644 | 44.6 | 7.9 | 4 (1188) | .694 | 34.6 | 1.0 | 152 (1610) |
Turn-Taking Prediction
Turn-taking prediction estimates when speakers will stop, continue, or enter a conversation, helping dialogue systems respond at appropriate moments.
See all 38 papers