cs.CLSep 4, 2026

TamilEOT: A Dataset and Model for Semantic End-of-Turn Detection in Tamil Telephone Speech

Authors: Santhoshkumar V

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

A voice agent has to decide, at every pause, whether the user has finished speaking. Without a model of the language that decision falls back to a fixed silence timeout: set it short and the agent interrupts, set it long and every turn pays the full wait. Open semantic end-of-turn detectors exist, but to our knowledge none covers a South Indian language. We release TamilEOT: 18,485 labelled turn boundaries cut from 116 real Tamil telephone conversations, and two audio-only detectors fine-tuned from Smart Turn v3. On a held-out split of 4,168 clips from 30 unseen calls, accuracy rises from 70.30% zero-shot to 83.71% (8.7 MB) and 86.13% (21 MB); ROC-AUC rises from 0.751 to 0.921. Both models run in under 150 ms single-threaded on a laptop CPU. We also report what building it cost. Rule-derived labels, checked against a blind human listening pass, were right 95.9% of the time on the positive class and 44.4% on the negative class, which is below chance, because the rule answered a different question than the model is asked. Replacing them with an audio-LLM labeller measured at 97.5% human agreement cost US$5.69. Of every training lever we measured, only encoder capacity moved the result; three runs at identical config and seed span 0.87 accuracy points, which is the floor below which none of our other deltas mean anything. Replaying the same labelled boundaries through the production VAD and streaming adapter costs a further 2.60 points, and 7.8% of boundaries are never surfaced to the model at all. Data, weights, code and every negative result are public.

Figures & tables

Table 1
labelleragreementon hold_intraest. cost, full pool
gemini-3.7-flash97.5%97.0%$5.77
gemini-3.1-pro-preview96.4%93.9%$41.17
gemini-3.6-flash96.4%97.0%$9.10
gemini-3.5-flash93.9%92.9%$11.82
gemini-3-flash-preview93.4%90.9%$17.09
gemini-2.5-flash91.8%90.8%$13.50
Table 3: Labeller bake-off on the 197 blind-listened clips. Six of seven clear the 90% bar. By McNemar the top three are statistically indistinguishable ( p=0.75, 0.73p=0.75,\ 0.73 ), so the tie was broken on price, not on the point estimate.
modelbuildparamssizeaccuracyROC-AUCFP/ NNp50, 1 thread
majority class ( complete )———63.08%0.500——
smart-turn-v3.0 , zero-shotint88.0M—65.64%0.7797.08%—
smart-turn-v3.2 , zero-shotint88.0M8.7 MB70.30%0.75113.44%—
TamilEOT-tinyfp328.0M32 MB83.35%0.9047.73%133 ms
TamilEOT-tinyint8 dyn8.0M8.7 MB83.71%0.9057.94%83 ms
TamilEOT-tinyint8 static8.0M8.4 MB79.32%0.8994.94%73 ms
Table 4: Test split: 4,168 clips from 30 held-out calls, threshold 0.5. FP/ NN is Smart Turn’s convention (see Section 1). Latency is p50, batch 1, one thread, inference only, measured on a single idle development laptop (Intel i5-12450H); add ∼12{\sim}12 ms for the mel front-end. Every build was timed the same way on the same machine, so the column is comparable within itself; it is not a deployment specification (Section 7 ). Released models in bold.
buckettinybase 
change−7.42-7.42−2.05-2.05speaker change already implies completion
change_midseg−1.34-1.34+0.45\mathbf{+0.45}first bucket base clears that tiny does not
hold_intra+31.59+31.59+33.42\mathbf{+33.42}58% of the set — the real EOT problem
hold_inter+31.88+31.88+35.25\mathbf{+35.25} 
disputed−11.76-11.76−6.17-6.17where the two witnesses disagree
Table 5: Accuracy relative to a majority-class policy on each bucket, in points.
levermeasured onΔ\Delta 
whisper-tiny →\rightarrow whisper-basetest accuracy+2.88\mathbf{+2.88}the only one that moved
distillation, base →\rightarrow tinytest accuracy+0.41+0.41inside the 0.87 spread
funnel relaxation, +19.9%+19.9\% train rowstest accuracy+0.03+0.03 
learning-rate retunedev AUC+0.003+0.003reversed once data grew
50% →\rightarrow 100% of data, at tinydev AUC+0.001+0.001flat — capacity-bound
undisputed-only trainingtest accuracy−5.11-5.11 
Table 6: Every training lever measured, ranked. Three separate levers bought about +0.003+0.003 AUC each; one bought six times that.
operating pointthresholdtest accuracytest FP/ NN 
tiny, inherited0.5083.35%7.73%released default
tiny, dev FPR ≤10%\leq 10\%0.7580.64%5.01% 
base, inherited0.5086.23%8.95%released default
base, dev FPR ≤10%\leq 10\%0.9282.41%4.51% 
Table 7: Thresholds picked on dev, measured once on test.
More in this topic

Turn-Taking Prediction

Turn-taking prediction estimates when speakers will stop, continue, or enter a conversation, helping dialogue systems respond at appropriate moments.

See all 37 papers

Explore similar work

CardsList
Sep 24, 2026

BanglaTurn: A Benchmark and Whisper-Based Model for End-of-Turn Detection in Bangla Speech

This paper presents BanglaTurn, a corpus for end-of-turn detection in Bangla conversational speech, and a model trained on it. The corpus holds 35,374 samples of 3 to 15 s of podcast speech, labelled for turn state by combining speaker diarization with an LLM pass, with every label then checked by a human annotator. The model pairs a Whisper encoder with task-specific classification heads. On a class-balanced test set drawn from a held-out podcast, it reaches 84.33% accuracy (95% CI 80.3 to 88.1) against 69.28% for the Smart-Turn v3 baseline, and lowers the false negative rate from 51.57% to 7.55% at the cost of a higher false positive rate. We report what encoder layer fine-tuning, multi-scale pooling and INT8 quantization each contribute, and latency stays within 165 to 191 ms end to end on CPU.

Mizbaul Haque Maruf

Vivasoft Limited, Dhaka, Bangladesh

Turn-Taking Prediction
AskPDF
Sep 10, 2026

Less can be More: What Aspects of Speech Drive End-of-Turn Detection

In conversational AI, detecting when a speaker has finished talking is crucial for natural turn taking. While recent work incorporates semantics, the relative contribution of different modalities remains unclear. We present a controlled ablation of acoustic, prosodic, and semantic signals for streaming end of turn detection using a lightweight trimodal classifier. Under identical training conditions, the acoustic prosodic combination achieves the best balance of accuracy and latency, achieving utterance F1 of 0.93 with 7.8% false alarms at 400ms median latency. Adding text increases premature detections without improving performance. Feature space analysis confirms that prosodic features have the strongest class separability, while text representations overlap substantially. These findings suggest that turn-taking is primarily conveyed through intonation and silence patterns rather than semantic completeness, enabling faster and more reliable systems without expensive text inference.

Rini Sharon, Manickavela A, Kadri Hacioglu +1

Uniphore, India

Speech ProsodyTurn-Taking PredictionTurn-Taking in Dialogue
AskPDF
Oct 15, 2026

VoxTurn: Answering Several Turn-Taking Questions in One Pass with One Model

A cascaded voice agent must decide, throughout each silence of its user, whether the user has finished their turn: answering too early cuts the user off, answering too late makes the agent slow. It often needs related decisions too: will the user soon go on, is speech during its own turn an interruption or a backchannel, is a sound speech at all. With open detectors, each extra decision costs another model or another call, and on the 14-language eot-bench they cut users off in more than a quarter of mid-turn pauses. Because they are trained on segmented utterances, this mismatch is the largest single source of error. We present VoxTurn, a 0.8B speech LLM that casts these decisions as multiple-choice questions, scores the hidden state of each option, and answers any set of them in one forward pass with a block-isolating attention mask, exactly as if asked one at a time. Trained on real conversational pauses with labels derived from their timing, and re-queried while a silence lasts, VoxTurn reaches 11.5% false cut-offs in English at a 300 ms latency budget, comparable to two commercial detectors, and a 14-language mean of 19.9%, the lowest among open models, at 8.3 ms per decision. In Pipecat and LiveKit it cuts off 17% and 9% of turns, and it ranks fifth of 24 in TurnBench end-of-turn detection without any TurnBench training data. The model, code and framework plugins are released.

Yang Xiao

The University of Melbourne

Turn-Taking PredictionSpeech Language ModelsVoice Agent Evaluation
AskPDF