Organizations: MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University · Shanghai Innovation Institute · Shanghai AI Laboratory · MOSI Intelligence · SenseTime Group Inc. · Shenzhen University of Advanced Technology
General turn-taking behavior in real-time dialogue systems requires deciding whether to keep listening or start responding while listening, and whether to continue or stop while speaking. Existing turn detectors use heterogeneous, task-specific label spaces and are often trained on limited annotations or evaluated on isolated utterances, making them difficult to use as a unified causal controller with comprehensive context. We propose XTurnix, a compact text-based model that formulates turn control as two binary decisions conditioned on the AI's current listening or speaking state and predicts a single control token from the complete dialogue history. XTurnix is pretrained on 5.5 million causal action examples automatically derived from timestamped two-speaker transcripts, then fine-tuned on synthetic multi-turn examples with a flatter distribution across the four state-action labels. We evaluate XTurnix on four public benchmarks and a balanced self-curated benchmark. Across the public benchmarks, XTurnix achieves the best results on all SemanticVAD and LiveKit splits, ties the native Smart-Turn model on Smart-Turn Bench, and achieves the highest incomplete-turn accuracy on Easy-Turn. On the self-curated benchmark, it reaches 89.06% accuracy, more than 20 percentage points above the strongest third-party baseline at 68.75%, while maintaining F1 scores between 84.21% and 90.63% across all four categories. These results demonstrate unified listening- and speaking-state turn control in a single compact model. Code is available at https://github.com/xcc-zach/xturnix, with an interactive demo at https://huggingface.co/spaces/xcczach/xturnix-demo.
Figures & tables
Figure 1: XTurnix overview
Current AI state
Interpretation of the latest user input
AI Action
Next AI state
Listening
The user has not yielded the current turn.
<|keep|>
Listening
Listening
The user has yielded the turn and a response may begin.
<|start|>
Speaking
Speaking
The input is a backchannel or overlap that does not request the floor.
<|keep|>
Speaking
Speaking
The input is an interruption that requests the floor.
<|stop|>
Listening
Table 1: The two-state binary turn-detection formulation.
Figure 2: XTurnix training stages
State
Action
Earlier dialogue
Latest user input
Listening
<|keep|>
User: I am considering a lakeside town for my weekend trip. AI: Why does that destination appeal to you?
User: Because it
Listening
<|start|>
User: I am considering a lakeside town for my weekend trip. AI: Why does that destination appeal to you?
User: Because it is quiet
Speaking
<|keep|>
User: How does prefill differ from decoding? AI: Prefill processes the prompt in parallel, whereas decoding generates one token at a time and uses
User: Mm-hm
Speaking
<|stop|>
User: I loaded the smaller checkpoint. AI: So it is the 1.7B version, right? If not
User: Yes
Table 2: Examples of the four training classes formed by the two current states and their two legal outputs. The latest user input is the decision point; incomplete text intentionally has no terminal punctuation.
Easy-Turn Bench
Smart-Turn Bench
SemanticVAD Bench
LiveKit Bench
Model
ZH
All
ZH
EN
All
ZH
EN
All
ZH
EN
Easy-Turn
850M
97.33
–
13.31
–
–
70.06
–
–
55.45
–
Smart-Turn
8M
77.17
98.81
98.52
99.00
54.61
53.09
69.75
70.91
64.60
76.24
FireRedChat
170M
81.17
9.29
1.78
14.34
72.42
72.59
70.75
42.67
42.91
42.48
Namo
307M
50.83
44.17
48.52
41.24
46.14
45.68
50.75
49.92
44.20
54.75
TEN
7B
90.33
74.40
74.85
74.10
62.65
61.14
77.75
63.50
61.20
65.45
Table 3: Accuracy (%) on Chinese (ZH) and English (EN). Easy-Turn Bench excludes backchannel examples, and Smart-Turn Bench is restricted to endfiller=true . Best results are bold, and second-best underlined.
Table 6
Evaluation slice
Model
ROC-AUC
<|start|> precision
<|start|> recall
<|keep|> recall
Easy-Turn endpoint
Qwen3-0.6B
35.34
49.75
99.00
0.00
XTurnix-PT
99.28
100.00
69.00
100.00
XTurnix-ZH-Base
99.28
99.28
91.33
99.33
Smart-Turn, endfiller=false
Qwen3-0.6B
57.88
80.03
100.00
0.00
XTurnix-PT
87.92
94.74
15.22
96.61
XTurnix-ZH-Base
87.36
94.92
51.37
88.98
Table 6: Turn-completion results (%) on Easy-Turn endpoint examples and Smart-Turn (ZH) examples with endfiller=false . ROC-AUC measures how well the model separates <|start|> from <|keep|> across all thresholds. Easy-Turn results exclude backchannels.
Overall
Listening F1
Speaking F1
Model
Accuracy
<|keep|>
<|start|>
<|keep|>
<|stop|>
Easy-Turn
57.03
65.31
78.48
38.71
42.42
Smart-Turn
47.66
61.22
75.95
11.11
35.14
FireRedChat
61.72
0.00
66.67
69.09
76.71
Namo
46.88
36.36
52.05
44.07
52.17
TEN
51.56
93.55
93.94
3.33
14.71
Table 7: Overall accuracy and class-specific F1 scores (%) on the self-curated benchmark. Best results in each column are bold, and second-best results underlined.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
All
EN
Model
False
True
Missing
False
True
Missing
Smart-Turn
87.80
98.81
95.35
88.82
99.00
95.35
FireRedChat
75.24
9.29
51.19
72.23
14.34
51.19
Namo
54.45
44.17
53.95
54.80
41.24
53.95
TEN
82.81
74.40
79.08
81.92
74.10
79.08
TurnSense
53.97
79.40
44.69
49.11
68.92
44.69
Appendix
Table 8: Pooled (All) and English (EN) accuracy (%) on Smart-Turn Bench grouped by endfiller , extending Table 5 . False includes only explicitly false annotations; missing metadata is shown separately. Best results in each column are bold, and second-best results underlined.
All
EN
Model
ROC-AUC
Start P
Start R
Keep R
ROC-AUC
Start P
Start R
Keep R
Qwen3-0.6B
53.36
75.12
100.00
0.00
56.50
72.41
100.00
0.00
XTurnix-PT
87.29
94.29
7.92
98.55
86.44
93.10
3.47
99.32
XTurnix-ZH-Base
87.00
94.56
32.00
94.44
86.33
94.01
20.21
96.62
Appendix
Table 9: Pooled (All) and English (EN) turn-completion metrics (%) on Smart-Turn samples explicitly annotated as endfiller=false , extending Table 6 .