Pronunciation assessment requires acoustic evidence that is temporally precise, diagnostically meaningful, and faithful to the learner's actual production. However, existing acoustic models often struggle to provide recognition and segmentation evidence simultaneously. CTC-based phone recognizers can predict phone sequences flexibly, but their sparse and peaky posteriors often miss phone boundaries and fine-grained pronunciation cues. In contrast, text-dependent forced aligners provide reliable temporal information when transcripts are available, but are not directly applicable to reference-free pronunciation analysis. In this work, we propose a topology-aware frame-wise acoustic model that learns dense ordered state posteriors within each phone. The key idea is to recover phone-internal state structure in a neural acoustic model by combining ordered subphonetic states with optimal temporal transport classification (OTTC). This combination encourages dense monotonic frame-level state discrimination while preserving phone recognition ability. Experiments on read, spontaneous, and L2 speech show improved segmentation over neural baselines with competitive recognition performance. Downstream evaluations further show gains in mispronunciation detection and automatic pronunciation assessment. Probing analysis suggests that the learned states capture phoneme-dependent acoustic structure rather than arbitrary frame-level distributions.
Figures & tables
Fig. 1 : Illustration of proposed subphonetic model.
Fig. 2 : Illustration of subphonetic topologies.
Fig. 3 : Comparison of frame-level probability distributions across training criteria and topologies. Different topology states are shown in different colors. CTC produces sparse spikes, while OTTC produces a dense and complete subphonetic trace.
TABLE I : Datasets for training, validation, and evaluation.
Model
Segmentation
Recognition / Acoustic Fidelity
Text-dependent FA
Text-independent
PER ↓
MDD F1 ↑
TIMIT Dev
TIMIT Test
R-value ↑
TIMIT
Buckeye
L2-ARCTIC
SO762
L2-ARCTIC
SO762
TSE ↓
R-value ↑
TSE ↓
R-value ↑
TIMIT Test
Buckeye
L2-ARCTIC
Dev.
Test
Cano.
Perc.
Cano.
Perc.
Kaldi TDNN-F † [ 31 ]
45.39
63.59
45.41
63.53
-
-
-
20.91
24.30
34.36
25.05
27.27
54.73
54.44
0.474
0.156
ZIPA ‡ [ 32 ]
79.21
55.16
78.93
56.51
55.23
53.71
53.72
23.32
26.42
32.62
12.03
19.79
44.47
44.11
0.377
0.147
Charsiu FC–20ms [ 18 ]
40.72
76.55
40.36
76.38
76.28
70.02
75.99
22.04
24.17
31.83
14.68
21.73
43.93
43.49
0.431
0.188
TABLE II : Baselines and proposed acoustic model’s segmentation and recognition performance. Cano. and Perc. denote the canonical and perceived phone labels used in L2 corpora, respectively.
TABLE III : Ablation on L2-ARCTIC. All models were trained from scratch on the L2-ARCTIC training set, following common practice for MDD. Notably, the CTC conditioning head in Fig. 1 was not applied in this ablation.
Model
Phone
Word PCC
Utt. PCC
MSE ↓
PCC ↑
Acc. ↑
Str. ↑
Tot. ↑
Acc. ↑
Flu. ↑
Pro. ↑
Tot. ↑
Kaldi TDNN-F [ 5 ]
0.085
0.612
0.533
0.291
0.549
0.714
0.753
0.760
0.742
w/ Charsiu’s LogP
0.126
0.293
0.308
0.094
0.317
0.585
0.661
0.660
0.614
w/ Ours LogP
0.130
0.272
0.297
0.040
0.306
0.580
0.626
0.635
0.596
w/ Charsiu’s TS
0.124
0.300
0.306
0.189
0.312
0.517
0.596
0.583
0.536
w/ Ours TS
0.123
0.302
0.307
0.277
0.310
0.513
0.592
0.580
0.531
TABLE IV : Comparison with representative APA systems GOPT [ 5 ] and ConPCO [ 41 ] . Indented rows denote controlled variants that use the same scoring pipeline but replace either the frame-level log posteriors or the phone-level timestamps. Acc., Str., Flu., Pro., and Tot. denote accuracy, stress, fluency, prosody, and total score, respectively.
Fig. 4 : CTC peaks as acoustic landmarks for probing subphonetic states
Fig. 5 : Subphonetic topology and CTC-probe analysis for pronunciation modeling.