OpenPA: Forced-alignment-free Multi-granular Pronunciation Assessment with Mixed-transcript Adaptation
Organizations: National Taiwan Normal University, Taipei, Taiwan
Abstract
Recent advances in automatic pronunciation assessment (APA) have driven a shift from scripted to free-form assessment. However, existing methods often rely on multi-stage pipelines with forced alignment and separate feature extractors, limiting their practicality across response settings. In light of this, we propose OpenPA, a novel APA framework that leverages Whisper's cross-attention mechanism to jointly predict word- and utterance-level pronunciation scores without forced alignment. Through mixed-transcript adaptation and adaptive substitution weighting, OpenPA accommodates heterogeneous transcripts across closed- and open-response settings within a single model. Trained solely on speechocean762, OpenPA achieves the strongest overall performance among evaluated systems under in-domain closed and simulated open-response conditions, while maintaining its performance zero-shot on MultiPA. An additional cross-task evaluation on oral-cancer speech suggests its potential applicability beyond non-native speech.
Figures & tables
| Transcript | Type | Training | Word Acc. (M+S/M/S)↑ | Utt. Avg.↑ |
|---|---|---|---|---|
| medium.en | GT-only | 0.416 / 0.479 / 0.118 | 0.765 | |
| (WER: 20.1% ) | Seen | Mixed | 0.504 / 0.471 / 0.287 | 0.779 |
| Full | 0.506 / 0.451 / 0.295 | 0.779 | ||
| large-v3 | GT-only | 0.458 / 0.546 / 0.152 | 0.764 | |
| (WER: 16.3% ) | Unseen | Mixed | 0.526 / 0.521 / 0.328 | 0.777 |
| Full | 0.526 / 0.504 / 0.335 | 0.780 |