A Native-Reference Phone-Class Geometry for Second-Language Pronunciation Analysis

Tina Raissi; Nhan Phan; Chenxiao Wang; Mikko Kurimo

cs.CLSep 24, 2026

A Native-Reference Phone-Class Geometry for Second-Language Pronunciation Analysis

Authors: Tina Raissi, Nhan Phan, Chenxiao Wang, Mikko Kurimo

Organizations: Department of Information and Communications Engineering, Aalto University, Espoo, Finland

Abstract

Automatic speaking assessment systems can provide holistic proficiency scores, but often lack interpretable measures that characterize pronunciation quality. We propose a native-reference phone-class geometry for measuring second language (L2) pronunciation deviation without requiring pronunciation labels, read-aloud prompts, or matched recordings of the same text from native and L2 speakers. Given a native speech corpus, we average frame-level self-supervised representations for each context-dependent phone-class and use singular value decomposition (SVD) to derive a compact native-reference coordinate system. For each L2 utterance, we compute the corresponding averages and project them into the native-reference space. We then demonstrate that the distances between L2 and native-reference coordinates for matched phone-classes show consistent negative correlations with holistic speaking proficiency on the Dev subset of the Speak and Improve Corpus 2025 (Spearman's $ρ\!=\!-0.53$ ) and with pronunciation quality on the learner subset of the English Read by Japanese Students dataset ( $ρ\!=\!-0.34$ ). These findings suggest that the proposed geometry captures acoustic-phonetic information relevant for proficiency rating while remaining applicable to spontaneous L2 speech without matched native recordings.

Figures & tables

Source	Feature	S&I		UME-ERJ
Source	Feature	Train	Dev	UME-ERJ
Raw	# Words	$0.55$	$0.57$	$-0.20$
Raw	Speaking rate	$0.54$	$0.52$	$\phantom{-}0.13$
$\mathcal{P}^{\star}$	Diphones	$0.60$	$0.62$	$-0.23$
$\mathcal{P}^{\star}$	Triphones	$0.60$	$0.62$	$-0.22$

Table 1 : Non-acoustic baselines. We report Spearman correlation

\rho

for S&I Train, S&I Dev, and UME-ERJ learner using LBS-4h native-reference corpus. Higher positive correlations indicate better agreement with proficiency level.

\mathcal{P}^{\star}

denotes the matched context-dependent phone-classes between native and L2 coordinates in our proposed method.

Source	Feature	S&I		UME-ERJ
Source	Feature	Train	Dev	UME-ERJ
Silence	VAD	$-0.36$	$-0.41$	$\phantom{-}0.10$
Silence	Alignment	$-0.60$	$-0.57$	$-0.07$
ASR	WER	$-0.22$	$-0.20$	$-0.24$
Raw	Mean	N/A		$-0.20$
Encoder	DTW	N/A		$-0.50$

Table 2 : Acoustic baselines for similar evaluations and datasets as in Table 1 . Here, lower negative correlations indicate better agreement with proficiency. The silence ratio is calculated by considering both voice activity detection (VAD) and silence segments from the alignment. Encoder averaging and DTW apply only to UME-ERJ , which provides parallel learner–native recordings.

Class	Avg.	Random Boundary Shift [ms]
Class	Avg.	0	20	40	60	80	120
Diphone	Center	$-0.46$	$-0.45$	$-0.42$	$-0.36$	$-0.30$	$-0.17$
Diphone	Full	$-0.51$	$-0.50$	$-0.47$	$-0.41$	$-0.33$	$-0.19$
Triphone	Center	$-0.55$	$-0.55$	$-0.52$	$-0.48$	$-0.42$	$-0.29$
Triphone	Full	$\mathbf{-0.56}$	$\mathbf{-0.56}$	$\mathbf{-0.54}$	$\mathbf{-0.51}$	$\mathbf{-0.46}$	$\mathbf{-0.34}$

Table 3 : Effect of random boundary shift in milliseconds (ms) for different phone-classes and averaging strategies (Avg.) using the LBS-4h native corpus. Values are Spearman

\rho

of the cosine distance on the S&I Train, averaged over three random seeds.

Native-Ref	S&I Train			UME-ERJ
Corpus	Euc	Maha	Cosine	Euc	Maha	Cosine
TIMIT	$-0.44$	$-0.43$	$-0.51$	$-0.25$	$-0.26$	$-0.13$
LBS-4h	$-0.52$	$-0.51$	$\bf-0.56$	$-0.27$	$\bf-0.28$	$-0.13$
UME-ERJ	$-0.35$	$-0.32$	$-0.48$	$-$

Table 4 : Spearman

\rho

for three native-reference (Native-Ref) corpora evaluated on S&I Train and UME-ERJ learner subset using three distance metrics: Euclidean (Euc), Mahalanobis (Maha), and cosine distance. UME-ERJ native is held out for testing, see Table 6 .

L2 Task	Distance	Spearman $\rho$
L2 Task	Measure	Full	Rnd.	SVD
S&I Train	Euc	$-0.50$	$-0.46$	$\bf-0.52$
	Maha	$-0.44$	$-0.46$	$\bf-0.51$
	Cosine	$\bf-0.58$	$-0.53$	$-0.56$
UME-ERJ	Euc	$\bf-0.28$	$-0.26$	$-0.27$
	Maha	$\bf-0.29$	$-0.26$	$-0.28$

Table 5 : Spearman’s

\rho

between distances and S&I Train proficiency scores or UME-ERJ learner pronunciation scores. We report Euclidean (Euc), Mahalanobis (Maha), and cosine distance metrics under three native-reference coordinate constructions: full (no projection), random (Rnd.) and SVD projections, both of rank

32

. All experiments use triphone classes with full-unit averaging. More negative values indicate better agreement with proficiency. The best correlation in each row is highlighted.

L2 Task	Native-Ref	$\boldmath{\|\boldsymbol{\mathcal{P}}^{\star}\|}$		Correlation
	Corpus	min	max	$\mathbf{r}$	$\boldsymbol{\rho}$
					Raw	Partial	Conf. Int.
							min	max
S&I Dev	LBS-4h	$9$	$394$	$-0.53$	$-0.53$	$-0.19$	$-0.56$	$-0.49$
UME-ERJ	UME-ERJ	$2$	$54$	$-0.37$	$-0.34$	$-0.27$	$-0.37$	$-0.31$

Table 6 : Final evaluation on S&I Dev and UME-ERJ learner using LBS-4h and UME-ERJ native-reference (Native-Ref) corpora. We report the range of matched phone-classes

|\mathcal{P}^{\star}|

, Pearson’s

r

, raw and partial Spearman’s

\rho

, for

|\mathcal{P}^{\star}|

and silence ratio as control variables, and the

95\%

bootstrap confidence interval (Conf. Int.) for raw

\rho

. More negative values indicate stronger agreement with proficiency.

Explore similar work

A Native-Reference Coordinate Geometry for L2 Pronunciation Deviation Using Self-Supervised Speech Models
Sep 23, 2026Tina Raissi, Nhan Phan, Mikko KurimoPronunciation Assessment Grapheme-To-Phoneme
Department of Information and Communications Engineering, Aalto University, Espoo, Finland
Self-supervised Speech Comparison for L2 Phone, Rhythm, and Intonation Scoring
Jul 15, 2026Stephen McIntosh, Reuben Smit, Daisuke Saito +2Grapheme-To-Phoneme Prosody
The University of Tokyo, Japan · Stellenbosch University, South Africa
Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal
Jun 18, 2026Syeda Faiza Ahmed Sara, Shammur Absar ChowdhuryPronunciation Assessment Surprisal
Qatar Computing Research Institute, Doha, Qatar

Sep 23, 2026, cs.CLJ/K move · Enter open · S save

A Native-Reference Coordinate Geometry for L2 Pronunciation Deviation Using Self-Supervised Speech Models

Tina Raissi, Nhan Phan, Mikko Kurimo

Department of Information and Communications Engineering, Aalto University, Espoo, Finland

Pronunciation Assessment Grapheme-To-Phoneme L2 Subspace

Open paper AskPDF arXiv

Self-supervised speech models encode rich phonetic information, but it remains unclear how to transform this information into interpretable metrics for second-language (L2) pronunciation assessment in spontaneous speech. We propose a native-reference coordinate geometry in which phone-class averages from native speech define a low-dimensional reference subspace, and L2 speech is evaluated by its distance to matching native phone-class coordinates. Unlike prior distance-based approaches, our method does not require parallel recordings with matched linguistic content or dedicated pronunciation labels. Across different self-supervised encoders and modeling choices, the resulting native-reference distances show negative Spearman correlations up to -0.5 with speaking proficiency, indicating that higher-proficiency speakers tend to lie closer to the native-reference space.

Abstract

Explore similar work

A Native-Reference Coordinate Geometry for L2 Pronunciation Deviation Using Self-Supervised Speech Models

Self-supervised Speech Comparison for L2 Phone, Rhythm, and Intonation Scoring

Light-weight Pronunciation Assessment via Discrete Speech Token Surprisal