Organizations: Signal Analysis and Interpretation Lab, University of Southern California, USA · Department of Linguistics, University of Southern California
Accent plays a crucial role in speaker identity and inclusivity in speech technologies. Existing accented text-to-speech (TTS) systems either require large-scale accented datasets or lack fine-grained phoneme-level controllability. We propose a accented TTS framework that combines phonological rules with a multilingual TTS model. The rules are applied to phoneme sequences to transform accent at the phoneme level while preserving intelligibility. The method requires no accented training data and enables explicit phoneme-level accent manipulation. We design rule sets for Spanish- and Indian-accented English, modeling systematic differences in consonants, vowels, and syllable structure arising from phonotactic constraints. We analyze the trade-off between phoneme-level duration alignment and accent as realized in speech timing. Experimental results demonstrate effective accent shift while maintaining speech quality.
Figures & tables
Spanish Rule
US IPA
Spanish IPA
1. Initial Consonant Substitution
\textipa /v \texttheta \textipa ð \textipa z \textdyoghlig /
\textipa /b s d s j/
2. Spanish Rhoticity
\textipa /r/
\textipa /r R/
3. Epenthesis (s-clusters)
\textipa /sp st sk/
\textipa /esp est esk/
4. Final Consonant Devoicing
\textipa /b d g z \textdyoghlig /
\textipa /p t k s \textteshlig /
5. Vowel Simplification
\textipa /I U @ A 2 E 3 O/
\textipa /i u a a a e e o a/
6. Monophthongization and Schwa
\textipa /eI oU @/
\textipa /e o a/
Table 1: Phonological transformation rules that demonstrates shifts from US English to Spanish- and Indic-accented English.
IPA
Examples
Rules
This very tall teacher closed the big park
US
\textipa /D Is v E r i tO l t i \textteshlig @ r k l O z d D @ b Ig pA rk./
SP
\textipa /d Is b E R i do l ti \textteshlig @ R g l o zd d @ bi k ba rk./
1,2,4,5,6
IN
\textipa /∥ [d Is w E R i to l t i \textteshlig @ R k l o z d ∥ [d @ b i g p a rk./
1,2,3,4,5
The little button was very broken.
US
\textipa / D @ l I R @ l b 2 tn w 2z v E r i br O k@n./
Table 2: Examples of applying phonological rules. The first column indicates which accented IPA they are and the third column indicates which rules were applied for the transformation
Figure 1: Synthesis Pipeline. We follow the synthesis and evaluation pipeline described in [ 13 ] . The key difference is that we condition the TTS model on a speaker embedding from the target language. In addition, we explicitly control whether duration alignment is applied between the American (US) phoneme sequence and the transformed target-accent phoneme sequence.
Probability
UK
SP
IN
UK With Alignment
80.28
1.23
0.17
UK Without Alignment
81.12
1.06
0.7
SP With Alignment
6.50
51.59
3.28
SP Without Alignment
3.9
50.86
2.73
IN With Alignment
0.82
6.08
86.4
Table 3: Effect of duration alignment on accentedness. Accent classification probabilities are reported for UK, Spanish (SP), and Indian (IN) conditions with and without phoneme-level duration alignment.
Probability
Similarity
Speech Quality
US
Target
US
Target
UTMOS
WER
CER
US Spk Emb.
73.8
-
0.71
-
4.38
3.42
2.83
+SP rules
25.97
-
0.50
-
4.39
24.91
13.56
+IN rules
16.85
-
0.42
-
4.38
32.66
18.32
SP Spk Emb.
26.6
23.7
0.73
0.47
3.73
7.99
5.64
+SP rules
2.97
51.59
0.38
0.60
3.87
13.13
7.79
Table 4: Overall effects of phonological rules on accentedness and speech quality. Accent probability and embedding similarity are reported for US and target accents, along with UTMOS, WER, and CER.
Accent Prob
Accent Sim
US ↓
SP ↑
US ↓
SP ↑
SP Spk Emb
32.46
22.86
0.70
0.32
+ Rule 1
29.80
22.20
0.70
0.37
+ Rule 2
21.13
23.00
0.66
0.32
+ Rule 3
30.50
22.78
0.69
0.32
+ Rule 4
28.47
25.22
0.69
0.35
Table 5: Effect of individual phonological rules on accentedness. Accent probability and embedding similarity are shown as each rule is added to the Spanish (SP) and Indian (IN) speaker embeddings.
Accuracy
Strength
Naturalness
US Spk Emb
1.000
4.04
4.09
+ SP rules
0.386
2.59
2.81
+ IN rules
0.229
2.88
2.73
SP Spk Emb
0.071
2.20
3.14
+ SP rules
0.757
3.08
2.94
IN Spk Emb
0.786
3.31
3.47
Table 6: Human evaluation of accent perception. We report listener judgments of accent accuracy, perceived accent strength (for correctly identified samples), and naturalness.
Figure 2: Human accent confusion matrix. Rows denote synthesis conditions (speaker embeddings with and without phonological rules), and columns represent listener-labeled accents.
Signal Analysis and Interpretation Lab, University of Southern California, USA · Thomas Lord Department of Computer Science, University of Southern California, USA · Department of Linguistics, University of Southern California