Organizations: School of Information Science, Beijing Language and Culture University, Beijing, China · School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen, China
While second language (L2) learners may acquire target syntactic word order, mapping this syntax onto appropriate prosodic structures remains a persistent challenge. This study investigates the fossilization and stability of the L2 syntax-prosody interface by comparing 67 native Mandarin speakers with 67 Vietnamese learners using the BLCU-SAIT corpus. By integrating C-ToBI boundary annotation with Dependency Grammar analysis, we examined both the quantity of prosodic boundaries and their mapping to syntactic relations. Results reveal a non-linear acquisition: although high-proficiency learners (VNH) converge to the native baseline in boundary quantity at the Major Phrase level (B3), their structural mapping significantly diverges. Specifically, VNH demote the prosodic boundary at the Subject-Verb (SBV) interface (Major Phrase B3 -> Prosodic Word B1), while erroneously promoting the boundary at the Verb-Object (VOB) interface (Prosodic Word B1 -> Major Phrase B3). This strategy allows learners to maintain high long phrasal output at the expense of structural accuracy. This results in a distorted prosodic hierarchy where the native pattern is inverted.
Figures & tables
\toprule Tag
Description
Example
\midrule SBV
subject-verb
I give her a bunch of flowers (I ← give)
VOB
verb-object
I give her a bunch of flowers (give → flowers)
IOB
indirect object
I give her a bunch of flowers (give → her)
FOB
fronting object
He reads any book (book ← read)
ATT
attribute
Red apple (red ← apple)
ADV
adverbial
Very beautiful (very ← beautiful)
Table 1: Harbin Institute of Technology Dependency Parsing Guidelines
\toprule Boundary Type
Speaker Type
No.of Boundary
\midrule Level:B1
CN group
225
VNH group
190
VNL group
341
\midrule Level:B2
CN group
155
VNH group
104
VNL group
173
Table 2: Number of prosodic boundaries
\toprule Contrast
Estimate
Std. Error
z value
p-value
\midrule (Intercept)
5.50472
0.06528
84.325
VNH
-0.12333
0.10862
-1.136
VNL
0.32677
0.11290
2.894
B2
-0.41671
0.09251
-4.505
B3
-1.25623
0.09324
-13.473
VNH:B2
-0.32482
0.15420
-2.106
**
Table 3: Negative binomial regression model results. Significance levels: *** for p-value <0.001, ** for p-value <0.01, * for p-value <0.05, . for p-value <0.1, and ns for p-value ≥ 0.1.
Figure 1: Deviation in Dependency Relation Frequency.( The zero line represents the CN as baseline. The bar length denotes the degree of deviation. Positive = Over-segmentation, Negative = Under-segmentation.)
\toprule Relation
Level
CN (Res)
VNL (Res)
VNH (Res)
\midrule ADV
B1
-0.45
-0.55
1.27
ATT
B1
0.13
0.25
-0.45
CMP
B1
1.78
1.49
-4.40
POB
B1
2.42
0.05
-4.20
SBV
B1
-3.60
0.18
6.02
VOB
B1
2.11
-0.37
-3.30
Table 4: Standardized Residuals of Syntactic Relations across Prosodic Boundaries. Values with absolute magnitude >1.96 (bold) indicate significant deviation from independence.
While it is well-established that prosody carries crucial cues for syntactic structure, the degree and nature of correspondence between these two domains remains contested. We investigate the syntax-prosody interface through an information-theoretic lens, quantifying the interaction between prosodic features and syntactic representations as their mutual information. We provide a general-purpose framework for estimating this quantity over large speech-text corpora using multimodal language models. Our framework is structure-agnostic and modular, insofar as it can be used to measure the contributions of individual prosodic features or components of structure. We evaluate the syntax-prosody relationship for two features (word duration and inter-word pauses) across two domains--read audiobooks and spontaneous conversations--both in English. Our results demonstrate that prosody contains measurable syntactic information, with prosodic features reducing syntactic uncertainty in spontaneous conversations by up to 10.2%. Our findings offer new empirical support for several theoretical accounts of the syntax-prosody interface.
Prosody plays an important role in speech translation, conveying information such as emphasis, emotion, and intent beyond lexical content. However, despite recent progress in expressive speech-to-speech translation (S2ST), little is known about how prosodic patterns are similar/different across languages. Understanding these cross-lingual similarities and differences is crucial for effectively incorporating prosody into expressive S2ST systems. In this work, we present the first fine-grained cross-lingual analysis of prosody using multilingual dubbing data across English-German, English-Spanish, and English-French language pairs. We analyze the similarity of pitch, energy, and temporal feature patterns between source and target speech and investigate the linguistic and alignment-related factors affecting this similarity. Our analysis reveals inherent cross-lingual correlations in prosodic structure between certain languages. The findings provide important insights into the transferability of prosody across languages and offer empirical guidance for future expressive speech-to-speech translation systems.
Haopeng Xie, Ismail Rasim Ulgen, Sofia Son +2
Center for Language and Speech Processing (CLSP), Johns Hopkins University Baltimore, MD, USA
This chapter explores the intricate interplay between intonation and tone in Mandarin Chinese varieties, focusing on f0, the primary acoustic cue for both intonation and tone. The main empirical base is intonation boundary phenomena, where intonation and tone intersect and influence each other in conveying a range of sentence-level linguistic functions -- such as question vs. statement -- and a rich array of speakers' attitudinal information. Theoretical models and emerging techniques are also discussed to account for the observed interactions of tonal aspects and boundary phenomena to convey multiple levels of communicative meanings.