Articulatory Entrainment and Coordination Complexity in Spontaneous Autistic and Non-autistic Dialogue
Authors: Thanushi Withanage, Carol Espy-Wilson, Elizabeth Redcay, Desi Jones, Noah Sasson
Organizations: Department of Electrical and Computer Engineering, University of Maryland College Park, USA · Department of Psychology, University of Maryland College Park, USA · Department of Psychology, The University of Texas at Dallas, USA
Articulatory entrainment, the adaptation of vocal tract coordination to facilitate interaction remains underexplored in spontaneous dialogue, particularly among autistic speakers. Many prior studies have utilized task-based, phoneme-level analyses with invasive measurement techniques. Here, we introduce a speaker-independent framework to quantify articulatory entrainment in spontaneous dyadic conversations among autistic and non-autistic adults using acoustic-to-articulatory inversion and coordination complexity metrics. We examine temporal changes in articulatory coordination across interaction. Non-autistic dyads exhibit increasing coordination complexity and stronger entrainment over time, autistic dyads show moderate effects, and mixed dyads demonstrate the least alignment. Greater articulatory entrainment correlates with higher self-reported conversational success, indicating increase in coordination complexity as a marker of effective social interaction.
Figures & tables
Figure 1: Coordination simulation and eigen spectra comparison for simple, complex and erratic speech
Figure 2: Eigen spectra for an individual for each quarter of the conversation
Figure 3: Difference eigen spectra for an individual for each quarter of the conversation with first quarter eigen spectra as the reference
Figure 4: (a) Mean & standard deviation variations of ω of NA-NA dyads vs. quarter of conversation for the CANDOR dataset. (b) Percentage of Conversations showing positive entrainment ( e>0 ) and high PCS in NA-NA dyads of CANDOR dataset. (c) Mean & standard deviation variations of ω(h) vs. two halves of conversations for different dyad groups (by speaker) in the UTD dataset. (d) Entrainment vs. dyad type in UTD dataset.
Individuals often align their speaking patterns with their interlocutors, a phenomenon linked to engagement and rapport. While well documented in task-oriented dialogues, less is known about entrainment in naturalistic, non-task and virtual settings. In this study, we analyze a large corpus of spontaneous dyadic Zoom conversations to examine how conversational dynamics relate to perceived interaction quality. We extract multimodal features encompassing turn-taking, pauses, facial movements, and acoustic measures such as pitch and intensity. Perceived conversational success was quantified via factor analysis of post-conversation ratings. Results demonstrate that entrainment reliably detected in spontaneous speech and correlates with higher perceived success. These findings identify key interactional markers of conversational quality and highlight opportunities for targeted interventions to foster more effective and engaging communication.
Thanushi Withanage, Elizabeth Redcay, Carol Espy-Wilson
Department of Electrical and Computer Engineering, University of Maryland College Park, MD, USA · Department of Psychology, University of Maryland College Park, MD, USA
With the proliferation of speech AI agents, understanding emotional entrainment in conversational interaction has become increasingly important. Emotional entrainment is shaped by social relationships and conversational context, influencing affective coordination over time. We introduce DyadEE, a dataset for emotional entrainment detection in dyadic speech interactions, containing both emotionally entrained conversations and synthetic interactions where entrainment is disrupted through partner swapping and emotion resynthesis. We further propose TRACE, a window-level framework that models dyadic interaction as ordered sequences of acoustic embeddings derived from emotion fine-tuned Whisper representations, treating each sample as an interaction trace rather than pooled utterances. Experimental results on DyadEE show that incorporating conversational context and relationship information improves emotional entrainment detection, with TRACE achieving the best accuracy of 97.01%.
Department of Computer Science, Johns Hopkins University, USA · Department of Electrical and Computer Engineering, Johns Hopkins University, USA · Amazon Research, UK +1
Speech assessment tools for speakers with speech pathology must be both accurate and interpretable if they are to be adopted in clinical practice. Existing reference-audio measures such as the Neural Acoustic Distance (NAD) reach high speaker-level correlations with listener intelligibility scores but operate on self-supervised features that are hard to interpret, providing only frame-level explanations. We propose ART-NAD, a reference-audio intelligibility metric that replaces the wav2vec2 features of NAD with vocal-tract constriction variables (tract variables, TVs) predicted from audio by a speaker-independent acoustic-to-articulatory inversion model trained on the same wav2vec2 features. ART-NAD is computed as the multivariate Dynamic Time Warping distance between the nine-channel quasi-TV trajectories of the test and one or more references. Across 20 reference-audio protocols spanning six pathological-speech datasets and five languages, ART-NAD with silence trimming (ART-NAD-FA) reaches the same average speaker-level Pearson correlation as NAD-FA (both r=0.71) on the same self-supervised backbone, with no significant per-protocol difference (Wilcoxon p=0.18), and is the strongest reference-audio metric on 6 of the 20 protocols. Beside the score itself, each TV channel visualizes which constriction deviates from the reference over time, providing interpretable information as to where articulation breaks down.
Bence Mark Halpern, Thomas Tienkamp, Defne Abur +1
Nagoya University Nagoya, Japan · University of Cologne Cologne, Germany · University of Groningen Groningen, the Netherlands