Speech Prosody

Follow this topic Ask about this topic

Speech prosody concerns the patterns of pitch, stress, rhythm, duration, and pausing that shape how speech is understood. Researchers analyze acoustic contours and phonological annotations, probe self-supervised speech representations, and build neural systems that predict or control prosodic features in tasks such as text-to-speech, tone recognition, and pronunciation assessment. Evaluation spans multilingual stress and tone tests, emotional speech perception, dialogue and translation, and comparisons with human judgments. A central challenge is that models may encode prosodic cues without reliably using or reproducing them, especially when fine-grained meaning depends on context, language, or speaker.

Latest papers

80 papers10 papers in the last four weeks, down 23% on the four weeks before. 0.1% of all new papers.

MAGIC-TTS: Fine-Grained Controllable Speech Synthesis with Explicit Local Duration and Pause Control
Apr 23, 2026Jialong Mai, Xiaofen Xing, Xiangmin XuTTS Synthesis Speech Prosody
South China University of Technology
Deep Supervised Contrastive Learning of Pitch Contours for Robust Pitch Accent Classification in Seoul Korean
Apr 21, 2026Hyunjung Joo, GyeongTaek LeeSpeech Prosody Supervised Contrastive Learning
Department of Linguistics, Rutgers University · Hanyang Institute for Phonetics and Cognitive Sciences of Language (HIPCS) · Department of Smart Factory, Gachon University
AI-Driven Analytics of Team-Teaching Talk: Acoustic Patterns across Experience, Cohorts and the Learning Design
Apr 19, 2026Yuchen Liu, Roberto Martinez-Maldonado, Riordan Alfredo +3Speech Prosody
Monash University, Australia · EPFL, Switzerland
Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark
Apr 12, 2026Arnon Turetzky, Avihu Dekel, Hagai Aronowitz +2TTS Synthesis TTS Evaluation
The Hebrew University of Jerusalem, Israel · IBM Research, Israel
Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Models
Apr 10, 2026Yizhong Geng, Yanliang Li, Jinghan Yang +4TTS Synthesis Speech Language Models
Beijing University of Posts and Telecommunications, Beijing, China · Beijing Logic Intelligence Technology, Beijing, China · University of California, USA +2
Lexical Tone is Hard to Quantize: Probing Discrete Speech Units in Mandarin and Yorùbá
Apr 8, 2026Opeyemi Osakuade, Simon KingDiscrete Speech Representation Learning Audio Tokenization
The Centre for Speech Technology Research, University of Edinburgh, UK
In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads
Apr 7, 2026Charlotte Pouw, Hosein Mohebbi, Afra Alishahi +1In-Context Learning Speech Language Models
ILLC, University of Amsterdam · CSAI, Tilburg University
Brain-to-Speech: Prosody Feature Engineering and Transformer-Based Reconstruction
Apr 7, 2026Mohammed Salah Al-Radhi, Géza Németh, Andon Tchechmedjiev +1EEG Decoding Neural Decoding
Department of Telecommunications and Artificial Intelligence, Budapest University of Technology and Economics, Budapest, Hungary · EuroMov Digital Health in Motion, University of Montpellier, IMT Mines Alès, Montpellier, France
Prosodic ABX: A Language-Agnostic Method for Measuring Prosodic Contrast in Speech Representations
Apr 2, 2026Haitong Sun, Stephen McIntosh, Kwanghee Choi +3Speech Prosody Self-Supervised Speech Representation Learning
The University of Tokyo, Japan · University of Texas at Austin, USA
Combining Masked Language Modeling and Cross-Modal Contrastive Learning for Prosody-Aware TTS
Mar 31, 2026Kirill Borodin, Vasiliy Kudryavtsev, Maxim Maslov +3TTS Synthesis Speech Prosody
MTUCI, Moscow, Russia
Covertly improving intelligibility with data-driven adaptations of speech timing
Mar 31, 2026Paige Tuttösí, Angelica Lim, H. Henny Yeung +2Speech Prosody TTS Synthesis
School of Computing Science, Simon Fraser University, Canada · Université Marie et Louis Pasteur, SUPMICROTECH, CNRS, institut FEMTO-ST, France · Department of Linguistics, Simon Fraser University, Canada
Assessing the Ability of Neural TTS Systems to Model Consonant-Induced F0 Perturbation
Mar 22, 2026Tianle Yang, Chengzhe Sun, Phil Rose +2TTS Synthesis Speech Prosody
University at Buffalo, Department of Linguistics, Buffalo, 14260, NY, United States · University at Buffalo, Department of Computer Science and Engineering, Buffalo, 14260, NY, United States · Australian National University, Emeritus Faculty, Canberra, 0200, ACT, Australia
Gesture2Speech: How Far Can Hand Movements Shape Expressive Speech?
Mar 20, 2026Lokesh Kumar, Nirmesh Shah, Ashishkumar P. Gudmalwar +1TTS Synthesis Controllable Speech Generation
Media Analysis, Sony Research India
Probing neural audio codecs for distinctions among English nuclear tunes
Mar 14, 2026Juan Pablo Vigneaux, Jennifer ColeSpeech Prosody Neural Audio Codecs
Northwestern University
Causal Prosody Mediation for Text-to-Speech:Counterfactual Training of Duration, Pitch, and Energy in FastSpeech2
Mar 12, 2026Suvendu Sekhar MohantyTTS Synthesis Speech Prosody
Arlington, Virginia, USA
Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input
Mar 6, 2026Changsong Liu, Tianrui Wang, Ye Ni +2Long-Form TTS Synthesis TTS Synthesis
Nanyang Technological University, Singapore · Tianjin University, China · Southeast University, China
ProKWS: Personalized Keyword Spotting via Collaborative Learning of Phonemes and Prosody
Mar 5, 2026Jianan Pan, Yuanming Zhang, Kejie HuangKeyword Spotting Representation Learning
College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou, China
Quantity Convergence, Quality Divergence: Disentangling Fluency and Accuracy in L2 Mandarin Prosody
Feb 26, 2026Yuqi Shi, Hao Yang, Xiyao Lu +1Speech Prosody Second Language Acquisition
School of Information Science, Beijing Language and Culture University, Beijing, China · School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen, China
UtterTune: LoRA-Based Target-Language Pronunciation Edit and Control in Multilingual Text-to-Speech
Aug 13, 2025Shuhei KatoTTS Synthesis Multilingual TTS Synthesis
Independent Researcher Tokyo, Japan
Balalaika: Data-Centric, Prosody-Aware Annotation Pipeline for Russian Speech
Jul 17, 2025Kirill Borodin, Nikita Vasiliev, Vasiliy Kudryavtsev +3Speech Processing Speech Prosody
lab260, Moscow Technical University of Communications and Informatics, Russia · BitmanagerAI, UAE

Apr 23, 2026J/K move · Enter open · S save

MAGIC-TTS: Fine-Grained Controllable Speech Synthesis with Explicit Local Duration and Pause Control

Jialong Mai, Xiaofen Xing, Xiangmin Xu

South China University of Technology

TTS Synthesis Speech Prosody Controllable Speech Generation

Open paper AskPDF arXiv

Fine-grained local timing control is still absent from modern text-to-speech systems: existing approaches typically provide only utterance-level duration or global speaking-rate control, while precise token-level timing manipulation remains unavailable. To the best of our knowledge, MAGIC-TTS is the first TTS model with explicit local timing control over token-level content duration and pause. MAGIC-TTS is enabled by explicit token-level duration conditioning, carefully prepared high-confidence duration supervision, and training mechanisms that correct zero-value bias and make the model robust to missing local controls. On our timing-control benchmark, MAGIC-TTS substantially improves token-level duration and pause following over spontaneous synthesis. Even when no timing control is provided, MAGIC-TTS maintains natural high-quality synthesis. We further evaluate practical local editing with a scenario-based benchmark covering navigation guidance, guided reading, and accessibility-oriented code reading. In this setting, MAGIC-TTS realizes a reproducible uniform-timing baseline and then moves the edited regions toward the requested local targets with low mean bias. These results show that explicit fine-grained controllability can be implemented effectively in a high-quality TTS system and can support realistic local timing-editing applications.