cs.SDNov 14, 2026

Auditory-Feedback-Conditioned Flow TTS for Implicit Lombard Adaptation

Authors: Yunya Wang, Satoshi Nakamura

Organizations: School of Data Science The Chinese University of Hong Kong, Shenzhen · School of AI The Chinese University of Hong Kong, Shenzhen

Abstract

Conventional text-to-speech (TTS) systems can lose intelligibility in noise environments because they cannot adapt to environmental changes. However, humans tend to increase their vocal efforts in noisy environments, a phenomenon known as the Lombard effect. Building on an auditory-feedback Lombard TTS, we investigate whether a pretrained Flow-based TTS can learn acoustic adaptation directly from auditory feedback without explicit prosodic control. Signal-to-noise ratio (SNR) feedback and automatic speech recognition (ASR) feedback are extracted from the noise-corrupted waveforms and injected into the Flow-matching condition for utterance refinement and causal block-level updates. We assess feedback benefits across prompting and noise conditions using plain/Lombard prompting, prompt-switching controls, and paired-finetuning. Experiments show that while finetuning provides strong robustness under in-domain conditions, auditory feedback yields additional cue-dependent benefits that generalize to unseen noise. Moreover, stronger power/F0 tracking alone does not guarantee better speech intelligibility or quality.

Figures & tables

Explore similar work

CardsList
  1. Noise-Aware Lombard Speech Synthesis via Flow-GRPO for Intelligibility Optimization

    Oct 14, 2026Zhaoyang Wang, Dingding Yao, Chengzhong Wang +1TTS Synthesis

  2. Loud and Clear: Dynamic Activation Steering for Improving Speech Intelligibility in Noisy Environments

    Oct 6, 2026Seymanur Akti, Alexander WaibelTTS SynthesisControllable Speech Generation

  3. Synthesizing the Lombard Effect: Multi-Level Control of Speech Clarity and Vocal Effort in TTS

    Jun 22, 2026Seymanur Akti, Alexander WaibelTTS SynthesisSpeech Prosody