cs.SDNov 9, 2026

WaveGen: Truly End-to-End Waveform Generation via Internal Spectral Trajectory Learning

Authors: Sang-Hoon Lee, Ha-Yeong Choi

Organizations: Department of Artificial Intelligence, Ajou University, Suwon, Korea · Frontier AI Lab, KT Corp., Seoul, Korea

Abstract

This work proposes WaveGen, a fully end-to-end diffusion-based text-to-waveform framework based on Spectral Forcing, which jointly learns an internal spectral trajectory and the target waveform trajectory within a single model, enabling direct end-to-end text-to-waveform generation. Unlike recent TTS pipelines, WaveGen does not rely on separately trained or externally decoded intermediate representations, such as neural audio codecs, Mel-spectrogram vocoders, or VAE-based latent models. In addition, WaveGen performs internal text-speech alignment within a single model, eliminating the need for external alignment modules such as duration predictors. To preserve a truly end-to-end formulation, our framework further avoids dependence on self-supervised representations, such as BERT-based text embeddings and Wav2Vec 2.0-based semantic representations, as well as semantic distillation methods for training optimization. To achieve this, we carefully design Spectral Forcing, a model architecture with disentangled diffusion heads, and a training objective based on a multi-scale spectral vv-loss. We further validate the scalability of end-to-end TTS models by scaling the model size from 0.1B to 0.6B parameters. Using a single architecture, open-source data, and one-stage training, without external modules such as duration predictor or REPA, WaveGen demonstrates fully end-to-end waveform generation achieving state-of-the-art performance on the Emilia training dataset. In particular, WaveGen achieves a WER of 1.47 and a SIM of 0.75 on the Seed-en benchmark. Source code and checkpoints are available at https://github.com/sh-lee-prml/WaveGen

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling

    Jun 2, 2026Wenxi Chen, Dongya Jia, Yushen Chen +11TTS SynthesisAudio Diffusion Models

  2. BareWave: Waveform-Native Flow-Matching Text-to-Speech

    Jun 8, 2026Wei Fan, Chao-Hong Tan, Qian Chen +5Flow MatchingTTS Synthesis

  3. Speaking Longer Than Learned: Training-Free Long-Form Generation for Full-Stream Text-to-Speech

    Oct 14, 2026Jeseung Lim, Jiseong Woo, Heeseung KimLong-Form TTS SynthesisTTS Synthesis