Paper ID: 2501.05787 • Published Jan 10, 2025
MARS6: A Small and Robust Hierarchical-Codec Text-to-Speech Model
Matthew Baas, Pieter Scholtz, Arnav Mehta, Elliott Dyson, Akshat Prakash, Herman Kamper
TL;DR
Get AI-generated summaries with premium
Get AI-generated summaries with premium
Codec-based text-to-speech (TTS) models have shown impressive quality with
zero-shot voice cloning abilities. However, they often struggle with more
expressive references or complex text inputs. We present MARS6, a robust
encoder-decoder transformer for rapid, expressive TTS. MARS6 is built on recent
improvements in spoken language modelling. Utilizing a hierarchical setup for
its decoder, new speech tokens are processed at a rate of only 12 Hz, enabling
efficient modelling of long-form text while retaining reconstruction quality.
We combine several recent training and inference techniques to reduce
repetitive generation and improve output stability and quality. This enables
the 70M-parameter MARS6 to achieve similar performance to models many times
larger. We show this in objective and subjective evaluations, comparing TTS
output quality and reference speaker cloning ability. Project page:
this https URL