Speaking Longer Than Learned: Training-Free Long-Form Generation for Full-Stream Text-to-Speech

Jeseung Lim; Jiseong Woo; Heeseung Kim

cs.SDOct 14, 2026

Speaking Longer Than Learned: Training-Free Long-Form Generation for Full-Stream Text-to-Speech

Authors: Jeseung Lim, Jiseong Woo, Heeseung Kim

Organizations: Department of Physics, University of Seoul, Seoul, Republic of Korea · Department of AI, University of Seoul, Seoul, Republic of Korea

Abstract

Recent full-stream text-to-speech (TTS) models enable low-latency personalized synthesis as input text arrives. However, sustaining generation beyond the lengths seen during training without additional training remains largely unexplored. We systematically analyze VoXtream2, which extends generation by periodically discarding past context while retaining the acoustic prompt. We find that this strategy does not prevent degradation: pronunciation accuracy deteriorates near the end of the training range due to the range of positions in which the model operates rather than the loss of past context, while speaker similarity gradually declines throughout generation. Based on these findings, we propose two training-free modifications that reduce the attention window and strengthen attention to the acoustic prompt. Experiments show that these modifications preserve pronunciation accuracy and speaker similarity throughout long-form generation while retaining the naturalness of the original model. Ablations further show complementary effects of the two modifications.

Figures & tables

Figure 1: Segment-wise WER and SPK-SIM of VoXtream2, together with the corresponding ground truth speech.

Figure 2: Expected number of phonemes advanced per frame versus position for

W=400

625

, and

900

. Shaded bands show standard errors over

8

randomly selected utterances.

Model	Metric	Objective Metrics								Subjective
		Overall	Segment (s)							MOS $\uparrow$
		Overall	0–30	30–60	60–90	90–120	120–150	150–180	180–210	MOS $\uparrow$
Ground Truth (GT)	WER $\downarrow$	1.86	1.73	1.62	1.93	1.99	1.78	1.66	2.18	$4.39\pm 0.05$
	SPK-SIM $\uparrow$	0.832	0.849	0.837	0.834	0.834	0.831	0.832	0.826	$4.39\pm 0.05$
[0.3pt/1.5pt] GT + Mimi Codec [ 2 ]	WER $\downarrow$	1.95	1.80	1.60	1.73	2.03	2.49	1.75	1.89	$4.10\pm 0.07$
	SPK-SIM $\uparrow$	0.754	0.771	0.760	0.756	0.757	0.753	0.754	0.748	$4.10\pm 0.07$

Table 1: Overall and segment-wise performance on long-form speech synthesis. Lower WER (%) and higher SPK-SIM and MOS indicate better performance. MOS is reported as mean

\pm

95% confidence interval on a 5-point scale.

Figure 3: SPK-SIM over time with

W=400

when the acoustic prompt is attended and when it is masked after

30

s. Error bars show standard errors over

8

randomly selected utterances.

Window size (No bias)
Setting	Value	WER (%) $\downarrow$	SPK-SIM $\uparrow$
Default	$W=625$	8.63	0.659
[0.3pt/1.5pt] Window	$W=500$	5.42	0.668
	$W=375$	4.82	0.683
	$W=250$	6.51	0.703
Attention bias ( $W=375$ )

Table 2: Ablation study on window size and attention bias.

W

b

, and

\psi_{\mathrm{target}}

denote the window size, bias strength, and target attention mass, respectively.

Explore similar work

MagpieTTS-LF: Inference-Time Long-Form Speech Generation Without Training on Long-Form data
Jun 16, 2026Subhankar Ghosh, Jason Li, Paarth Neekhara +4TTS Synthesis Speech Prosody
NVIDIA Corporation, USA
Taming Long-form Text-to-Speech
Sep 15, 2026Rongxiang Wang, Berkin Durmus, Aysegul Orhon +2TTS Synthesis TTS Evaluation
Argmax, Inc. · University of Virginia · Bilkent University
VoXtream2: Full-stream TTS with dynamic speaking rate control
Mar 13, 2026Nikita Torgashov, Gustav Eje Henter, Gabriel SkantzeStreaming TTS Synthesis Zero-Shot TTS
Department of Speech, Music and Hearing, KTH Royal Institute of Technology, Sweden

Jun 16, 2026J/K move · Enter open · S save

MagpieTTS-LF: Inference-Time Long-Form Speech Generation Without Training on Long-Form data

Subhankar Ghosh, Jason Li, Paarth Neekhara, Shehzeen Hussain+3

NVIDIA Corporation, USA

TTS Synthesis Speech Prosody

Open paper AskPDF arXiv

Neural Text-to-Speech (TTS) systems achieve remarkable quality on short utterances but long-form speech generation shows prosodic drift, speaker inconsistencies and sentence boundary artifacts. Existing approaches either compress sequences, increase context length or naively concatenate independently synthesized chunks. We present an inference-time approach called MagpieTTS-LF that enables MagpieTTS to produce coherent long-form speech without model retraining. Our method introduces three key innovations: (1) soft attention priors to guide monotonic alignment while preserving past and future context; (2) a stateful inference algorithm that maintains context across sentence chunks, ensuring prosodic continuity; (3) history-aware text encoding that uses past text for discourse-level prosodic planning. Experiments on long texts show significant improvements in long-range intelligibility, prosodic coherence, speaker consistency, and boundary naturalness compared to other baselines.

Abstract

Explore similar work

MagpieTTS-LF: Inference-Time Long-Form Speech Generation Without Training on Long-Form data

Taming Long-form Text-to-Speech

VoXtream2: Full-stream TTS with dynamic speaking rate control