EditVoice: Variable-Length Non-Autoregressive Zero-Shot TTS and Speech Editing with Edit Flows

Hongyao Deng; Wenhao Guan; Xuetao Lin; Peijie Chen; Weijie Wu; Lin Li; Qingyang Hong

cs.SDSep 24, 2026

EditVoice: Variable-Length Non-Autoregressive Zero-Shot TTS and Speech Editing with Edit Flows

Authors: Hongyao Deng, Wenhao Guan, Xuetao Lin, Peijie Chen, Weijie Wu, Lin Li, Qingyang Hong

Organizations: School of Informatics, Xiamen University, China · School of Electronic Science and Engineering, Xiamen University, China

Abstract

Recent non-autoregressive (NAR) zero-shot text-to-speech (TTS) models generate in parallel but typically require the target sequence length to be specified before generation. We introduce EditVoice, to our knowledge the first variable-length NAR zero-shot TTS model, which uses Edit Flows to jointly update speech content and sequence length through insertions, deletions, and substitutions. EditVoice adopts speech-infilling training, which unifies zero-shot TTS and text-based speech editing and allows both prefix and suffix speech prompt placements at inference. We introduce Complementary Prompt Sampling (CPS) to leverage the complementary Edit Flow predictions induced by the two prompt placements. We further find that EditVoice can edit source and model-generated speech beyond its training sources. We use this generalization for end-to-end editing and training-free post-generation refinement. With the Edit Flow model trained on 10K h of GigaSpeech, EditVoice demonstrates competitive zero-shot TTS performance on Seed-TTS Eval EN and LibriSpeech-PC and speech editing performance on RealEdit.

Figures & tables

Autoregressive
Model	Params.	Train (h)	Seed-TTS EN			LibriSpeech-PC			NMOS $\uparrow$	SMOS $\uparrow$	T2S RTF $\downarrow$
			WER $\downarrow$	SIM-o $\uparrow$	UTMOS $\uparrow$	WER $\downarrow$	SIM-o $\uparrow$	UTMOS $\uparrow$
Ground Truth	–	–	–	–	–	–	–	–	3.77	4.11	–
CosyVoice2 † [ 5 ]	0.5B	167K	2.58	0.659	4.17	1.80	0.655	4.39	3.88	3.84	0.5313
CosyVoice3 [ 6 ]	0.5B	1M	2.00	0.697	3.97	1.77	0.695	4.30	–	–	–
VoxCPM [ 30 ]	0.5B	1.8M	1.86	0.729	3.82	1.92	0.719	4.21	–	–	–

Table 1: Zero-shot TTS results. For EditVoice , 10K h denotes the GigaSpeech data used to train the Edit Flow model.

\dagger

denotes systems using the same S3Tokenizer2 tokenizer and CosyVoice 2 detokenizer as EditVoice ; bold and underline denote the best and second-best synthesized results. T2S RTF measures only text-to-semantic-token generation.

Cascade editing
Model	Type	WER $\downarrow$	SIM-o $\uparrow$	MOSN $\uparrow$	UTMOS $\uparrow$
Ground Truth	–	5.59	–	3.22	3.40
FluentSpeech [ 13 ]	NAR	5.80	0.949	3.18	2.68
VoiceCraft [ 20 ]	AR	5.89	0.973	3.14	3.34
SSR-Speech [ 24 ]	AR	4.75	0.986	3.20	3.36
EditVoice	NAR	4.40	0.981	3.22	3.35

Table 2: Objective RealEdit results; bold and underline denote the best and second-best results within each setting.

56 generation NFE
Inference	WER $\downarrow$	SIM-o $\uparrow$	UTMOS $\uparrow$
Prefix	1.65%	0.653	4.08
Suffix	1.66%	0.656	4.08
CPS w/o Prompt Consistency	1.61%	0.658	4.08
CPS	1.54%	0.659	4.11
56 generation + 8 refinement NFE

Table 3: CPS and refinement ablation on Seed-TTS Eval EN.

Explore similar work

CosyEdit2: Speech-Editing-Oriented Reinforcement Learning Unlocks Better Zero-Shot TTS
May 25, 2026Junyang Chen, Yuhang Jia, Hui Wang +3Flow-Matching Text-To-Speech Edit
College of Computer Science, Nankai University · College of Artificial Intelligence, Nankai University
CosyEdit: Unlocking End-to-End Speech Editing Capability from Zero-Shot Text-to-Speech Models
Jan 8, 2026Junyang Chen, Yuhang Jia, Hui Wang +2Flow-Matching Text-To-Speech Edit
College of Computer Science, Nankai University, Tianjin, China
dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model
Aug 2, 2026Hankun Wang, Bohan Li, Shi Lian +7Utterances Autoregressive Model
X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University · Xiaohongshu Inc.

May 25, 2026, cs.SDJ/K move · Enter open · S save

CosyEdit2: Speech-Editing-Oriented Reinforcement Learning Unlocks Better Zero-Shot TTS

Junyang Chen, Yuhang Jia, Hui Wang, Jiaming Zhou+2

College of Computer Science, Nankai University · College of Artificial Intelligence, Nankai University

Flow-Matching Text-To-Speech Edit

Open paper AskPDF arXiv

Speech editing and zero-shot Text-to-Speech (TTS) share a similar generative foundation conditioned on speech prompts, yet speech editing demands far stricter local acoustic consistency with surrounding unedited content. While prior work has shown that Supervised Fine-Tuning (SFT) enables TTS models to acquire functional editing capability, this approach remains fundamentally bottlenecked by imperfect paired editing data and coarse-grained optimization signals. To address these limitations, we propose CosyEdit2, a speech editing model built on a two-stage post-training framework that progresses from supervised editing initialization to editing-oriented Group Relative Policy Optimization (GRPO) over target-speech-free data. Extensive experiments demonstrate that CosyEdit2 not only substantially advances speech editing performance, but also unlocks better zero-shot TTS capability, revealing a deeper mutual relationship between the two tasks. Audio samples are available at https://cjy1018.github.io/CosyEdit2.

Abstract

Explore similar work

CosyEdit2: Speech-Editing-Oriented Reinforcement Learning Unlocks Better Zero-Shot TTS

CosyEdit: Unlocking End-to-End Speech Editing Capability from Zero-Shot Text-to-Speech Models

dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model