Towards Efficient Simultaneous Inverse Text Normalization with Pretrained Text-to-Text Language Model and Read-Tag-Write Policy

Kiet Anh Hoang; Khanh Le; Bao Nguyen; Linh Pham; Dung Vo; Thai Tran; Mai Nguyen; Tri Nguyen

cs.CLOct 14, 2026

Towards Efficient Simultaneous Inverse Text Normalization with Pretrained Text-to-Text Language Model and Read-Tag-Write Policy

Authors: Kiet Anh Hoang, Khanh Le, Bao Nguyen, Linh Pham, Dung Vo, Thai Tran, Mai Nguyen, Tri Nguyen,

Organizations: UNEY, Switzerland

Abstract

Streaming Inverse Text Normalization (ITN) is vital for converting spoken-form outputs from streaming Automatic Speech Recognition into formatted written text. Existing streaming ITN methods rely on hybrid systems combining neural tagging with expert-crafted finite-state transducer rules, limiting scalability across domains and languages. While end-to-end models offer superior scalability by learning directly from data, standard encoder-decoder architectures are inherently non-streaming due to global attention. In this paper, we propose an efficient streaming end-to-end ITN system, adapting a pretrained text-to-text model to leverage its robust linguistic knowledge. To enable streaming, we introduce architectural adaptations, a specialized training strategy, a Read-Tag-Write decoding policy, and inference optimizations. Experiments on Vietnamese datasets show accuracy comparable to non-streaming baselines, outperforming hybrid methods while satisfying real-time latency requirements.

Figures & tables

Figure 1: Overview of the proposed streaming end-to-end ITN framework using the Read–Tag–Write policy. The streaming encoder utilizes Dynamic Right Context masking to process incoming chunks of spoken-form tokens, while a joint tagger predicts {B, I, O} labels. Verbatim tokens (O) are passed through instantly to minimize latency. Upon the completion of a normalization span, the autoregressive decoder is selectively triggered, generating normalized text by cross-attending to the relevant encoder hidden states ( $H_{0:m}$ ) and conditioning on the written-form prefix $\hat{y}_{pre}$ .

						With PnC			Without PnC
	Model	Pretrained	c	r	F 1	I-WER	NI-WER	WER	I-WER	NI-WER	WER
Non-stream	$B_{0}$	$\times$	$\infty$	$\infty$	–	14.20	1.51	4.61	8.28	0.54	1.11
	$B_{1}$	EnViT5	$\infty$	$\infty$	–	11.69	1.30	3.84	5.17	0.39	0.74
	$B_{2}$	EnViT5	$\infty$	$\infty$	97.34	10.80	1.32	3.63	5.03	0.16	0.52
	AdapITN [ 2 ]	EnViBert	$\infty$	$\infty$	96.65	156.88	2.50	40.08	14.58	0.40	1.45
	$S_{C}$ (ours)	EnViT5	{3,4,5}	{0,1,2}	96.62	11.48	1.31	3.79	5.83	0.18	0.60

Table 1: Evaluation of ITN performance with and without punctuation and capitalization (PnC). Parameters

c

and

r

denote chunk and right-context sizes during inference. All metrics are reported as percentages (%). Model categories:

B

(Non-streaming baselines),

H

(Hybrid streaming models), and

S

(Proposed end-to-end streaming variants).

Figure 2: Per-chunk latency breakdown (Encoder vs. Total) for four streaming configurations across chunk sizes of 3, 4, and 5.

	Model	I-WER	NI-WER	WER
Non-stream	$B_{0}$	37.30	2.44	10.94
	$B_{1}$	34.09	2.24	10.01
	$B_{2}$	35.33	2.28	10.34
	$S_{C}$	36.18	2.27	10.54
Stream	$H_{2}$	47.56	2.75	13.67
	Wait- $k$	41.16	2.48	11.91

Table 2: Real-world ITN performance on noisy ASR transcripts. All metrics are reported as percentages (%).

Figure 3: Impact of truncation probability $p_{\text{trunc}}$ on the Word Error Rate for both offline and streaming inference of our model.

Lang.	Backbone	Params	Stream	WER (%)
				ASR-L	ASR-O
Vi	BartPho	132M	$\times$	4.61	11.07
	BartPho	132M	$\checkmark$	6.31	12.84
	EnViT5	275M	$\times$	3.84	10.01
	EnViT5	275M	$\checkmark$	4.54	11.02
				W/ PnC	W/o PnC

Table 3: Generalizability of ITN performance across architectures and languages. Values represent average WER (%) across

c\in\{3,4,5\}

and

r\in\{0,1,2\}

. Vietnamese evaluation uses ASR-L (ground-truth spoken text) and ASR-O (noisy ASR transcripts); English evaluation uses With and Without PnC.

Efficient Language Model Inference

Efficient language model inference reduces the compute, memory, latency, and energy needed to generate useful outputs.

See all 318 papers

Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input
Mar 6, 2026Changsong Liu, Tianrui Wang, Ye Ni +2Long-Form TTS Synthesis TTS Synthesis
Nanyang Technological University, Singapore · Tianjin University, China · Southeast University, China
TASTE-Streaming: Towards Streamable Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
Mar 12, 2026Liang-Hsuan Tseng, Hung-yi LeeSpeech Language Models Audio Tokenization
Graduate Institute of Communication Engineering, National Taiwan University
Streaming T5-based Text-to-Speech Synthesis with Limited Lookahead
Jun 20, 2026Muyang Du, Jason Roche, Junjie LaiTTS Synthesis Streaming TTS Synthesis
1NVIDIA, China · 2NVIDIA, USA

Mar 6, 2026J/K move · Enter open · S save

Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input

Changsong Liu, Tianrui Wang, Ye Ni, Yizhou Peng+1

Nanyang Technological University, Singapore · Tianjin University, China · Southeast University, China

Long-Form TTS Synthesis TTS Synthesis Streaming TTS Synthesis Speech Language Models Speech Prosody

Open paper AskPDF arXiv

Streaming TTS that receives streaming text is essential for interactive systems, yet this scheme faces two major challenges: unnatural prosody due to missing lookahead and long-form collapse due to unbounded context. We propose a prosodic-boundary-aware post-training strategy, adapting a pretrained LLM-based TTS model using weakly time-aligned data. Specifically, the model is adapted to learn early stopping at specified content boundaries when provided with limited future text. During inference, a sliding-window prompt carries forward previous text and speech tokens, ensuring bounded context and seamless concatenation. Evaluations show our method outperforms CosyVoice-Style interleaved baseline in both short and long-form scenarios. In long-text synthesis, especially, it achieves a 66.2% absolute reduction in word error rate (from 71.0% to 4.8%) and increases speaker and emotion similarity by 16.1% and 1.5% relatively, offering a robust solution for streaming TTS with incremental text.

Abstract

Efficient Language Model Inference

Explore similar work

Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input

TASTE-Streaming: Towards Streamable Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling

Streaming T5-based Text-to-Speech Synthesis with Limited Lookahead