cs.SENov 5, 2026

Ninada: Sub-Clause Split-Graph VITS Streaming and Capability-Routed 24 kHz Speech Synthesis Across Twelve Scripts

Authors: Hemanth HM

Abstract

End-to-end VITS text-to-speech models synthesize natural speech in a single forward pass, yet standard ONNX exports execute monolithically over an entire clause, delaying Time-To-First-Audio (TTFA) to several seconds and spiking activation memory. Client-side deployment across Indic scripts faces four further bottlenecks: 138.8 MB float32 checkpoints per language strain browser caches, mobile WebGPU stalls for 6-14 s compiling convolution shaders while still running slower than real time (1.44×1.44\times RTF), small WebAssembly pthread pools stall at barriers on big-LITTLE mobile CPUs, and mobile Web Audio pipelines drop frames or mute output under AudioWorklet contention and iOS WKWebView rules. We present Ninada, an open-source 24 kHz TTS system covering English, Classical Sanskrit, and ten Indian languages (sa, en, hi, kn, mr, bn, ta, te, ml, gu, pa, or) across a capability-routed 48-checkpoint fleet: 24 studio HiFi-GAN v1 specialists (27.9 MB W8A32+FP16 ONNX) for desktop/WebGPU and 24 distilled c8c8_384 iSTFTNet specialists (21.7 MB ONNX, 2.64×2.64\times fewer vocoder MACs via an exact NOLA dual-window ConvTranspose1d inverse STFT with zero complex ops) for mobile WASM, plus a 0.0 MB zero-dependency Klatt fallback. At runtime, Ninada splits a single ONNX file in memory in <15< 15 ms into prior, flow, and windowed vocoder sub-sessions streaming 12-frame halos with raised-cosine crossfades. On N=480N = 480 held-out 24 kHz studio utterances, Ninada achieves 3.22 dB 0-8 kHz MCD (3.09 dB mobile iSTFTNet; 1.33 dB 24-voice distillation MCD, vs. 5.17 dB for the 1,526.8 MB 16 kHz Meta MMS fleet) and 6.07 dB 8-12 kHz error, delivering 89-155 ms audible startup on desktop WebGPU and 538-649 ms zero-underrun streaming at 0.75-0.76x RTF on a physical Google Pixel 10 Pro XL.

Figures & tables

Explore similar work

Nov 10, 2026

Swanada: Split-Encoder Capability-Routed Conformer CTC and Sub-30 MB On-Device Speech Recognition Across Twelve Scripts

Client-side automatic speech recognition (ASR) across Indic and Classical scripts eliminates cloud transcription costs, network tail latency, and server-side audio retention, yet neural exports face four systems bottlenecks in browsers and mobile runtimes: (1) monolithic per-language Conformer checkpoints require ~177 MiB per script (> 2.1 GiB across 12 scripts), exhausting CacheStorage quotas; (2) static relative positional encoding tables embed 10.2-20.5 MB of FP32 constants while 1x1 Conv1d layers trigger slow or unsupported ConvInteger kernels in WebAssembly SIMD; (3) sliding-window CTC flickers on boundary tokens; and (4) mobile WebAudio pipelines suffer 44.1/48 kHz sample-rate mismatches and silent AudioWorklet stalls under iOS WKWebView. We present Swanada, an open-source 16 kHz streaming ASR system spanning Classical Sanskrit, Indian English, and ten Indian languages (sa, en, hi, kn, mr, bn, ta, te, ml, gu, pa, or) across browser WebAssembly, Node.js, and Python. Swanada pairs capability-routed shared INT8 Conformer encoders (a 17-layer 177.38 MiB Studio tier at d=512d = 512 for desktop/servers and a distilled 14-layer 25.35 MiB Mobile tier at d=256d = 256 for phones) with twelve hot-swappable 515.0 KiB native-script linear CTC heads (257×512257 \times 512), cutting 12-language storage by 11.6×11.6\times (183.4 MiB) and 67.8×67.8\times (31.39 MiB; 25.85 MiB first load). Runtime protobuf tail surgery exposes the shared representation in <2< 2 ms, while dynamic positional synthesis, MatMul pointwise convolutions, and post-quantization QInt8 head calibration preserve sub-30 MB accuracy. Across N=480N = 480 held-out and N=1,280N = 1,280 open public test clips (FLEURS, LibriSpeech, GLOBE, AMI, wild Sanskrit), Swanada achieves 5.43% held-out and 8.87% 12-language open CER on Studio (18.56% and 19.23% on Mobile) at 0.0097×0.0097\times-0.0264×0.0264\times RTF, with 30.6 ms median TTFT and 0.00% committed-prefix churn.
Sep 24, 2026

BanglaKontho: Closing the Long-Form Gap in Bangla Text-to-Speech

Bangla, the seventh most spoken language in the world, remains under-resourced for neural text-to-speech. Public Bangla speech corpora are dominated by short read-prompt utterances collected for speech recognition, leaving long-form prosody and consistent single-speaker narration uncovered. We present BanglaKontho, a single-speaker Bangla TTS corpus of 20 hours derived from professional audiobook recordings: 7,050 segmented utterances with verified transcripts at 24 kHz. We also release a reusable Bangla text normalizer covering Bangladeshi-style digit grouping, currency and date expressions, Danda punctuation and Unicode normalization, together with the full preprocessing pipeline. An MB-iSTFT-VITS baseline trained from scratch reaches 9.5% WER and 4.46 naturalness MOS, against 16.0% and 3.16 for the same architecture retrained on the 12-hour IndicTTS-Bn corpus. The corpus is released openly under CC BY-NC 4.0.
Sep 8, 2026

TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context

Text-to-speech systems often face a trade-off between natural prosody and efficient inference: higher perceptual quality usually comes at increased computational cost and latency. We present TontaubeV1, a model that preserves natural prosody while enabling streaming from a single consumer GPU. Speech is encoded by the hierarchical DualCodec representation at 12.5 Hz, which separates a semantic stream from successive acoustic refinements. Our design assumes that prosodic structure is largely established when the semantic stream is generated, and allocates capacity accordingly: a Qwen3-1.7B-derived transformer predicts that stream and thereby the utterance duration, while three progressively smaller Qwen3-0.6B-derived transformers each add one acoustic refinement. Text is tokenized per character rather than by subword. Paired text and audio markers at shared positions support long-form generation with bounded context, and overlapping DualCodec reconstructions are mapped into the VibeVoice acoustic latent space and decoded causally, enabling streaming despite DualCodec's noncausal decoder. The model accepts up to one minute of reference audio for voice conditioning and is designed primarily for English and German, with additional multilingual support. The four predictors total 2.9B parameters; on a single RTX 5090 the streaming path reaches about 200 ms to first audio. In separate non-streaming measurements, the end-to-end real-time factor (RTF) is 0.08 for one input and the aggregate RTF is 0.02 across eight concurrent inputs. In a blinded study, 40 listeners preferred TontaubeV1's prosody over Fish Audio S2 Pro in 70.25% of 400 comparisons (95% CI: 64.2-75.7%). On an LLM-as-a-judge benchmark, it matches ElevenLabs Flash v2.5 and outperforms the April 2026 Gradium API and Cartesia Sonic 3 on prosody. The weights are released on Hugging Face under the Tontaube Community Model License 1.0.