ufakzeka-kulak: Building and Evaluating a 31.6M-Parameter Turkish Speech Recognizer from Scratch

Sait Furkan Teke

cs.CLNov 2, 2026

ufakzeka-kulak: Building and Evaluating a 31.6M-Parameter Turkish Speech Recognizer from Scratch

Authors: Sait Furkan Teke

Organizations: ufak AI

Abstract

ufakzeka-kulak is an open Turkish speech-to-text model of 31.6M parameters, a FastConformer encoder with a token-and-duration transducer (TDT) and a CTC head, trained from scratch and released under Apache-2.0. It writes cased, punctuated Turkish with numbers as words. Its 4,977 training hours come from sources whose licences allow commercial use; 281 hours are human-transcribed, and for the other 4,696 the training text is an uploader subtitle or a machine transcript, kept only where a second, independent reading of the audio agrees with it. The recipe, checkpoint, labelling and export choices were made on development data, and the test sets were scored once, in a release gate fixed before training. With greedy TDT decoding the normalised word error rate (WER) is 0.0921 (95 percent interval 0.0792 to 0.1055) on FLEURS Turkish test and 0.0916 (0.0873 to 0.0958) on Common Voice 27.0 Turkish test, lower than Whisper base and Whisper small on both sets with paired intervals wholly below zero; Whisper large-v3 (greedy) scores 0.0604 and 0.0767. On conversational speech the WER is 0.3807. We also report punctuation and casing, degraded audio, and how closely each shipped export reproduces the trained model.

Figures & tables

				hours by text origin
source	licence	utterances	hours	subtitle	checked
Common Voice Scripted Speech 27.0 Turkish	CC0 1.0	64,064	67.90
Common Voice Spontaneous Speech 5.0 Turkish	CC0 1.0	29	0.10
antalia-voice-corpus	CC BY 4.0	892	4.16
ISSAI Turkish Speech Corpus, train split	CC BY 4.0	178,087	209.06
YODAS ( yodas0.10 and yodas0.00 ) and YODAS v3	CC BY 3.0	780,714	4,696.21	369.62	4,326.59

Table 1: The training pool. Hours count each recording once; speed-perturbed copies and repeats of the mix are not counted. For the YODAS recordings the hours are split by the origin of the training text, the uploader’s subtitle (subtitle) or a checked machine transcript (checked).

compared choice	effect	relative	FLEURS-only interval	outcome
CR-CTC instead of intermediate CTC	$+0.0115$	$+7.0$ %	[ $+0.0069$ , $+0.0152$ ]	not adopted
CTC-side weight 0.1 instead of 0.3	$+0.0020$	$+1.2$ %	[ $-0.0031$ , $+0.0048$ ]	not adopted
BPE-dropout 0.1 instead of none	$+0.0062$	$+3.7$ %	[ $-0.0001$ , $+0.0095$ ]	not adopted
random gain, original level	$+0.00005$	$+0.03$ %	[ $-0.0069$ , $+0.0057$ ]	adopted
random gain, at $-20$ and $-30$ dB	$-0.0173$	$-9.6$ %		adopted

Table 2: Design pilots at step 14,500 on the development sets (Common Voice 27.0 dev and FLEURS dev, WER pooled over both). The effect is the mean of the runs with the choice minus the mean of the runs without it, four runs against four for the factorial rows and one run against the three baseline seeds for gain; positive is worse. The gain effect at the attenuated levels has

z=-4.19

against the seed noise and bootstrap error.

set	utt.	WER	CER	base WER	Whisper base	Whisper small
				Whisper	ufakzeka-kulak WER minus
FLEURS test	743	0.0921 [0.0792, 0.1055]	0.0335	0.2461	$-0.1541$ [ $-0.1653$ , $-0.1430$ ]	$-0.0421$ [ $-0.0506$ , $-0.0340$ ]
CV 27.0 test	11,874	0.0916 [0.0873, 0.0958]	0.0242	0.3287	$-0.2372$ [ $-0.2440$ , $-0.2302$ ]	$-0.0885$ [ $-0.0930$ , $-0.0843$ ]
Real-TurnTurk	1,532	0.3807 [0.3270, 0.4640]	0.2787	0.7242	$-0.3435$ [ $-0.4258$ , $-0.3080$ ]	$-0.1367$ [ $-0.1544$ , $-0.1164$ ]
MediaSpeech	2,511	0.1042 [0.1013, 0.1073]	0.0422	0.3282	$-0.2240$ [ $-0.2302$ , $-0.2173$ ]	$-0.1248$ [ $-0.1319$ , $-0.1180$ ]

Table 3: Test results of ufakzeka-kulak, normalised WER and CER, NeMo float32, greedy TDT; CV: Common Voice; utt.: utterances (2.61, 14.54, 1.74 and 9.99 hours). Whisper base and small use the better of greedy and Whisper’s own decoding per set. Only FLEURS and Common Voice gate; Real-TurnTurk has 6 speakers, so its interval is wide; MediaSpeech has no speaker ids, so its interval treats utterances as independent and may be too narrow. For reference, Whisper large-v3 (greedy) scores 0.0604 [0.0501, 0.0718] on FLEURS test, 0.0767 [0.0722, 0.0810] on Common Voice test and 0.4366 on Real-TurnTurk; no paired test was run against it.

		WER, no marks				capital	references ending
set	PC-WER	or casing	PuncER	CaseER	PER	F1	in a mark
FLEURS test	0.1508	0.0890	0.4223	0.1878	0.4134	0.8634	0.9960
Common Voice 27.0 test	0.1132	0.0963	0.1385	0.0530	0.1379	0.9691	0.9832
Real-TurnTurk	0.4960	0.3838	0.8337	0.3817	0.8647	0.3116	1.0000

Table 4: Punctuation and casing on the test sets (lower is better except capital F1). The WER without marks or casing removes only marks and capitals, with the reference numbers written as the model writes them, so it differs from the normalised WER of Table 3 . The last column describes the references, not the model.

copy	FLEURS test	Common Voice 27.0 test	Real-TurnTurk
Opus, 6,000 bit/s	$+0.0517$ [ $+0.0458$ , $+0.0579$ ]	$+0.0752$ [ $+0.0712$ , $+0.0792$ ]	$+0.0803$ [ $+0.0604$ , $+0.0920$ ]
Opus, 12,000 bit/s	$+0.0043$ [ $+0.0015$ , $+0.0075$ ]	$+0.0110$ [ $+0.0090$ , $+0.0130$ ]	$+0.0135$ [ $+0.0114$ , $+0.0160$ ]
telephone band, A-law	$+0.0373$ [ $+0.0312$ , $+0.0436$ ]	$+0.0491$ [ $+0.0460$ , $+0.0522$ ]	$+0.0377$ [ $+0.0307$ , $+0.0481$ ]
$-20$ dB	$+0.0045$ [ $+0.0021$ , $+0.0070$ ]	$+0.0009$ [ $-0.0003$ , $+0.0021$ ]	$+0.0022$ [ $-0.0005$ , $+0.0060$ ]
$-30$ dB	$+0.0205$ [ $+0.0163$ , $+0.0250$ ]	$+0.0051$ [ $+0.0035$ , $+0.0069$ ]	$+0.0040$ [ $-0.0001$ , $+0.0112$ ]

Table 5: WER change on degraded copies of the test audio, copy minus original, same model, paired interval.

		utterances with NeMo’s ids		WER minus NeMo
file	runtime	CV dev (11,864)	FLEURS dev (338)	CV dev	FLEURS dev
transformers TDT, CTC	transformers	11,864, 11,864	338, 338	0.0000	0.0000
sherpa-onnx TDT, int8, gain 8	sherpa-onnx	9,709	174	$+0.0030$	$+0.0022$
sherpa-onnx CTC, int8, gain 8, pad	sherpa-onnx	9,339	142	$+0.0032$	$+0.0031$
GGUF TDT, f32	parakeet.cpp	11,748	306	$-0.0002$	$+0.0007$
GGUF TDT, q8_0	parakeet.cpp	11,555	297	$-0.0001$	$+0.0018$

Table 6: Shipped exports against NeMo 3.0.0 on the development sets (CV: Common Voice 27.0), with transformers 5.20.1, sherpa-onnx 1.13.8, parakeet.cpp v0.6.1, and parakeet-mlx 0.5.3 on mlx 0.32.4. The sherpa-onnx rows are the int8 files the examples load; the float32 copies also ship.

Automatic Speech Recognition

Automatic speech recognition converts spoken language into text across speakers, languages, domains, and recording conditions.

See all 493 papers

ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch
Sep 18, 2026Sait Furkan TekeLanguage Modeling Language Model Pretraining
ufak AI
FreyaTTS Technical Report
Jul 10, 2026Ahmet Erdem Pamuk, Ömer Yentür, Ahmet Tunga Bayrak +2Non-Autoregressive Text Generation TTS Synthesis
Freya Team
ufakzeka-karar: An Open Turkish Typed-Decision Model with Order-Invariant Option Scoring
Oct 5, 2026Sait Furkan TekeLLM Evaluation Confidence Estimation in Language Models
ufak AI

Sep 18, 2026J/K move · Enter open · S save

ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch

Sait Furkan Teke

ufak AI

Language Modeling Language Model Pretraining Small Language Models LLM Evaluation Language Model Safety Evaluation Byte-Level Language Model

Open paper AskPDF arXiv

We describe ufakzeka-1, a 151M-parameter (182M with embeddings) decoder-only Turkish language model pretrained from scratch on 13.5B tokens of openly licensed text and instruction-tuned for chat, at a total cost of about $286 in cloud GPU, API and notebook time. The contribution is not the model's capability, which is what a model this size can be expected to have, but the record of building and measuring it: a Turkish byte-level tokenizer at 1.77 tokens per word, a three-stage pretraining schedule, a post-training mixture of openly licensed and generated data, and an evaluation battery of release gates, a rule-checked sweep of 5,508 conversations, judged conversations and hand tests, all with prompts held out from the training data, enforced by decontamination inside the data build and by a checked-in invariant script we run before each build. We report three findings that we believe transfer to other small-model efforts: a safety gate that had been "fixed" with training data written from its own questions read 64/64 while the honest figure was 34/64; training-seed variance was as large as the spread across every recipe we tried, so single-seed comparisons at this scale are uninformative; and data rounds repaired only what was absent from the data, while identity tracking over long context and multi-turn arithmetic did not move across any data change we tried, which we read as limits of the model size rather than gaps in the data, a reading the next, larger model will test. Weights, the data recipe, the evaluation code and the spend ledger are released under Apache-2.0.

Abstract

Automatic Speech Recognition

Explore similar work

ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch

FreyaTTS Technical Report

ufakzeka-karar: An Open Turkish Typed-Decision Model with Order-Invariant Option Scoring