Harf-Speech: A Clinically Aligned Framework for Arabic Phoneme-Level Speech Assessment

Asif Azad; MD Sadik Hossain Shanto; Mohammad Sadat Hossain; Bdour Alwuqaysi; Sabri Boughorbel; Yahya Bokhari; Abdulrhman Aljouie; Ayah Othman Sindi

eess.ASMar 11, 2026

Harf-Speech: A Clinically Aligned Framework for Arabic Phoneme-Level Speech Assessment

Authors: Asif Azad, MD Sadik Hossain Shanto, Mohammad Sadat Hossain, Bdour Alwuqaysi, Sabri Boughorbel, Yahya Bokhari, Abdulrhman Aljouie, Ayah Othman Sindi,

Organizations: Ministry of Defense, Saudi Arabia · Ability Center, Saudi Arabia · University of Rochester, USA

Abstract

Automated phoneme-level pronunciation assessment is vital for scalable speech therapy and language learning, yet validated tools for Arabic remain scarce. We present Harf-Speech, a modular system scoring Arabic pronunciation at the phoneme level on a clinical scale. It combines an MSA phonetizer, a fine-tuned speech-to-phoneme model, Levenshtein alignment, and a blended scorer using longest common subsequence and edit-distance metrics. We fine-tune three ASR architectures on Arabic phoneme data and benchmark them with zero-shot multimodal models; the best, OmniASR-CTC-1B-v2, achieves 8.92% phoneme error rate. Three certified speech-language pathologists independently scored 40 utterances for clinical validation. Harf-Speech attains a Pearson correlation of 0.791 and ICC(2,1) of 0.659 with mean expert scores, outperforming existing end-to-end assessment frameworks. These results show Harf-Speech yields clinically aligned, interpretable scores comparable to inter-rater expert agreement.

Figures & tables

Figure 1: Overview of the Harf-Speech methodology. The example reference word shown in the figure translates to “be prepared”

Model	Params	Type	PER	RTF
Gemini-3-flash	unknown	ZS	17.31%	0.394
Gemini-3-pro	unknown	ZS	15.07%	10.748
Qwen3-ASR-1.7B	1.7B	FT	16.79%	0.130
Wav2Vec2-LV60-CV	317M	FT	13.58%	0.009
OmniASR-CTC-1B-v2	1B	FT	8.92%	0.004

Table 1: Phoneme error rate (PER) and real-time factor (RTF; inference time divided by audio duration) comparison across models. ZS = zero-shot; FT = fine-tuned.

Figure 2: Pairwise scatter plots on the 0–5 clinical scale. Points are colored by absolute disagreement (green = close, red = far). The shaded band marks $\pm$ 0.5 tolerance around perfect agreement.

Inter-SLP
Pair	PCC	SCC	ICC (2,1)	MAE	RMSE	Exact (%)	$\pm$ 1 (%)
SLP 1 vs SLP 2	.858	.625	.846	0.56	0.95	61.5	84.6
SLP 2 vs SLP 3	.927	.875	.922	0.39	0.64	69.2	94.9
SLP 1 vs SLP 3	.903	.625	.896	0.50	0.75	53.8	94.9
vs. SLP 1
Harf-Speech	.798	.491	.640	0.84	1.15	43.6	76.9

Table 2: Full pairwise agreement metrics on the 0–5 clinical scale, grouped by SLP target. Bold values indicate the better result between Harf-Speech and Azure within each group.

Speech Processing

Speech processing develops computational methods for recognizing, synthesizing, analyzing, and transforming spoken audio.

See all 410 papers

A Fusion-Aware Two-Stage Framework for Mispronunciation Detection and Diagnosis in Low-Resource Modern Standard Arabic
Jun 23, 2026Jing Yang, Shuqing Zhang, Yongyi Deng +5Domain Adaptation Mispronunciation Detection and Diagnosis
School of Electronic Information, Wuhan University, Wuhan Hubei, 430072, China · School of Computing and Information Systems, University of Melbourne, Melbourne, Australia · Northwestern Polytechnical University +1
Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech Recognition
Sep 28, 2026Omid Ghahroodi, Anas Madkoor, Dima Faris Al Saudi +37ASR Evaluation Arabic NLP
QCRI, HBKU · Qatar University · UDST +15
SHAMS: An Audio-Grounded Pronunciation Benchmark for Levantine Arabic
Oct 1, 2026Ben Sapirstein, Roy Mattar, Guy Mor-Lan +3ASR Evaluation Arabic NLP
Reichman University · Independent Researcher · The Hebrew University of Jerusalem +2

Jun 23, 2026J/K move · Enter open · S save

A Fusion-Aware Two-Stage Framework for Mispronunciation Detection and Diagnosis in Low-Resource Modern Standard Arabic

Jing Yang, Shuqing Zhang, Yongyi Deng, Pan Li+4

School of Electronic Information, Wuhan University, Wuhan Hubei, 430072, China · School of Computing and Information Systems, University of Melbourne, Melbourne, Australia · Northwestern Polytechnical University+1

Domain Adaptation Mispronunciation Detection and Diagnosis Speech Processing Automatic Speech Recognition Low-Resource Language Processing

Open paper AskPDF arXiv

Accurate phoneme recognition is pivotal for mispronunciation detection and diagnosis (MDD) in modern standard Arabic (MSA), yet remains constrained by data scarcity and the synthetic-real domain gap. This work proposes a two-stage end-to-end framework. It integrates a pre-trained encoder with causal dilated temporal convolutional networks to preserve fine-grained phonetic variations. A hierarchical two-stage strategy first learns general mappings from native/synthetic corpora, then adapts to scarce real learner data to mitigate domain shift without over-correction. Prediction stability is further enhanced via multi-checkpoint ensemble inference with N-gram rescoring. Evaluated on the QuranMB.v2 test set, our system achieves an F1-score of

0.7201

, a

63.1

% relative improvement over baseline (

0.4414

). This performance ranks at the top of the IqraEval.2 Challenge, establishing a new state-of-the-art for low-resource MSA in MDD.

Abstract

Speech Processing

Explore similar work

A Fusion-Aware Two-Stage Framework for Mispronunciation Detection and Diagnosis in Low-Resource Modern Standard Arabic

Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech Recognition

SHAMS: An Audio-Grounded Pronunciation Benchmark for Levantine Arabic