Lipi-Ghor: A Large-Scale, Diarized, Multi-Domain Bengali Speech Corpus and a Scalable Pipeline for Building It
Authors: Sanjid Hasan, A H M Fuad, Risalat Labib, Bayazid Hasan Simon
Organizations: Department of Electrical and Electronic Engineering, KUET, Khulna, Bangladesh · Department of Computer Science and Engineering, BUET, Dhaka, Bangladesh
Bengali, spoken by over 230 million people, remains severely under-resourced in spoken language research. Existing open Bengali ASR corpora are confined to short, read or crowd-sourced speech and lack speaker diarization and long-form audio. We present Lipi-Ghor, a large-scale weakly-supervised Bengali speech corpus comprising 885 hours of audio sourced from 998 YouTube videos across 596 unique channels, spanning 17 diverse domains including audiobooks, drama, religious sermons, news, and regional dialects. Each recording is annotated in SSTT format (Speaker, Speech, Transcription, Timestamp), which combines automated diarization with Bengali YouTube caption tracks. To our knowledge, Lipi-Ghor is the first Bengali speech corpus to pair large-scale coverage with speaker-diarized, long-form audio annotations, both of which are absent from existing open Bengali resources. We also present the construction methodology as a replicable pipeline for any language with a sufficient captioned YouTube presence. Targeted channel curation across a handful of rich domains yields corpus scales that were previously out of reach for low-resource languages. We benchmark four widely-used ASR models under zero-shot and fine-tuned conditions. All four degrade consistently on emotive, dialectal, and long-form content. Because caption targets are noisy, we also release a human-annotated gold evaluation split, verified by 17 volunteer annotators over 30 hours of human-in-the-loop correction and stratified across all 17 domains. We release our original contributions (video-ID lists, diarization output, pipeline code, and the gold annotations) under CC-BY-4.0, and distribute caption-derived transcripts for research use only; no raw audio is redistributed.
Figures & tables
Figure 1: The Lipi-Ghor construction pipeline. Four stages turn a curated set of Bengali YouTube channels into a speaker-diarized, SSTT-annotated corpus. The methodology is language-agnostic; only Stage 1 curation is language-specific.
Corpus
Hrs
Dom.
Diar.
Long
Common Voice bn
∼ 40
1
✗
✗
BengaliAI Speech
∼ 1200 ∗
few
✗
✗
OOD-Speech (test)
23
17
✗
✗
BengaliLoop (eval)
small
few
✓
✓
Lipi-Ghor
885
17
✓
✓
Table 1: Open Bengali speech resources by scale, domain count, speaker diarization, and long-form audio. ∗ Predominantly short, read/controlled speech.
Figure 2: Pareto view of audio duration by domain. Seven of the seventeen domains account for roughly 83% of the corpus, reflecting where Bengali caption coverage is densest on YouTube.
Property
Value
Total Videos
998
Unique Channels
596
Total Duration
885.34 hours
Avg. Duration/Video
∼ 52 minutes
Domains
17
Total Word Tokens
5,832,680
Table 2: Lipi-Ghor corpus summary. ∗ CC-BY-4.0 applies to our original contributions; caption-derived transcripts are for research use only (§ 9 ).
Property
Value
Diarized recordings
1,019
Speaker-attributed segments
984,647
Speaker labels (all recordings)
9,056
Speech / non-speech
639 h / 243 h
Speech ratio (mean)
0.76
Speakers per rec. (median, IQR)
4 (1–12)
Table 3: Conversational structure of the diarized corpus. Speech ratio is the fraction of each recording covered by at least one speaker turn, averaged over recordings.
Domain
Spk
Solo
Turn
SR
Seg
Audio-book
1
62
3.4
0.74
1.83
Talk-show
5
2
20.1
0.86
2.38
Movie
24
14
20.0
0.58
0.95
Natok
18
2
21.6
0.56
1.14
Podcast
3
10
21.9
0.85
1.73
Waz
2
32
5.8
0.84
2.21
Table 4: Conversational structure by domain. Spk = median speakers per recording; Solo = % of single-speaker recordings; Turn = mean speaker turns per minute; SR = mean speech ratio; Seg = median segment duration (s). Ordered by total duration.
Figure 3: Cumulative distribution of diarized segment durations (log-scaled x-axis). Nearly two-thirds of the 984,647 segments are shorter than two seconds.
Domain
WER
CER
Cartoon
28.57
25.85
Vlog
14.17
4.07
Commentary
10.81
5.33
News
10.34
6.57
Movie
7.32
5.28
Waz
6.51
3.88
Table 5: Domain-wise disagreement between the weakly supervised YouTube transcripts and the human-corrected gold references. WER and CER are corpus-level percentages calculated separately for each domain.
Figure 4: Global WER for the four zero-shot systems (blue) and two fine-tuned models (orange). Fine-tuned Faster Whisper Medium reaches 20.70%, below the best zero-shot system (Hishab TITU, 30.51%).
Model
WER
CER
Hishab TITU Conformer-L
30.51
18.23
Meta MMS 1B
43.06
21.16
Seamless M4T-v2
66.71
45.54
Whisper large-v3
84.53
72.92
Table 6: Zero-shot ASR performance on Lipi-Ghor (%).
Domain
TITU
MMS
SM4T
Wh-v3
Audio-book
21.02
33.04
57.09
80.87
News
28.03
41.68
57.83
86.23
Talk-show
29.88
43.38
64.81
86.17
Waz
35.75
45.02
74.27
85.15
Natok
44.37
60.14
73.94
89.01
Comedy/Satire
44.56
56.18
85.37
88.64
Table 7: Domain-specific WER (%) for representative domains. SM4T = Seamless M4T-v2.
Figure 5: Zero-shot WER across all 17 Lipi-Ghor domains, sorted by Hishab TITU’s error rate. The three Bengali-capable systems degrade together as speech becomes more spontaneous and dialectal; Whisper large-v3 fails near-uniformly across domains. Table 7 lists a representative subset.
Figure 6: Median diarized segment duration vs. zero-shot WER (Hishab TITU) across the seven domains of Table 7 . Marker area proportional to domain hours.
Quartile
TITU
MMS
SM4T
Wh-v3
Short
30.54
41.98
66.11
85.01
Medium
29.14
41.79
64.49
84.25
Long
28.46
40.74
63.55
84.00
Very Long
32.60
45.53
70.13
84.93
Table 8: WER (%) by utterance duration quartile.
Model
WER
CER
Fast Conformer (A)
24.67
15.56
Faster Whisper Med. (B)
20.70
11.90
Moonshine-Base Bengali †
21.54
10.79
Table 9: Fine-tuned model performance on Lipi-Ghor (%). † Hasan and Rahman (2026) ; RTF 0.0053.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Domain
Videos ∗
Hours
Audio-book
277
254.37
Talk-show
176
142.40
Movie
74
122.18
Natok (Drama)
95
78.66
Podcast
62
57.59
Waz (Sermons)
66
47.89
Appendix
Table 10: Domain distribution of Lipi-Ghor. ∗ Per-domain counts are raw video counts before cross-domain deduplication (1,019 total); deduplication yields 998 unique videos. Hours are on the deduplicated corpus.
Caption Type
Videos
Percentage
Manually created captions
102
10.22%
Auto-generated captions
896
89.78%
Total
998
100.00%
Appendix
Table 11: Distribution of YouTube Caption Types in Lipi-Ghor
Figure 7: The Lipi-Ghor annotation interface. Clockwise from top left: the embedded video player, the waveform with a per-speaker diarization ribbon beneath it, the speaker roster for the recording, and the segment list where each row carries a speaker assignment, timestamps, and an editable Bengali transcript. Annotators can split, merge, or delete segments and reassign speakers from the dropdown. Identifying details of the source video, including the video identifier and all channel branding, are masked for double-blind review; annotators saw the unmasked frame.
Although Automatic Speech Recognition (ASR) in Bengali has seen significant progress, processing long-duration audio and performing robust speaker diarization remain critical research gaps. To address the severe scarcity of joint ASR and diarization resources for this language, we introduce Lipi-Ghor-882, a comprehensive 882-hour multi-speaker Bengali dataset. In this paper, detailing our submission to the DL Sprint 4.0 competition, we systematically evaluate various architectures and approaches for long-form Bengali speech. For ASR, we demonstrate that raw data scaling is ineffective; instead, targeted fine-tuning utilizing perfectly aligned annotations paired with synthetic acoustic degradation (noise and reverberation) emerges as the singular most effective approach. Conversely, for speaker diarization, we observed that global open-source state-of-the-art models (such as Diarizen) performed surprisingly poorly on this complex dataset. Extensive model retraining yielded negligible improvements; instead, strategic, heuristic post-processing of baseline model outputs proved to be the primary driver for increasing accuracy. Ultimately, this work outlines a highly optimized dual pipeline achieving a ∼0.019 Real-Time Factor (RTF), establishing a practical, empirically backed benchmark for low-resource, long-form speech processing.
Bengali is spoken by over 230 million people yet remains severely under-served in automatic speech recognition (ASR) and speaker diarization research. In this paper, we present our system for the DL Sprint 4.0 Bengali Long-Form Speech Recognition (Task1) and Bengali Speaker Diarization Challenge (Task2). For Task1, we propose a data-centric pipeline that constructs a high-quality training corpus from Bengali YouTube audiobooks and dramas \cite{tabib2026bengaliloop}, incorporating LLM-assisted language normalization, fuzzy-matching-based chunk boundary validation, and muffled-zone augmentation. Fine-tuning the tugstugi/whisper-medium model on approximately 21,000 data points with beam size 5, we achieve a Word Error Rate (WER) of 16.751 on the public leaderboard and 15.551 on the private test set. For Task2, we fine-tune the pyannote.audio community-1 segmentation model with targeted hyperparameter optimization under an extreme low-resource setting (10 training files), achieving a Diarization Error Rate (DER) of 0.19974 on the public leaderboard, and .26723 on the private test set. Our results demonstrate that careful data engineering and domain-adaptive fine-tuning can yield competitive performance for Bengali speech processing even without large annotated corpora.
We describe our end-to-end system for Bengali long-form speech recognition (ASR) and speaker diarization submitted to the DL Sprint 4.0 competition on Kaggle. Bengali presents substantial challenges for both tasks: a large phoneme inventory, significant dialectal variation, frequent code-mixing with English, and a relative scarcity of large-scale labelled corpora. For ASR we achieve a best private Word Error Rate (WER) of 0.37738 and public WER of 0.36137, combining a BengaliAI fine-tuned Whisper medium model with Demucs source separation for vocal isolation, silence-boundary chunking, and carefully tuned generation hyperparameters. For speaker diarization we reach a best private Diarization Error Rate (DER) of 0.27671 and public DER of 0.20936 by replacing the default segmentation model inside the pyannote.audio pipeline with a Bengali-fine-tuned variant, pairing it with wespeaker-voxceleb-resnet34-LM embeddings and centroid-based agglomerative clustering. Our experiments demonstrate that domain-specific fine-tuning of the segmentation component, vocal source separation, and natural silence-aware chunking are the three most impactful design choices for low-resource Bengali speech processing.
MD. Sagor Chowdhury, Adiba Fairooz Chowdhury
Department of Computer Science and Engineering Chittagong University of Engineering and Technology Chattogram, Bangladesh