Bengali is spoken by over 230 million people yet remains severely under-served in automatic speech recognition (ASR) and speaker diarization research. In this paper, we present our system for the DL Sprint 4.0 Bengali Long-Form Speech Recognition (Task1) and Bengali Speaker Diarization Challenge (Task2). For Task1, we propose a data-centric pipeline that constructs a high-quality training corpus from Bengali YouTube audiobooks and dramas \cite{tabib2026bengaliloop}, incorporating LLM-assisted language normalization, fuzzy-matching-based chunk boundary validation, and muffled-zone augmentation. Fine-tuning the tugstugi/whisper-medium model on approximately 21,000 data points with beam size 5, we achieve a Word Error Rate (WER) of 16.751 on the public leaderboard and 15.551 on the private test set. For Task2, we fine-tune the pyannote.audio community-1 segmentation model with targeted hyperparameter optimization under an extreme low-resource setting (10 training files), achieving a Diarization Error Rate (DER) of 0.19974 on the public leaderboard, and .26723 on the private test set. Our results demonstrate that careful data engineering and domain-adaptive fine-tuning can yield competitive performance for Bengali speech processing even without large annotated corpora.
Figures & tables
Fig. 1: Chunk boundary validation pipeline. For each chunk, the last 5 seconds of audio are sent to Gemini for endpoint word prediction. Fuzzy matching then selects the best boundary word from a candidate list spanning the current and next chunk.
Model
WER
Notes
Wav2Vec2 (arijitx)
∼ 100
Merged/missing words
Whisper large-v3
75.0
No Bengali fine-tune
IndicWav2Vec Bengali
> 40
ai4bharat
BanglaASR small
—
Garbage output
BanglaASR (5 epochs)
∼ 50
Trained on comp. data
Indic Conformer 600M
—
No output produced
TABLE I: Baseline Model Comparison (No Fine-Tuning)
Parameter
Value
Epochs
12
Batch size (per device)
16
Gradient accumulation steps
1
Learning rate
2×10−5
LR scheduler
Cosine
Warmup steps
700
TABLE II: Training Hyperparameters (Full Fine-Tune, A100)
Fig. 2: Training loss over 12 epochs of full fine-tuning on the A100.
Fig. 3: Evaluation loss over 12 epochs of full fine-tuning on the A100.
Fig. 4: Evaluation WER during fine-tuning. The dotted line marks the base model WER (34.8%). The model converges to 16.751% by epoch 12 ( checkpoint-12000 ).
Beam Size
WER (%)
Inference Time
1
18.42
14 min 53 s
4
∼ 17.5
∼ 20 min
5
16.751
24 min 7 s
TABLE III: Beam Size vs. WER and Inference Time (Test Set, Fine-Tuned Model)
Strategy
WER
Manual chunking
41.8
Model internal stride
∼ 39.0
WhisperX (batch=64)
40.6
WhisperX + normalization
35.9
Vocal extraction (HTDemucs)
34.8
TABLE IV: Chunking Strategy Comparison (tugstugi base model)
System Configuration
DER
WhisperX (chunk=5s, agg. merge)
0.273
WhisperX (chunk=2s)
0.248
Pyannote 3.1 (raw)
0.410
Pyannote 3.1 + segment merging
0.290
Community-1 + heavy merge
0.272
Community-1 + FT (10 epochs)
0.199
TABLE V: Diarization System Progression (DER)
System
Public
Private
tugstugi base (no FT)
34.80
—
50% data, H200, 5 epochs
21.893
—
+ vocal extraction (HTDemucs)
21.00
—
Full FT, A100, 12 epochs
16.751
15.551
TABLE VI: ASR WER Progression
System
Params
WER
FT?
Whisper large-v3
1,550M
75.0
No
IndicWav2Vec Bengali
317M
> 40.0
No
BanglaASR small (5 ep.)
244M
∼ 50.0
Yes
tugstugi + normalization
769M
35.9
No
tugstugi + vocals + normalization
769M
34.8
No
Ours (tugstugi FT, beam=5)
769M
16.751
Yes
TABLE VII: ASR Benchmark — Competition Audio
System
DER (train set)
Pyannote 3.1 (raw)
0.410
WhisperX (best config)
0.248
Community-1 + heavy merge
0.272
Community-1 + FT (ours)
0.199
Community-1 + FT + HPO (ours)
0.194
Public leaderboard: 0.18999 (after boundary rounding)