Reading comprehension systems for low-resource languages face significant challenges in handling unanswerable questions. These systems tend to produce unreliable responses when correct answers are absent from context. To solve this problem, we introduce NCTB-QA, a large-scale Bangla question answering dataset comprising 87,805 question-answer pairs extracted from 50 textbooks published by Bangladesh's National Curriculum and Textbook Board. Unlike existing Bangla datasets, NCTB-QA maintains a balanced distribution of answerable (57.25%) and unanswerable (42.75%) questions. NCTB-QA also includes adversarially designed instances containing plausible distractors. We benchmark three transformer-based models (BERT, RoBERTa, ELECTRA) and demonstrate substantial improvements through fine-tuning. BERT achieves 313% relative improvement in F1 score (0.150 to 0.620). Semantic answer quality measured by BERTScore also increases significantly across all models. Our results establish NCTB-QA as a challenging benchmark for Bangla educational question answering. This study demonstrates that domain-specific fine-tuning is critical for robust performance in low-resource settings.
Figures & tables
Dataset
QA Pairs
Contexts
Answerable
Unanswerable
Domain
NCTB-QA (Ours)
87,805
10,070
50,270 (57.25%)
37,535 (42.75%)
Educational
TigerLLM ( Raihan and Zampieri )
10M tokens
163 textbooks
100%
0%
Educational
BanglaRQA ( Ekram et al. )
14,889
3,000
11,258 (75.6%)
3,631 (24.4%)
General
BEnQA ( Shafayat et al. )
5,161
–
100%
0%
Educational
NCTB( Khondoker et al. )
3,000
–
3,000 (100%)
0%
Educational
BanglaQA ( Shahriar et al. )
178,012
15,997 articles
142,181 (79.87%)
35,831 (20.13%)
News
Table 1: Comparison of NCTB-QA with existing Bangla question–answering datasets. NCTB-QA uniquely combines large-scale, educational domain specificity, and a relatively balanced distribution of answerable and unanswerable questions.
Figure 1: Overview of the content extraction and QA generation pipeline. Markdown textbooks are cleaned, segmented, and thematically grouped into coherent contexts, which are then processed by Gemini to generate structured QA pairs in JSON format.
Figure 2: Sample of Dataset
Metric
Value
Total QA pairs
87,805
Answerable questions
50,270 (57.25%)
Unanswerable questions
37,535 (42.75%)
Contextual segments
10,070
Source textbooks
50
Grade levels
1–10
Table 2: Overall statistics of the NCTB-QA dataset.
Figure 3: Distribution of word count per context in NCTB-QA. The distribution spans from 103 to 1,871 words, with the majority of contexts containing between 150–400 words.
Figure 4: Distribution of word count per question in NCTB-QA. Questions are predominantly concise, with most containing between 7–12 words.
Figure 5: Distribution of word count per answer in NCTB-QA. The right-skewed distribution shows most answers are brief, while some require extended responses.
Figure 6: Distribution of answer start positions (character index) in NCTB-QA. Answers are distributed throughout contexts, mitigating positional bias.
Figure 7: Subject-wise distribution of answerable (blue) and unanswerable (red) questions in NCTB-QA, showing consistent answerability ratios across domains.
Split
Unanswerable
Answerable
Train
Factual
7,359
9,223
Confirmation
15
67
Causal
2,092
2,792
Comparative
1,900
1,607
Negation
3,441
4,116
Table 3: Distribution of question types across train, validation, and test splits in NCTB-QA. The consistent proportions across splits ensure balanced evaluation of different reasoning capabilities.
Hyperparameter
BERT
RoBERTa
ELECTRA
Pre-trained model
bert-base-cased
roberta-base
google/electra-base-discriminator
Max sequence length
384
384
384
Document stride
128
128
128
Train batch size
2
2
4
Eval batch size
4
4
8
Gradient accumulation steps
16
16
8
Table 4: Training hyperparameters for BERT, RoBERTa, and ELECTRA extractive QA models.
Model
Stage
EM
F1
BERTScore
BERT
Before
0.059
0.150
0.665
After
0.509
0.620
0.880
RoBERTa
Before
0.427
0.428
0.667
After
0.450
0.469
0.867
ELECTRA
Before
0.395
0.406
0.664
After
0.411
0.550
0.825
Table 5: Model performance on NCTB-QA test set before and after fine-tuning
Model
Stage
Factual
Confirm.
Causal
Compar.
Negation
Others
BERT
Before
0.150
0.148
0.156
0.126
0.159
0.163
After
0.613
0.679
0.517
0.767
0.673
0.610
RoBERTa
Before
0.396
0.437
0.440
0.672
0.482
0.380
After
0.443
0.500
0.444
0.688
0.497
0.438
ELECTRA
Before
0.379
0.410
0.424
0.617
0.449
0.355
After
0.551
0.575
0.454
0.701
0.601
0.501
Table 6: F1 scores across question types before and after fine-tuning. All models show consistent gains, with BERT achieving the largest improvements.
Question answering studies how AI systems interpret questions and produce accurate, evidence-grounded answers across text, dialogue, and multimodal data.