Organizations: Data Science Research Lab, Department of Statistics, University of Rajshahi, Rajshahi-6205, Bangladesh · Department of Statistics, Mawlana Bhashani Science and Technology University, Tangail-1902, Bangladesh · Department of Statistics, University of Rajshahi, Rajshahi-6205, Bangladesh
For millions of users in developing economies who depend on mobile banking as their primary gateway to financial services, app quality directly shapes financial access. The study analyzed 5,652 Google Play reviews in English and Bangla (filtered from 11,414 raw reviews) for four Bangladeshi government banking apps. The authors used a hybrid labeling approach that combined use of the reviewer's star rating for each review along with a separate independent XLM-RoBERTa classifier to produce moderate inter-method agreement (kappa = 0.459). Traditional models outperformed transformer-based ones: Random Forest produced the highest accuracy (0.815), while Linear SVM produced the highest weighted F1 score (0.804); both were higher than the performance of fine-tuned XLM-RoBERTa (0.793). McNemar's test confirmed that all classical models were significantly superior to the off-the-shelf XLM-RoBERTa (p < 0.05), while differences with the fine-tuned variant were not statistically significant. DeBERTa-v3 was applied to analyze the sentiment at the aspect level across the reviews for the four apps; the reviewers expressed their dissatisfaction primarily with the speed of transactions and with the poor design of interfaces; eJanata app received the worst ratings from the reviewers across all apps. Three policy recommendations are made based on these findings - remediation of app quality, trust-centred release management, and Bangla-first NLP adoption - to assist state-owned banks in moving towards improving their digital services through data-driven methods. Notably, a 16.1-percentage-point accuracy gap between Bangla and English text highlights the need for low-resource language model development.
Figures & tables
App
Raw
Clean
Avg. Rating
Sonali e-Wallet
7,306
4,347
3.81
Agrani Smart
2,434
1,432
3.27
eJanata
1,232
925
2.55
Rupali e-Bank
442
340
3.49
Total
11,414
7,044
3.52
TABLE I: Dataset Statistics per Application
Fig. 1: Distribution of raw and cleaned reviews across the four applications.
Fig. 2: Aspect-based sentiment polarity across six service dimensions.
Model
Acc.
Prec.
Rec.
W-F1
95% CI
Random Forest
0.815
0.801
0.815
0.803
[.778, .829]
Linear SVM
0.809
0.808
0.809
0.804
[.781, .829]
Log. Reg.
0.804
0.801
0.804
0.798
[.773, .823]
Naïve Bayes
0.814
0.801
0.814
0.795
[.772, .821]
XLM-R (FT)
0.769
0.834
0.769
0.793
[.772, .818]
XLM-R (OTS)
0.683
0.839
0.683
0.740
[.716, .764]
TABLE II: Model Performance on Test Set (Weighted F1)
Comparison
χ2
p-value
Sig.
vs. Logistic Reg.
76.11
< 0.001
Yes
vs. Linear SVM
82.97
< 0.001
Yes
vs. Random Forest
80.82
< 0.001
Yes
vs. Naïve Bayes
81.85
< 0.001
Yes
TABLE III: McNemar’s Test: OTS XLM-RoBERTa vs. Classical Models