Machine Learning-Based Detection of Mobile Money Scan Messages in Ghana: A Comparative Evaluation of Naive Bayes, Support Vector Machine, and Randon Forest Classifiers

Ibrahim Salifu; Muhaisin Tiyumba Nantogmah

cs.LGOct 29, 2026

Machine Learning-Based Detection of Mobile Money Scan Messages in Ghana: A Comparative Evaluation of Naive Bayes, Support Vector Machine, and Randon Forest Classifiers

Authors: Ibrahim Salifu, Muhaisin Tiyumba Nantogmah

Organizations: Department of Computer Science, University of Technology and Applied Sciences, Navrongo, Ghana · Department of Computer Science, University for Development Studies, Tamale, Ghana

Abstract

Mobile money (MoMo) is Ghana's preferred retail payment channel, recording GHC518.4 billion in transactions in December 2025 and over 26.7 million active accounts (Graphic Online, 2026). This growth has coincided with increased social-engineering fraud over Short Message Service (SMS) and Unstructured Supplementary Service Data (USSD), where fraudsters impersonate network operators, agents, relatives, or regulators to obtain personal identification numbers (PINs), identity data, or funds. We present a supervised machine-learning pipeline for classifying scam and legitimate Ghanaian MoMo SMS messages using a labelled corpus of 220 messages across 18 scam and legitimate subcategories. Domain-specific normalisation of currency amounts, phone numbers, and USSD codes preceded TF-IDF vectorisation of unigrams and bigrams. Using stratified five-fold cross-validation and a 20% held-out test set, we trained and evaluated Multinomial Naive Bayes, linear support vector machine (SVM), and Random Forest classifiers using accuracy, precision, recall, F1-score, Matthews correlation coefficient (MCC), and area under the receiver operating characteristic curve (AUC-ROC). Held-out accuracy, F1, and AUC were 1.00 for Naive Bayes and linear SVM; cross-validated accuracy averaged 96.6% (+/-3.3), 94.9% (+/-4.6), and 92.0% (+/-4.2). Discriminative n-grams reflected imperative urgency ("send," "claim," "please send") and PIN solicitation ("PIN," "pincode"); legitimate messages featured confirmatory and cautionary language ("balance," "successful," "do not share"). Findings are contextualised in Sub-Saharan African SMS fraud detection, small template-based corpus ceiling effects, and a pathway toward larger naturalistic multilingual corpora. Interpretable classical classifiers achieve high accuracy on structured MoMo scam text and suit carrier- or handset-level filtering in low-resource, on-device settings.

Figures & tables

Category	Class	Count	Example theme
transaction_alert	legitimate	22	Successful withdrawal/payment / bill-pay confirmation
otp_notice	legitimate	22	One-time verification code with non-disclosure reminder
loan_notice	legitimate	21	Genuine loan approval, no upfront fee required
loan_scam	scam	18	“Pre-approved” loan requiring an activation/processing fee
fake_prize	scam	18	Lucky-customer / lottery reward requiring PIN entry
family_emergency	scam	18	Impersonated relative requesting urgent funds

Table 1: Dataset composition by class label and message category.

Figure 1: (a) Overall class distribution (scam vs. legitimate). (b) Distribution of messages across the 18 fine-grained message categories.

Figure 2: (a) Character-length distribution by class. (b) Word-count distribution by class (box plot); scam messages are marginally longer on average ( $M=125.5$ characters) than legitimate messages ( $M=114.3$ characters), consistent with scam scripts’ tendency to embed elaborate justification narratives.

Model Architecture	Test Accuracy	Test F1-Score	Model Size (MB)	Avg. Latency (ms)	Peak RAM (MB)	On-Device Feasibility
Multinomial Naïve Bayes	100.0%	1.000	0.18 MB	0.42 ms	12.4 MB	Optimal (Ultra-lightweight)
Linear SVM	100.0%	1.000	0.34 MB	0.85 ms	14.1 MB	Optimal (High efficiency)
Random Forest (300 trees)	97.7%	0.983	14.20 MB	12.60 ms	48.2 MB	Moderate (Noticeable overhead)
DistilBERT Baseline (Ref.)	98.2%	0.981	255.00 MB	88.50 ms	320.0 MB	Impractical without dedicated NPU/Cloud

Table 2: Comparative evaluation of model predictive performance, computational footprint, and inference latency for deployment feasibility.

Figure 3: Held-out test-set performance across five evaluation metrics for all three classifiers. Naïve Bayes and the linear SVM achieve perfect scores on every metric; Random Forest trails marginally on precision and MCC owing to a single false-positive misclassification.

Figure 4: Distribution of 5-fold cross-validation accuracy on the training partition. Individual fold scores are overlaid as points; Naive Bayes shows the highest median accuracy and Random Forest the widest spread, indicating comparatively greater sensitivity to fold composition.

Figure 5: Confusion matrices on the held-out test set ( $n=44$ : 16 legitimate, 28 scam) for Naive Bayes, linear SVM, and Random Forest.

Figure 6: Receiver Operating Characteristic curves for all three classifiers on the held-out test set, plotted against the random-classifier reference line.

Figure 7: Top 15 scam-indicative (a) and legitimate-indicative (b) unigram/bigram features ranked by Naive Bayes log-probability ratio between classes.

Figure 8: Top 20 features by Gini importance in the fitted Random Forest classifier, corroborating the Naive Bayes lexical analysis (urgency verbs, PIN/USSD references, and currency-amount placeholders dominate scam-side importance).

Financial Fraud Detection

Financial fraud detection identifies suspicious transactions, accounts, claims, and behavior to prevent losses and guide investigation.

See all 139 papers

Detecting Money Laundering in Rwandan Mobile Money: A Machine Learning Framework
Aug 15, 2026Emmanuel Nahimana, Yaé Ulrich GabaAnti-Money Laundering Detection Anti-Money Laundering
African Institute for Mathematical Sciences (AIMS) Senegal, KM 2, Route de Joal, Mbour, Senegal · AI Research and Innovation Nexus for Africa (AIRINA Labs), AI.Technipreneurs, Cotonou, Bénin · Sefako Makgatho Health Sciences University, Pretoria, South Africa
FraudSMSWalker: Benchmarking Agentic Large Language Models for SMS-to-Webpage Fraud Detection
Jun 15, 2026Y. H. Zhou, Z. M. Ma, Y. J. Zhou +12Web Agent Benchmarks LLM Agent Evaluation
ReCAST: Restoration-aware Cascaded Stage-wise Training for Obfuscated SMS Risk Classification
Sep 4, 2026Jieyun Huang, Yi Shen, Kaikai Zhao +7Text Classification Adversarial Robustness
Unicom Data Intelligence, China Unicom · China Unicom Data Science & Artificial Intelligence Research Institute, China Unicom

Aug 15, 2026J/K move · Enter open · S save

Detecting Money Laundering in Rwandan Mobile Money: A Machine Learning Framework

Emmanuel Nahimana, Yaé Ulrich Gaba

African Institute for Mathematical Sciences (AIMS) Senegal, KM 2, Route de Joal, Mbour, Senegal · AI Research and Innovation Nexus for Africa (AIRINA Labs), AI.Technipreneurs, Cotonou, Bénin · Sefako Makgatho Health Sciences University, Pretoria, South Africa

Anti-Money Laundering Detection Anti-Money Laundering Financial Fraud Detection

Open paper AskPDF arXiv

Mobile money has widened financial access across Sub-Saharan Africa and enlarged the surface for money-laundering and terrorism-financing (ML/TF) activity in ecosystems dominated by high-volume, low-value transactions. Rwanda is a case in point: several million active mobile-money users, telecom-led wallets on the MTN and Airtel networks, and a Financial Intelligence Centre (FIC) supervising transaction streams whose scale exceeds static rule-based monitoring. This paper develops and evaluates a transaction-monitoring framework aligned to the Rwandan AML/CFT regime under (i) extreme class imbalance (~0.1% prevalence), (ii) scarce and delayed labels, and (iii) bounded investigator capacity. Using SAML-D, a synthetic dataset of 9,504,852 transactions with 17 laundering typologies, we engineer account-centric behavioural features (rolling velocity, net-flow directionality, counterparty diversity, burstiness) and benchmark supervised classifiers (Logistic Regression, Random Forest, LightGBM), unsupervised anomaly detectors (Isolation Forest, Local Outlier Factor), a dense autoencoder, and a late-fusion meta-learner. Evaluation is operational: PR-AUC, recall at a calibrated ~90%-precision point, recall at top-K%, and alerts per 10,000. On the chronologically held-out test period, LightGBM attains PR-AUC = 0.0469, capturing 64 laundering cases at precision ~0.89 with 0.51 alerts per 10,000; the fusion stacker reaches PR-AUC = 0.0477 at precision ~0.91 and 0.46 alerts per 10,000, recovering 59 true positives. We map score bands to Rwanda-relevant analyst workflows and STR/SAR escalation, and outline a staged path from synthetic prototyping to real-data validation with the National Bank of Rwanda and FIC. The contribution is operational: a governance-aware pipeline and evaluation protocol calibrated to the constraints of an African mobile-money regulator, not a new algorithm.

Abstract

Financial Fraud Detection

Explore similar work

Detecting Money Laundering in Rwandan Mobile Money: A Machine Learning Framework

FraudSMSWalker: Benchmarking Agentic Large Language Models for SMS-to-Webpage Fraud Detection

ReCAST: Restoration-aware Cascaded Stage-wise Training for Obfuscated SMS Risk Classification