cs.LGOct 29, 2026

Machine Learning-Based Detection of Mobile Money Scan Messages in Ghana: A Comparative Evaluation of Naive Bayes, Support Vector Machine, and Randon Forest Classifiers

Authors: Ibrahim Salifu, Muhaisin Tiyumba Nantogmah

Organizations: Department of Computer Science, University of Technology and Applied Sciences, Navrongo, Ghana · Department of Computer Science, University for Development Studies, Tamale, Ghana

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Mobile money (MoMo) is Ghana's preferred retail payment channel, recording GHC518.4 billion in transactions in December 2025 and over 26.7 million active accounts (Graphic Online, 2026). This growth has coincided with increased social-engineering fraud over Short Message Service (SMS) and Unstructured Supplementary Service Data (USSD), where fraudsters impersonate network operators, agents, relatives, or regulators to obtain personal identification numbers (PINs), identity data, or funds. We present a supervised machine-learning pipeline for classifying scam and legitimate Ghanaian MoMo SMS messages using a labelled corpus of 220 messages across 18 scam and legitimate subcategories. Domain-specific normalisation of currency amounts, phone numbers, and USSD codes preceded TF-IDF vectorisation of unigrams and bigrams. Using stratified five-fold cross-validation and a 20% held-out test set, we trained and evaluated Multinomial Naive Bayes, linear support vector machine (SVM), and Random Forest classifiers using accuracy, precision, recall, F1-score, Matthews correlation coefficient (MCC), and area under the receiver operating characteristic curve (AUC-ROC). Held-out accuracy, F1, and AUC were 1.00 for Naive Bayes and linear SVM; cross-validated accuracy averaged 96.6% (+/-3.3), 94.9% (+/-4.6), and 92.0% (+/-4.2). Discriminative n-grams reflected imperative urgency ("send," "claim," "please send") and PIN solicitation ("PIN," "pincode"); legitimate messages featured confirmatory and cautionary language ("balance," "successful," "do not share"). Findings are contextualised in Sub-Saharan African SMS fraud detection, small template-based corpus ceiling effects, and a pathway toward larger naturalistic multilingual corpora. Interpretable classical classifiers achieve high accuracy on structured MoMo scam text and suit carrier- or handset-level filtering in low-resource, on-device settings.

Figures & tables

CategoryClassCountExample theme
transaction_alertlegitimate22Successful withdrawal/payment / bill-pay confirmation
otp_noticelegitimate22One-time verification code with non-disclosure reminder
loan_noticelegitimate21Genuine loan approval, no upfront fee required
loan_scamscam18“Pre-approved” loan requiring an activation/processing fee
fake_prizescam18Lucky-customer / lottery reward requiring PIN entry
family_emergencyscam18Impersonated relative requesting urgent funds
Table 1: Dataset composition by class label and message category.
Figure 1: (a) Overall class distribution (scam vs. legitimate). (b) Distribution of messages across the 18 fine-grained message categories.
Figure 2: (a) Character-length distribution by class. (b) Word-count distribution by class (box plot); scam messages are marginally longer on average ( M=125.5M=125.5 characters) than legitimate messages ( M=114.3M=114.3 characters), consistent with scam scripts’ tendency to embed elaborate justification narratives.
Model ArchitectureTest AccuracyTest F1-ScoreModel Size (MB)Avg. Latency (ms)Peak RAM (MB)On-Device Feasibility
Multinomial Naïve Bayes100.0%1.0000.18 MB0.42 ms12.4 MBOptimal (Ultra-lightweight)
Linear SVM100.0%1.0000.34 MB0.85 ms14.1 MBOptimal (High efficiency)
Random Forest (300 trees)97.7%0.98314.20 MB12.60 ms48.2 MBModerate (Noticeable overhead)
DistilBERT Baseline (Ref.)98.2%0.981255.00 MB88.50 ms320.0 MBImpractical without dedicated NPU/Cloud
Table 2: Comparative evaluation of model predictive performance, computational footprint, and inference latency for deployment feasibility.
Figure 3: Held-out test-set performance across five evaluation metrics for all three classifiers. Naïve Bayes and the linear SVM achieve perfect scores on every metric; Random Forest trails marginally on precision and MCC owing to a single false-positive misclassification.
Figure 4: Distribution of 5-fold cross-validation accuracy on the training partition. Individual fold scores are overlaid as points; Naive Bayes shows the highest median accuracy and Random Forest the widest spread, indicating comparatively greater sensitivity to fold composition.
Figure 5: Confusion matrices on the held-out test set ( n=44n=44 : 16 legitimate, 28 scam) for Naive Bayes, linear SVM, and Random Forest.
Figure 6: Receiver Operating Characteristic curves for all three classifiers on the held-out test set, plotted against the random-classifier reference line.
Figure 7: Top 15 scam-indicative (a) and legitimate-indicative (b) unigram/bigram features ranked by Naive Bayes log-probability ratio between classes.
Figure 8: Top 20 features by Gini importance in the fitted Random Forest classifier, corroborating the Naive Bayes lexical analysis (urgency verbs, PIN/USSD references, and currency-amount placeholders dominate scam-side importance).
More in this topic

Financial Fraud Detection

Financial fraud detection identifies suspicious transactions, accounts, claims, and behavior to prevent losses and guide investigation.

See all 139 papers

Explore similar work

CardsList
Aug 15, 2026

Detecting Money Laundering in Rwandan Mobile Money: A Machine Learning Framework

Mobile money has widened financial access across Sub-Saharan Africa and enlarged the surface for money-laundering and terrorism-financing (ML/TF) activity in ecosystems dominated by high-volume, low-value transactions. Rwanda is a case in point: several million active mobile-money users, telecom-led wallets on the MTN and Airtel networks, and a Financial Intelligence Centre (FIC) supervising transaction streams whose scale exceeds static rule-based monitoring. This paper develops and evaluates a transaction-monitoring framework aligned to the Rwandan AML/CFT regime under (i) extreme class imbalance (~0.1% prevalence), (ii) scarce and delayed labels, and (iii) bounded investigator capacity. Using SAML-D, a synthetic dataset of 9,504,852 transactions with 17 laundering typologies, we engineer account-centric behavioural features (rolling velocity, net-flow directionality, counterparty diversity, burstiness) and benchmark supervised classifiers (Logistic Regression, Random Forest, LightGBM), unsupervised anomaly detectors (Isolation Forest, Local Outlier Factor), a dense autoencoder, and a late-fusion meta-learner. Evaluation is operational: PR-AUC, recall at a calibrated ~90%-precision point, recall at top-K%, and alerts per 10,000. On the chronologically held-out test period, LightGBM attains PR-AUC = 0.0469, capturing 64 laundering cases at precision ~0.89 with 0.51 alerts per 10,000; the fusion stacker reaches PR-AUC = 0.0477 at precision ~0.91 and 0.46 alerts per 10,000, recovering 59 true positives. We map score bands to Rwanda-relevant analyst workflows and STR/SAR escalation, and outline a staged path from synthetic prototyping to real-data validation with the National Bank of Rwanda and FIC. The contribution is operational: a governance-aware pipeline and evaluation protocol calibrated to the constraints of an African mobile-money regulator, not a new algorithm.

Emmanuel Nahimana, Yaé Ulrich Gaba

African Institute for Mathematical Sciences (AIMS) Senegal, KM 2, Route de Joal, Mbour, Senegal · AI Research and Innovation Nexus for Africa (AIRINA Labs), AI.Technipreneurs, Cotonou, Bénin · Sefako Makgatho Health Sciences University, Pretoria, South Africa

Anti-Money Laundering DetectionAnti-Money LaunderingFinancial Fraud Detection
AskPDF
Jun 15, 2026

FraudSMSWalker: Benchmarking Agentic Large Language Models for SMS-to-Webpage Fraud Detection

SMS fraud is increasingly cross-channel: a message directs the user to a webpage, and the final risk depends on how the SMS claim aligns with the page content and requested user action. However, existing evaluations either focus on message-only smishing classification or expose URL and domain cues that allow models to rely on reputation shortcuts. To address this gap, we introduce FraudSMSWalker, a controlled benchmark for URL-masked SMS-to-webpage fraud judgment. FraudSMSWalker contains 699 bilingual chains, including 332 fraudulent and 367 benign cases, across ten service scenarios. The model-visible input consists of the SMS context and sanitized webpage evidence, while raw URLs, hosts, domains, IPs, redirects, and reputation metadata are withheld. The benchmark further includes hard benign cases whose pages contain login, payment, verification, or account-management elements that are plausible under the service context but also appear in scam flows. We evaluate nine web agents under masked browser-agent protocols and conduct URL-visibility ablations. The results show that current agents can detect suspicious cues, but struggle to preserve benign recall and often produce positive predictions that are weakly supported by the observed evidence. These findings position FraudSMSWalker as a benchmark for measuring whether web agents can make fraud judgments that remain both accurate and evidence-grounded when direct reputation shortcuts are suppressed. The associated code and dataset are accessible at the anonymous link.

Y. H. Zhou, Z. M. Ma, Y. J. Zhou +12

Web Agent BenchmarksLLM Agent EvaluationFinancial Fraud Detection
AskPDF
Sep 4, 2026

ReCAST: Restoration-aware Cascaded Stage-wise Training for Obfuscated SMS Risk Classification

Fraudulent messages sent via Short Message Service (SMS) are increasingly obfuscated to evade cost-conscious classifiers in production systems. In Chinese SMS, attackers can exploit a wide range of carefully crafted obfuscation strategies to hide risk-bearing phrases while preserving human readability, making direct classification brittle under real-world latency and throughput constraints. We propose ReCAST, a Restoration-aware Cascaded Stage-wise Training framework for robust obfuscated Chinese SMS classification. ReCAST distills a large teacher model's de-obfuscation ability into a smaller deployable student model by supervising obfuscated span detection, obfuscation type prediction, and text restoration, and then uses the restoration-aware student for downstream risk classification. Experiments on an internally constructed real-world Chinese SMS benchmark show that ReCAST substantially improves classification performance over directly trained baselines under obfuscation. The results suggest that restoration-aware distillation offers a practical path toward robust SMS risk classification with smaller deployable models under production-oriented constraints.

Jieyun Huang, Yi Shen, Kaikai Zhao +7

Unicom Data Intelligence, China Unicom · China Unicom Data Science & Artificial Intelligence Research Institute, China Unicom

Text ClassificationAdversarial RobustnessKnowledge Distillation
AskPDF