An Optimised Greedy-Weighted Ensemble Framework for Financial Loan Default Prediction
Authors: Ezekiel Nii Noye Nortey, Jones Asante-Koranteng, Marcellin Atemkeng, Theophilus Ansah-Narh, David Mensah, Rebecca Davis, Ravenhill Adjetey Laryea
Organizations: Department of Statistics and Actuarial Science, University of Ghana, P. O. Box LG 115, Legon, Accra, Ghana. · Department of Statistics, Tali Graduate school, Dominion University College, P. O. Box LG 80, Legon, Accra,Ghana. · Department of Mathematics, Rhodes University, P.O. Box 94, Makhanda, 6140, Grahamstown, South Africa. · National Institute for Theoretical and Computational Sciences (NITheCS), Stellenbosch 7600, South Africa. · Ghana Space Science and Technology Institute, Ghana Atomic Energy Commission, P. O. Box LG 80, Legon, Accra, Ghana. · Department of Mathematics and Actuarial Science, Pentecost University, P.O. Box KN 1739, Kaneshie, Accra, Ghana. · Department of Economics and Actuarial Science, University of Professional Studies, P. O. Box LG 149, Legon, Accra, Ghana.
Accurate prediction of loan defaults is a central challenge in credit risk management, particularly in modern financial datasets characterised by nonlinear relationships, class imbalance, and evolving borrower behaviour. Traditional statistical models and static ensemble methods often struggle to maintain reliable performance under such conditions. This study proposes an Optimised Greedy-Weighted Ensemble framework for loan default prediction that dynamically allocates model weights based on empirical predictive performance. The framework integrates multiple machine learning classifiers, with their hyperparameters first optimised using Particle Swarm Optimisation. Model predictions are then combined via a regularised greedy weighting mechanism. At the same time, a neural-network-based meta-learner is employed within stacked-ensemble to capture higher-order relationships among model outputs. Experiments conducted on the Lending Club dataset demonstrate that the proposed framework improves predictive performance compared with individual classifiers. The BlendNet ensemble achieved the strongest results with an AUC of 0.80, a macro-average F1-score of 0.73, and a default recall of 0.81. Calibration analysis further shows that tree-based ensembles such as Extra Trees and Gradient Boosting provide the most reliable probability estimates, while the stacked ensemble offers superior ranking capability. Feature analysis using Recursive Feature Elimination identifies revolving utilisation, annual income, and debt-to-income ratio as the most influential predictors of loan default. These findings demonstrate that performance-driven ensemble weighting can improve both predictive accuracy and interpretability in credit risk modelling. The proposed framework provides a scalable data-driven approach to support institutional credit assessment, risk monitoring, and financial decision-making.
Figures & tables
Paradigm
Core Principle
Strengths
Key Limitations for Credit Risk Prediction
Bagging
Independent models trained on bootstrap samples and aggregated through voting or averaging
Reduces variance and improves model stability
Typically assigns equal or heuristic weights to base learners, limiting adaptability under concept drift and evolving borrower behaviour
Boosting
Sequential training where each learner focuses on correcting previous errors
Strong predictive accuracy and ability to capture nonlinear patterns
Sequential dependence may amplify noise or instability; primarily reweights observations rather than dynamically reallocating model importance
Stacking
Meta-learner combines predictions from heterogeneous base models
High predictive flexibility and ability to exploit model diversity
Model contributions are often implicit and opaque; limited mechanisms for adapting model influence under changing data regimes
Proposed GWE
Performance-driven greedy optimisation of ensemble weights
Adaptive model weighting, improved interpretability, and robustness to evolving data distributions
Designed to overcome static weighting, sequential dependence, and opacity limitations present in traditional ensembles
Table 1: Comparison of ensemble paradigms and their limitations in credit risk modelling
Table 2: Comparison of hyperparameter spaces and optimal values for different base models tuned using PSO
Figure 3.1 : Flowchart illustrating the BlendNet meta-learning process within the stacked ensemble framework.
Items
Description
0
loan_amnt
The specified loan amount requested by the borrower
1
term
The quantity of installments for the loan
2
int_rate
Loan’s interest rate
3
installment
The borrower’s monthly payment obligation
4
grade
Loan grade assigned by LC
5
sub_grade
Subgrade designation for LC-assigned loans
Table 3: Description of loan variables.
Figure 4.1 : Distribution of missing values across features with gaps in the Lending Club dataset.
count
mean
std
min
25%
50%
75%
max
loan_amnt
335867.00
14522.76
8386.60
1000.00
8000.00
12400.00
20000.00
40000.00
int_rate
335867.00
13.80
4.51
5.32
10.64
13.35
16.78
30.99
installment
335867.00
445.16
251.59
28.75
263.78
388.20
583.79
1533.81
annual_inc
335867.00
75817.04
61972.24
5000.00
47000.00
65000.00
90000.00
8706582.00
dti
335867.00
17.72
8.15
0.00
11.63
17.25
23.42
380.53
open_acc
335867.00
11.60
5.18
1.00
8.00
11.00
14.00
90.00
Table 4: Summary statistics for loan default related variables.
Figure 4.2 : Violin plots showing the distribution and variability of key numerical features in the Lending Club dataset. The plots reveal the central tendency, spread, and potential outliers within each feature, providing insights into the characteristics of the borrowers and loans.
Figure 4.3 : Heatmap of Pearson correlation coefficients among key numerical features in the Lending Club dataset, emphasising statistically significant relationships influencing loan default risk.
Figure 4.4 : Comparison of loan outcomes–fully paid or charged off-based on borrower-provided number of payments (Panel A), home ownership status (Panel B), loan verification status (Panel C), and loan request category (Panel D).
Figure 4.5 : The left panel shows the distribution of fully paid and charged-off loans across different employment lengths. The right panel displays the top 30 job titles that were afforded a loan, highlighting the variety of professions represented in the dataset.
Figure 5.1 : Outlier Detection and Correction using the BCP-HI Pipeline: (Left) The original signal (annual_inc) showing significant spikes indicative of potential outliers. (Middle) The original signal with detected outliers marked as red dots. (Right) The corrected signal after outlier detection and correction displays a smoother, more reliable trend for further analysis.
Figure 5.2 : Comparison of loan status plots: (a) illustrates the biased plot, while (b) showcases the unbiased plot, highlighting the impact of addressing data imbalance in depicting accurate loan status distributions.
X′=Φ−1(F(X)),
Algorithm 5 Minority Class Enhancement Algorithm Using Statistical Data Augmentation
D={(X1,Y1),(X2,Y2),…,(XN,YN)}
Algorithm 6 Recursive Feature Elimination with Model Classifier
Figure 5.3 : Feature importance analysis using RFE across four ensemble learning models–RF, LightGBM, XGBoost, and CatBoost. The plot highlights the most influential features for loan default prediction, showcasing the consistency and model-specific preferences for key predictors.
Figure 5.4 : Performance of classification models across different numbers of features using Recursive Feature Elimination with Cross-Validation.
Figure 6.1 : Performance comparison of base models and ensemble methods for predicting financial loan defaults: Confusion matrices highlighting classification accuracy across “Fully Paid” (Class 1 ) and “Charged Off” (Class 0 ) loans.
Figure 6.2 : Classification reports for base models and ensemble methods in loan default prediction. The figure compares precision, recall, and F1-scores for “Fully Paid” (Class 1) and “Charged Off” (Class 0) loans across all evaluated models.
Figure 6.3 : Bootstrapped ROC curves with 95% confidence intervals for all evaluated models. BlendNet achieves the highest AUC ( 0.80±0.10 ), indicating superior discriminatory capability compared with both traditional classifiers and alternative ensemble strategies.
Figure 6.4 : Calibration curves for all evaluated models. ExtraTrees and Gradient Boosting demonstrate the best calibration behaviour, while several models show deviations from the ideal probability calibration line.
Credit risk default prediction remains a cornerstone of risk management in the financial industry. The task involves estimating the likelihood that a borrower will fail to meet debt obligations, an objective critical for lending decisions, portfolio optimization, and regulatory compliance. Traditional machine learning models such as logistic regression and tree-based ensembles are widely adopted for their interpretability and strong empirical performance. However, modern credit datasets are high-dimensional, heterogeneous, and noisy, increasing overfitting risk in monolithic models and reducing robustness under distributional shift. We introduce STRIKE (Stacking via Targeted Representations of Isolated Knowledge Extractors), a feature-group-aware stacking framework for structured tabular credit risk data. Rather than training a single monolithic model on the complete dataset, STRIKE partitions the feature space into semantically coherent groups and trains independent learners within each group. This decomposition is motivated by an additive perspective on risk modeling, where distinct feature sources contribute complementary evidence that can be combined through a structured aggregation. The resulting group-specific predictions are integrated through a meta-learner that aggregates signals while maintaining robustness and modularity. We evaluate STRIKE on three real-world datasets spanning corporate bankruptcy and consumer lending scenarios. Across all settings, STRIKE consistently outperforms strong tree-based baselines and conventional stacking approaches in terms of AUC-ROC. Ablation studies confirm that performance gains stem from meaningful feature decomposition rather than increased model complexity. Our findings demonstrate that STRIKE is a stable, scalable, and interpretable framework for credit risk default prediction tasks.
Swattik Maiti, Ritik Pratap Singh, Fardina Fathmiul Alam
Science Academy, University of Maryland, College Park MD, USA · Department of Computer Science, University of Maryland, College Park MD, USA
Credit-default prediction is an important task in financial decision making. Traditional methods use fitted classifiers such as logistic regression and random forests on tabular features. Large language models (LLMs) have recently been applied to this task through prompting. In this work we study how a fitted classifier and an LLM can be combined for credit-default prediction. We distinguish telling the LLM to imitate a classifier from using the classifier to build the prompt. We hypothesize that a fitted classifier can supply the ranking ability that an LLM prompt lacks. We experiment on the Default of Credit Card Clients dataset, and report recall, F1, and the area under the ROC and precision-recall curves, with bootstrap confidence intervals. We observe that a few-shot LLM has the highest recall (0.47) and F1 (0.50) of any single model but ranks worse than a random forest (AUC-ROC 0.72 against 0.79). Instructing the LLM to imitate a classifier gives no significant change. Pruning the prompt to the classifier's eight most important features raises recall by 0.071 and F1 by 0.032. Adding the classifier's predicted probability to the prompt raises the LLM's AUC-ROC from 0.72 to 0.78, matching the random forest, while keeping 0.118 higher recall than it. The reverse composition, and the use of several classifiers, do not help. We thus recommend a simple classifier-guided prompt for LLM-based credit prediction.
Rishi Datta, Lavanya Prahallad
Amador Valley High School · Pleasanton, California, USA · Research Spark Hub Inc. +1
Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters, in tension with supervisory expectations that adverse decisions be explainable. A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale. We build such a system end to end and test whether the second half of the promise holds. The predictive component is a multi-scale stacking ensemble fusing four differently regularised gradient-boosting learners with a residual network through a neural meta-learner trained on out-of-fold predictions. On a public 32,581-application credit dataset it reaches test ROC-AUC 0.9539 (95% CI [0.9462, 0.9616]) and PR-AUC 0.9137, beating the best single model by Delta-AUC = 0.0143 (p = 0.016 under a conservative independence assumption). Our central finding is asymmetric. The ranking gain is real but operationally small: at the F1-optimal threshold the ensemble avoids only six additional missed defaults out of 1,422 against a tuned random forest, cutting cost-weighted loss by under 2%. The narrative layer fails in a way prompt engineering alone does not fix. In an audited case the model named three factors as risk-increasing that the supplied attributions scored as risk-reducing, omitted the dominant driver, and introduced a feature never given to it. We trace this to properties we measure rather than assume: SHAP and LIME agree on which features matter (overlap@10 = 0.80) but not on their order (tau = 0.43, p = 0.18), and the attribution sign for the model's most sensitive input is near a coin flip across applicants (modal-sign share 0.53). Calibration (ECS = 0.117) and perturbation stability (DPD = 0.078) both fall short of our own thresholds. Constrained prompting is necessary but not sufficient: grounding must be verified after generation, not assumed.
Gregorius Reynaldi Pratama, Kuo-Kun Tseng
School of Computer Science and Technology Harbin Institute of Technology, Shenzhen Shenzhen, China