Calibrated Credit Intelligence: Shift-Robust and Fair Risk Scoring with Bayesian Uncertainty and Gradient Boosting

Srikumar Nayak

q-fin.RMMar 6, 2026

Calibrated Credit Intelligence: Shift-Robust and Fair Risk Scoring with Bayesian Uncertainty and Gradient Boosting

Authors: Srikumar Nayak

Organizations: Incedo Inc, USA

Abstract

Credit risk scoring must support high-stakes lending decisions where data distributions change over time, probability estimates must be reliable, and group-level fairness is required. While modern machine learning models improve default prediction accuracy, they often produce poorly calibrated scores under distribution shift and may create unfair outcomes when trained without explicit constraints. This paper proposes Calibrated Credit Intelligence (CCI), a deployment-oriented framework that combines (i) a Bayesian neural risk scorer to capture epistemic uncertainty and reduce overconfident errors, (ii) a fairnessconstrained gradient boosting model to control group disparities while preserving strong tabular performance, and (iii) a shiftaware fusion strategy followed by post-hoc probability calibration to stabilize decision thresholds in later time periods. We evaluate CCI on the Home Credit Credit Risk Model Stability benchmark using a time-consistent split to reflect real-world drift. Compared with strong baselines (LightGBM, XGBoost, CatBoost, TabNet, and a standalone Bayesian neural model), CCI achieves the best overall trade-off between discrimination, calibration, stability, and fairness. In particular, CCI reaches an AUC-ROC of 0.912 and an AUC-PR of 0.438, improves operational performance with Recall@1%FPR = 0.509, and reduces calibration error (Brier score 0.087, ECE 0.015). Under temporal shift, CCI shows a smaller AUC-PR drop from early to late periods (0.017), and it lowers group disparities (demographic parity gap 0.046, equal opportunity gap 0.037) compared to unconstrained boosting. These results indicate that CCI produces risk scores that are accurate, reliable, and more equitable under realistic deployment conditions.

Figures & tables

Panel of Figure 1 — Fig. 1: CCI pipeline: preprocessing feeds a Bayesian scorer and a fairness-constrained booster; their outputs are fused under shift checks and then calibrated for reliable risk probabilities.

Model	AUC-ROC $\uparrow$	AUC-PR $\uparrow$	Recall@1%FPR $\uparrow$	Brier $\downarrow$	ECE $\downarrow$
Logistic Regression [ 2 ]	$0.865\pm 0.003$	$0.332\pm 0.004$	$0.410\pm 0.005$	$0.102\pm 0.001$	$0.032\pm 0.002$
XGBoost [ 7 ]	$0.892\pm 0.002$	$0.401\pm 0.003$	$0.468\pm 0.004$	$0.095\pm 0.001$	$0.024\pm 0.001$
LightGBM [ 13 ]	$0.898\pm 0.002$	$0.413\pm 0.003$	$0.482\pm 0.003$	$0.092\pm 0.001$	$0.022\pm 0.001$
CatBoost [ 8 ]	$0.899\pm 0.002$	$0.418\pm 0.003$	$0.487\pm 0.003$	$0.091\pm 0.001$	$0.021\pm 0.001$
TabNet [ 14 ]	$0.890\pm 0.002$	$0.405\pm 0.003$	$0.471\pm 0.004$	$0.094\pm 0.001$	$0.023\pm 0.001$
BNN (uncertainty) [ 4 ]	$0.888\pm 0.002$	$0.398\pm 0.004$	$0.459\pm 0.004$	$0.093\pm 0.001$	$0.020\pm 0.001$

TABLE I: Main results on Home Credit (Model Stability). Higher is better for AUC-ROC/AUC-PR/Recall@1%FPR (

\uparrow

), and lower is better for Brier/ECE (

\downarrow

). Values are mean

\pm

std over repeated runs.

Model	Early AUC-PR $\uparrow$	Late AUC-PR $\uparrow$	Drop $\downarrow$
LightGBM [ 13 ]	0.426	0.392	0.034
Fair-GBDT [ 12 ]	0.422	0.392	0.030
CCI (Ours)	0.448	0.431	0.017

TABLE II: Stability under temporal shift (AUC-PR). Smaller drop from early to late indicates better robustness.

Model	$\boldsymbol{\Delta_{\mathrm{DP}}}$ $\downarrow$	$\boldsymbol{\Delta_{\mathrm{EO}}}$ $\downarrow$
LightGBM	$0.083\pm 0.004$	$0.066\pm 0.004$
BNN	$0.078\pm 0.004$	$0.063\pm 0.004$
Fair-GBDT	$0.052\pm 0.003$	$0.041\pm 0.003$
CCI (Ours)	$\mathbf{0.046\pm 0.003}$	$\mathbf{0.037\pm 0.003}$

TABLE III: Fairness metrics (lower is better). Values are mean

\pm

std across runs.

Fig. 2: AUC-PR comparison across baselines and CCI (values aligned with Table I ).

Fig. 3: Reliability diagram (calibration curve). CCI is closer to the diagonal, indicating better calibration (lower ECE/Brier).

Fig. 4: AUC-PR trend across evaluation weeks (distribution shift). CCI shows a smaller decline over time.

Fig. 5: Fairness–accuracy trade-off (Demographic parity gap vs. AUC-PR). CCI provides a strong balance between the two.

Explore similar work

Feb 2, 2024

A Distributionally Robust Optimisation Approach to Fair Credit Scoring

Credit scoring has been catalogued by the European Commission and the Executive Office of the US President as a high-risk classification task, in light of the potential harms of making loan approval decisions based on models that would be biased against certain groups. To address this concern, recent credit scoring research has considered a range of fairness-enhancing techniques put forward by the machine learning community to reduce bias and unfair treatment in classification systems. While the definition of fairness or the approach they follow to impose it may vary, most of these techniques, however, disregard the robustness of the results. This can create situations where unfair treatment is effectively corrected in the training set, but when producing out-of-distribution classifications, unfair treatment is incurred again. Instead, in this paper, we will investigate how to apply Distributionally Robust Optimisation (DRO) methods to credit scoring, thereby empirically evaluating how they perform in terms of fairness, ability to classify correctly, and the robustness of the solution against changes in the marginal proportions. In so doing, we find DRO methods to provide a substantial improvement in terms of fairness, with almost no loss in predictive performance. These results thus indicate that DRO can improve fairness in credit scoring, provided that further advances are made in efficiently implementing these systems. In addition, our analysis suggests that many of the commonly used fairness metrics are not ideally suited to the credit scoring setting, as they evaluate performance at a single classification threshold.

Pablo Casas, Huan Yu, Christophe Mues

Southampton Business School, University of Southampton, Southampton, SO16 7QF, United Kingdom

Distribution Shift Robustness Financial Services Algorithmic Fairness Distributionally Robust Optimization Group Fairness Credit Scoring

Ask PDF

Oct 8, 2026

When Do External Priors Help Low-Resource Structured Prediction? A Ranking-Level Alignment Study

Cold-start credit scoring (scarce labels, immature features, or minimal capacity) motivates injecting external priors (teacher models, LLMs, or domain experts) into training. We study Ranking Prior Alignment (RPA), a model-agnostic objective that aligns a model's ranking distribution to an external prior via a temperature-scaled KL loss with an annealed weight,

\mathcal{L} = \mathcal{L}_{\text{task}} + γ(t)\,\text{KL}(P_{\text{agent}} \| P_{\text{model}})

, and requires no external model at inference. Across tree-ensemble and multiple-instance-learning (MIL) instantiations on the public Amex dataset and an industrial merchant-credit dataset (1.5M+ merchants), we ask when such priors genuinely transfer information rather than merely act as a regularizer. Our central finding: the prior's quality

π

(its rank correlation with the label) governs transfer only when the alignment granularity matches the granularity at which

π

is measured. On cross-sample tree alignment (Amex), a high-

π

teacher yields

+0.028

AUC over the no-alignment baseline (paired bootstrap 95% CI

[0.022, 0.035]

) whereas zero-information priors (uniform, random) yield none, and

Δ

AUC tracks

π

(XGBoost Pearson

r = 0.985

); this replicates on a second public dataset (LendingClub). On within-bag MIL-attention alignment, the same high-

π

teacher transfers nothing: it is significantly worse than its own within-bag shuffle (net

-0.017

, 95% CI

< 0

, sign

p = 0.021

over 10 seeds), while zero-information priors give the only significant gains: the KL term collapses to pure entropy regularization. We distill these results into a two-factor

N \cdot \mathcal{C}

decision rule (gains concentrate at low sample size and capacity and vanish as either grows) and a scale-controlled paired protocol that separates information transfer from regularization.

Qiye Lu, Jiang Ji, Liang Zhang

Ant International China

Credit Scoring

Ask PDF

Sep 29, 2026

CredWise: A Controlled Agentic Decision-Intelligence Framework for Explainable and Auditable Credit-Risk Assessment

Credit-risk prediction is important in banking, but a prediction alone does not explain why an applicant is risky or how it should be combined with other evidence. This paper presents CredWise, a decision-support framework that integrates credit-risk prediction, probability calibration, explainable artificial intelligence, policy retrieval, SQL analytics, and controlled agent-based workflows. An XGBoost model is trained on Lending Club data (1,345,310 loans, 18 features) using a temporal split: 2007--2016 for training, 2017 for validation, and 2018 for testing. On the 2018 test set, the calibrated model achieved a ROC-AUC of 0.7109, PR-AUC of 0.2993, F1-score of 0.3714, and accuracy of 65.44%. Calibration reduced the Brier score from 0.2157 to 0.1273 and the expected calibration error from 0.2862 to 0.0585. SHAP explanations were temporally stable, with a Spearman correlation of 0.9959 between 2017 and 2018 feature rankings. On 28 labeled queries covering nine policy sections, FAISS achieved the best Hit@1 (0.929) and MRR (0.964), while all three retrieval methods reached Hit@5 = 1.0. Agent routing achieved 95.6% accuracy (43 of 45 cases), and the SQL benchmark scored 1.0 on exact-match, execution-success, and result-match across six cases. These results show that CredWise can combine predictions, explanations, policy evidence, and structured analytics in one controlled workflow. It is an academic research prototype, and final decisions remain with a human reviewer.

Aakash Kumar Tiwari

Department of Mathematics, Indian Institute of Technology Kharagpur, Kharagpur 721302, West Bengal, India

AI-Assisted Decision Making Credit Scoring

Ask PDF