Explainability of Complex AI Models with Correlation Impact Ratio

Poushali Sengupta; Rabindra Khadka; Sabita Maharjan; Frank Eliassen; Yan Zhang; Shashi Raj Pandey; Pedro G. Lind; Anis Yazidi

Quantity	Value / Computation
$n^{\prime}$	$5$
$\mathbf{f}_{i}$	$[1,2,2,3,4]$
$\mathbf{y}^{\prime}$	$[0.8,1.1,0.9,1.3,1.5]$
Means	$\hat{f}_{i}=2.4,\ \hat{y}^{\prime}=1.12$
Mid-mean	$m_{i}=(\hat{f}_{i}+\hat{y}^{\prime})/2=1.76$
Numerator	$n^{\prime}\!\big[(\hat{f}_{i}-m_{i})^{2}+(\hat{y}^{\prime}-m_{i})^{2}\big]=4.096$

Appendix

TABLE S2 : Toy CIR example.

Fig. S1 : BlockCIR: correlated features are grouped by domain and summarized to

z_{B}

, which is compared with the model output

y^{\prime}

. Each

\mathrm{CIR}(z_{B},y^{\prime})

quantifies the group’s overall contribution, yielding interpretable, non-redundant attributions.

Fig. S2 : Multi-output ExCIR via CCA. Inputs and outputs are projected onto canonical directions

z=\mathbf{X}^{\prime}w^{\star}

and

s=\mathbf{Y}^{\prime}u^{\star}

, then

\mathrm{CIR}(z,s)

is computed on the canonical pair. For vector outputs, use

s=\mathbf{Y}^{\prime}u^{\star}

; for class-conditioned explanations, use

s=\mathbf{Y}^{\prime}w_{c}

for class

c

. Scores are invariant to well-conditioned linear reparameterizations of

\mathbf{Y}^{\prime}

.

Fig. S3 : Linear case: CCA and ExCIR produce nearly identical per-feature scores.

Fig. S4 : Nonlinear case: ExCIR elevates nonlinear drivers (

x_{0},x_{1},x_{2}

), whereas CCA suppresses them.

Fig. S5 : Nonlinear pattern captured by ExCIR but not CCA—the sinusoidal driver

x_{0}

.

Fig. S6 : Nonlinear pattern captured by ExCIR but not CCA—the quadratic driver

x_{1}

.

Fig. S7 : Nonlinear pattern captured by ExCIR but not CCA—the step-function driver

x_{2}

.

Fig. S8 : 3D feature-space for the sinusoid driver (

x_{0}

). ExCIR’s nonlinear map reveals a helical structure enabling

R^{2}=0.21

.

Fig. S9 : 3D feature-space for the quadratic driver (

x_{1}

). ExCIR straightens the parabola, enabling

R^{2}=0.65

.

Metric	Value
Spearman $\rho$ (CCA vs ExCIR)	$0.979$
$p$ -value	$3.09\times 10^{-8}$

Appendix

TABLE S3 : Linear regime ranking agreement.

Method	@3	@5	@8
CCA ( $\|corr\|$ )	$0.33$	$0.20$	$0.25$
ExCIR (feature-space)	$\mathbf{0.67}$	$\mathbf{0.60}$	$\mathbf{0.38}$

Appendix

TABLE S4 : Nonlinear regime: Precision@k for true nonlinear drivers.

Driver	CCA $\|r\|$	ExCIR $R^{2}$
$x_{0}$ (sinusoid)	$0.018$	$0.211$
$x_{1}$ (quadratic)	$0.041$	$0.647$
$x_{2}$ (step)	$0.189$	$0.038$

Appendix

TABLE S5 : Input-space CCA vs feature-space ExCIR for nonlinear drivers.

Fig. S10 : Methodology Schema We compute a bounded Correlation Impact Ratio (CIR) per feature to quantify co-movement with predictions. To keep explanations with less computational cost, we train a lightweight model on a distributionally similar subset and validate transfer so CIR agrees with the full model. The resulting ranking is used for top-

k

retraining, monitoring, and audits.

Fig. S11 : (a) shows how projection and embedding distances converge, indicating that the lightweight model’s output aligns as a rotated and translated version of the original model’s output, preserving accuracy, (b) compares kernel density estimates for output distributions in both models, showing nearly identical positions with minor alignment shifts indicated by differences in kurtosis, (c) compares the final output distributions of the lightweight and original models, demonstrating that the lightweight model mirrors the original’s output, maintaining similar accuracy.

Method	No. of Features	Accuracy (%)
SHAP-Ranked Features	6	56.2
ExCIR/ExCIR-LW Ranked Features	6	62.7
SHAP-Ranked Features	8	56.23
ExCIR/ExCIR-LW Ranked Features	8	65.1

Appendix

TABLE S7 : Comparison of predictive accuracy between SHAP and ExCIR/ExCIR-LW-ranked features.

Fig. S12 : Distribution of

L_{2}

distances between baseline and perturbed ExCIR profiles for CAU–EEG (

M=100

trials).

Fig. S13 : Top-

k

sufficiency. Test accuracy when keeping only the

k

highest-ranked features per method.

Fig. S14 : Top-

k

sufficiency. Test accuracy when keeping only the

k

highest-ranked features per method.

Fig. S15 : Necessity curves. Test accuracy after removing the top-

m

features.

Fig. S16 : Noise robustness. ExCIR agreement with its own baseline under small i.i.d. noise.

Fig. S17 : Correlation stress test. Method–to–method Spearman agreement as within–block correlation rises.

Fig. S18 : Experiment 5 (agreement–cost). Pareto scatter of wall–clock time vs. CIR rank correlation (full vs. lightweight). Marker size encodes top–8 overlap. The operating point at

f{=}0.20

attains perfect agreement at minimal cost.

Property	SHAP	LIME	ExCIR
Requires model gradients	✗	✗	✓
Requires perturbation/sampling	✓	✓	✗
Observation-only support	✗	✗	✓
Runs on edge devices	✗	✗	✓
Constant memory per feature	✗	✗	✓
Bounded attribution score	✗	✗	✓

Appendix

TABLE S8 : System Readiness Comparison: ExCIR vs. SHAP and LIME

Fig. S19 : Experiment 6 (runtime scaling). End–to–end time grows with the fraction of rows kept. At

f{=}0.20

we already match the full CIR ranking (see Fig. 30(a) ) with a much smaller cost.

Fig. S20 : Calibration and threshold stability. Left: reliability diagram. Right: accuracy vs. decision threshold.

Fig. S21 : Drift sensitivity. Change in ExCIR under simulated drift (positive bars indicate increased importance).

Fig. S22 : Block whitening. ExCIR before/after whitening the tire block.

Grouping	Top- $k$ overlap	$\Delta\tau$	Comment
Domain-informed (tire/control/powertrain)	1.00	0.00	reference
Correlation clustering (auto)	0.88	$-0.03$	similar heads
Mis-grouped control (swap tire $\leftrightarrow$ powertrain)	0.81	$-0.06$	head preserved

Appendix

TABLE S9 : BLOCKCIR grouping sensitivity (Vehicular val). We report group scores, Top-

k

overlap (vs. domain blocks), and

\Delta\tau

.

Fig. S23 : Uncertainty bands. Median ExCIR with 95% bootstrap CIs for top features.

Fig. S24 : Counterfactual sanity. Average predicted risk vs. single–feature nudges.

Fig. S25 : Summary figures for accuracy, runtime, and fidelity.

(A) Accuracy–cost
Method / Model	Kind	Time (s)	Acc	Drop	$\rho_{s}$	Top-10
GBM (baseline predictor)	fit		$0.701$	$0.000$
ExCIR–LW (20%)	explain	$0.005$	$0.701$	$0.000$	$0.94$	$0.98$
ExCIR–LW (30%)	explain	$0.007$	$0.701$	$0.000$	$0.95$	$1.00$
ExCIR–LW (50%)	explain	0.008	0.701	0.000	0.96	1.00
LIME + TinyGBM (20 $\times$ 2)	fit	$1.70$	$0.698$	$0.003$	$0.54$	$0.50$

Appendix

TABLE S11 : Head-to-head accuracy–cost and comparator budget sensitivity (Vehicular).

Method	Top-8 ranked features (high $\rightarrow$ low)
CIR (full & LW)	age $\rightarrow$ gfp_value $\rightarrow$ unlabeled $\rightarrow$ B_gev $\rightarrow$ C_gev $\rightarrow$ D_gev $\rightarrow$ A_gev $\rightarrow$ F_gev
SHAP	age $\rightarrow$ C_occurrences $\rightarrow$ D_occurrences $\rightarrow$ F_occurrences $\rightarrow$ A_occurrences $\rightarrow$ B_occurrences $\rightarrow$ F_gev $\rightarrow$ B_gev

Appendix

TABLE S12: Top-8 ranked features per method.CAU–EEGLeft→right = higher→lower importance. CIR(LW) aligns with CIR(full), preserving lightweight consistency.

Figure 49

Fig. S27 : Necessity and randomization sanity checks for ExCIR. ( a ) Feature–removal (“necessity”) curves show how test accuracy decreases as the top

m

ranked features are progressively removed and the model retrained. ExCIR exhibits the steepest accuracy drop, confirming that its highest-ranked features are the ones the model relies on most strongly. ( b ) Randomization sanity test evaluates rank stability under label shuffling and model re-initialization. As expected, correct recomputation on perturbed models drives rank correlation toward 0, restoring sanity; the earlier flat result was traced to reused baseline predictions (see text for details).

Fig. S28 : Noise robustness and correlated-block recovery in ExCIR. ( a ) Rank-stability analysis under additive feature noise shows that ExCIR maintains high Spearman correlation and nearly perfect Top-10 overlap with the noise-free baseline, demonstrating robustness to small perturbations at evaluation time. ( b ) Synthetic correlated-blocks experiment verifies that ExCIR correctly recovers the underlying block hierarchy (B1

>

B2

>

B3), highlighting its ability to identify dominant correlated groups and preserve meaningful ordering among them.

Fig. S29 : Agreement under growing within-group correlation: ExCIR vs SHAP/LIME (Spearman rank correlation).

Fig. S30 : Agreement–cost trade-off and runtime scaling in lightweight ExCIR. ( a ) Agreement–cost sweep showing Spearman rank correlation and Top-

k

overlap between ExCIR-LW and full ExCIR across varying lightweight fractions. Even at 20–30% of the validation rows, rank agreement exceeds 0.9 with minimal compute time. ( b ) Runtime scaling curve illustrates that execution time grows linearly with lightweight fraction

f

, confirming the sub-linear trade-off between fidelity and cost for the vehicular study.

Fig. S31 : Model calibration and threshold stability for ExCIR. ( a ) Calibration curve on the test split shows predicted probabilities closely following the diagonal, indicating well-calibrated model confidence. ( b ) Accuracy–vs–threshold plot demonstrates a broad, smooth optimum, ensuring that ExCIR explanations remain reliable across a range of decision thresholds.

Fig. S32 : Runtime scaling and drift sensitivity of ExCIR. ( a ) ExCIR response to a controlled distributional drift shows the most affected feature groups (e.g., tires, grade, and powertrain load) becoming more salient, confirming interpretability under data shifts. ( b , d ) ExCIR runtime scales linearly with both the number of features

d

and samples

n

, consistent with its single-pass closed-form computation. ( c , e ) In contrast, SHAP runtimes remain nearly flat in

n

but increase sharply with

d

, as shown for a fixed sampling budget of

\sim

800 point. Together these results demonstrate ExCIR’s efficient scaling with dataset size and its stability under moderate distributional drift.

Fig. S33 : Multi-class extension, whitening, and spurious correlation behavior in ExCIR. ( a ) Multi-class ExCIR shows class-wise CIR values aggregated across categories, where the same dominant features recur with modest variation, confirming the robustness of the multi-output formulation. ( b ) Whitening within correlated feature blocks enhances ExCIR separability, yielding cleaner within-group contrast and improved interpretability. ( c ) Spurious correlation test compares environments A and B: after residualization, ExCIR correctly demotes the spurious driver and restores expected directional behavior, highlighting reliability under confounding and distributional shifts.

Fig. S34 : Uncertainty quantification and counterfactual sanity checks for ExCIR. ( a ) Non-parametric bootstrap confidence intervals show narrow 95% bands for the leading features, indicating strong stability and low variance in ExCIR rankings across resamples. ( b ) Counterfactual sanity curves illustrate model responses under small, realistic perturbations: increasing speed markedly raises predicted risk, increased brake pressure has a mild effect, and higher tire pressure reduces risk—confirming that ExCIR’s global attributions align with domain intuition.

Fig. S35 : Multi-output and image-patch ExCIR evaluations (2×2 panel). ( a ) Directional sensitivity along canonical perturbations shows near-linear response for salient pixels. ( b ) Patch-level ExCIR maps reveal spatially coherent relevance across classes. ( c ) Calibration robustness under remixing confirms ranking stability across softmax perturbations. ( d ) CIR distribution placeholder for illustrating variation in joint influence across patches or outputs.

Fig. S36 : Top-10 Jaccard Overlap between class-wise ExCIR rankings on Digits. Diagonal dominance indicates intra-class consistency, while off-diagonal values reflect cross-class explanation divergence.

Fig. S37 : Top-10 Jaccard Overlap between class-wise ExCIR rankings on Digits. Diagonal dominance indicates intra-class consistency, while off-diagonal values reflect cross-class explanation divergence.

Fig. S38 : AOPC curves and class-level ExCIR heatmap. Panel (a) shows insertion/deletion behavior under ExCIR rankings; panel (b) visualizes global importance for the “dog” class.

Fig. S39 : Left: a validation image. Middle: the same global ExCIR map from Fig. 38(b) . Right: overlay. This overlay is illustrative: the map is global (average over many images), not an instance-specific saliency.

Condition	Sufficiency $\uparrow$	Top-8 overlap $\uparrow$
Random labels	0.10	0.13
Random features	0.12	0.15
Constant model	0.11	0.12
Trained (baseline)	0.71	1.00

Appendix

TABLE S18 : Negative controls (Vehicular, val).

Fig. S40 : Performance summary of ExCIR across eight evaluation dimensions (Q1–Q8). Each bar represents the normalized score (1–5 scale) for a specific evaluation criterion—fidelity, validity, sufficiency, robustness, group dynamics, efficiency, multi-output stability, and significance. All scores

\geq 4.7/5.0

indicate strong, stable performance across datasets and evaluation settings.

Aspect	Status quo (SOTA)	ExCIR (ours)
Computation	Sampling/perturbation-heavy; cost grows with $k$ (e.g., SHAP $\sim 2^{k}$ )	Closed-form, observation-only; one-time $\mathcal{O}(n^{3})$ then $\mathcal{O}(n)$ per feature; independent of $k$
Ranking, sufficiency	Local-slope emphasis; unclear/unstable global order	Performance-aligned ranking; higher top- $k$ sufficiency (compact subsets)
Deployment	Full-data-only pipelines; computationally costly explanations	similar lightweight environment keeps all features, preserves ranking/accuracy.
Calibration	Unbounded, hard to compare across runs	Bounded CIR $\in[0,1]$ with sensitivity link; comparable across datasets/models/time

Appendix

TABLE S21 : ExCIR novelties vs. SOTA in one view.

Feature	True Role	MI Rank	ExCIR Rank
$X_{1}$	Predictive	1	1
$X_{2}$	Redundant	2	3
$X_{3}$	Noise	3	2

Appendix

TABLE S26 : Feature-importance under redundancy (synthetic experiment).

Item	Setting
Data split	45,000 training / 5,000 validation / 10,000 official test
Architecture	CIFAR-adapted ResNet-18
Input stem	$3\times 3$ convolution, stride 1; no initial max-pooling
Normalization	CIFAR-10 channel means and standard deviations
Training augmentation	Random crop with padding 4; horizontal flip
Optimizer	SGD with Nesterov momentum

Appendix

TABLE S27 : Complete CIFAR-10 ResNet-18 training recipe.

Seed	Selected epoch	Validation accuracy (%)	Test accuracy (%)
7	194	95.36	95.13
17	186	95.16	95.02
27	189	94.94	95.48
Mean $\pm$ SD	–	$95.15\pm 0.21$	$95.21\pm 0.24$

Appendix

TABLE S28 : Clean CIFAR-10 predictive performance across independent seeds. The best epoch is selected using validation accuracy; test accuracy is then measured on the official held-out test partition.

$\sigma$	Test accuracy (%)	Spearman vs. clean	Top-8 Jaccard
0.00	95.13	1.00	1.00
0.01	95.01	1.00	1.00
0.03	94.67	1.00	1.00
0.05	93.97	1.00	1.00
0.10	89.85	1.00	1.00

Appendix

TABLE S29 : CIFAR-10 predictive accuracy and ExCIR ranking stability under evaluation-only Gaussian noise.

Fig. S41 : Frozen-model CIFAR-10 robustness. The left axis reports test accuracy; the right axis reports ExCIR rank stability relative to clean inputs. Gaussian noise is added in normalized input space only during evaluation. Predictive accuracy decreases with noise intensity, while Spearman correlation and Top-8 Jaccard similarity remain 1.00 over the tested range.

Fig. S42 : Fixed-seed qualitative CIFAR-10 examples. Five correctly classified truck images (left) and their projected class-conditioned ExCIR patch-score distributions (right) are shown together with predicted confidence. The examples were selected from correctly classified test instances before inspecting their maps.

Fig. S43 : Token-level ExCIR visualization for a correctly classified rec.sport.baseball test document. Bars show the ten largest local ExCIR scores. The most influential observed terms include team , Baltimore , fan , and Orioles .

Fig. S44 : Runtime comparison on CIFAR-10 with ResNet-18. ExCIR is the fastest method in the non-linear vision setting.

Aspect	Status quo (SOTA)	ExCIR (ours)
Computation	Shapley family: exact $\mathcal{O}(2^{k})$ (number of features $k$ ). KernelSHAP: $\mathcal{O}(m\,k)$ model calls ( $m$ perturbation samples). TMC-SHAP: $\mathcal{O}(m\,k)$ (Monte Carlo paths). TreeSHAP: $\mathcal{O}(TLk)$ (trees $T$ , max depth $L$ ). GradientSHAP/IG/DeepSHAP: $\mathcal{O}(m\,k)$ backprop passes. HSIC/MI estimators: typically $\mathcal{O}(n^{2}k)$ (pairwise kernels).	Closed-form, observation-only. One-time covariance: $\mathcal{O}(k^{3}{+}p^{3})$ ; streaming covariances $\mathcal{O}(nk)$ . Per-feature scoring (single pass): $\mathcal{O}(nk)$ , independent of perturbations or resampling.
Ranking, sufficiency	Local/perturbation-driven; global order unstable under correlation and noise.	Performance-aligned global ranking; higher top- $k$ sufficiency with compact subsets; correlation-aware.
Deployment	Perturbation-heavy pipelines; repeated model evaluations; full data required.	Lightweight-transfer; single-pass, low-memory; preserves ranking under subsampling (20-40% data).
Calibration	Unbounded scores; difficult cross-run comparison; sensitive to retraining.	Bounded CIR $\in[0,1]$ with MI-linked upper bound; comparable across datasets or models; stable under sampling & feature noise.

Per-feature ExCIR (validation)				Block (Group) ExCIR
Rank	Feature	Group	CIR	Group (rank)	GroupCIR
1	brake	Control	0.127	Control (1)	0.428
2	tire_rr	Tires	0.119	Environment (2)	0.226
3	rpm	Powertrain	0.119	Dynamics (3)	0.207
4	road_grade	Environment	0.118	Powertrain (4)	0.177
5	maf	Powertrain	0.118	Tires (5)	0.143

Metric	Value	Interpretation
Validation Accuracy	$97.6\%$	Base classifier performance
Test Accuracy	$96.1\%$	Generalization check
Kendall– $\tau$ (after remix)	$0.91$	Rank invariance under $Y^{\prime}M$
Top–8 overlap	$0.88$	Leader preservation
Top–10 overlap	$0.85$	Cross-class consistency
Relative Runtime	$1.08\times$	Over scalar ExCIR

Explainability of Complex AI Models with Correlation Impact Ratio

Abstract

Figures & tables

Appendix

Explore similar work

MCIR: A Feature Dependence-Aware Explainability Method with Reliability Guarantees

A Human-Centered Validation of the Explainability-Performance Coefficient

Stabilising Explainability Fragility in Cybersecurity AI: The Impact and Mitigation of Multicollinearity in Public Benchmark Datasets

Dataset	Model	Accuracy	Method	Time (s)
CIFAR-10	ResNet-18	$95.15\pm 0.21$ %	ExCIR	0.16
			BlockCIR	0.27
			MI	3.14
			SHAP	30.12
			LIME	1.34
20 Newsgroups	Logistic Reg.	89.3%	ExCIR	0.59