Robustness, Cost, and Attack-Surface Concentration in Phishing Detection

Julian Allagan; Mohamed Elbakary; Zohreh Safari; Weizheng Gao; Gabrielle Morgan; Essence Morgan; Vladimir Deriglazov

cs.LGMar 19, 2026

Robustness, Cost, and Attack-Surface Concentration in Phishing Detection

Authors: Julian Allagan, Mohamed Elbakary, Zohreh Safari, Weizheng Gao, Gabrielle Morgan, Essence Morgan, Vladimir Deriglazov

Organizations: Department of Mathematics, Computer Science, and Engineering Technology Elizabeth City State University, Elizabeth City, NC 27909, USA

Abstract

Phishing detectors built on engineered website features attain near-perfect accuracy under i.i.d.\ evaluation, yet deployment security depends on robustness to post-deployment feature manipulation. We study this gap through a cost-aware evasion framework that models discrete, monotone feature edits under explicit attacker budgets. Three diagnostics are introduced: minimal evasion cost (MEC), the evasion survival rate $S(B)$ , and the robustness concentration index (RCI). On the UCI Phishing Websites benchmark (11,055 instances, 30 ternary features), Logistic Regression, Random Forests, Gradient Boosted Trees, and XGBoost all achieve $\mathrm{AUC}\ge 0.979$ under static evaluation. Under budgeted sanitization-style evasion, robustness converges across architectures: the median MEC equals 2 with full features, and over 80% of successful minimal-cost evasions concentrate on three low-cost surface features. Feature restriction improves robustness only when it removes all dominant low-cost transitions. Under strict cost schedules, infrastructure-leaning feature sets exhibit 17-19% infeasible mass for ensemble models, while the median MEC among evadable instances remains unchanged. We formalize this convergence: if a positive fraction of correctly detected phishing instances admit evasion through a single feature transition of minimal cost $c_{\min}$ , no classifier can raise the corresponding MEC quantile above $c_{\min}$ without modifying the feature representation or cost model. Adversarial robustness in phishing detection is governed by feature economics rather than model complexity.

Figures & tables

Feature Group	Examples	$-1{\to}0$	$-1{\to}1$	$0{\to}1$	$-1{\to}0$	$-1{\to}1$	$0{\to}1$	Time Horizon
		Base Schedule			Strict Schedule
Surface	URL_of_Anchor, SFH,	1	2	1	1	2	1	hours to days
	Prefix_Suffix, SSLfinal_State
Semi-domain	Domain_Reg_Length,	3	6	3	3	6	3	days to weeks
	Google_Index
Infrastructure	web_traffic, DNSRecord,	4	8	4	4	$\infty$	$\infty$	weeks to months

Table 1: Feature manipulation cost schedules with operational time horizons. Costs are calibrated by the time-to-effect principle: 1 unit

\approx

changes feasible within a day; 4 units

\approx

multi-week external accumulation.

Panel of Figure 1 — Figure 1: Cost-aware adversarial robustness framework with MEC, survival curves, and attack-surface concentration.

Feature set	Schedule	$c_{\min}$	$\hat{\alpha}_{c_{\min}}$	Observed median MEC
Full	base	1	0.31	2
Full	strict	1	0.33	2
RA-8	base	1	0.28	2
RA-8	strict	1	0.15	2
VA-7b	base	1	0.62	1

Table 2: Empirical mass at the cost floor across feature configurations.

Model	Accuracy	AUC	Phishing TPR
Logistic Regression	0.927	0.979	0.900
Random Forest	0.950	0.993	0.913
GBDT	0.953	0.990	0.932
XGBoost	0.965	0.995	0.953

Table 3: Held-out classification performance (Full feature set, threshold

=0.5

	Median MEC		RCI 3
Model	Full $\mathcal{P}_{0}$	Intersection	Full $\mathcal{P}_{0}$	Intersection
Logistic Regression	2	2	0.96	0.96
Random Forest	2	2	0.85	0.84
GBDT	2	2	0.89	0.89
XGBoost	2	2	0.82	0.82

Table 4: Robustness metrics on full

\mathcal{P}_{0}

versus the 300-instance cross-model intersection (Full/base).

Features	Sched.	Model	Acc	FRI	MEC	[Q1,Q3]	RCI 3	FT1	NoEv
Full	base	Logit	.927	.076	2	[1,2]	.961	.850	0%
Full	base	RF	.950	.092	2	[2,3]	.843	.580	0%
Full	base	GBDT	.953	.076	2	[2,2]	.892	.370	0%
Full	base	XGB	.965	.092	2	[2,3]	.815	.440	0%
Full	strict	Logit	.927	.077	2	[1,2]	.975	.847	0%
Full	strict	RF	.950	.093	2	[2,3]	.843	.540	0%

Table 5: Robustness across feature sets and schedules. NoEvasion reports infeasible mass within

B_{\max}=18

Model	Median MEC [95% CI]	FRI [95% CI]	RCI 3 [95% CI]
Logistic Regression	2 [2, 2]	0.076 [0.068, 0.084]	0.96 [0.94, 0.97]
Random Forest	2 [2, 2]	0.092 [0.082, 0.101]	0.84 [0.80, 0.87]
GBDT	2 [2, 2]	0.076 [0.069, 0.083]	0.89 [0.86, 0.92]
XGBoost	2 [2, 3]	0.092 [0.083, 0.101]	0.82 [0.78, 0.85]

Table 6: 95% bootstrap confidence intervals (200 resamples, Full/base, 300-instance intersection).

Panel of Figure 2 — Figure 2: Evasion survival curves. RA-8/strict exhibits a persistent plateau corresponding to instances whose dominant low-cost path is blocked. Shaded bands (omitted for clarity) are narrow: 95% bootstrap intervals for $S(2)$ span $\pm 0.04$ across configurations.

Panel of Figure 3 — Figure 3: First-edit concentration by feature set and schedule. RA-8 exhibits near-total concentration on SSLfinal_State.

Panel of Figure 4 — Figure 4: RA-8/strict survival stratified by SSLfinal_State initial value. A persistent infeasible tail appears when the bottleneck feature is already at $+1$ .

Panel of Figure 5 — Figure 5: Accuracy versus median MEC. All architectures converge to the effective cost floor, consistent with Corollary 3.1 .

		$S(B{=}2)$				$S(B{=}4)$
Config	Model	Q50	Q100	Q500	Exact	Q50	Q100	Q500	Exact
Full/base	Logit	.28	.24	.23	.22	.05	.03	.02	.02
Full/base	GBDT	.30	.26	.23	.22	.08	.05	.03	.02
Full/base	XGB	.34	.30	.29	.28	.07	.05	.03	.03
RA-8/str	GBDT	.48	.44	.43	.42	.24	.21	.19	.19
RA-8/str	XGB	.50	.46	.44	.42	.26	.22	.20	.19

Table 7: Evasion survival

S(B)

under query-limited greedy search versus exact MEC.

$\lambda_{\mathrm{surf}}$	Median MEC	RCI 3	Top-3 features
1 (base)	2	0.89	URL_of_Anchor, SSLfinal_State, SFH
2	4	0.87	URL_of_Anchor, SSLfinal_State, SFH
3	6	0.84	URL_of_Anchor, SSLfinal_State, SFH
4	8	0.82	URL_of_Anchor, SSLfinal_State, SFH

Table 8: Median MEC and concentration under surface cost scaling (Full/base, GBDT).

Perturbation	Median MEC	RCI 3	Notes
$\lambda_{\mathrm{semi}}=0.5$	2	0.89	Surface paths remain cheapest
$\lambda_{\mathrm{semi}}=2$	2	0.89	Semi-domain rarely on optimal path
$\lambda_{\mathrm{infra}}=0.5$	2	0.88	Surface transitions dominate
$\lambda_{\mathrm{infra}}=2$	2	0.90	Infrastructure edits avoided
SSL $\to$ semi-domain (Full)	2	0.84	Alternative surface paths used
SSL $\to$ semi-domain (RA-8)	3	0.95	Bottleneck cost increases

Table 9: Extended cost sensitivity (Full/base, GBDT unless noted).

Phishing Detection

See all 16 papers

Adversarial Robustness of Phishing Email Detection: A Comparative Study of TF-IDF + Logistic Regression and Fine-Tuned DistilBERT
Jul 20, 2026Tanveer Ahmed, Seyedali PourmoafilAdversarial Robustness Cybersecurity
School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield, Herts AL10 9AB, United Kingdom
Phishing the Phishers with SpecularNet: Hierarchical Graph Autoencoding for Reference-Free Web Phishing Detection
Mar 2, 2026Tailai Song, Pedro Casas, Michela MeoPhishing Detection Graph Neural Networks
Politecnico di Torino Turin, Italy · AIT Austrian Institute of Technology Vienna, Austria
Feature-Context Consistency: Unsupervised Adversarial Detection with Drift Stability on Attributed Graphs
Oct 12, 2026Firas Bayram, Maxime CordyGraph-Based Anomaly Detection Anomaly Detection
SnT, University of Luxembourg Luxembourg

Jul 20, 2026J/K move · Enter open · S save

Adversarial Robustness of Phishing Email Detection: A Comparative Study of TF-IDF + Logistic Regression and Fine-Tuned DistilBERT

Tanveer Ahmed, Seyedali Pourmoafil

School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield, Herts AL10 9AB, United Kingdom

Adversarial Robustness Cybersecurity Phishing Detection Adversarial Text Attacks

Open paper AskPDF arXiv

Phishing emails remain one of the most persistent cybersecurity threats, and machine-learning classifiers are widely used to detect them. Most reported detection accuracies, however, are measured on clean, in-distribution test data rather than on emails deliberately altered to evade detection. This paper reports a controlled, pairwise comparison of two phishing-detection approaches a TF-IDF + Logistic Regression baseline and a fine-tuned DistilBERT transformer trained on a unified corpus of 82,255 emails drawn from six public datasets and evaluated under three conditions: normal in-distribution, synthetic phishing, and adversarial phishing. Both models exceeded 98% accuracy on clean data yet degraded sharply under adversarial testing: TF-IDF + LR fell to 64.00% (a 34.59-percentage-point drop) and DistilBERT fell to 63.64% (a 35.40-percentage-point drop) a gap of only 0.36 percentage points, equivalent to a single email in the 275-sample adversarial test set. LIME, SHAP, and attention-rollout analysis indicate the two models relied on different evidence yet showed similar vulnerability. Pairwise error analysis shows the models agreed on 54.9% of adversarial samples but each made a similar number of exclusive errors (24 and 25 respectively), indicating partly complementary rather than identical failure modes. The results show that clean-data accuracy does not predict adversarial robustness, and that adversarial testing should be a standard part of phishing-detection evaluation.

Abstract

Phishing Detection

Explore similar work

Adversarial Robustness of Phishing Email Detection: A Comparative Study of TF-IDF + Logistic Regression and Fine-Tuned DistilBERT

Phishing the Phishers with SpecularNet: Hierarchical Graph Autoencoding for Reference-Free Web Phishing Detection

Feature-Context Consistency: Unsupervised Adversarial Detection with Drift Stability on Attributed Graphs