Citation

Ahamed, Tanvir and Kakon, Shawon Chakrabarty and Farid, Fahmid Al and Uddin, Jia and Abdul Karim, Hezerul (2026) An integrated evaluation protocol for adversarial robustness, generalization, and explanation stability in URL-based phishing detection. Frontiers in Computer Science, 8. ISSN 2624-9898

Text An integrated evaluation protocol for adversarial robustness, generalization, and explanation stability in URL-based phishing detection.pdf - Published Version Restricted to Repository staff only Download (1MB)

Abstract

Introduction: The reliability of phishing Uniform Resource Locator (URL) detectors under adversarial URL rewriting, domain shift, and explanation instability remains insufficiently understood. This study proposes an integrated robustness evaluation protocol for URL-based phishing detection, which integrates structured adversarial perturbation, unseen attack-family generalization, compositional attack effects, explanation stability, and external vulnerability transfer. Methods: The protocol tests four representative model families: Logistic Regression, XGBoost, CharCNN, and BERT-base, using 235,370 validated URLs from PHIUSIIL, consisting of 100,520 phishing and 134,850 benign URLs, along with 49,121 PhishTank-validated phishing URLs for external validation. Results: All models performed well on the clean test sets, ranging from 0.9962 to 0.9984, but their robustness decreased substantially under realistic URL mutations. Subdomain injection degraded the strong performance of Logistic Regression, XGBoost, and CharCNN to around 0.432, indicating collapse to the phishing-prevalence floor. BERT was highly susceptible to homoglyph, padding, and path-based perturbations. Leave-one-family-out evaluation also showed poor transfer to unseen subdomain attacks for both Logistic Regression and XGBoost, with Robustness Degradation Index values of 0.535 and 0.565, respectively. Explanation stability also suffered, with SHAP top-K Jaccard similarity dropping to 0.526-0.535 under subdomain perturbation. Discussion: These results provide a solid benchmark for evaluating robustnessaware phishing URL detection for achieving deployable reliability under realistic adversarial and non-IID settings.

Item Type:	Article
Uncontrolled Keywords:	Phishing URL detection, structural perturbations
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Divisions:	Faculty of Artificial Intelligence & Engineering (FAIE)
Depositing User:	Ms Rosnani Abd Wahab
Date Deposited:	01 Oct 2026 04:32
Last Modified:	01 Oct 2026 04:32
URII:	http://shdl.mmu.edu.my/id/eprint/16774

Downloads

Edit (login required)