Citation

Mahamud, Eram and Assaduzzaman, Md and Fahad, Nafiz and Liew, Tze Hui and Ohidujjaman, Ohidujjaman (2026) Dual-level knowledge distillation from vision transformer to Swin transformer for fine-grained brain tumor classification: A 44-class MRI benchmark. Journal of Pathology Informatics, 23. p. 100703. ISSN 21533539

Text Dual-level knowledge distillation from vision transformer to Swin transformer for fine-grained brain tumor classification_ A 44-class MRI benchmark.pdf - Published Version Restricted to Repository staff only Download (1MB)

Abstract

Accurate subtype classification of brain tumors from MRI requires simultaneous discrimination across histological category and imaging modality. We propose a dual-level knowledge distillation (DLKD) framework that transfers logitlevel soft targets (temperature T = 4.0, weight α = 0.6) and feature-level representations via a learnable projector from a frozen vision transformer-Base teacher (86.6 M parameters) to a compact Swin-Tiny student (28.3 M parameters), evaluated on a 44-class brain tumor MRI benchmark (4479 images) spanning 15 histological subtypes across three imaging modalities. Teacher and student are trained under an identical protocol matched augmentation, class-balancing (oversampling to 255 images/class), and optimization on an 80/10/10 stratified split. Averaged across five random seeds, the proposed student achieves 97.9% ± 0.2% test accuracy and 0.978 ± 0.003 macro F1, compared with 95.25% ± 0.3% for the teacher, 95.8% ± 0.3% for a no-distillation student baseline, 96.4% ± 0.2% for logit-only distillation, and 96.5%± 0.3% for feature-only distillation. Each distillation component individually improves over baseline (+0.6 pp. and +0.7 pp. respectively), and their combination yields the largest gain (+2.1 pp. over baseline; +2.65 pp. over the teacher), indicating the two objectives are complementary rather than redundant. Four widely used convolutional neural network baselines trained under the same protocol EfficientNet-B0 (94.4%), ResNet-50 (93.5%), DenseNet-121 (92.8%), and MobileNetV3-Large (91.6%) all trail the distilled student. We report four threats to validity: (1) patient identifiers are unavailable in the source data, so cross-modality leakage of the same patient's T1, T1C+, and T2 scans across splits cannot be excluded; (2) the Kaggle-sourced dataset may contain acquisition artifacts or shortcuts that inflate performance across all models; (3) the student's gains over the teacher and baseline, while consistent with typical knowledge-distillation results, still warrant external replication; (4) the smaller 10% test split leaves some classes with as few as 5–7 test images, making their per-class F1 unreliable. This work establishes a knowledge-distillation benchmark on the 44-class taxonomy and provides methodological infrastructure for future patient-stratified studies; the reported numbers should not be interpreted as clinical-grade performance until these threats are addressed.

Item Type:	Article
Uncontrolled Keywords:	Knowledge distillation, Brain tumor classification, Vision transformer
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Divisions:	Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Rosnani Abd Wahab
Date Deposited:	02 Oct 2026 03:11
Last Modified:	02 Oct 2026 03:11
URII:	http://shdl.mmu.edu.my/id/eprint/16828

Downloads

Edit (login required)