Citation

Lim, Jia Min and Lim, Kian Ming and Lee, Chin Poo (2027) MP-DistillFormer: Multimodal prototype distillation with self-supervised transformer for few-shot fine-grained classification. Expert Systems with Applications, 332. p. 133659. ISSN 0957-4174

Text 1-s2.0-S0957417426025674-main.pdf - Published Version Restricted to Repository staff only Download (6MB)

Abstract

Few-shot fine-grained image classification remains challenging due to limited supervision and the need to distinguish subtle differences among visually similar categories. We propose MP-DistillFormer, a three-stage framework that enhances discrimination and generalization through multimodal prototype distillation and self-supervised refinement. In the first stage, a CNN-based teacher is trained with stochastic augmentation to capture diverse local features. In the second stage, knowledge is distilled into our proposed TeSMo-KAN student model. TeSMo-KAN unifies convolutional tokenization for spatial precision, lightweight local refinement for detail preservation, and nonlinear decision modeling within a Transformer backbone. To enrich semantic representation, TeSMo-KAN is guided by multimodal prototypes that fuse complementary visual and textual embeddings, enabling more discriminative learning under few-shot settings. Finally, a rotation-based self-supervised fine-tuning stage improves robustness under data-scarce conditions. Extensive experiments on three fine-grained benchmarks including CUB-200-2011, Stanford Dogs, and Stanford Cars demonstrate that MP-DistillFormer consistently outperforms state-of-the-art methods in both 1-shot and 5-shot scenarios. The source code is available at https://github.com/annym-ai00/MP-DistillFormer.

Item Type:	Article
Uncontrolled Keywords:	Few-shot fine-grained image classification, Multimodal prototype fusion, Self-supervised learning, Knowledge distillation, Few-shot learning
Subjects:	Q Science > Q Science (General)
Divisions:	Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	03 Sep 2026 03:26
Last Modified:	03 Sep 2026 03:26
URII:	http://shdl.mmu.edu.my/id/eprint/16597

Downloads

Edit (login required)