Citation

Liu, Wei Han and Lim, Kian Ming and Ong, Thian Song and Lee, Chin Poo (2024) NegCosIC: Negative Cosine Similarity-Invariance- Covariance Regularization for Few-Shot Learning. IEEE Access, 12. pp. 52867-52877. ISSN 2169-3536

Text 11.pdf - Published Version Restricted to Repository staff only Download (1MB)

Abstract

Few-shot learning continues to pose a challenge as it is inherently difficult for visual recognition models to generalize with limited labeled examples. When the training data is limited, the process of training and fine-tuning the model will be unstable and inefficient due to overfitting. In this paper, we introduce NegCosIC: Negative Cosine Similarity-Invariance-Covariance Regularization, a method that aims to improve the mean accuracy from the perspective of stabilizing the fine-tuning process and regularizing variance. NegCosIC incorporates a negative simple cosine similarity loss to stabilize the parameters of the feature extractor during fine-tuning. In addition, NegCosIC integrates invariance loss and covariance loss to regularize the embeddings in order to reduce overfitting. Experimental results demonstrate that NegCosIC is able to bring substantial improvements over the current state-of-the-art methods. An indepth worse case analysis is also conducted and shows that NegCosIC is able to outperform state-of-theart methods on worst case accuracy. The proposed NegCosIC achieved 2.15% and 2.13% higher accuracy on miniImageNet 1-shot and 5-shot tasks, 3.22% and 2.67% higher accuracy on CUB 1-shot and 5-shot tasks, and 2.13% and 7.74% higher accuracy on CIFAR-FS 1-shot and 5-shot tasks in terms of worst-case accuracies.

Item Type:	Article
Uncontrolled Keywords:	Few-shot learning, negative cosine similarity, invariance, covariance, regularization.
Subjects:	Q Science > QC Physics > QC350-467 Optics. Light
Divisions:	Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Nurul Iqtiani Ahmad
Date Deposited:	02 May 2024 07:16
Last Modified:	02 May 2024 07:16
URII:	http://shdl.mmu.edu.my/id/eprint/12415

Downloads

Edit (login required)