Citation

Tan, Chun Keat and Lim, Kian Ming and Lee, Chin Poo and Chang, Roy Kwang Yang and Alqahtani, Ali (2023) SDViT: Stacking of Distilled Vision Transformers for Hand Gesture Recognition. Applied Sciences, 13 (22). p. 12204. ISSN 2076-3417

Text 57.pdf - Published Version Restricted to Repository staff only Download (3MB)

Abstract

Hand gesture recognition (HGR) is a rapidly evolving field with the potential to revolutionize human–computer interactions by enabling machines to interpret and understand human gestures for intuitive communication and control. However, HGR faces challenges such as the high similarity of hand gestures, real-time performance, and model generalization. To address these challenges, this paper proposes the stacking of distilled vision transformers, referred to as SDViT, for hand gesture recognition. An initially pretrained vision transformer (ViT) featuring a self-attention mechanism is introduced to effectively capture intricate connections among image patches, thereby enhancing its capability to handle the challenge of high similarity between hand gestures. Subsequently, knowledge distillation is proposed to compress the ViT model and improve model generalization. Multiple distilled ViTs are then stacked to achieve higher predictive performance and reduce overfitting. The proposed SDViT model achieves a promising performance on three benchmark datasets for hand gesture recognition: the American Sign Language (ASL) dataset, the ASL with digits dataset, and the National University of Singapore (NUS) hand gesture dataset. The accuracies achieved on these datasets are 100.00%, 99.60%, and 100.00%, respectively.

Item Type:	Article
Uncontrolled Keywords:	Sign language
Subjects:	H Social Sciences > HV Social pathology. Social and public welfare. Criminology > HV697-4959 Protection, assistance and relief > HV697-3024 Special classes > HV1551-3024 People with disabilities Including blind, deaf, people with physical and mental disabilities
Divisions:	Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Nurul Iqtiani Ahmad
Date Deposited:	04 Jan 2024 01:46
Last Modified:	04 Jan 2024 01:46
URII:	http://shdl.mmu.edu.my/id/eprint/12022

Downloads

Edit (login required)