Citation

Lee, Chin Poo and Tan, Shing Chiang and Lim, Kian Ming and Tan, Alan Wee Chiat (2019) Isolated sign language recognition using Convolutional Neural Network hand modelling and Hand Energy Image. Multimedia Tools and Applications. pp. 1-28. ISSN 1380-7501

Text 62.pdf - Published Version Restricted to Repository staff only Download (4MB)

Abstract

This paper presents an isolated sign language recognition system that comprises of two main phases: hand tracking and hand representation. In the hand tracking phase, an annotated hand dataset is used to extract the hand patches to pre-train Convolutional Neural Network (CNN) hand models. The hand tracking is performed by the particle filter that combines hand motion and CNN pre-trained hand models into a joint likelihood observation model. The predicted hand position corresponds to the location of the particle with the highest joint likelihood. Based on the predicted hand position, a square hand region centered around the predicted position is segmented and serves as the input to the hand representation phase. In the hand representation phase, a compact hand representation is computed by averaging the segmented hand regions. The obtained hand representation is referred to as “Hand Energy Image (HEI)”. Quantitative and qualitative analysis show that the proposed hand tracking method is able to predict the hand positions that are closer to the ground truth. Similarly, the proposed HEI hand representation outperforms other methods in the isolated sign language recognition.

Item Type:	Article
Uncontrolled Keywords:	Sign language recognition, Convolutional Neural Network, Hand Energy Image,Hand gesture recognition
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Divisions:	Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	07 Mar 2022 02:51
Last Modified:	07 Mar 2022 02:51
URII:	http://shdl.mmu.edu.my/id/eprint/9217

Downloads

Edit (login required)