Citation

Lim, Kian Ming and Tan, Alan Wee Chiat and Tan, Shing Chiang (2016) A feature covariance matrix with serial particle filter for isolated sign language recognition. Expert Systems with Applications, 54. pp. 208-218. ISSN 0957-4174

Text 1-s2.0-S0957417416300112-main.pdf Restricted to Repository staff only Download (2MB)

Abstract

As is widely recognized, sign language recognition is a very challenging visual recognition problem. In this paper, we propose a feature covariance matrix based serial particle filter for isolated sign language recognition. At the preprocessing stage, the fusion of the median and mode filters is employed to extract the foreground and thereby enhances hand detection. We propose to serially track the hands of the signer, as opposed to tracking both hands at the same time, to reduce the misdirection of target objects. Subsequently, the region around the tracked hands is extracted to generate the feature covariance matrix as a compact representation of the tracked hand gesture, and thereby reduce the dimensionality of the features. In addition, the proposed feature covariance matrix is able to adapt to new signs due to its ability to integrate multiple correlated features in a natural way, without any retraining process. The experimental results show that the hand trajectories as obtained through the proposed serial hand tracking are closer to the ground truth. The sign gesture recognition based on the proposed methods yields a 87.33% recognition rate for the American Sign Language. The proposed hand tracking and feature extraction methodology is an important milestone in the development of expert systems designed for sign language recognition, such as automated sign language translation systems.

Item Type:	Article
Uncontrolled Keywords:	Sign language recognition, Feature covariance matrix, Serial particle filter
Subjects:	P Language and Literature > P Philology. Linguistics
Divisions:	Faculty of Information Science and Technology (FIST) Faculty of Engineering and Technology (FET)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	07 Feb 2018 13:00
Last Modified:	07 Feb 2018 13:00
URII:	http://shdl.mmu.edu.my/id/eprint/6672

Downloads

Edit (login required)