Citation

Raihan, Md. Johir and Labib, Mainul Islam and Jim, Abdullah Al Jaid and Tiang, Jun Jiat and Biswas, Uzzal and Nahid, Abdullah-Al (2024) Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People. Sensors, 24 (16). p. 5351. ISSN 1424-8220

Text Bengali-Sign_ A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People.pdf - Published Version Restricted to Repository staff only Download (16MB)

Abstract

Sign language is undoubtedly a common way of communication among deaf and nonverbal people. But it is not common among hearing people to use sign language to express feelings or share information in everyday life. Therefore, a significant communication gap exists between deaf and hearing individuals, despite both groups experiencing similar emotions and sentiments. In this paper, we developed a convolutional neural network–squeeze excitation network to predict the sign language signs and developed a smartphone application to provide access to the ML model to use it. The SE block provides attention to the channel of the image, thus improving the performance of the model. On the other hand, the smartphone application brings the ML model close to people so that everyone can benefit from it. In addition, we used the Shapley additive explanation to interpret the black box nature of the ML model and understand the models working from within. Using our ML model, we achieved an accuracy of 99.86% on the KU-BdSL dataset. The SHAP analysis shows that the model primarily relies on hand-related visual cues to predict sign language signs, aligning with human communication patterns.

Item Type:	Article
Uncontrolled Keywords:	Convolutional neural network
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Divisions:	Faculty of Engineering (FOE)
Depositing User:	Ms Nurul Iqtiani Ahmad
Date Deposited:	01 Oct 2024 00:37
Last Modified:	01 Oct 2024 00:37
URII:	http://shdl.mmu.edu.my/id/eprint/12987

Downloads

Edit (login required)