Citation

Utomo, Wargijono and Suhanda, Yogasetya and Ar-Rasyid, Harun and Dharmalau, Andy (2025) Indonesian Language Sign Detection using Mediapipe with Long Short-Term Memory (LSTM) Algorithm. Journal of Informatics and Web Engineering, 4 (3). pp. 245-258. ISSN 2821-370X

Text 1541-Article Text-21523-1-10-20250914.pdf - Published Version Restricted to Repository staff only Download (1MB)

Abstract

People with disabilities mostly communicate using sign language, but the public still has little understanding of the Indonesian Sign Language System (ISLS). This causes obstacles in daily interactions. Advances in artificial intelligence technology, especially artificial neural networks, open opportunities in sign language recognition, but are still in the development stage. This study aims to build a ISLS sign language recognition model using the LSTM approach and MediaPipe Hands. The method of collecting hand keypoint data, 25 sequences per gesture, and 36 alphabetic and numeric gestures. The dataset is divided into three categories, namely 80% training, 10% validation, and 10% testing. The model developed to handle sequential data from hand gestures using the LSTM architecture. The results of the study can be shown model accuracy of 97.1%, average macro precision of 97%, recall of 96.6%, and F1-score of 96.4% and weighted average precision of 97.4%, recall of 97.1%, and F1-score of 97%. The results show that the combination of LSTM and MediaPipe can detect ISLS gestures with high accuracy. This can be used as a potential solution in automatic sign language translation, so that this model can improve the inclusiveness of communication for people with disabilities. Further research can be developed using a more accurate hand recognition framework, as well as improving data pre-processing, and exploring deep learning (DL) methods such as SSD, YOLO, or Faster-RCNN. In addition, pose and facial recognition can be added to improve accuracy in gesture recognition more comprehensively.

Item Type:	Article
Uncontrolled Keywords:	Sign language, Indonesian sign language system
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Depositing User:	Nor Afiqah Mohd Adnan
Date Deposited:	11 Nov 2025 01:35
Last Modified:	11 Nov 2025 01:35
URII:	http://shdl.mmu.edu.my/id/eprint/14856

Downloads

Edit (login required)