Citation

Nisha, Salma Jahan and Salih, N. D. and Mohd-Isa, Wan Noorshahida (2025) Sentence-based Sign Language Recognition using CNN and NLP. In: 2025 Multimedia University Engineering Conference, MECON 2025, 21 July 2025 - 23 July 2025, Cyberjaya, Malaysia.

Text 473.pdf - Published Version Restricted to Repository staff only Download (909kB)

Abstract

This paper presents a comprehensive real-time sign language gesture recognition framework using a combination of Convolutional Neural Networks (CNNs) and Natural Language Processing (NLP). Although individual letter level sign recognition has seen considerable advancements, interpreting sign language sentences remains a challenging and underexplored area. Our proposed approach utilizes a CNN-based model to capture both spatial and temporal features from real-time inputs, transforming dynamic sequences of hand gestures into coherent short textual sentences using NLP. Leveraging key Python libraries such as Tensorflow, OpenCV, LLM and LabelImg, the system efficiently extracts important key point features to recognize gestures and convert them into sentences in sign language. Experimental evaluations demonstrate the potential of our framework as possible means to reduce communication barriers and expand the beneficial possibilities in technologies for individuals with hearing impairments.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Deep Learning, Artificial Intelligence, Gesture, Communication, Hybrid System, CNN, NLP
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines
Divisions:	Faculty of Creative Multimedia (FCM)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	19 Mar 2026 01:04
Last Modified:	19 Mar 2026 01:04
URII:	http://shdl.mmu.edu.my/id/eprint/15496

Downloads

Edit (login required)