Amogh Bysani

What is the effectiveness of machine learning models, specifically Convolutional Neural Networks (CNNs) for image tensor analysis and Recurrent Neural Networks (RNNs) such as Long Short-Term Memory (LSTM) models in fostering effective communication between American Sign Language (ASL) and English through real-time sign language recognition and translation?

Started Apr. 12, 2023

Abstract or project description

This research project aims to develop a real-time sign language recognition system using deep learning models to foster effective communication between American Sign Language (ASL) and English. The system takes input as video feeds, capturing hand gestures converted into image tensors. The research investigates the effectiveness of Convolutional Neural Networks (CNNs) in analyzing image tensors to extract relevant spatial features of sign language gestures. Additionally, Recurrent Neural Networks (RNNs) like Long Short-Term Memory (LSTM) models are utilized to understand temporal dependencies in the sequence of gestures. The central research question explores the potential of machine learning models, namely CNNs and LSTMs, to bridge the communication gap between ASL and English. The system's real-time capabilities and accuracy in translating sign language gestures into text and voice output is assessed through extensive experimentation and dataset analysis. The ultimate goal is to develop accessible technology that enables seamless communication and inclusivity for individuals with hearing impairments, preferably as a plugin in video call tools such as Zoom or as a standalone app.