AI-Based Deaf Companion System for Bridging Communication Between Deaf and Hearing Communities

• 1Anna University Chennai, Assistant Professor, Loyola Institute of Technology and Science, Nagercoil, Tamilnadu, IN

Deaf individuals face significant difficulties in communicating with others in society, as only a small number of them possess knowledge of and utilize sign language for communication. In general, deaf individuals use sign language or text to interact or communicate with others. While these methods are effective within the deaf community, they face significant limitations when trying to communicate with the hearing community. The main contribution of this project is to develop and build the deaf companion System (DCS) to enable two-way communication between non-deaf and normal people in Indian Sign Language using Temporal Convolutional Network (TCN). The proposed system has three modules; the sign recognition module (SRM) that recognizes the signs of a deaf individual which were integrated into the sign translation with Multilingual Interpreter System, the speech recognition using Hidden Markov Model and synthesis module (SRSM) that processes the speech of a non-deaf individual and converts it to text, and an Avatar module (AM) to generate and perform the corresponding sign of the non-deaf speech.