Speech To Sign Language Translator For Hearing Impaired

An application which takes in live speech or audio recording as input, converts it into text and displays the relevant Indian Sign Language images or GIFs.

• Front-end using EasyGui.
• Speech as input through microphone using PyAudio.
• Speech recognition using Google Speech API and Sphix(for offline use).
• Text Preprocessing using NLP.
• Dictionary based Machine Translation.

To run the application.

1. Open the Downloads folder and then open the terminal.
2. From the terminal, run the main1 and main2 python file using the command python main.py.
3. The application interface appears on the screen.
4. Hit the record button to start taking speech as input.
5. Any speech recorded is then processed and respective outputs are shown accordingly.
6. To exit the application using speech, say goodbye.

Sign language is a visual language that is used by deaf people as their mother tongue. Unlike acoustically conveyed sound patterns, sign language uses body language and manual communication to fluidly convey the thoughts of a person. Due to considerable time required in learning the Sign Language, it becomes difficult to communicate with these specially abled people, and thus creates a communication gap.

##ProblemStatement Dumb people use hand signs to communicate, hence normal people face problem in recognizing their language by signs made. Hence there is a need of the systems which recognizes the different signs and conveys the information to the normal people.

Objective

This Audio to Sign Language converter aims at :

• Providing information access and services to deaf people in Indian sign language.
• Developing a scalable project which can be extended to capture whole vocabulary of ISL through manual and non-manual signs

It can be developed as a desktop or mobile application to enable specially abled people to communicate easily and effectively with others

Sign language is a visual language that is used by deaf people as their mother tongue. Unlike acoustically conveyed sound patterns, sign language uses body language and manual communication to fluidly convey the thoughts of a person. Due to considerable time required in learning the Sign Language, people find it difficult to communicate with these specially abled people, creating a communication gap. Thus, we propose an application which takes in live speech or audio recording as input, converts it into text and displays the relevant Indian Sign Language images or GIFs.

Algorithm

Audio to Sign Language Translator

1. Start
2. Getting the Speech
   1. Listen for 1 second and calibrate the energy threshold for ambient noise levels.
   2. Listen the Speech using Microphone. Now the energy threshold is already set to a good value, and we can reliably catch speech right away.
3. Recognise the Speech.
4. Convert Speech to Text.
   1. Make the Text to lowercase for further manipulation.
5. Detected Text
   1. If “goodbye” then exit. 2.Else if Detected Text in predefined Dictionary Words. Display respective GIFs of the Phrase.
   2. Else Count the Letters of the Word/Phrase.
      1. Display the Visual of the phrase with some delay of Actions.
   3. Continue all the steps from Step 3, and continue till the Speech Ends.
6. If Error in Step 2, That is if no Speech Detected then display error message “Could not listen”.

Due to considerable time required in learning the Sign Language, people find it difficult to communicate with these specially abled people, creating a communication gap. Thus the Audio to Sign Language converter is important and significant because it helps in providing information access and services to deaf people in Indian sign language and develops a scalable project which can be extended to capture whole vocabulary of ISL through manual and non-manual signs. It also can be developed as a desktop or mobile application to enable specially abled people to communicate easily and effectively with others.

##Result The amount of time it took the translator to convert speech to sign language using the command line has been recorded. Depending on the length of the sentence we speak, the speech recognizer takes time. Converting the text into a sentence takes only a short amount of time during the text analysis.It takes 18.80 seconds to convert the speech “Hello,Good Evening” to sign language (from the start of audio recording to translation).

The project before enhancement and modification was cloned from Technoveters This project is now modified for better and enhanced speech recognition. Also added the program to work in offline mode.