SIGNUM
Real-time ASL-to-Speech translation software
Recognize. Identify. Respond.
American Sign Language (ASL) is the sixth most used language in the US. It presented an interesting opportunity for us to create an impressive but meaningful piece of software.
Signum is a near real-time American Sign Language (ASL) translation tool that uses computer vision to recognize and track a user's gestures and then uses a learned model to identify the ASL character most closely correlated to that gesture. For more information, check out our project website or look at our project poster.
Recognize
Scan environment and extract ASL hand symbols.
Input: Raw ASL hand symbol
Output: Filtered ASL symbol
Classify
Use neural network to classify the ASL symbol.
Input: Filtered ASL symbol
Output: Closest word/alphabet to ASL symbol
Respond
Use the classification output and convert it to speech.
Input: Closest word/alphabet to ASL symbol
Output: Spoken word/alphabet corresponding to the ASL symbol