Wearable-tech glove translates sign language into speech

UCLA bioengineers have designed a glove-like device that can translate American Sign Language (ASL) into English speech in real time though a smartphone app.

“Our hope is that this opens up an easy way for people who use sign language to communicate directly with non-signers without needing someone else to translate for them,” says Jun Chen, an assistant professor of bioengineering at the UCLA Samueli School of Engineering and the principal investigator on the research. “In addition, we hope it can help more people learn sign language themselves.”

The system includes a pair of gloves with thin, stretchable sensors that run the length of each of the five fingers. These sensors, made from electrically conducting yarns, pick up hand motions and finger placements that stand for individual letters, numbers, words, and phrases.

© UCLA | https://www.ucla.edu

The device then turns the finger movements into electrical signals, which are sent to a small circuit board worn on the wrist. The board transmits these signals wirelessly to a smartphone that translates them into spoken words at the rate of about one word per second.

The researchers also added adhesive sensors to testers' faces – in between their eyebrows and on one side of their mouths – to capture facial expressions that are a part of ASL.

Previous wearable systems that offered translation from ASL were limited by bulky and heavy device designs or were uncomfortable to wear.

The device developed by the UCLA team is made from lightweight and inexpensive but long-lasting, stretchable polymers. The electronic sensors are also flexible and inexpensive.

In testing the device, researchers worked with four people who are deaf and use ASL. The wearers repeated each hand gesture 15 times, and a custom machine-learning algorithm turned these gestures into the letters, numbers, and words they represent. The system recognized 660 signs, including each letter of the alphabet and numbers 0 through 9.