October 15, 2014

Researchers at the Georgia Institute of Technology in Atlanta have developed Glassware that allows for real-time speech captioning on Google Glass for the deaf or hearing impaired. Coupled with the companion Captioning on Glass (CoG) app for Android devices (iOS coming soon), one can use the smartphone’s built-in microphone for accurate voice transcription which is then streamed to the Glass display.

The idea for the app came about when Professor Jim Foley discovered he was having trouble hearing and thought Glass may help. The Institute’s Textual Computer Group, lead by Foley’s colleague Professor Thad Starner, decided to use a smartphone app in concert with the developed Glassware since this divide-and-conquer approach has a number of advantages.

“Glass has its own microphone, but it’s designed for the wearer,” said Starner. The Glass microphone may also pick up too much background noise from the wearer’s surroundings. “The mobile phone puts a microphone directly next to the speaker’s mouth, reducing background noise and helping to eliminate errors.”

Starner goes on to say that physically holding the smartphone in the speaker’s hand helps the speaker concentrate more effectively when speaking full sentences. Transcribed speech is streamed in real-time to the Glass wearer.

In the event there are errors in speech recognition, the speaker can use the CoG app to see a list of alternative captioning options and choose the best transcription. There are also options through both the CoG app and Glass to enlarge or shrink the size of the captioned text displayed on Glass to make it easier for the wearer to read. Coming from a user’s perspective, Professor Foley clarifies:

“This system allows wearers like me to focus on the speaker’s lips and facial gestures. If hard-of-hearing people understand the speech, the conversation can continue immediately without waiting for the caption. However, if I miss a word, I can glance at the transcription, get the word or two I need and get back into the conversation.”

The team is working on another project, Translations for Glass, that provide the same functionality of the CoG app, but instead takes spoken word in one language and streams the translation in another language to the Glass display in real time. You can find more information about the CoG system and even provide feedback or suggestions by visiting the team’s website here.

Source: cnet

Image Source: Georgia Tech Video

– See more at:


1 Comment

  1. Cyn says:

    There are apps out there already to help the hearing impaired or deaf understand on a one on one conversations with speech recognition using a tablet or phone.
    What needs to be developed using speech recognition is a device that allows the HOH or deaf to understand speech in at conference room meeting, classroom or other large group settings with people sitting at a distance. Sending the text to their tablet, phone screen or laptop via a central mic placed at the center of the group. An individually hand held mic is not new. Though nice for one on one it’s group settings that most HOH/ deaf want to participate in fully. Most people will not invest in Glass while also wearing a hearing aid as its too many devices resting on your head. If you need to read captions b/c you cannot hear you need a hearing aid. While hearing aids do not give you perfect hearing again they do help depending on the type, user and assistive listening technology.

Post a Comment