October 15, 2014
Researchers at the Georgia Institute of Technology in Atlanta have developed Glassware that allows for real-time speech captioning on Google Glass for the deaf or hearing impaired. Coupled with the companion Captioning on Glass (CoG) app for Android devices (iOS coming soon), one can use the smartphone’s built-in microphone for accurate voice transcription which is then streamed to the Glass display.
The idea for the app came about when Professor Jim Foley discovered he was having trouble hearing and thought Glass may help. The Institute’s Textual Computer Group, lead by Foley’s colleague Professor Thad Starner, decided to use a smartphone app in concert with the developed Glassware since this divide-and-conquer approach has a number of advantages.
“Glass has its own microphone, but it’s designed for the wearer,” said Starner. The Glass microphone may also pick up too much background noise from the wearer’s surroundings. “The mobile phone puts a microphone directly next to the speaker’s mouth, reducing background noise and helping to eliminate errors.”
Starner goes on to say that physically holding the smartphone in the speaker’s hand helps the speaker concentrate more effectively when speaking full sentences. Transcribed speech is streamed in real-time to the Glass wearer.
In the event there are errors in speech recognition, the speaker can use the CoG app to see a list of alternative captioning options and choose the best transcription. There are also options through both the CoG app and Glass to enlarge or shrink the size of the captioned text displayed on Glass to make it easier for the wearer to read. Coming from a user’s perspective, Professor Foley clarifies:
“This system allows wearers like me to focus on the speaker’s lips and facial gestures. If hard-of-hearing people understand the speech, the conversation can continue immediately without waiting for the caption. However, if I miss a word, I can glance at the transcription, get the word or two I need and get back into the conversation.”
The team is working on another project, Translations for Glass, that provide the same functionality of the CoG app, but instead takes spoken word in one language and streams the translation in another language to the Glass display in real time. You can find more information about the CoG system and even provide feedback or suggestions by visiting the team’s website here.
Image Source: Georgia Tech Video