In TED Talk, Deaf engineer debuts AI model that transcribes sign language to text in seconds

On the TED stage in Atlanta, Georgia, Adam Munder speaks to Hasiba Haq through his AI American Sign Language translation software: Omnibridge

As a Deaf man, Adam Munder has long been advocating for communication rights in a world that chiefly caters to hearing people. 

The Intel software engineer and his wife — who is also Deaf — are often unable to use American Sign Language in daily interactions, instead defaulting to texting on a smartphone or passing a pen and paper back and forth with service workers, teachers, and lawyers. 

It can make simple tasks, like ordering coffee, more complicated than it should be. 

But there are life events that hold greater weight than a cup of coffee. 

Recently, Munder and his wife took their daughter in for a doctor’s appointment — and no interpreter was available. 

Adam Munger, a white man with brown hair, wears a light blue button down shirt. He has rectangular glasses and speaks through American Sign Language on stage at TED. Beside him is a translator in a black suit jacket.
Adam Munder speaks at TEDNext. October 22 - 24, 2024. Atlanta, GA. Photo: Erin Lubin / TED

To their surprise, their doctor said: “It’s alright, we’ll just have your daughter interpret for you!” 

In a TED Talk at TEDNext in Atlanta, Georgia, Munder took the audience back to this pivotal day at the doctor’s office. 

“Can you imagine going to a doctor’s appointment, and being told by the doctor that your 7-year-old is going to relay the information to you?” Munder asked the crowd in ASL, as an interpreter translated.

That day at the doctor’s office came at the heels of a thousand frustrating interactions and miscommunications — and Munder is not isolated in his experience.

“Where I live in Arizona, there are more than 1.1 million individuals with a hearing loss,” Munder said, “and only about 400 licensed interpreters.”

Adam Munger, a white man with brown hair, wears a light blue button down shirt. He has rectangular glasses and speaks through American Sign Language on stage at TED
Adam Munder speaks at TEDNext. October 22 - 24, 2024. Atlanta, GA. Photo: Gilberto Tadday / TED

In addition to being hard to find, interpreters are expensive. And texting and writing aren’t always practical options — they leave out the emotion, detail, and nuance of a spoken conversation. 

ASL is a rich, complex language with its own grammar and culture; a subtle change in speed, direction, facial expression, or gesture can completely change the meaning and tone of a sign. 

“Writing back and forth on paper and pen or using a smartphone to text is not equivalent to American Sign Language,” Munder emphasized. “The details and nuance that make us human are lost in both our personal and business conversations.”

His solution? An AI-powered platform called Omnibridge

“My team has established this bridge between the Deaf world and the hearing world, bringing these worlds together without forcing one to adapt to the other,” Munder said. 

Trained on thousands of signs, Omnibridge is engineered to transcribe spoken English and interpret sign language on screen in seconds. 

To test an alpha version of his invention, Munder welcomed TED associate Hasiba Haq on stage. 

Adam Munger, a white man with brown hair, wears a light blue button down shirt. He has rectangular glasses and speaks through American Sign Language on stage at TED. Across from him is a woman, Hasiba Haq
Adam Munder speaks at TEDNext. October 22 - 24, 2024. Atlanta, GA. Photo: Gilberto Tadday / TED

“I want to show you how this could have changed my interaction at the doctor appointment, had this been available,” Munder said. 

He went on to explain that the software would generate a bi-directional conversation, in which Munder’s signs would appear as blue text and spoken word would appear in gray. 

At first, there was a brief hiccup on the TED stage. Haq, who was standing in as the doctor’s office receptionist, spoke — but the screen remained blank. 

“I don’t believe this; this is the first time that AI has ever failed,” Munder joked, getting a big laugh from the crowd. “Thanks for your patience.”

After a quick reboot, they rolled with the punches and tried again.

Haq asked: “Hi, how’s it going?” 

Her words popped up in blue. 

Munder signed in reply: “I am good.” 

His response popped up in gray. 

On the TED stage in Atlanta, Georgia, Adam Munder speaks to Hasiba Haq through his AI American Sign Language translation software: Omnibridge. Behind them are screens displaying their real-time conversation
Adam Munder speaks at TEDNext. October 22 - 24, 2024. Atlanta, GA. Photo: Gilberto Tadday / TED

Back and forth, they recreated the scene from the doctor’s office. But this time Munder retained his autonomy, and no one suggested a 7-year-old should play interpreter. 

Munder’s TED debut and tech demonstration didn’t happen overnight — the engineer has been working on Omnibridge for over a decade. 

“It takes a lot to build something like this,” Munder told Good Good Good in an exclusive interview, communicating with our team in ASL. “It couldn't just be one or two people. It takes a large team, a lot of resources, millions and millions of dollars to work on a project like this.” 

After five years of pitching and research, Intel handpicked Munder’s team for a specialty training program. It was through that backing that Omnibridge began to truly take shape. 

But Munder’s journey started long before then — he has dreamed of technology like this since he was a child. 

Growing up, his grandparents were voracious storytellers with rich histories, and he was eager to hear their stories — but his mother wasn’t always available to bridge the gap between her son and her parents. 

Adam Munger, a white man with brown hair, wears a light blue button down shirt. He has rectangular glasses and speaks through American Sign Language on stage at TED. Beside him is a translator in a black suit jacket.
Adam Munder speaks at TEDNext. October 22 - 24, 2024. Atlanta, GA. Photo: Gilberto Tadday / TED

“I wanted to be able to talk to them, but I didn't always have that ability to do that,” Munder recalled. “So there's a lot of frustration growing up in not being able to express myself, not being able to be understood, not [being] able to understand what was going around me.” 

For Munder, Omnibridge is not just a gift to his younger self, but a future service to the Deaf community at large.

“Our dream is that the technology will be available to everyone, everywhere,” Munder said. “I feel like three to four years from now, we're going to have an app on a phone. Our team has already started working on a cloud-based product, and we're hoping that will be an easy switch from cloud to mobile to an app.” 

In order to achieve that dream — of transposing their technology to a smartphone — Munder and his team have to play a bit of a waiting game. Today, their platform necessitates building the technology on a PC, with an AI engine. 

“A lot of things don't have those AI PC types of chips,” Munder explained. “But as the technology evolves, we expect that smartphones will start to include AI engines. They'll start to include the capability in processing within smartphones. It will take time for the technology to catch up to it, and it probably won't need the power that we're requiring right now on a PC.” 

Adam Munger, a white man with brown hair, wears a light blue button down shirt. He has rectangular glasses and speaks through American Sign Language on stage at TED
Adam Munder speaks at TEDNext. October 22 - 24, 2024. Atlanta, GA. Photo: Gilberto Tadday / TED

At its heart, Omnibridge is a testament to the positive capabilities of artificial intelligence. 

But it is more than a transcription service — it allows people to have face-to-face conversations with each other. There’s a world of difference between passing around a phone or pen and paper and looking someone in the eyes when you speak to them. 

It also allows Deaf people to speak ASL directly, without doing the mental gymnastics of translating their words into English.

“For me, English is my second language,” Munder told Good Good Good. “So when I write in English, I have to think: How am I going to adjust the words? How am I going to write it just right so somebody can understand me? It takes me some time and effort, and it's hard for me to express myself actually in doing that. This technology allows someone to be able to express themselves in their native language.” 

Ultimately, Munder said that Omnibridge is about “bringing humanity back” to these conversations. 

“We’re changing the world through the power of AI, not just revolutionizing technology, but enhancing that human connection,” Munder said at the end of his TED Talk. 

“It’s two languages,” he concluded, “signed and spoken, in one seamless conversation.”

Header image courtesy of Gilberto Tadday / TED

Article Details

October 25, 2024 9:38 AM
Two images side by side. On the left, a person cuts into a Thanksgiving turkey. On the right, a map shows a turkey's journey from Mayberry, NC to Plymouth, MA.

Interactive turkey tracker follows carbon emissions of Thanksgiving birds 'from farm to fork'

The tool follows a single turkey’s journey across the supply chain, with optimizations to help reduce its environmental impact.
A thick, green Janis crystal absorbs water

'Smart crystals' that harvest water from the air could combat water scarcity, NYU researchers say

The organic crystals were modeled after desert plants and animals, which have evolved to have water-absorbing capabilities.
No items found.

Want to stay up-to-date on positive news?

The best email in your inbox.
Filled with the day’s best good news.