Her name is Ann. She suffered a brainstem stroke at the age of 30 and it left her deeply paralyzed. The reason for the stroke is still mysterious. The condition resulted in loss of control of all muscles in her body. Ann was unable to breathe. Ann is 47 years now. After 18 years of the injury, even though she cannot speak verbally on her own, thanks to AI technology, Ann can communicate and able to speak through an avatar using AI technology that translates her brain signals into facial expression along with her speech!
Speech is the ability to express ideas and thoughts through spoken words. Speech loss after an injury eventually can impair communication and can cause social isolation.
Until recently, patients were able to communicate slowly by relying on slow speech synthesizers. These speech synthesizers spelling out words by using eye tracking or facial movement. Using this till recently Ann was able to communicate using this movement tracking technology where she can select letters of up to 14 words a minute. Even though this method is useful, communication is highly limited. With the new technology- the brain computer interfaces (BCIs) could transfer lives of people who are unable to speak like Ann due to strokes or ALS (Amyotrophic lateral sclerosis).
How Ann is able to speak and communicate more now?
Tiny electrodes implanted on Ann’s brain surface to detect electrical activity in the part of the brain that controls speech and face expressions and movements. Signals translated directly to digital avatar. Avatar expresses speech and facial movement of all sorts – frustration, smile, frowning, surprise, laugh etc.
This research work carried out at University of California, San Fransisco (UCSF). According to Prof Edward Chang, who is the lead of this research work “our goal is to restore full, embodied way of communicating, which is really the most natural way for us to talk with others. These advancements bring us much closer to making this a real solution for patients” Research team implanted a paper-thin rectangle of 253 electrodes on Ann’s brain surface which is critical for the speech. These electrodes intercepted the brain signals. Ann worked with a research team to train the AI algorithm to detect brain signals for different speech sounds using various phrases repeatedly.
Ann’s voice was personalized in avatar by using her wedding speech. The AI system learned 39 distinctive sounds and Chat GPT- style language model was used for translating the signals into sentences. The test run had 500 phrases and it generated brain to text at a rate of 78/ minute compared to natural conversation where 110-150 words/ minutes spoken. The next step in this innovation is to create a wireless version of the BCI to implant beneath the skull.
According to researchers’ accuracy, speed and sophistication of this technology suggests that that this AI Avatar speech system is now can be used in practical purposes for patients.
D. David Moses, one of the co -author of the research said, “Giving people the ability to freely control their own computers and phones with this technology would have profound effects on their independence and social interactions.”
Says Ann “My brain feels funny when it hears my synthesized voice. It’s like hearing an old voice. I want patients to see me and know that their lives are not over. I want to show them that disabilities don’t need to stop us or slow us down,” Ann is currently helping researchers at UC San Francisco and UC Berkeley develop new brain-computer technology that could one day allow people like her to communicate more naturally through a digital avatar that resembles a person.
For more on this innovation see this video: How a Brain Implant and AI Gave a Woman with Paralysis Her Voice Back – YouTube
References:
https://www.universityofcalifornia.edu/
https://www.nature.com/
https://regina.ctvnews.ca/
https://www.technologyreview.com/
Image credit: UCSF -University of California, San Fransisco
