Image Credit: Peshkova / Shutterstock.com
Scientists have converted brain activity into text with the help of Artificial Intelligence (AI). The team at the University of California, San Francisco, recorded data from neurons that were activated while people were speaking to devise their method which could see those with locked in syndrome regaining a form of communication.
Training AI to Read Neural Activity
The California-based team published their findings this week in the journal Nature Neuroscience. In their paper, the team describes how they successfully developed an AI-based system that can convert brain signals into text, in the future, this may allow those who have lost the use of speech to communicate once more.
Four participants were recruited to take part in the study. These participants already had an array of electrodes implanted into their brains to help monitor their epileptic seizures.
During the study, participants were required to read a set of 50 sentences aloud. They did this a number of times while the team tracked their neural activity. A machine-learning algorithm then studied the neural data, using it to convert the brain activity into a numerical code.
The system then checked that the numbers generated were specifically linked to speech aspects by comparing the actual recorded audio to the sounds that the system predicted from small segments of brain activity. This ensured the specificity of the system. Next, the numbers converted in a second part of the AI system which converted the strings of digits into word sequences.
In the beginning, the system generated sentences that made no sense. However, the system continued to compare its outputs against the words that were actually read out to improve its accuracy. Over time, the system improved, learning which words were related to specific patterns of neural activity, and understanding which words were likely to follow each other.
The next part of the study saw researchers testing the system further, having it generate written text using brain activity alone (brain activity generated from reading speech).
While the system was not 100% accurate, it proved itself to be more accurate than previous systems had achieved. The accuracy was higher or lower depending on the person the system trained on, showing variability between participants. However, the error rates of the AI system were lower than that of professional human transcribers, 3% compared with 5%. Although, the algorithm was trained on a very limited set of sentences, unlike transcribers who work with unlimited data input.
A Highly Effective System
Scientists are excited at what has been achieved by the team at the University of California because it demonstrates that the AI system can be trained in less than 40 minutes and gain high levels of accuracy. Using conventional methods, millions of hours of training would be required to achieve the same effect.
The levels of accuracy achieved in this research are higher than those previously recorded by other methods. However, it is important to note that there is much more research and development needed before the system can be effectively used to help severely disabled people regain communication skills. This is because currently, the method relies on being trained on the brain activity generated from the participant speaking out loud, something that those with locked in syndrome or those who have suffered severe strokes cannot do.
Giving Communication Back to Those with Disabilities
Although the system cannot currently be used to help those with no speech communicate once more. This application of the technology is not outside of the realms of possibility. Scientists view this work as a potential basis for speech prosthesis. With more work, the system could advance to the point where it can understand neural activity related to inner speech without first needing to be trained on the person actually reading out sentences first. This could offer a way for people who have locked in syndrome to communicate with the world again.
Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.