"Verbal mind reading" using OpenBCI tech
Hello everyone, thrilled to be here. I'd be grateful if you could help. Maybe you know a relevant existing research project, or have any kind of helpful recommendations - I'd be grateful for anything!
I work with children diagnosed with the most extreme cases of traumatic brain injury and brain-based disorders.
All the kids at our school are non-verbal, and they typically have trouble even making most basic coordinated movements with their hands. Many of them can't control the movement of their eyes.
AND YET - many of them understand speech and attempt to communicate.
I'd like to explore the possibility of creating the following research project:
I want to attempt to scan the EEG and maybe other signals (EMG, EKG - honestly, I don't know what I need to scan) - associated with words of English language thought by a test subject.
My hypothesis is that when a person thinks a word, there have to be some kind of readable micro-signals from the brain, and possibly facial and other muscles, that can be recognized as a certain pattern. If that pattern is reasonably consistent for the test subject, we may be able to pinpoint such consistent pattern for something like 1,000 words - and that would allow our test subject to communicate by controlling the display of these words on the computer screen, or for example having those words spoken by a computer voice.
So, to rephrase, I want to experiment to try to scan verbal thoughts via BCI into computer, essentially enabling non-verbal kids to communicate verbally with this assistive technology.
So, my questions are: What device / devices would you recommend I need to purchase in order to be able to conduct this type of experiment at quality? Do you know if any attempts had been made using the technology in this way (perhaps there are some documents I could get familiar with)? What else could you share that you feel might help? Thank you in advance!
Comments
Hi Dimitri,
The short answer is that Brain Computer Interfaces, especially the non-invasive type (EEG electrodes on scalp) used in the majority of BCI research -- are not yet capable of 'verbal mind reading'. There is some early research going on in this larger area, but it is not yet accurate and requires extensive pre-training of the machine-learning models.
This paper shows an invasive approach with six different words, and only 79% or 23% accuracy (for the two patients).
https://medicalxpress.com/news/2024-05-braincomputer-interface-decode-words-spoken.html
Also using invasive intercranial electrodes,
https://www.sciencedirect.com/science/article/pii/S1053811923000617?via=ihub
Another research using fMRI approach,
https://www.vox.com/future-perfect/2023/5/4/23708162/neurotechnology-mind-reading-brain-neuralink-brain-computer-interface
The above are just some examples. There is a class of non-invasive BCIs that use VEP, Visual Evoked Potentials, that might apply to 'some' of your subjects. My concern is that you state: "Many of them can't control the movement of their eyes." The VEP based BCIs work by having the subject select from various on-screen 'menus' of actions, based on what display screen pictures or objects they are focusing on. Below are some links you can look at.
https://openbci.com/forum/index.php?p=/discussion/3894/mindaffect-eeg-cvep-bci-robot-control-tutorial-by-pyotr-vozniak
https://www.google.com/search?as_q=MindAffect&as_sitesearch=openbci.com
https://openbci.com/community/mind-controlled-robot-openbci-mindaffectbci-maqueen-v2/
https://openbci.com/forum/index.php?p=/discussion/2783/mindaffect-announces-open-source-release-of-their-cvep-bci-speller-games-etc
Finally we should mention some systems doing recognition on 'subvocalization' detection of facial muscles during imagined speech.
https://www.google.com/search?q=subvocalization+imagined+speech
https://en.wikipedia.org/wiki/Subvocal_recognition
Regards, William