The page you link to, is likely not as relevant as actual published research above.
Some adventurous experimenters are asking ChatGPT to process their raw EEG files. But this is only picking out the most obvious features.
"ChatGPT can actually interpret Muse Data. This is a paradigm shift for the whole neurotech industry! I plan on making a lot more videos like this. Feel free to email me back with suggestions and ideas on what devices/projects you would like to see related to AI brainwave decoding."
Dr. Cody Rall is a psychiatrist with a channel, TechForPsych which reviews consumer brain tech.
I kind of like Dr. Cody. He's somewhat "pop"... popping in my social media for sure. I can see the excitement with ChatGPT. I think as he gets to Tensorflow and Eden AI and azure etc. likely he will feel that this derails him from his core niche, because ChatGPT 4 is easy and impressive on doing... whatever it can do. But when you want to do something specific, beyond the - as you say - extracting the most obvious features, it's still hard.
Back to Tensorflow, it's nice if we can have some open source code we can look at. Some Google results are interesting, I can see many attempts for classification and a relevant Kaggle competition. Sometimes the code might be a bit too advanced. I think the approach of most people who come from AI is closer to Dr. Cody's i.e. let's see what we can do easily with ML on EEG data, instead of what people in this forum often try to do, which is try to do exactly what they want, using available technology. Real-time low latency estimation of brain rhythms with deep neural networks is an example of published research where they used ML to achieve predefined research goals. I wish I could see their source code (and I didn't have to pay $40 for the paper). Would love to see more code released, in general.
Mind-reading AI can translate brainwaves into written text
A system that records the brain's electrical activity through the scalp can turn thoughts into words with help from a large language model – but the results are far from perfect
By James Woodford 12 December 2023
An AI can decode brainwave recordings to predict the words someone is reading
Using only a sensor-filled helmet combined with artificial intelligence, a team of scientists has announced they can turn a person’s thoughts into written words.
In the study, participants read passages of text while wearing a cap that recorded electrical brain activity through their scalp. These electroencephalogram (EEG) recordings were then converted into text using an AI model called DeWave.
Chin-Teng Lin at the University of Technology Sydney (UTS), Australia, says the technology is non-invasive, relatively inexpensive and easily transportable.
While the system is far from perfect, with an accuracy of approximately 40 per cent, Lin says more recent data currently being peer-reviewed shows an improved accuracy exceeding 60 per cent.
In the study presented at the NeurIPS conference in New Orleans, Louisiana, participants read the sentences aloud, even though the DeWave program doesn’t use spoken words. However, in the team’s latest research, participants read the sentences silently.
Last year, a team led by Jerry Tang at the University of Texas at Austin reported a similar accuracy in converting thoughts to text, but MRI scans were used to interpret brain activity. Using EEG is more practical, as subjects don’t have to lie still inside a scanner.
The DeWave model was trained by looking at lots of examples where brain signals match up with specific sentences, says team member Charles Zhou at UTS.
“For instance, when you think about saying ‘hello’, your brain sends out certain signals,” says Zhou. “DeWave learns how these signals relate to the word ‘hello’ by seeing many examples of these signals for different words or sentences.”
Once DeWave understood the brain signals well, the team connected it to an open-source large language model (LLM), akin to the AI that powers ChatGPT.
“This LLM is like a brainy writer that can make sentences. We tell this writer to pay attention to the signals from DeWave and use them as a guide to create sentences,” says Zhou.
Finally, the team trained both DeWave and the language model together to get even better at writing sentences based on the EEG data.
With further refinement, the researchers predict that the system could revolutionise communication for people who have lost speech, such as those affected by a stroke, and could also have applications in robotics.
Craig Jin at the University of Sydney says he is impressed with the work by Lin’s team. “It’s excellent progress,” he says.
“People have been wanting to turn EEG into text for a long time and the team’s model is showing a remarkable amount of correctness. Several years ago, the conversions from EEG to text were complete and utter nonsense.”
The holy grail of BCI - will have to read it. Ok - I had a look -
words might produce similar brain wave patterns
effectively convey the intended sentiment
I'm wondering if they catch the sentiment or subtle motor-related signals from subvocalization. Especially since the paper is very "words" focused. If you search the paper, "thoughts" rarely appears. The more informal New Scientist article rushes to put "thoughts" into the headline of course. In any case, this is very impressive! Way to go!
EEG waves are collected with a 128-channel system under a sampling rate 500Hz through a frequency band filter from 0.1Hz to 100Hz
Using magnetoencephalography (MEG), a non-invasive neuroimaging technique in which thousands of brain activity measurements are taken per second, we showcase an AI system capable of decoding the unfolding of visual representations in the brain with an unprecedented temporal resolution. https://ai.meta.com/blog/brain-ai-image-decoding-meg-magnetoencephalography/
Even this unreal research from Meta (I don't want to know how much the MEG infra costed, and how much post-processing it took) I think it likely fits on visual cortex activity data.
embedding of natural images in pretrained deep nets linearly account for the neuronal responses to these images in the cortex
Probably what it says is that when we "imagine" things, somewhere in the brain, an actual representation of an image is formed. The results are way too perfect. Thankfully they don't seem to go as far as to say that every thought is represented as an image.
Comments
I've seen some @vinnyMS1 's code here: https://openbci.com/forum/index.php?p=/discussion/comment/19444/#Comment_19444
Hi Winstead and LookFwd,
There are many papers in this subject area of ML and EEG:
https://www.google.com/search?q=machine+learning+with+eeg
The page you link to, is likely not as relevant as actual published research above.
Some adventurous experimenters are asking ChatGPT to process their raw EEG files. But this is only picking out the most obvious features.
Dr. Cody Rall is a psychiatrist with a channel, TechForPsych which reviews consumer brain tech.
https://www.youtube.com/@CodyRallMD/videos
https://www.google.com/search?q=analyze+raw+EEG+with+chatgpt
William
I kind of like Dr. Cody. He's somewhat "pop"... popping in my social media for sure. I can see the excitement with ChatGPT. I think as he gets to Tensorflow and Eden AI and azure etc. likely he will feel that this derails him from his core niche, because ChatGPT 4 is easy and impressive on doing... whatever it can do. But when you want to do something specific, beyond the - as you say - extracting the most obvious features, it's still hard.
Back to Tensorflow, it's nice if we can have some open source code we can look at. Some Google results are interesting, I can see many attempts for classification and a relevant Kaggle competition. Sometimes the code might be a bit too advanced. I think the approach of most people who come from AI is closer to Dr. Cody's i.e. let's see what we can do easily with ML on EEG data, instead of what people in this forum often try to do, which is try to do exactly what they want, using available technology. Real-time low latency estimation of brain rhythms with deep neural networks is an example of published research where they used ML to achieve predefined research goals. I wish I could see their source code (and I didn't have to pay $40 for the paper). Would love to see more code released, in general.
https://www.google.com/search?q=reading+thoughts+from+EEG
https://www.newscientist.com/article/2408019-mind-reading-ai-can-translate-brainwaves-into-written-text/
https://openreview.net/pdf?id=WaLI8slhLw
https://profiles.uts.edu.au/Chin-Teng.Lin
The holy grail of BCI - will have to read it. Ok - I had a look -
I'm wondering if they catch the sentiment or subtle motor-related signals from subvocalization. Especially since the paper is very "words" focused. If you search the paper, "thoughts" rarely appears. The more informal New Scientist article rushes to put "thoughts" into the headline of course. In any case, this is very impressive! Way to go!
Nice hardware and long time wiring (I guess)
For those just wanting a proven cVEP based BCI, MindAffect is free open source.
https://openbci.com/forum/index.php?p=/discussion/2783/mindaffect-announces-open-source-release-of-their-cvep-bci-speller-games-etc
https://mindaffect-bci.readthedocs.io/
Even this unreal research from Meta (I don't want to know how much the MEG infra costed, and how much post-processing it took) I think it likely fits on visual cortex activity data.
Probably what it says is that when we "imagine" things, somewhere in the brain, an actual representation of an image is formed. The results are way too perfect. Thankfully they don't seem to go as far as to say that every thought is represented as an image.