Methods for signal validation / EEG "phantom heads"
Hi all, been great to see this community grow so much.
I'm curious to hear what kind of techniques people are using for verifying and validating that their systems are working properly.
E.g., dealing with the "ground truth" problem when it comes to debugging, and more complex signal analyses.
You need a way to make sure your OpenBCI actually works!
In MRI, CT, PET... pretty much all other biomedical imaging... there are so called "phantom" devices that are used to calibrate the system by proving a known signal. Like a big sphere of oil (MRI) or cylinders of different density materials (CT).
In EEG there is no real community-adopted standard methods. Over the years I've seen (and personally used) everything from just connecting leads together directly (not ideal) to using a mildly resisting strip conductor, to (my own favorite) a cantaloupe or other melons with nails jammed in them that you pass a current through and stick leads on it like a real head.
some labs have made up some pretty elaborate devices like taking a real human skull, re-hydrating it, adding electrodes and sealing it in an electrolyte solution. Others have made full head models from carbon fiber mixtures (@chipaudette knows what I'm talking about!). These are great but not feasible for your average DIY guy!
The whole point is, ways to avoid having to use a real human head to get a signal... because you can't control what is coming out of somebody's noggin!
What have you done?
This comes up a lot when dealing with seemingly random noise issues. Normal EEG looks an aweful lot like noise if you don't know what you're looking for...
I'm curious to hear what kind of techniques people are using for verifying and validating that their systems are working properly.
E.g., dealing with the "ground truth" problem when it comes to debugging, and more complex signal analyses.
You need a way to make sure your OpenBCI actually works!
In MRI, CT, PET... pretty much all other biomedical imaging... there are so called "phantom" devices that are used to calibrate the system by proving a known signal. Like a big sphere of oil (MRI) or cylinders of different density materials (CT).
In EEG there is no real community-adopted standard methods. Over the years I've seen (and personally used) everything from just connecting leads together directly (not ideal) to using a mildly resisting strip conductor, to (my own favorite) a cantaloupe or other melons with nails jammed in them that you pass a current through and stick leads on it like a real head.
some labs have made up some pretty elaborate devices like taking a real human skull, re-hydrating it, adding electrodes and sealing it in an electrolyte solution. Others have made full head models from carbon fiber mixtures (@chipaudette knows what I'm talking about!). These are great but not feasible for your average DIY guy!
The whole point is, ways to avoid having to use a real human head to get a signal... because you can't control what is coming out of somebody's noggin!
What have you done?
This comes up a lot when dealing with seemingly random noise issues. Normal EEG looks an aweful lot like noise if you don't know what you're looking for...
Comments
Zeynep and Scott Makeig are collaborators of ours, actually our lab partly sponsored that paper ;-)
Really I just want to get some discussion going on what people are using and how they are addressing this problem. We're working on cheap, easy methods, and my goal is to open-source release our full plans and mixing methods within the very near future, but with the crowd here I'd love to know if somebody has better ideas.
Regarding the quote above, that was a reference to the Emotiv Epoch system. We have had very good luck with it, regarding everything except (1) timing issues when using 3rd-party triggers and secondary devices, and (2) long-term comfort.
Now, yes I know - $350 is teh consumer model that dosn't give access to teh raw data, you have to spend $750 for that...
There have been a number of articles comparing Emotiv against others, some of which I think have been discussed here (definitely over on the Emotiv forums) but here is a comparative analysis we did a few years ago
https://www.researchgate.net/publication/263286124_A_Comparison_of_Electroencephalography_Signals_Acquired_from_Conventional_and_Mobile_Systems
Short story - overall for ERPs we found the Emotiv about the same... once you addressed the timing issues, and rejected the slightly higher # of noisy trials....
What is important though is that you use a material that is at least moderately resistive. E.g. don't use a sheet of steel or copper.
A bowl of Jello works ;-). So does a cantaloupe. Some of the new conductive-filament (carbon)-doped 3d printing materials could do the trick too.
My point to starting thsi thread was to hear discussions from other people out in teh world as to what they use, and for what specific purposes.
... I could probably talk for days about all the ideas I have about what potential uses are out there but I'd rather hear from others.
Either of you guys recommend a really good conductive filament for possible 3D printing of comb electrodes? I think @Conor and Joel @biomurph tried some tests and the test 'combs' only worked if gel was applied. Wondering if Ninjaflex could incorporate some of these conductive fibers in a possible new type of elastomeric filament.
David can you post a pdf link to your recent report on your lab's conductive elastomer electrodes and the phantoms / test setup you use?
William
Dear folks,
I joined the site most specifically to revive this thread..
I noticed David has since, true to his word, open-sourced instructions for a phantom head creation (thanks David! awesome site!!). Also, multiple articles have been released describing 3D printed phantom heads (https://vista.cs.technion.ac.il/wp-content/uploads/2018/09/TsiMunBroISBI18.pdf and https://www.nature.com/articles/s41598-017-05006-8#Sec2 are just a couple).
I'm interested to know - following all this, has an industry standard or common practice has started to emerge?
Also, I noticed that all of the projects above include the use of some degradable material (usually ballistic gel) is involved in creating the outer layer, unlike the article discussed above in which William was involved (https://www.researchgate.net/publication/230714956_Creation_of_a_Human_Head_Phantom_for_Testing_of_Electroencephalography_Equipment_and_Techniques) which, as I understand, uses no gel of sorts and is thus built to last.
Why is this? Is this structure extremely hard to reproduce or was it found to be not as stable?
Any answers you can provide would be most appreciated and either way thank you all for the value you are creating.
Omer, if David (ratlabguy) does not post a further comment here, you can email / message him directly by clicking on his username (above the date line in his comments), then click the Message button. That will forward directly to his email address. He is generally busy on DOD projects.
Chip has not posted on the forum here for some time, but it's possible. He is still at Creare as far as I know and is involved with an open source hearing aid project, with Joel Murphy (co-founder of OpenBCI).
https://shop.tympan.org/pages/the-team
If you do get further info on phantoms from David or elsewhere, feel free to post those. Here is the open source phantom you referred to:
https://osf.io/qrka2/
William
Here's the phantom that we made (notice OpenBCI sneaking in on the far left):
We used it to play back EEG signals recorded from this guy:
The key challenge (after making the head) is calibrating it so that you know which combination of electrodes on the inside of the head you need to drive (and with what polarity and intensity) in order to get the desired signals from the scalp.
For some relatively simple cases, we ended up doing kinda OK:
Chip
When we were working with this head phantom, the challenge that we never fully overcame is that the DC baseline of our EEG signals wandered around a lot when using the phantom. Our human subject gave pretty stable near-DC signals over our relatively-short recording durations. The head phantom showed a lot more near-DC variability, even over these short periods.
We saw this with two EEG systems (one OpenBCI system and one from ANT) so I don't think that it was the EEG system. I think that it was the interaction of the EEG system, the EEG gel, and the head phantom.
If you don't care about really low-frequency performance, it's fairly easy to filter out this low frequency stuff (most of the time). But, if you **do** care about very low-frequency EEG, you might find this to be a troublesome problem!
Chip
Here are two projects / papers discovered by Scott Yuen last week:
https://www.mdpi.com/1424-8220/21/14/4658/htm
"A Long-Lasting Textile-Based Anatomically Realistic Head Phantom for Validation of EEG Electrodes"
https://osf.io/qrka2/
"Open EEG Phantom"
I'm familiar with this paper. While what they propose is quite useful, what the authors have completely overlooked is that gelatins are a model because it is an ionic substrate roughly akin to skin. I say roughly because its still moist and not a great analog, but currently (IMO) the easiest and cheapest thing you can do that is semi-realistic.
This is important because a LARGE part of the magic of getting a good signal with electrical sensing is the skin-to-electrode junction. In the normal situation it is a semi-capacitive relationship where ions fluctuating at your skin (which have arise from internal brain fluctuations) have to excite electrons on the other side of the junction (electrode). Conventionally AgCl is used to make this happen... and a big part of the reason why most so-called "dry" electrodes suck is because they have no mechanism for connecting ionic flux to electronic. E.g. it is purely capacitive.
The point of all of this rambling is that it means if you want to provide or test the efficacy of an electrode recording system, you really need to have this non-ideal situation replicated - an ion-electrode interface.
In the above paper, however, they totally short-circuit this by using an electrically conductive cloth membrane. Functionally this is no different that a wire, and really at this point you might as well just take a wire and connect a function generator right to the EEG electrode. The head model buys you nothing at all, except a nice shape to set a cap on. I guess it would be good for testing timing and such, or if your just need to confirm "yes my system works" for some pre-made form factor.
Here's an alternative idea: If you don't want to mix up gelatins or make a phantom head - go to the butcher and buy $5 worth of pig skin. Then attach a function generator to that.
https://ieeexplore.ieee.org/abstract/document/8717106
"tACS generator as method for evaluating EEG electrodes: Initial validation using pig skin" Publisher: IEEE
David, thanks much for your comments and feedback. I'll relay these and your paper to Scott Yuen.
Attached below a link to the pdf of your 2019 paper.
"tACS generator as method for evaluating EEG electrodes: Initial validation using pig skin" Publisher: IEEE
https://openbci.com/forum/uploads/editor/s5/u74iv0g3y18i.pdf
William
David, do you have any comments on the Creare phantom (posted in earlier comments above). Link:
https://www.researchgate.net/publication/339697951_Design_and_Demonstration_of_a_Head_Phantom_for_Testing_of_Electroencephalography_EEG_Equipment
https://ieeexplore.ieee.org/document/9475461