functions linked to 10-20 locations. aBCI: affective BCI

ellenvolumetricellenvolumetric
edited March 2015 in General Discussion
There are a lot of experienced people on this forum, and that's great. Its also important to hear from people who are not in electronics or software development such as myself. I have just used some of the tutorials to make the OpenBCI board and software work, and will be sending my comments and suggestions to Joel and Conner - i.e. what is outright wrong in the tutorials, and what needs further explanation. This is what opensource is about.

When I have used proprietary EEG software, at least I knew what certain parts of the brain were mapped for, i.e, simple things like attention and focus.. When you look at the graphic of the brain in OpenBCI you see round circles that are supposed to correspond to some sort of brain activity. Here is my very simple question - When you look at graphic of a head with all those circles and numbers in them, (Both in the OpenBCI graphic and in the traditional graphic that everyone uses with numerous circles) what, exactly does that mean? Does it correspond to any specific event in the human brain? If I place an electrode on, for example, circle 14 on someone's head, what is that corresponding to? Or not? If I want to map anger, what spot on the brain is that? Does such a thing exist?

I want to use a range of  emotional responses to trigger certain visual and auditory events, and I know from working with NeuroSky that can be accomplished with meditation and focus. So now that I can get a signal from OpenBCI, how do I interpret it past "alpha"? Couldn't you find an "alpha" signal coming off any random part of the brain? Or Beta, or Theta or Delta? If the Emotive can figure that out, how does that relate to OpenBCI? And can BrainBay help with this, or is that totally off topic.


Comments

  • wjcroftwjcroft Mount Shasta, CA
    edited February 2015
    Ellen, hi.

    Mapping brain locations to functions, hubs, networks -- is a HUGE research area. Only approximations can be achieved with surface EEG. Here's a 'guideline' table, with links to Brodmann areas if you want more detail.

    https://sites.google.com/site/biofeedbackpages/function-map

    More alpha is produced in the occipital and parietal regions, so that is a better location for your sensor if you are doing alpha training. However lower amounts of alpha can be found in other locations.

    The classifiers used in consumer commercial gear are likely to be fairly simple minded. So for example:
    • 'meditative' would likely be triggering on increased levels of alpha and reduced beta, theta.
    • 'focus' would likely be shown with increased beta and reduced slow waves (alpha and theta).
    There is a neurofeedback tutorial available with BrainBay that reinforces (rewards) for alpha and inhibits beta.

    William
  • wjcroftwjcroft Mount Shasta, CA
    Also checkout these great posts by @chipaudette on his blog,

    http://eeghacker.blogspot.com/2013/10/eeg-frequency-bands-jorge-ochoa.html

    http://eeghacker.blogspot.com/2014/04/detecting-concentration.html

    http://eeghacker.blogspot.com/2013/12/eeg-while-meditating.html

    ---

    re: detecting emotions, you may find some ideas here,

    https://www.google.com/search?q=eeg+detect+emotion

    EMOTION DETECTION FROM EEG SIGNALS: CORRELATING CEREBRAL CORTEX ACTIVITY WITH MUSIC EVOKED EMOTION (2014 thesis)
    http://mtg.upf.edu/system/files/publications/Erim-Yurci-Master-Thesis-2014.pdf

    Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns
    http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0095415

  • ellenvolumetricellenvolumetric
    Thank you so much, very helpful!  When I worked with the NeuroSky I took Bluetooth to Processing to OSC to MAX/MSP and it was successful. I will read all of this, update the forum if it needs to be updated, and at a late date hope to code all of this into Processing, and out to MAX once again. 
  • wjcroftwjcroft Mount Shasta, CA
    Ellen, thanks. It would be great to have an OSC / MAX / PureData connection for OpenBCI as well. (Should be able to avoid the Processing step). If you hear of / or get that to work, please let us know. We also have a Musical Applications thread; those folks would be very happy with OSC connectivity.

    http://openbci.com/forum/index.php?p=/discussion/98/musical-applications

    William
  • ellenvolumetricellenvolumetric
    William:
    I am a visual artist and though I work with sound, I do not compose it, though I do hijack it sometimes. When Conor first started working with OpenBCI I asked him to write something for MAX (PureData) but it never happened either because he was too busy, did not care, or did not know MAX well enough, I don't know. But I was all over that two years ago.

    In my proof of concept with Neuro Sky here are the steps I took and on a very rudimentary level it worked.

    1. Take Bluetooth Signal from Neuro Sky
    2. Use Processing with driver to handle Bluetooth into Processing. Set parameters for Attention and Meditation
    3. Send those parameters to send out
    4. Code MAX with CMAT UDP Receive (for whatever reason plain old UDP Receive does not work)
    5. Use mediation and attention to start and stop short movies. 
    That is a synopsis without the actual code.

    I can't see OpenBCI going into MAX without a driver, and I don't know who will write that, it won't be me, that's for sure. With my same scenario its easy to adapt it to sound.  Also word on the street is to try and bring OpenBCI into OpenFrameWorks, and I will know more about that in the future as I am also playing around with OpenFrameworks. 

    I looked at the music thread you suggested and it is very interesting, but it seems to be along the same sine waves/ saw tooth wave, etc. use of information to produce sound. I am taking a decidedly different approach - more curated. 

    For example, if I can figure out what is an "anger" or "fear" spot on the brain (I am tired of just attention and focus) I would use that response not to create a generative graphic, which is what most people think of, but a pre-curated, pre composed 30 second visual clip of a thematic representation of 'anger'.  So I guess that is like saying for sound people that if you feel anger, then the sound is an 'angry' sound - pre-recorded. I will use the brain impulses to launch pre-recorded data banks of visual and sonic information. But if I am unable to do that, then I will have to rethink or retool my approach. 

    Ellen
  • jfreyjfrey
    Hello,

    The problem here is that you need 2 different things:

    a. send data through OSC or other protocol (the easy part)
    b. do signal processing so that signal has meaning

    It's part b that's tricky, because there is no such signal as "Meditation" per se in EEG, and as such from OpenBCI. This is what @wjcroft refers to. Meditation, anger, and all of that are high-level constructs, that need... high-level processing.

    With neurosky headset -- and with other consumer products for that matters -- you do have something called "meditation". But what is it really? I do not know what neurosky is doing when you call their API in your step 2. It may be over-the-top signal processing or random values. 

    The purpose of OpenBCI is not to give you polished constructs, but raw EEG that you will have to work out. The GUI is merely a tool to gauge EEG signals, you cannot interpret many things with it directly, therefore a driver is not enough.

    From what I understand, you don't want to deal with signal processing, so you don't need an OpenBCI-MAX pipeline. What you need is to pipe EEG signals to another software, that will do the magic (or *try* to do the magic, we're never really sure what we are measuring with EEG), and *then* use this interpreted signal for music. If neurosky's algorithms are open, you could use run them on OpenBCI signals. Or use other software and toy with your own signal processing -- and yes, I will mention OpenViBE as I know people using it for music + it has OSC in the dev version :D

    Such software could also be used -- hopefully with little prior knowledge -- to exact basic spectral (ie: alpha, beta, etc.) or temporal (electrodes location) features. If you don't mind "experimental" music, or kinda random video clips, then you could start with that and later improve on the meaning as you dig in the literature :)

    (sorry if you already had all the big picture in mind, I realize I got a bit off track, my professorial tone being summoned by the "neurosky == meditation" keyword :D)
  • ellenvolumetricellenvolumetric
    Jfrey:
    Ok, I want to get very basic, and very real. In my two times of using OpenBCI the tape did not stick very well on the electrodes and they fell off. Also there was music in the room and other people and events and they skewered the use (I was at a hardware hack) and signals. So do you have to be in an isolation tank to use it?

    Question number 1
    Have you actually used OpenBCI to do anything??? Have you used the spider claw headset?  Does it work, because all I have seen here in New York is Conor wearing his cloth headband and using blinks and muscle flexing, which looks cool in a demo. I have seen a Makerbot printout of the headset, but I have not seen it in action - at all. Could someone please show me a video of a successful prototype of something or other that they actually did? I saw some guys at the hack moving an inflatable shark around (Sharknado) and someone else trying to land a drone in the hallway, but it was manic and I could not figure out what they were doing.

    Question number 2
    BrainBay looks like Isadora and it appears compelling. What is its relationship to OvenVibe - or not. OpenVibe seems somewhat Python based, and that is a road I am not traveling down.  BrainBay at least seems to work with MAX, but OpenVibe totally not. Also sending data through OSC or the other part is not that easy, and have you done it? It took me three months to make it all work and after I did I felt like I should declare a national holiday complete with a parade it was such a big deal.

    Look, I don't know epistemologically or otherwise what is defined as 'meditation' and truthfully I'm not that concerned. I know when I meditate it gets picked up by a sensor and I can do something with it. I'm not a doctor or a neuroscientist so its not that critical, no one is going to live or die over what I am investigating.

    Ellen 
  • jfreyjfrey
    edited February 2015
    You may know that you are meditating, but it's harder for the computer, for all it knows you could just be smiling ;) No one will die, but some could be deceived by the difference.

    There is already some documented applications. You can refer to Chip's blog http://eeghacker.blogspot.fr/ ... or to the one post entry of mine http://blog.jfrey.info/ , where I describe how to setup a "P300 speller", that is to say to write using your brain activity -- sorry, no video, it did not seem pertinent at that time, you'll have to take my word for it. It looks like that:


    I also experienced difficulty to maintain electrodes onto my scalp only with Ten20 paste, fortunately I have a "real" EEG cap, as you can see on the picture. The final version of the 3D print headset is not yet available, hopefully soon it'll be fixed.

    OpenViBE is not python based, under the hood it's coded in C++. You *could* use python or lua scipt inside the program, but in fact a regular user will just move boxes around to play with EEG; it's a visual programming environment. As such it is a "competitor" of BrainBay :D  Here is a (outdated) video that present the software:



    The strength of the software is that it can manage the whole brain-computer interface pipeline: acquisition, filtering, machine learning (the real job), output. The idea for you would be to pipe the output of the classifiers -- the "engines" that measure high-level signals -- to the outside world. OSC support is in the dev version, next release (1.0!) coming very soon.

    I deeply understand that you could be frustrated because OpenBCI is not "plug and play". Part of the problem is because there are lots of fantasies around EEG. While you do not need to be in a tank to use EEG, an act as simple as clenching your teeth will pollute all the signal. Even with medical grade equipment (30k$ in my lab), EEG is prone to muscular artifacts. We have to learn how to deal with that in order to use "real" brain signals. And it's hard to comprehend, even for hobbyist, even for scientists (!). Fortunately there are resources on Internet to help us to get start with the theory, eg in the "family": http://www.autodidacts.io/

    The other part of the problem is that at the moment there is little done for the general public regarding EEG and brain-computer-interface. We are several people working actively on it, trying to document and facilitate the transition. This is also why we need your kind of input, to know what to improve, how to communicate without giving false information or -- more importantly -- without giving false hopes.

    To conclude, a gift especially for you. "Sending raw EEG with OSC is easy"...well... here is the proof: one commit and 40 lines of codes added this afternoon to my dev branch of OpenBCI_Python. Spent roughly 1h, including reworking  my architecture and my very first attempt to use OSC. Still, nothing to show off, the ground was prepared by others. If fact it's *using* those OSC signal in the outside world that took most of my afternoon, each one has its area of expertise ;)
  • wjcroftwjcroft Mount Shasta, CA
    Posted a bit more on the quick OSC bridge that @jfrey setup, over on the Musical thread,

    http://www.openbci.com/forum/index.php?p=/discussion/comment/1680/#Comment_1680

    Allows use with VVVV, MAX, PureData, etc. Whatever 'performance' type apps that OSC can talk to.

    ellenvolumetric , I would take a look at the 2014 thesis by Yurci (Feb 18 post above). It gives a short survey of the various emotion detection algorithms and paper references that he looked at. In short, this is a very tough area of research. You may want to snag a signal processing geek to help you out with this aspect. You'll need a lot of sensors, I'm not sure 8 channels is enough. Your signal flow might go something like OpenViBE for analysis and then over the OSC to your other visualization / sonification apps.

    If you have Max already, OSC is how you get the EEG data into it.
  • ellenvolumetricellenvolumetric
    William:
    The thesis looks absolutely fantastic. 
    All of the posts are great. However the physical frailty of the wires in the electrodes, the oddness of the lack of understanding what an EEG on a spot on the brain means is challenging. When you suggest I use more than 8 channels, that is not within the realm of realistic arts practice. OpenVibe is for Windows so its not even in the running.

    The real issue is headset, wires, readings and meanings, movement and silence.

    I have already made a prototype or proof of concept taking NeuroSky into Processing and out to MAX via OSC to turn on and off video clips so I know such things can be done, because I did it. I don't want a signal processing geek and I don't want a lot of sensors. I want to make a creative practice performance in an immersive environment and I know I will, its just a question of what technology to use. I do want a MAX geek and I have one to make a random database of images, and not using SQL Lite. 

    I am learning a lot about the brain and signal processing on this forum, and that's great. I'm just questioning if OpenBCI can step up to the task I am asking it to perform, that's all.
    Ellen
  • jfreyjfrey
    OpenViBE is for Windows... and Linux, which is the most important platform as we all know. What, you had Mac in mind maybe? :D 

    You're absolutely right when you say that EEG, at this moment, is not very compatible with an artistic performance -- we have the same problem in our lab when we try to make "practical" applications with the general public; installation, calibration and signal processing are a big issue. Too tedious for regular people. However, I still believe that OpenBCI can help you to replicate and improve what you have with the neurosky. You may for example create a fancy 3D headset design, using maybe only 3/4 electrodes and a saline solution. The 3D printed *electrodes* amaze me (see the post on the Community page), I dream of a headset in one piece that would hold OpenBCI board and integrate the electrodes at the same time -- if only the triple head 3D printer we received in our lab would start working... There's work ahead as all in here is about DIY -- or more precisely "DIO" in this case -- but this is what make the journey such a thrill :)


  • wjcroftwjcroft Mount Shasta, CA
    edited March 2015
    Ellen, Jeremy, hi.

    Certainly you'll want some sort of headset or EEG cap. It's looking like the g.tec g.GAMMAcap2 is a pretty good interim solution until next generation headsets (using dry active sensors) surface. The g.tec cap can likely be used with a number of electrode configurations: (1) Jeremy's 3D printed holders for injected gel applications. (2) no-gel/no-paste saline solution pads with Ag-AgCl buttons. (3) the dry Ag-AgCl combs that Karl @kzurn is trial-ing (see electrode thread). These 'dry' combs can also be injected with gel if needed through the hole on the top.

    And there are also the classic Electro-caps, 19 channels, injected gel. Cost of the Electro-cap and g.tec cap is comparable. (Around $230). Both elastic caps fit a range of head sizes. But they offer a large size as well for those of us swell-headed guys.  :-)

    Who knows how many electrodes are going to be needed for your application? If you look over that list of prior research in Yurci's thesis, there is a wide range of percentage accuracy figures for emotion classification. Yurci himself is using all 14 channels of the Emotiv EPOC, and achieves superior accuracy compared with the earlier algorithms. I'm not sure his system though was live & realtime, vs. post processing.

    I'm trying to get a copy of the 2012 Ramirez paper, "Detecting Emotion from  EEG Signals Using the Emotiv Epoc Device". It's accurate in the 80% range, and was only using 4 sensors.

    http://link.springer.com/chapter/10.1007/978-3-642-35139-6_17

    Also see this list of papers on the Emotiv site,

    http://emotiv.com/paper/?SHOWALL_1=1

    re: signal processing engineering. You or your partner are going to have to jump into this if you want to use the open research. Of course another alternative is to just use the proprietary approach with the EPOC headset and their closed emotion classifier. Don't know what the accuracy is on that.

    re: OpenViBE on Mac (via VirtualBox free virtual machine environment). We will hopefully have that going soon, some minor glitches currently.

    Remember your Neurosky setup was doing very simple classification based on relative amounts of alpha and beta bands. True emotion detection is still a research topic, but looking more practical as time goes on. Provided you have some 'training' time available to adapt the detection to a subject(!)

    William
  • wjcroftwjcroft Mount Shasta, CA
    edited March 2015
    Oops, looks like Emotiv's built in algorithms are not nearly as generalized as some of the research papers, see quote appended below.

    Also saw another EPOC research paper trying a range of classifiers; I guess the Emotiv paper list link on previous post is a bit dated,

    http://link.springer.com/chapter/10.1007/978-3-642-34500-5_47

    This Honors Report from the University of Canterbury concludes that EPOC classifier is "unsuitable" for their application. Basically confirms what Emotiv themselves say in below quote.

    http://www.cosc.canterbury.ac.nz/research/reports/HonsReps/2013/hons_1302.pdf

    ----

    Comment by Emotiv staff on their built in classifier performance:

    https://www.emotiv.com/forum/forum4/topic848/messages/

    The question about accuracy is a bit like nailing down jelly (jello for our US readers ;)) How do you measure emotions in the first place so you can compare the outputs and come up with a number? Our internal measures ran between 65% and 100% depending on the emotion, the subject and possibly the phase of the moon. Excitement is pretty solid - it's quite easy to measure objectively and we figure it's over 80% accurate for over 80% of people. Engagement/Boredom is also pretty good and again it's not hard to measure objectively. Frustration is ok but gets confused with anger a little - they frequently occurred in our training data at the same time. Meditation is really good for those in our testing group who could genuinely meditate, but how many kinds of meditation are there?

    Those are the four basic emotions we measure. There will be more.

    The output for each emotion is a floating point number between zero and one. We self-scale the output based on historical patterns of each individual user, so it takes a few hours for the system to settle down for a given subject. Self-scaling provides a useful within-subject scale but makes it very difficult to compare subjects (land a plane with no engines on a river with barely an increased heart rate - our system will show you to be hyper-excited as the plane hits the water, even though objectively you're a lot calmer than I am when someone cuts me off at a trffic light - which will max me out).

    Depends what your PhD dissertation is about whether I'd recommend basing your entire future on our Affectiv suite
  • ellenvolumetricellenvolumetric
    William:
    I find your resources amazing. 
    Why don't you have a look at this video of the work of my friend, and a Professor at my school in Hong Kong, Samson Young.


    also you can watch here a piece on did with a 12 camera motion capture studio, body mapping and telematics

    I just read Yurci's thesis, and though it is helpful, it is not the direction I will be moving in. Also I am watching the UCSD on-line lecture course which is most helpful.

    I run an art hack every week and am about to start playing around with the Muse this week or next.

    I don't need more than 4 or 5 locations to work with on the brain as I want to have 4 or 5 screens projecting. I don't care if the emotions are deeply accurate, just approximations. If I can extract the data to OSC, I will know what to do with it.

    Ellen
  • wjcroftwjcroft Mount Shasta, CA
    Ellen, thanks.

    Enjoyed watching both videos, Beethoven & Brainwaves do seem to inspire each other(!) :-)  I do music improvisation myself, using the framework of the Music for People organization, founded by cellist David Darling.

    I think you'll be right at home with the Muse, it plugs directly into OSC.

    There are also now annual conferences using facial expressions to detect emotion,


    William
  • wjcroftwjcroft Mount Shasta, CA
    edited March 2015
    He just retweeted a conference that is very aligned with your interests,

    http://www.affective-sciences.org/aBCI2015

    aBCI: Affective BCI. (affect = Emotions detection.) That's the 4th conference in the series, so likely the previous ones are online as far as programs / papers, etc.

    I wasn't aware of the 'aBCI' term myself until now. Great to know there are many researchers in this field.


  • ellenvolumetricellenvolumetric
    William:
    Muse is not OSC capable, unless you deal with its app, and I don't even know about it because I did not use the app. I actually just used the MUSE, and someone I knew knew the developer so I contacted them, and they said the OSC version to work on a computer is not even ready yet for release. So don't believe the hype.The thing had trouble pairing, and its sensors on the headband area were nothing more than tape on a thin metal band - very rickety. 

    The twitter feed from Conor is ok, but not where I need to go. I think the icmi.acm.org is much better and at least addresses what I want to deal with. 

    It's great you are so up on this stuff, I find your links very helpful.
    Ellen
  • wjcroftwjcroft Mount Shasta, CA
    Ellen, this repository implies that Muse can stream to OSC. It's not authored by InteraXon, but by a separate developer. And there are other Muse / OSC toolkits out there apparently, by external devs.

    Since Jeremy has made the OpenBCI / Python OSC plugin available, have you looked into trying some tests with that?

Sign In or Register to comment.

© Vanilla Keystone Theme 2026