My SSVEP BCI, 16 LED matrix

nekrodezynfekatornekrodezynfekator Poznan, Poland
edited January 2016 in Research
Hi,
I'm working on my phd thesis. I designed and built BCI system using 8-channel OpenBCI.
image
I'm using my own photostimulator with RGB diode, Arduino UNO and board with 4xTLC5940. My BCI can recognize 8 different stimuli with accuracy >90% (it is work on that). In near future i want to extend the number of stimuli to 16 or more.
This is my control panel written in C#:
image

I will try to publish my papier in which I describe in details my BCI, but if u have any questions I will anwser here.

Comments

  • wjcroftwjcroft Mount Shasta, CA
    edited January 2016
    Marcin, impressive.

    Below are a few questions. Here's an earlier thread pointing to your Github. Is the C# function that reads the OpenBCI stream, available to other C# programmers? I think you may be our primary C# guy(!)


    Some questions:
    • What kind of behaviors or outputs are you controlling with your SSVEP?
    • So you have 8 distinct flash patterns that can be recognized, with combinations of flash rate and matrix patterns? What would be some examples?
    • Curious about the ribbon cables, that's lots of wires, about 20 per cable and it looks like 4 ribbons.
    • The graph shown on the bottom photo, what do the bars represent, is this an FFT? What are the axes?
    • Looks like you are mainly using green or bluegreen for colors.

    Cheers,

    William


  • GitHub contains early version of my app. I think that I will rewrite it and then share it for OpenBCI community. It will provide a way to collect and display data.

    What kind of behaviors or outputs are you controlling with your SSVEP?

    I'm controlling rows of LEDs.
    image
    For example: if computer recognizes that I watched "B" diode, he will turn on red diode below the green one.

    So you have 8 distinct flash patterns that can be recognized, with combinations of flash rate and matrix patterns? What would be some examples?

    I'm using sinusoidal waveform and patterns which are flashing with: 11,12,13,14,15,16,17,18 Hz. I tried another frequencies: 31-38 and 41-48, but without any success (to signal processing I used method described on http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2266836/).

    Curious about the ribbon cables, that's lots of wires, about 20 per cable and it looks like 4 ribbons.

    I know that the system is oversized. It can control 64 LED. Originally I wanted to build full qwerty keyboard. It's one of my future plans ;)

    The graph shown on the bottom photo, what do the bars represent, is this an FFT? What are the axes?

    No, it is not FFT. It's energy of each of the 8 frequencies (check out the article). 

    Looks like you are mainly using green or bluegreen for colors.

    Yes. I my previous research I checked that green and blue provides best SNR and user comfort.


    It is a little bit disappointing that it didn't work out with higher frequencies.


  • wjcroftwjcroft Mount Shasta, CA
    Interesting. So your LEDs 'flash', but not with square wave on / off cycles. Instead you modulate the brightness with a smooth sine wave pattern? How would you contrast those two styles of intensity modulation, as far as ease of detection? Does the smooth modulation give a cleaner signal? Less 'ringing' or harmonics?

    Thanks for that paper reference.

    In your bar graph above, the top graph it seems obvious which choice is being made (16hz?) But in the bottom bar graph, it's not clear which frequency 'wins'. Why does it show two bar graphs at once. It may help us visualize how this works if you eventually post a Youtube of the array in action with some narration.

    Best regards,

  • I checked that sin wave generates better SNR. I compared 3 waveforms: sine, square and sawtooth. Differences between sine and square was unsignificant.
    In my graph, top graph represents energy of fundamental harmonics, bottom represents energy of first harmonics. BCI can be run in many different configurations. For example, I can chose what should be analyzed: fundamental or fundamental and first harmonics. The second case provide better recognition accuracy but significantly lower time to pattern recognize.
  • nekrodezynfekator,
    Very Impressive!
    - What's your EEG montage (electrode locations, reference, ground, re-reference)? 
    - What's the bandwidth of the filter for picking up each frequency?
    - In the graph, what is the y-axis scale?
    - How long do you need to look at a diode for it to be picked up reliably?
    Thx,
    D
  • wjcroftwjcroft Mount Shasta, CA
    I also wanted to mention Joel Eaton @j_loe here, who is an expert on music BCI's, (see links on thread below), and who has built LED matrix SSVEP control stations,


    Joel, it would seem that Nek's improvement in signal/noise level by using sine wave intensity modulation (vs binary on/off) -- may be a new advancement in LED SSVEP. Do you know of any other groups using non-binary intensity modulation? Could this same intensity modulation be applied to LCD based / checkerboard SSVEPs?

    William


  • @wjcroft,
    nekrodezynfekator wrote "I checked that sin wave generates better SNR. I compared 3 waveforms: sine, square and sawtooth. Differences between sine and square was unsignificant."
    So unless there's a typo, he seems to say that difference between sine and square are insignificant.

  • wjcroftwjcroft Mount Shasta, CA
    edited February 2016
    @qwer1304, hi. Hmm, I read his sentence as saying that his SSVEP performs better (higher SNR, signal to noise ratio), but that the fundamental output (selected band) was the same as square. Otherwise why would he emphasize this in his previous statement: "I'm using sinusoidal waveform and patterns which are flashing with..."

    Marcin, can you clarify why you chose to use the sine wave intensity modulation?

  • wjcroft, could very well be.
    Note that since luminosity is only positive, the "sine" is either riding on a DC (1/2[sin(wt) + 1]) or is a rectified sine (abs(sin(wt)).

  • wjcroftwjcroft Mount Shasta, CA
    It would need to riding on the half amplitude offset, was my understanding. In order to give a smoother FFT / filter peaks at the measured frequencies. The rectified sine (even full wave rectified) would produce more harmonics.

    Marcin, let us know the details...

    Incidentally, a similar sine modulated intensity is used in some light therapy applications, to make the effect less harsh.
  • nekrodezynfekatornekrodezynfekator Poznan, Poland
    edited February 2016
    I use sine wave to control LEDs becouse: 
    I think that it's easier to obtain accurate sine wave using arduino than square wave,
    I was inspired by: http://www.pnas.org/content/112/44/E6058.abstract .They also use sine wave to generate visual flickers.
    and I was hoping for a higher first harmonic SNR. It should make easier to recognize a target.

    correct me please if I'm wrong
  • nekrodezynfekator, perhaps this message got lost:
    - What's your EEG montage (electrode locations, reference, ground, re-reference)? 
    - What's the bandwidth of the filter for picking up each frequency?
    - In the graph, what is the y-axis scale?
    - How long do you need to look at a diode for it to be picked up reliably
    Thx,
    D
  • What's your EEG montage (electrode locations, reference, ground, re-reference)? 

    electrode locations: O1, O2, P3, P4, Pz
    reference: Oz
    ground: right earlobe
    re-reference: ?? could u explain? 

    What's the bandwidth of the filter for picking up each frequency?

    i used 8 band-pass filters: x-0.2Hz to x+0.2Hz, where x is specific frequency 

    In the graph, what is the y-axis scale?

    Its It's energy of each of the 8 frequencies (check out the article: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2266836/ ). 

    How long do you need to look at a diode for it to be picked up reliably

    0.5-3.5s 
    mean: 2.7s
  • nekrodezynfekator. The paper uses AMUSE for BSS. I tried to run it from ICALAB, but it's too old for my Matlab 2015b and there're no .m sources to rebuild the .p files.
    Did you you AMUSE and if so - did you manage to run it in Matlab?
    Thx,
    D
  • I have the same problem. ICALAB is too old for my matlab. I didn't implement any BSS algorithm.
  • wjcroftwjcroft Mount Shasta, CA
    edited February 2016
    Found the Youtube demo of the paper mentioned, plus other links to this paper. They are achieving an average of 50 to 60 characters per second (cps) recognized. But I think that is in controlled trials where they are pushing for speed; in practical usage the cps would likely be more leisurely. Nonetheless this is a major advancement in SSVEP technology. Funding for this research came from China as well as several U.S. agencies: DARPA, Army, Navy. So perhaps there will be open source versions of this speller showing up eventually.



    https://www.google.com/search?q="High-speed+spelling+with+a+noninvasive+brain–computer+interface"

    http://english.cas.cn/newsroom/research_news/201511/t20151102_155042.shtml

  • Note that this is a synchronous system (i.e., you know precisely when the target begins to flash and know the user watches it), otherwise you loose the phase information. Not sure how practical this is.
  • wjcroftwjcroft Mount Shasta, CA
    > Note that this is a synchronous system...

    For those who are not completely "locked in", in other words, have some muscles they can control -- it seems to me that the speller could be setup such that a voluntary muscle contraction could be used to trigger each letter. The user would look at the key, then contract. Then the SSVEP would commence to recognize that letter. Pretty simple and similar to the way we use fingers on keyboards. (Hunt and peck typers.)

    Even in locked in cases, I think eye-blink could be used.

  • The pnas paper verified the method for accurately generating arbitrary frequencies and phase on a monitor with a 60 Hz refresh rate (as opposed to LEDs) using the following formula:

    s(f,theta,i) = 1/2 * {1 + sin(2πf * (i / RefreshRate) + theta)}, where:
    s(f,theta,i) is the stimulus sequence corresponding to frequency f and phase theta for frame index i and RefreshRate is the refresh rate of the monitor (e.g., 60 Hz) modulating the luminance of the screen.

  • Which pin on my OpenBCI I should use for re-reference?
  • wjcroftwjcroft Mount Shasta, CA
    Marcin, hi.

    @qwer1304 wrote:

    "What's your EEG montage (electrode locations, reference, ground, re-reference)?"

    He was just asking if you had performed any re-referencing after the EEG was collected from your original reference. It's an optional step and not needed in your case since you already have found an optimal reference location for your SSVEP protocol. I assume after trying a number of reference locations.

    Re-referencing is not associated with a 'pin' or amplifier channel. It's a software operation performed on the original data collected.

    http://sccn.ucsd.edu/wiki/Chapter_04:_Preprocessing_Tools#Re-referencing_the_data

    William
  • Hi
    I faced to this article :"Fully Online Multicommand Brain-Computer Interface with
    Visual Neurofeedback Using SSVEP Paradigm"
    but i cant find any codes for it.
    can somebody help me?


  • Can any body please help me
    I want to know how can we make flickering speller to be presented on monitor screen?
    Any help, suggestions or demo codes will be appreciated.

    Thanks
  • wjcroftwjcroft Mount Shasta, CA
    Just ran across this BCI group in Japan with many SSVEP and cVEP (code-modulated VEP) Youtube demo videos.

    http://about.bci-lab.info/demos

    In addition to VEP demos, they include some BCIs based on auditory or tactile stimulation.

    One of their latest cVEP projects is nominated in the 2016 BCI-Awards,

    http://www.bci-award.com/

Sign In or Register to comment.