Timestamps to sync stimulus and EEG vs external hardware trigger

skiria8skiria8
edited December 2018 in Cyton
I'm working on a P300-based appilcation where data should be synced with the stimulus displayed on the screen. I can't use hardware triggers.
I read the EEG data from the serial port and have a high precision timmer that logs the exact time when the stimulus is displayed. I can also log the exact time when the first eeg data is received or when the request to start openBCI is sent (after writting 's' to the serial port to start OpenBCI). 

What would be the best way to sync eeg with the stimulus? 
Is it possible to get the internal clock of OpenBCI for each sample? Or at certain controlled times during the session (I see some sort of timestamp at Cyton data format but I am not sure how to use it, any help will be appreciated if that's the best method). 
«1

Comments

  • wjcroftwjcroft Mount Shasta, CA
    Skiria, hi.

    So it sounds like some stimulus is presented on a screen to the subject. Could this same screen or another monitor, have a section of the screen (say lower right corner), where a photo sensor is attached with tape? Then that light detection can be picked up by an analog channel on the Cyton,



    [using SDK command '/2', and one of the analog pins]


    Regards,

    William

  • wjcroftwjcroft Mount Shasta, CA
    edited December 2018
    To clarify the Adafruit image. You would connect the positive voltage line to the Cyton Vdd pin socket on J3. Ground to Gnd, and your photocell analog output to D12. I assume your Cyton has the female header block soldered to J3.

    The image shows a 'breadboard', but the circuit is so simple, it could be soldered together, and then the wires taped or shrink-wraped, so as to not contact each other. Ask your local electronics guy to do it for you.

    To plug into the Cyton sockets, your guy should have some leads like this, or you can cut some of these to make your own. For example, solder your circuit from above to 3 copper stranded wires of say 4 to 6 feet. Splice on 3 male connectors to the bare ends, insulate with tape or shrink-wrap. Label each wire so you know the correct destination. Then plug into Cyton J3.


    We don't know the precise analog voltages you will get, but you can determine that by experimentation. You should be able to distinguish the 'off' state of your LCD display trigger, from the 'on' state, by different voltages. Pick a threshold value that guarantees trigger when illuminated.

  • skiria8skiria8
    I can't use hardware markers, it's one of the limitations I have. I was looking for a solution that synchronizes EEG and stimulus via software triggers or timestamps.
    I am using high precision timers to control and log the stimulus.
  • wjcroftwjcroft Mount Shasta, CA
    The question is, even if you have access to the time markers in Cyton, how do you sync those with your known time event?  You need some sort of correlation. I don't believe the time mark feature of Cyton is going to solve your sync issue.

    I don't understand the 'limitation'. Can you elaborate? You are not altering any lab equipment, just using adhesive tape to attach a photoresistor to your stimulus screen.

    LabStreamingLayer is intended provide such timing correlations. But in my opinion, the optical trigger is your surest bet. Do you understand the simple circuit I outlined? It would take less than an hour to solder together.
  • skiria8skiria8
    The application will be distributed to users that can buy OpenBCI but can't solder the photoresistor.

    I am currently using LSL but for simplicity for the users I'd rather have a standalone app.

    I can log the time when start is sent to OpenBCI and when first sample is received. Not sure if that will help. I also have the packet counter.
    Maybe latency could be estimated sending several pulses (if that's possible) to the board and comparing it with the log timestamps (from the computer)? Could that compensate jitter?
    Any idea how openvibe or LSL works?
  • wjcroftwjcroft Mount Shasta, CA
    edited December 2018
    These guys wrote a white paper, but use an external parallel port to CREATE a trigger signal channel,





    ----

    re: "but can't solder the photosensor". I don't know how many of these you are planning to distribute. But electronic technicians that work on contract are available. If you made a bulk order of say 10 of the cables, the cost might come down to around $20 to $50 each, parts plus labor. As I said, someone who knows how to use a soldering iron could assemble in less than an hour. A skilled tech could assemble them probably in less than 20 minutes (each).
  • skiria8skiria8
    edited December 2018
    I'm trying different appraoches for a software trigger but I also built the photoresistor to measure latency and jitter. 

    If I understood it correctly, I only need to send /2 to the board and I should read the trigger. Then, what would be the easiest way to test it? What is the best way to select the threshold to detect when the stimulus is On and Off? 
    I used a 200Ohm resistor, not sure if it's the most appropiate. 
  • wjcroftwjcroft Mount Shasta, CA
    Skiria, hi.

    I would suggest taping it up to the right corner of your primary or secondary LCD display. Then alternately turning on/off a square area under the sensor. So you have both 'black' and 'white' to compare to. See what kind of values you get, then selecting a threshold value should be easy. 

    For testing, if you don't want to write a program, just use an image drawing program to create two different jpeg or png files, white.png and black.png.

    You will probably find 'black' is not totally off. Nor 'white' totally on. But selecting the appropriate threshold value, be able to distinguish.

    Why did you choose a 200 ohm resistor? The Adafruit article suggests either 10K or 1K. Are you using the same photoresistor as Adafruit? If not, have you measured the resistance of it with an ohmmeter, both illuminated and covered up? 

    Regards,
  • skiria8skiria8
    Hi wjcroft, thanks for your quick reply

    No, I'm not using the same Adafruit photoresistor. I am measuring 16kOhm for a black image on screen and 2kOhm for a white image. Would 2k be a good resistor? I'm not sure how that is calculated. 

    Another question, do you know how to identify the firmware version of the OpenBCI board? According to the OpenBCI tutorial (http://docs.openbci.com/Tutorials/06-External_Trigger_Cyton_Example) it requires v 3.1.0 or later, I'm not sure exactly what version of v3 I have. I have the Arduino IDE so I can update to a newer version if required. 
  • wjcroftwjcroft Mount Shasta, CA
    Use the formula on this page,


    Vo = Vcc ( R / (R + Photocell) )

    Vcc = 3.3V

    And plug in your pull down resistor, either 2K or 10K, (as R), and 16K or 2K for your photoresistor value. That will let you calculate the Vo resulting voltage output that will be measured by your D12 analog input pin.

    re: checking Cyton firmware version, I answered that on the other thread. You can check it with the terminal emulator built into the Arduino IDE.

    Regards,


  • skiria8skiria8
    Not working. It seems like it's not reding from the photoresistor. It shows very small pulses 0 to max. 

    I have firmware v3.1and I tried testing the digital input with OpenBCI GUI and pressing the PROG button (as explained in the tutorial). It just shows random numbers. Pushing the button doesn't seem to do anything.Any ideas?
  • wjcroftwjcroft Mount Shasta, CA
    edited February 2019
    With the PROG button, did you set board mode '/2' ?

  • wjcroftwjcroft Mount Shasta, CA
    edited February 2019
    The two bytes, / and 2 must be sent within a short interval. Otherwise the Cyton times out and throws out the command bytes.

  • wjcroftwjcroft Mount Shasta, CA
    Are you using the Widget to set board mode? That is preferred to sending the commands manually.

  • skiria8skiria8
    Yes, I'm using the widget in the OpenBCI GUI.
  • wjcroftwjcroft Mount Shasta, CA
    Are you using the latest GUI / Hub? Have you tried running it from Processing IDE?

  • wjcroftwjcroft Mount Shasta, CA
    This issue was fixed with latest release,


  • skiria8skiria8
    edited January 2019
    I downloaded the latest release of the GUI and it's still doing the same. 

    Since the GUI doesn't seem to work, What would be the easiest way to try it without using the GUI?  

    In my standalone app, I read the data from the serial port. I can read the data in the 3 AUX bytes, but how can I change the mode to '/2'?  If I change to mode '/2" via Arduino, the board powers off and it seems that it returns to default mode when I turn it on. Is that expected?

  • skiria8skiria8
    edited January 2019
    In fact, it seems the issue might be either the board or the GUI not changing to Digital mode.

    I did the following test: 
    1) turn on digital mode via GUI (v 4.0.3 windows 64bit)
    2)  press PROG button for ~0.5sec
    3) Stop streaming
    The values showing on the GUI are kinda random. When I check the .txt file, I see mostly the values to 0 

    These are the values I get
     0.032, 0.030, 0.974
    0.000, 0.000, 0.000
    0.000, 0.000, 0.000
    0.000, 0.000, 0.000
    0.000, 0.000, 0.000
    0.000, 0.000, 0.000
    0.000, 0.000, 0.000
    0.000, 0.000, 0.000
    0.000, 0.000, 0.000
    0.000, 0.000, 0.000
    0.028, 0.030, 0.976  (10 samples of 0, one of non-zeros, very similar values)

    Any suggestions?
  • skiria8skiria8
    edited January 2019
    I finally figured out the problem and it works! Version 3.x GUI.

    To put the board on analog mode, what do you recommend?
    - modify the DefaultBoard example & upload it via Android IDE to have the board on Analog mode by default
    Or
    - send '/2' at the begining. I assume the process should be: OpenPort > send '/2' -> wait for success message (documentation says I should receive a message but I just receive 'success'?) -> send 'b' to start streaming ->read data. Does that seem correct?
  • wjcroftwjcroft Mount Shasta, CA
    edited January 2019
    Skiria, that's great.

    Can you elaborate on your "figured out the problem", in case others get stuck in a similar situation?

    Either way you mention can be used to get into analog mode. Your choice.

    What values are you seeing for light vs. dark in the Aux data? And can you pick a suitable threshold?

    Regards,

    William
  • skiria8skiria8
    Ups, I realized I forgot to reply.

    The latest openBCI GUI v.3.3.1 worked for me. The beta version (v 4.0.2 and 4.0.3) did not work. I was getting random values, both for digital and analog modes.

    The other problem I had was that the board was resetting and switching to default mode(accelerometer mode). I have a standalone app so I send now the '/2' command and wait to receive the 'success' message before starting the stream.

    With a resistor of 10k, I get values ~300 for a black image and ~800 for a white image. I set the threshold to 550.
  • wjcroftwjcroft Mount Shasta, CA
    @skiria8, thanks much.

    Can you add your comments (such as OS version) to this issue I just opened?


    Thanks,

    William
  • QozQoz UCSD
    @skiria8 if you wanted a good software solution for markers, I recommend the lab streaming layer (LSL). It's very easy to create an LSL stream from the Cyton using the OpenBCI GUI or OpenBCI_LSL, and you can acquire that data stream online for real-time P300-based applications. LSL has wrappers for various common languages, so you could create the P300 task itself using a language that has support for LSL (such as Python), and send a marker via LSL which will contain accurate timestamps in order to synchronize with your EEG data. For collecting offline, the LabRecorder actually handles this synchronization for you which makes piloting designs very straightforward. I have personally used both the Ganglion and the Cyton to create real-time BCIs using LSL.
  • wjcroftwjcroft Mount Shasta, CA
    @Qoz, thanks. I did mention LSL to Skiria on previous comments of this thread. Perhaps not giving the best example links as I could. 

    Would you have a link to some of your P300 BCIs on Github or elsewhere? Or have other links?

    Question: how does the LSL system and 'marker' technology account for the various (and variable) buffering delays that are present in the Cyton and Ganglion radio channels, serial ports, OS driver levels, etc. In the case of Cyton, it is sending radio packets every 4 milliseconds, containing the next 8 ch sample (250 samples/sec.) However those radio packets then get turned into USB traffic at the dongle, and are further buffered by the VCP Virtual Com Port, serial port emulation.

    Is there some type of calibration phase where these delay factors are quantized? (Such a calibration could not be guaranteed to maintain in light of varying CPU loads.)

    I do understand that LSL uses a time stamping mechanism, but these time stamps only occur after the final serial port byte stream reaches the app. By that time the Cyton channel samples in the packet will be at least 4 ms or 8 ms later than app "real-time", and possibly more than that. In the case of Ganglion, the radio packets are sent about 100x per second, each containing about two 4 channel samples. So 10 ms per packet, but 5 ms per sample with a 5 ms jitter.

    I'm just curious how LSL sorts all this out. Perhaps there are also tutorials online showing P300 examples with markers.

    Regards,

    William

  • QozQoz UCSD
    @wjcroft Sorry, I did not thoroughly read this thread, I was browsing the forums on my phone and was just skipping through. I apologize for the lack of effort on my part.

    I have not made the full extent of my data and code public, but I plan on doing so as part of a larger project in the near future, and will make a thread when that time comes.

    To answer your questions, which are key concerns, LSL relies purely on the timestamp mechanism you correctly described. In order to correct for the delay created by the various factors you mentioned, I conducted tests using a photosensor connected to the Ganglion to determine the total latency of the system. I then created a P300-based BCI using the Ganglion, correcting for the delay programmatically given the results of my test, not using some feature of LSL. As you correctly pointed out, there is a good amount of jitter in the Ganglion's signals, and I have found this difficult to deal with, which is why I started working with the Cyton. Unfortunately, I have not gone through the same testing procedure with the Cyton as I have been using it for more rudimentary and trivial projects where timing is not a massive concern (asynchronous alpha detection). That being said, this is one of the next items on my chopping block as I am looking to use the Cyton in various academic and professional capacities.

    Assuming you have control over the environment you're collecting data in, the delay should be relatively constant with about 1ms of jitter, which is tolerable. I really appreciate your comments and concerns as these are issues I'm actively tackling, and it's nice to know there seems to be a demand for solutions.

    All this to say, I believe hardware solutions for markers are superior when available, although a photosensors and other hardware solutions for marker detection are not feasible in certain projects (such as having a mobile device for stimulus display).

  • wjcroftwjcroft Mount Shasta, CA
    edited February 2019
    In a previous post on this thread I mentioned the Adafruit light detection circuit using a photoresistor,


    This works and has decent response. However I recently found out that photoresistors can have a slower response time / rise time than you might expect. In the milliseconds range, possibly around 5 or 10 ms. Cyton sample rate is 250 sps, which is 4 ms per sample. So I would caution anyone using the Adafruit suggested circuit, that you may have to subtract back in time one or two samples to have optimal alignment of your trigger.

    It turns out there are ways to bias photodiodes (vs photoresistors), to get much faster response times, in the order of a few microseconds. One particular integrated device, the TSL257, has a response of 200 microseconds, or 0.2 milliseconds; still adequate for EEG ERP. Additionally this device would connect with no other components, just the three wires from the Cyton: Vdd, Gnd, and sensor output. It's also cheap, less than $3. Looks like a viable alternative over the Adafruit suggestion.

    Datasheet:


    Page at Digikey:


    This device is VERY light sensitive, it may be necessary to reduce the amount of light reaching the device from your screen or room. Check what you get back testing for black and white levels. If it is getting too much light, then the 'black' level might appear similar to be 'white', meaning the chip is saturating. In this case you can reduce the light reaching the chip. This could be done by wrapping it with dark tape, with perhaps a small pinhole facing the screen.

    Regards,

    William
  • skiria8skiria8
    This is very useful information. Thanks so much for updating the thread! 

    Regarding the response time of the photoresistor: Do you know if the response time is constant? 
    If it's constant (for example with a latency of 5ms but no jitter), it's easy to compensate just subtracting 5ms. Even without subtracting it, although ERPs will be delayed 5ms, it should not affect performance as training and inference is done using the same photoresistor. If it has jitter, then it can be a serious problem. 

    Have you tried the TSL257? 
  • wjcroftwjcroft Mount Shasta, CA
    Yes, photoresistor response time is constant, so you can compensate. I don't know the exact response time, you can look that up in the datasheet for your sensor. 
  • skiria8skiria8
    I bought the TSL257  but I'm getting very high voltages (bigger than 1020, max value in OpenBCI). I can only get values around 950 if I completely cover it and apply some pressure. Do you have any idea if this can be fixed? Maybe with an extra resistor (althought Im not sure if response time will change then)?
Sign In or Register to comment.

© Vanilla Keystone Theme 2026