Sending EEG and accelerometer data to LSL through BrainFlow [resolved]

superhenrikkesuperhenrikke Norway
edited April 2021 in General Discussion

Hello,
I have an 8 channel Cyton board and I want to send both EEG and accelerometer data to LSL. (I have multiple other sensors as well, and I want all of them synchronized). Previously we used pyOpenBCI and OpenBCI_LSL, but since these are deprecated, I wrote a new code with BrainFlow. The code seems to give the correct data, but I wanted to do a sanity check of the code before starting to collect any actual data. Code below:

import argparse
import time
import numpy as np

import brainflow
from brainflow.board_shim import BoardShim, BrainFlowInputParams, LogLevels, BoardIds
from brainflow.data_filter import DataFilter, FilterTypes, AggOperations

from pylsl import StreamInfo, StreamOutlet

BoardShim.enable_dev_board_logger()

params = BrainFlowInputParams()
params.serial_port = 'COM3'

board = BoardShim(BoardIds.CYTON_BOARD.value, params) # added cyton board id here
board.prepare_session()

board.start_stream()
board.config_board('/2')  # enable analog mode only for Cyton Based Boards!    # added from example in docs
BoardShim.log_message(LogLevels.LEVEL_INFO.value, 'start sleeping in the main thread') # is this needed? 

# define lsl streams
# Scaling factor for conversion between raw data (counts) and voltage potentials:
SCALE_FACTOR_EEG = (4500000)/24/(2**23-1) #uV/count
SCALE_FACTOR_AUX = 0.002 / (2**4) 
# Defining stream info:
name = 'OpenBCIEEG'
ID = 'OpenBCIEEG'
channels = 8
sample_rate = 250
datatype = 'float32'
streamType = 'EEG'
print(f"Creating LSL stream for EEG. \nName: {name}\nID: {ID}\n")

info_eeg = StreamInfo(name, streamType, channels, sample_rate, datatype, ID)
chns = info_eeg.desc().append_child("channels")
for label in ["AFp1", "AFp2", "C3", "C4", "P7", "P8", "O1", "O2"]:
    ch = chns.append_child("channel")
    ch.append_child_value("label", label)

info_aux = StreamInfo('OpenBCIAUX', 'AUX', 3, 250, 'float32', 'OpenBCItestAUX')
chns = info_aux.desc().append_child("channels")
for label in ["X", "Y", "Z"]:
    ch = chns.append_child("channel")
    ch.append_child_value("label", label)

outlet_aux = StreamOutlet(info_aux)
outlet_eeg = StreamOutlet(info_eeg)

# construct a numpy array that contains only eeg channels and aux channels with correct scaling
# this streams to lsl
while True:
    print('get continiuous Data From the Board')
    data = board.get_current_board_data(1) # this gets data continiously

    # create scaled data
    scaled_eeg_data = data[1:9]*SCALE_FACTOR_EEG
    scaled_aux_data = data[10:13]*SCALE_FACTOR_AUX
    print(scaled_eeg_data)
    print(scaled_aux_data)
    print('------------------------------------------------------------------------------------------')

    outlet_eeg.push_sample(scaled_eeg_data)
    outlet_aux.push_sample(scaled_aux_data)


#When using the Python / LSL sending program, you must put the Cyton into either 'digital' or 'analog' (Aux mode), via a serial port SDK command:
#https://docs.openbci.com/docs/02Cyton/CytonSDK#board-mode
#The default "board mode", is to send the Accelerometer data at reduced rate.

Questions:
1. Line 20. Is it correct to put Cyton in Analogue mode? Why? Will this send accelerometer data at the correct rate (as opposed to the reduced rate mentioned in this post)
2. Line 58 and 59. Here I only include the 8 EEG channels, and the 3 accelerometer “channels”. Will I lose any info when not including the other “rows”/channels/packets/bytes/header/footer (not sure what the correct terminology is here)? In other words, is it safe to “discard” the rest?
3. Am I getting X, Y, X accelerometer data (in that order)?
4. Are the scale factors correct? E.i., will collect EEG data have unit uV and collected accelerometer data have unit m/s2 ?
5. Line 21. Is this line of code needed?

Thanks!
Regards, Henrikke

Comments

  • retiututretiutut Louisiana, USA
    edited April 2021

    Lines 58 and 59 are probably not correct.... I think....

    Take a look at this python code used for testing purposes: https://github.com/OpenBCI/OpenBCI_GUI/blob/master/Networking-Test-Kit/LSL/brainflow_lsl.py

    I think you should be using get_exg_channels() and get_accel_channels() to reliably fetch data instead of hardcoding these numbers:

     scaled_eeg_data = data[1:9]*SCALE_FACTOR_EEG
     scaled_aux_data = data[10:13]*SCALE_FACTOR_AUX
    

    @Andrey1994 anything else to mention or comment?

    After you get this working, I would like to include this code with the Public GUI in the networking test folder. Thanks for sharing!

  • retiututretiutut Louisiana, USA
    edited April 2021

    Line 20. Is it correct to put Cyton in Analogue mode? Why? Will this send accelerometer data at the correct rate (as opposed to the reduced rate mentioned in this post)

    This is also incorrect. Default mode for Cyton uses Accelerometer data. You do not need Line 20 based on what you have shared.

  • superhenrikkesuperhenrikke Norway
    edited April 2021

    @retiutut said:
    Lines 58 and 59 are probably not correct.... I think....

    Take a look at this python code used for testing purposes: https://github.com/OpenBCI/OpenBCI_GUI/blob/master/Networking-Test-Kit/LSL/brainflow_lsl.py

    I think you should be using get_exg_channels() and get_accel_channels() to reliably fetch data instead of hardcoding these numbers:

    Thanks! I changed this. Updated code below.

    import argparse
    import time
    import numpy as np
    
    import brainflow
    from brainflow.board_shim import BoardShim, BrainFlowInputParams, LogLevels, BoardIds
    from brainflow.data_filter import DataFilter, FilterTypes, AggOperations
    
    from pylsl import StreamInfo, StreamOutlet
    
    BoardShim.enable_dev_board_logger()
    
    params = BrainFlowInputParams()
    params.serial_port = 'COM3'
    
    board = BoardShim(BoardIds.CYTON_BOARD.value, params) # added cyton board id here
    board.prepare_session()
    
    board.start_stream()
    board.config_board('/2')  # enable analog mode only for Cyton Based Boards!    # added from example in docs
    #BoardShim.log_message(LogLevels.LEVEL_INFO.value, 'start sleeping in the main thread') # is this needed? 
    
    eeg_chan = BoardShim.get_eeg_channels(BoardIds.CYTON_BOARD.value)
    aux_chan = BoardShim.get_accel_channels(BoardIds.CYTON_BOARD.value)
    print('EEG channels:')
    print(eeg_chan)
    print('Accelerometer channels')
    print(aux_chan)
    
    # define lsl streams
    # Scaling factor for conversion between raw data (counts) and voltage potentials:
    SCALE_FACTOR_EEG = (4500000)/24/(2**23-1) #uV/count
    SCALE_FACTOR_AUX = 0.002 / (2**4) 
    # Defining stream info:
    name = 'OpenBCIEEG'
    ID = 'OpenBCIEEG'
    channels = 8
    sample_rate = 250
    datatype = 'float32'
    streamType = 'EEG'
    print(f"Creating LSL stream for EEG. \nName: {name}\nID: {ID}\n")
    
    info_eeg = StreamInfo(name, streamType, channels, sample_rate, datatype, ID)
    chns = info_eeg.desc().append_child("channels")
    for label in ["AFp1", "AFp2", "C3", "C4", "P7", "P8", "O1", "O2"]:
        ch = chns.append_child("channel")
        ch.append_child_value("label", label)
    
    info_aux = StreamInfo('OpenBCIAUX', 'AUX', 3, 250, 'float32', 'OpenBCItestAUX')
    chns = info_aux.desc().append_child("channels")
    for label in ["X", "Y", "Z"]:
        ch = chns.append_child("channel")
        ch.append_child_value("label", label)
    
    outlet_aux = StreamOutlet(info_aux)
    outlet_eeg = StreamOutlet(info_eeg)
    
    # construct a numpy array that contains only eeg channels and aux channels with correct scaling
    # this streams to lsl
    while True:
        print('get continiuous Data From the Board')
        data = board.get_current_board_data(1) # this gets data continiously
    
        # create scaled data
        scaled_eeg_data = data[eeg_chan]*SCALE_FACTOR_EEG
        scaled_aux_data = data[aux_chan]*SCALE_FACTOR_AUX
        print(scaled_eeg_data)
        print(scaled_aux_data)
        print('------------------------------------------------------------------------------------------')
    
        outlet_eeg.push_sample(scaled_eeg_data)
        outlet_aux.push_sample(scaled_aux_data)
    
    
    #When using the Python / LSL sending program, you must put the Cyton into either 'digital' or 'analog' (Aux mode), via a serial port SDK command:
    #https://docs.openbci.com/docs/02Cyton/CytonSDK#board-mode
    #The default "board mode", is to send the Accelerometer data at reduced rate.
    

    @retiutut said:

    Line 20. Is it correct to put Cyton in Analogue mode? Why? Will this send accelerometer data at the correct rate (as opposed to the reduced rate mentioned in this post)

    This is also incorrect. Default mode for Cyton uses Accelerometer data. You do not need Line 20 based on what you have shared.

    I tried to remove this line, however this resulted in a traceback and I wasn't able to send data to LSL. The data type seems to be the same () regardless of whether it's in analogue mode or not. Traceback (most recent call last): File "C:\.......\stream_eeg_brainflow_lsl_v2.py", line 81, in <module> outlet_eeg.push_sample(scaled_eeg_data) File "C:\........\Anaconda3\envs\lslenv\lib\site-packages\pylsl\pylsl.py", line 450, in push_sample handle_error(self.do_push_sample(self.obj, self.sample_type(*x), TypeError: only size-1 arrays can be converted to Python scalars
    I based the script of the example here (https://brainflow.readthedocs.io/en/stable/DataFormatDesc.html#openbci-specific-data), which also put Cyton in analogue mode. This was also the case in the post mentioned (https://openbci.com/forum/index.php?p=/discussion/2484/different-sampling-rate-between-eeg-signal-and-aux-signal-using-lsl-from-gui-workaround). From this (https://docs.openbci.com/docs/02Cyton/CytonExternal) I understand the difference between analog and digital mode (besides the obvious analog vs digital) is that we also read accelerometer data at 250 Hz, instead of at 25 Hz. Is there no other difference?

  • I also changed the sampling rate of the LSL accelerometer stream to 25 Hz (line 49 above), but this did not change anything. Still the same traceback.

  • Also, I noticed the documentation says EEG data are returned as uV (https://brainflow.readthedocs.io/en/stable/DataFormatDesc.html?highlight=uV#units-of-measure), so I assume this means the scale factor is not needed? What is the unit for accelerometer data then?

  • retiututretiutut Louisiana, USA

    You need the Cyton in default mode to get values from the Accelerometer. Analog and Digital signals are not what you want.

  • retiututretiutut Louisiana, USA
    edited April 2021

    The error is likely here. data[eeg_chan] returns an array and you are multiplying an entire array by 1 number.

        scaled_eeg_data = data[eeg_chan]*SCALE_FACTOR_EEG
        scaled_aux_data = data[aux_chan]*SCALE_FACTOR_AUX
    
  • retiututretiutut Louisiana, USA
    edited April 2021

    Sorry. That's not the fix. It was kind of hard to read the error log as a paragraph. I see now that Python allows multiplying an array by a scalar.

    Please review the following code that helps transcribe data into the chunk format for LSL. I think it will fix the error that happens when trying to push_sample().

    for i in range(len(data[0])):
                queue.put(data[:,i].tolist())
            elapsed_time = local_clock() - start_time
            required_samples = int(srate * elapsed_time) - sent_samples
            if required_samples > 0 and queue.qsize() >= required_samples:    
                mychunk = []
    
                for i in range(required_samples):
                    mychunk.append(queue.get())
                stamp = local_clock() - fw_delay 
                outlet.push_chunk(mychunk, stamp)
                sent_samples += required_samples
            time.sleep(1)
    

    Found here https://github.com/OpenBCI/OpenBCI_GUI/blob/master/Networking-Test-Kit/LSL/brainflow_lsl.py

    I think this can be a great resource for users interested in LSL! Once you get it working, I would still like to upload a version of this code to an OpenBCI repo for others to use.

  • I will have another look and go at it. I'll ask the LSL community as well, and see if they have any suggestions.
    Thanks! No problem uploading a version of the code to a OpenBCI repo, but I have to get it working first :)

  • @retiutut said:
    You need the Cyton in default mode to get values from the Accelerometer. Analog and Digital signals are not what you want.

    When I tested analog mode (the code in the original post), the accelerometer data seemed to be correct. See photo:

    The following photo is from an iteration before this, where I used only data = board.get_current_board_data(1) in the while loop and streamed 24 channels to LSL.


    So, is this accelerometer data incorrect, and what is wrong with it?

  • retiututretiutut Louisiana, USA
    edited April 2021

    I promise that you need the Cyton in default mode to check the Accelerometer.

    https://docs.openbci.com/docs/02Cyton/CytonDataFormat#firmware-version-200-fall-2016-to-now-1

  • Hm, ok. When in default mode I still get the traceback I mentioned (also below). I don't know why or how to fix it. I'm sorry, but I don't know what to make of the example that sends data in chunks. I would like to send data continuously.

    Traceback (most recent call last): File "C:\.......\stream_eeg_brainflow_lsl_v2.py", line 81, in <module>
        outlet_eeg.push_sample(scaled_eeg_data) 
    File "C:\........\Anaconda3\envs\lslenv\lib\site-packages\pylsl\pylsl.py", line 450, in push_sample 
        handle_error(self.do_push_sample(self.obj, self.sample_type(*x),
     TypeError: only size-1 arrays can be converted to Python scalars
    
  • retiututretiutut Louisiana, USA
    edited April 2021

    That error is not related to setting the board mode, it's with transcribing the data format to LSL.

    At this point, I may try to edit the code you have shared and then upload something that should work for you. I wanted to see if you could fix it first as a learning experience.

    Let me see if I can get this working for you! We are here to help.

    :smile:

  • That would be great!
    I really wanted to fix it as a learning experience as well. ;) This is basically what I have been trying to do for the past four days.

  • I was made aware of that get_current_board_data() doesn't remove samples from the buffer, so I'm pulling the same sample repeatedly. They suggested to use get_board_data() . However, when I use get_board_data() I'm not getting any data. It is all empty vectors.

  • retiututretiutut Louisiana, USA

    @superhenrikke I am dealing with a family emergency so I may not be able to make this quickly. This is on my to-do list.

  • Thanks, and thanks for letting me know. Hope all goes well with you and your family!

  • I managed to get some help and we used the code example you provided to send data in chunks. So the following script works.

    import argparse
    import time
    import numpy as np
    
    import brainflow
    from brainflow.board_shim import BoardShim, BrainFlowInputParams, LogLevels, BoardIds
    from brainflow.data_filter import DataFilter, FilterTypes, AggOperations
    
    from pylsl import StreamInfo, StreamOutlet
    #from queue import Queue
    
    BoardShim.enable_dev_board_logger()
    
    params = BrainFlowInputParams()
    params.serial_port = 'COM3'
    
    board = BoardShim(BoardIds.CYTON_BOARD.value, params) # added cyton board id here
    srate = board.get_sampling_rate(BoardIds.CYTON_BOARD.value)
    board.prepare_session()
    
    board.start_stream()
    
    
    eeg_chan = BoardShim.get_eeg_channels(BoardIds.CYTON_BOARD.value)
    aux_chan = BoardShim.get_accel_channels(BoardIds.CYTON_BOARD.value)
    print('EEG channels:')
    print(eeg_chan)
    print('Accelerometer channels')
    print(aux_chan)
    
    # define lsl streams
    # Scaling factor for conversion between raw data (counts) and voltage potentials:
    SCALE_FACTOR_EEG = (4500000)/24/(2**23-1) #uV/count
    SCALE_FACTOR_AUX = 0.002 / (2**4) 
    # Defining stream info:
    name = 'OpenBCIEEG'
    ID = 'OpenBCIEEG'
    channels = 8
    sample_rate = 250
    datatype = 'float32'
    streamType = 'EEG'
    print(f"Creating LSL stream for EEG. \nName: {name}\nID: {ID}\n")
    
    info_eeg = StreamInfo(name, streamType, channels, sample_rate, datatype, ID)
    chns = info_eeg.desc().append_child("channels")
    for label in ["AFp1", "AFp2", "C3", "C4", "P7", "P8", "O1", "O2"]:
        ch = chns.append_child("channel")
        ch.append_child_value("label", label)
    
    info_aux = StreamInfo('OpenBCIAUX', 'AUX', 3, 250, 'float32', 'OpenBCItestAUX')
    chns = info_aux.desc().append_child("channels")
    for label in ["X", "Y", "Z"]:
        ch = chns.append_child("channel")
        ch.append_child_value("label", label)
    
    outlet_aux = StreamOutlet(info_aux)
    outlet_eeg = StreamOutlet(info_eeg)
    
    # construct a numpy array that contains only eeg channels and aux channels with correct scaling
    # this streams to lsl
    while True:
        data = board.get_board_data() # this gets data continiously
    
        # don't send empty data
        if len(data[0]) < 1 : continue
    
        eeg_data = data[eeg_chan]
        aux_data = data[aux_chan]
        #print(scaled_eeg_data)
        #print(scaled_aux_data)
        #print('------------------------------------------------------------------------------------------')
        eegchunk = []
        for i in range(len(eeg_data[0])):
            eegchunk.append((eeg_data[:,i]*SCALE_FACTOR_EEG).tolist()) #scale data here
        outlet_eeg.push_chunk(eegchunk)
        auxchunk = []
        for i in range(len(aux_data[0])):
            auxchunk.append((aux_data[:,i]*SCALE_FACTOR_AUX).tolist()) #scale data here
        outlet_aux.push_chunk(auxchunk)    
    
  • As you'll see I have included the scale factors, but I'm still unsure if they should be used. In the code samples (https://brainflow.readthedocs.io/) there were no scale factors, but OpenBCI docs (https://docs.openbci.com/docs/02Cyton/CytonDataFormat#interpreting-the-eeg-data) says they need to be used. Any hint on which to use?

  • wjcroftwjcroft Mount Shasta, CA

    As you'll see I have included the scale factors, but I'm still unsure if they should be used.

    The Brainflow examples are correct. This scaling is ALREADY being done inside Brainflow library.

    William

  • retiututretiutut Louisiana, USA
    edited April 2021

    @superhenrikke said:
    As you'll see I have included the scale factors, but I'm still unsure if they should be used. In the code samples (https://brainflow.readthedocs.io/) there were no scale factors, but OpenBCI docs (https://docs.openbci.com/docs/02Cyton/CytonDataFormat#interpreting-the-eeg-data) says they need to be used. Any hint on which to use?

    @wjcroft said:

    As you'll see I have included the scale factors, but I'm still unsure if they should be used.

    The Brainflow examples are correct. This scaling is ALREADY being done inside Brainflow library.

    William

    I think we should update the Doc to say this information. I can confirm that these scale factors are declared in GUI 5.0.4 but aren't used for live data. It must be taken into account in BrainFlow. In the GUI, The ScaleFactorEEG is used when reading directly from a file saved to SD card with Cyton.

    https://github.com/brainflow-dev/brainflow/blob/8d1fb74b7f69cb65ba3835a5d6a91325c86b457f/src/board_controller/openbci/inc/cyton.h#L12

  • retiututretiutut Louisiana, USA

    CytonDataFormat Doc has been updated!

  • Great, thanks guys!!
    Best regards, Henrikke

  • Hi @superhenrikke, you may take a look at my repo: openbci-brainflow-lsl. The script allows to record both EXG and AUX data, plus you can use it with Arduino sending exernal markers. It works on Windows (I did not test it on Mac or Linux) with Cyton/Cyton + Daisy.

  • retiututretiutut Louisiana, USA
    edited May 2021

    @marles77 Nice!!!! I think we might want to put this in the official OpenBCI Github account in a new or existing repo.

    I have already included the script found in this thread in the GUI's "Networking Test Kit".

    https://github.com/OpenBCI/OpenBCI_GUI/blob/focus-widget-2021/Networking-Test-Kit/LSL/brainflow_lsl_cyton_accel.py

    @marles77 your example seems much more complete and usable.

  • marles77marles77 Poland
    edited May 2021

    Hi @retiutut

    I think we might want to put this in the official OpenBCI Github account in a new or existing repo.

    Sure, no problem.
    BTW, I found that the solution from openbci gui repo using queue module causes considerable timing problems. I don't know yet why, but it generates a ~1 sec. delay of EEG data which makes time locking almost impossible. Time.sleep()s have nothing to do with that, obviously. I performed a simple experiment with a button attached to Arduino (sending triggers via LSL as well) and electrodes placed over the first dorsal interosseous muscle to investigate whether or not it is a systematic error. Below is an averaged EMG (~100 repeats) showing that the muscle contraction is delayed in reference to button clicks.

    image

    However, when I removed queues from the code pushing chunks, muscle potentials started to match button markers quite perfectly:

    image

    This is probably fixable, but I resigned from using queue, anyway.
    Regards
    Marcin

Sign In or Register to comment.