The various versions of the data files - how to work with them
Hi,
I acquired my OpenBCI Cyton, Daisy, and Headset back in May of 2017. I am working with the files in MATLAB and want to ensure that I am working with the raw data, not compressed data.
I am working with files from August 2017 (OpenBCI-RAW-2017-12-25_16-55-58.txt). In the header I am to see that they were recorded at 250Hz. If memory serves me correct, limitations in the Bluetooth bandwidth required that the Cyton use an algorithm to encode the samples when creating these files. What was the equation / system?
In September 2017, I switched to the BDF format. I know that I can use the program EDF Browser to convert the files. Can we do this using the OpenBCI software yet?
In December 2017 I switched back to the RAW format (OpenBCI-RAW-2017-12-25_16-55-58.txt), which now shows a sample rate of 125Hz. I believe these files were not using any encoding?
During the same time I also used the CSV format (SDconverted-2017-12-25_19-40-26.csv), and I cannot recall what needs to be done with these. I have to convert these using the OpenBCI interface, correct? Were these the actual recorded values?
I later switched to writing the data directly to the MicroSD card on the Cyton. Much better! I use the OpenBCI interface to convert the file. I have already established decoding routines for this file format that I believe are accurate (here).
Thanks,
Don
I acquired my OpenBCI Cyton, Daisy, and Headset back in May of 2017. I am working with the files in MATLAB and want to ensure that I am working with the raw data, not compressed data.
I am working with files from August 2017 (OpenBCI-RAW-2017-12-25_16-55-58.txt). In the header I am to see that they were recorded at 250Hz. If memory serves me correct, limitations in the Bluetooth bandwidth required that the Cyton use an algorithm to encode the samples when creating these files. What was the equation / system?
In September 2017, I switched to the BDF format. I know that I can use the program EDF Browser to convert the files. Can we do this using the OpenBCI software yet?
In December 2017 I switched back to the RAW format (OpenBCI-RAW-2017-12-25_16-55-58.txt), which now shows a sample rate of 125Hz. I believe these files were not using any encoding?
During the same time I also used the CSV format (SDconverted-2017-12-25_19-40-26.csv), and I cannot recall what needs to be done with these. I have to convert these using the OpenBCI interface, correct? Were these the actual recorded values?
I later switched to writing the data directly to the MicroSD card on the Cyton. Much better! I use the OpenBCI interface to convert the file. I have already established decoding routines for this file format that I believe are accurate (here).
Thanks,
Don
Comments
In digging around on the forums, I found a link that helps describe how to handle the RAW files generated when using the Daisy board:
Our 16 channel system allows for control of individual settings for all
16 channels, and data is retrieved from both ADS1299 IC at a rate of
250SPS. The current bandwidth limitations on our serial radio links limit
the number of packets we can send. To solve for this, we are sending
data packet at the same rate of 250SPS, and alternating sample packets
between the on Board ADS1299 and the on Daisy ADS1299. The method takes
an average of the current and most recent channel values before sending
to the radio. The first sample sent will be invalid because there is no
previous sample to average it with. After this, on odd sample numbers, the Board ADS1299 values are sent, and on even sample numbers, the Daisy ADS1299 samples are sent.
In what I see / understand, when I read the above and review the files, the system records samples from the Daisy board on even samples, and from the Cyton on odd samples. On each sample line in the file, the first 8 are always from the Cyton and the last 8 from the Daisy board. On even samples the first 8 values are simply a repeat of the prior values - the last 8 are updated from the Daisy board. On odd samples, the first 8 values are updated from the Cyton - the last 8 are simply repeated from the prior values.
Question: Does that not mean that my actual sample rate is only 125Hz?
While I see that there is a routine that you can follow to up-sample, my thoughts are that this is just a smoothing routine and will not serve to actually reconstruct the missing samples.
Thanks,
Don
Thank you! I have updated my website with a script to convert from when we are using the Bluetooth data capture method. I appreciate your knowledge and guidance, could I ask you to review it for me and advise as per accuracy?
The section is in the notes just below the picture of the T8 electrode signal after data conditioning. I only provided sample code to convert, condition, and separate the EEG data into separate variables. www.generalfox.com/EEG_Research
Of course anyone willing to review and comment is welcome to offer comments!
Thanks,
Don