script to convert SD card file

Is there a script available (C, python, etc) to convert the SD card file (hex format) to voltages, etc without having to use the OpenBCI GUI?

Comments

  • wjcroftwjcroft Mount Shasta, CA
    Armz, hi.

    See the code from lines 1461 to 1651 in,


    You could write a standalone Processing app with just this section. Let us know if you do that with a link to your Github.

    William

  • Hi wjcroft,
    I am also looking for a script to convert SD card files (hex) to decimal.
    Is there a new link to the above code?
    Thanks,
    Sam

  • wjcroftwjcroft Mount Shasta, CA
    edited October 2023

    Richard @retiutut, do you know what section of the GUI code now handles decoding of SD card hex data? The link above no longer works because that filename 'DataLogging.pde', no longer exists. I do see the file 'DataSourceSDCard.pde'...

    Thanks, William

  • retiututretiutut Louisiana, USA
    edited October 2021

    Correct. Let me break down what happens in the new DataSourceSDCard.pde (https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/DataSourceSDCard.pde)!

    parseRow(String[] row, int numChannels) parses the values for all channels which exist as a "row" in the CSV file: https://github.com/OpenBCI/OpenBCI_GUI/blob/4ab8cf7e024aed367e0cca07f02ad4e3b4366836/OpenBCI_GUI/DataSourceSDCard.pde#L76

    parseInt24Hex(String hex) converts the values for each channel: https://github.com/OpenBCI/OpenBCI_GUI/blob/4ab8cf7e024aed367e0cca07f02ad4e3b4366836/OpenBCI_GUI/DataSourceSDCard.pde#L287

    And then there is a Processing method called unhex(): https://processing.org/reference/unhex_.html

  • Hi,
    @retiutut, Is there any python script to do so (SD Card hex to decimal). Since direct conversion is not the actual way to do so.

  • plutopluto mars
    edited November 2021

    Hi All,
    Hope you all are doing well.
    1. I just saw a repo (depricated) named: PyOpenBCI, does this python script converts the SD Card hex code to decimal values? If yes, I would really like to know on how to do it?

    1. Also, if I just take a raw value from the SD Card hex values, what is the way to convert those hex values into the actual decimal values (since direct conversion of those hex values did not give same results as the converted files from Processing SD Card converter).

    @wjcroft, I will be grateful for your help.

  • retiututretiutut Louisiana, USA

    I have made an issue on the GUI Github repo to address this.
    https://github.com/OpenBCI/OpenBCI_GUI/issues/1014

  • plutopluto mars
    edited November 2021

    Thanks for the help @retiutut.
    As you have mentioned above, the use of parseInt24Hex(String hex), under that I could see one unhex function, can you describe what does it do?

  • plutopluto mars
    edited November 2021

    Hi,
    I have two files with me, SD Card log, and converted of the same, let us take a value in SD Card log for example:
    F260C9, this is converted as: -19954.006, HOW?
    How do I know, how this is converted into that particular value?
    Can you tell any simple method to interpret this, not specifically any code, any simple explanation is welcome enough.

    @wjcroft, please help me out on this, kinda urgent.

    Thanks & Regards,
    Pluto

  • wjcroftwjcroft Mount Shasta, CA

    @pluto, hi. I merged your new thread into this existing one. It helps to put related questions into similar threads.

    https://docs.openbci.com/Cyton/CytonDataFormat/#24-bit-signed-data-values

    William

  • Thanks @wjcroft @retiutut, I could figure out the conversion while looking at the conversion file and then comparing with original hex, got this tip from firmware that those are negative hex, and else was easy to implement, thanks a lot for your kind support.

  • plutopluto mars
    edited December 2021

    Hi,
    I am seeing that while conversion from hex to decimal in SD Card log, there is an scale factor used.
    I am aware it depends on Vref and Gain, and I have considered same as what is used by OpenBCI and having few questions about it which are as follow:

    1. Till what decimal place is the scale factor used for conversion? (Since I could see while converting from python script of mine vs converted by OpenBCI gives different results from 4th or 5th decimal places, sometimes 2nd and 3rd too)

    2. If possible, can you give me the exact value of the scale factor computed? (Since I actually can not print the scale factor used from OpenBCI, and this would be helpful to compare mine scale factor vs the one used)

    @wjcroft, @retiutut I will be grateful for your help on this.

    Thanks and Regards,
    Pluto

  • wjcroftwjcroft Mount Shasta, CA

    @pluto, hi.

    I merged your 'scale factor' thread, into this existing thread, because the general topic is still conversion of the hex file.

    The calculation, including scale factor, is given here,

    https://docs.openbci.com/Cyton/CytonDataFormat/#interpreting-the-eeg-data

    William

  • wjcroftwjcroft Mount Shasta, CA

    As stated at that link, the default value is "0.02235 microVolts per count". In other words, each additional count of the low order bit of the raw sample value, adds two hundredths of a microvolt. Since EEG below a microvolt is mostly considered noise, the low order digits in the scale factor (0.00035) have low relevance.

  • plutopluto mars
    edited December 2021

    Thanks a lot for you help @wjcroft,
    I just tested it with this scenario:

        scale_factor = 0.02235
        hex_string_to_convert = FA8CFA
        Equivalent decimal value converted by OpenBCI GUI 4.2.0 = -7982.3887
        Equivalent decimal value converted by my script =         -7981.7661 (This value considers the above scale factor used)
    

    Previously I was using scale factor without any round off, and I got results as below:

        scale_factor is:  0.022351744455307063
        hex_string_to_convert = FA8CFA
        Equivalent decimal value converted by OpenBCI GUI 4.2.0 = -7982.3887
        Equivalent decimal value converted by my script =         -7982.38909034599
    

    In the second case where I did not round off the scale factor, the values are still different from 3rd decimal places itself, and even if I round off my value (decimal of hex) to 4 decimal place, I still won't get the same result as openbci.

    What is that I am missing??

  • wjcroftwjcroft Mount Shasta, CA

    Pluto, hi.

    The minor differences plus or minus an absolute microvolt, really don't make much difference, as previously mentioned. A single microvolt level signal in EEG is usually to close to noise level. Also, all your conversions are using the same scale value, so it is the relative microvolt amplitude levels within the same recording, that are important.

    However I do note that your sign seems to be wrong in some of the results you show, unless that is a typo on your part.

    Richard @retiutut may want to make a note that the docs page showing the sample 24 and 16 bit conversion code, has a subtle typo. That likely got introduced in one of the docusaurus hosting conversions. This is a link showing what the code samples used to look like:

    https://www.titanwolf.org/Network/q/a46315a4-d0c9-44e0-a7b4-0f6749094598/y

    Whereas the current example shows <! instead of <<:

    https://docs.openbci.com/Cyton/CytonDataFormat/#24-bit-signed-data-values

    Regards, William

  • plutopluto mars
    edited December 2021

    Hi William,
    Yes that was a typo from my side of missing -ve sign in openbci conversion. I have corrected that now.

    Considering about the values, since we have once source file for the log (from SD Card), and the values are pre-recorded, the conversion should provide me the same results, since it is not realtime deal with EEG signal.
    Let us consider the value as just a value. And particular conversion of some value should always give me same result, not different right? Like for example FF will always give me -1 everywhere.

    What I mean to say is, if we have one particular value (let us take for example: FA8CFA), knowing it is negative hex it will always have one particular value, considering till what value we take the deicmal.

    From my manual calculation, I am getting the exact same result as my script, which is not matching from 3rd decimal of OpenBCI.

    So how can both values differ? (Considering at the end both are just hex values)

    Regards,
    Pluto

  • wjcroftwjcroft Mount Shasta, CA

    Do you understand what I am saying about the extremely low relevance of single microvolt or sub microvolt EEG changes? Use whatever conversion factor you desire. If you can find one that matches exactly the GUI conversion, cool. But don't sweat about single microvolt or sub-microvolt differences. EEG is just noise at that amplitude level.

    I suggest consulting the OpenBCI source code if you want to see exactly what steps were used to derive the scale value.

  • I would like a python script to convert hexadecimal data (recorded in Cyton) recorded on an SD card to decimal.
    Thank you in advance.

  • wjcroftwjcroft Mount Shasta, CA

    Hi @macoffee, I merged your new thread into this existing thread on the same subject.

    See some of the previous comments. I believe there is still an issue open on this subject,

    https://github.com/OpenBCI/OpenBCI_GUI/issues/1014

    Richard @retiutut may have further comments. If you look at previous comments above, you can see the source code link,

    https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/DataSourceSDCard.pde

    @retiutut said:
    Correct. Let me break down what happens in the new DataSourceSDCard.pde (https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/DataSourceSDCard.pde)!

    parseRow(String[] row, int numChannels) parses the values for all channels which exist as a "row" in the CSV file: https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/DataSourceSDCard.pde#L76

    parseInt24Hex(String hex) converts the values for each channel: https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/DataSourceSDCard.pde#L287

    And then there is a Processing method called unhex(): https://processing.org/reference/unhex_.html

  • I've made a python script sd_convert.py, you can modify it for your needs https://github.com/roflecopter/openbci-psg

Sign In or Register to comment.