SD card logging interval asked for, not reflected in recording
While logging 16 channel EEG data (Cyton+Daisy) + 3 channel Analog Data (Analog read turned on) to SD card we observed the following behavior:
1. For 5 mins of SD card logging, we are getting a log for only 3 mins.
2. Similarly, for 15 mins of SD card logging, we are getting 9 mins of data log.
i.e. "SD card logs for only 60% of the programmed time".
Why is this issue happening? And how do we resolve this?
@wjcroft Kindly help us in resolving this.
Comments
Richard @retiutut, do you have any insights or experience on SD card not recording the interval intended?
Did I understand that @pluto is decoding his own hex SD card records? So it may be possible this is a glitch in his decoder.
An issue has been made in the GUI repo to re-assess the SD card functionality on the Cyton, I have just linked this forum thread to that issue. https://github.com/OpenBCI/OpenBCI_GUI/issues/893
Since this is slated for the next major update 5.1.0 (currently in development), I can try to replicate this. @pluto Thank you for sharing enough details to carry out a test and try to replicate.
Key Details for this test:
Hi @wjcroft and @retiutut,
Thanks for your reply.
William, I am not using any SD card hex decoder here.
William & Richard, We have not only used OpenBCI GUI (v5.0.8) but also Cyton SDK commands to stream and record the data in SD Card, in both the above cases we found out data recorded in SD Card to be only 60 % of the mentioned time.
An observation from our side: While recording data using OpenBCI GUI (For 5 mins of recording), we observed the stream gets stopped after 3 mins (approx), and the Red Status LED on dongle also gets turned off. For 15 mins of recording, the same thing happens after 9 mins (i.e., after 60 % of the set time).
And even without turning on the Analog read mode, we are observing the same behavior.
Regards,
Pluto
Hi William @wjcroft, and Richard @retiutut,
Any pointers from your side will be of great help in resolving this.
Awaiting for your response.
Thank you.
Could you use an obvious work-around, of asking for the next larger recording interval? Any resolution of this will require finding the bug in the firmware and fixing it. Most OpenBCI engineering staff is occupied with Galea development.
Does this bug only occur with 16 channel recording? So the counting error has something to do with the extra processing with Daisy?
Does the timing work if the Analog read mode data is turned off? That sounds like a likely culprit. What are you using the Aux channels for?
@pluto Thank you for your post.
i will bring this cyton SD card streaming/recording data issue to the attention of the engineers tomorrow morning.
This will likely require a firmware-level fix, and could take weeks. In meantime, can you work with the data that is saved directly to your computer? Can I ask why SD card recording is necessary for your research?
Hi William (@wjcroft),
Yes, as of now we are recording using extended time intervals.
Thanks for suggesting few inputs for testing out, here are the observations:
We have also tested few cases from our side, the results are appended below:
Observation from firmware modification: The stream was successfully recorded for entire selected duration available. It was done for both 5 mins and 15 mins of recordings.
Congrats. Post the changed section of code. Your multiplication takes into account the number of channels? Because 8 channels was working ok before.
William, no, the multiplication is only for defining the block write size of SD Card. We have not touched anything related to 8 channels.
The 8 channel recording test was done with original firmware provided by OpenBCI.
As per our 3 test cases in which the recordings was found to be correct. Out of which one is by disabling daisy, and another one is by disabling SD Card log, third one is test with 16 channels with modified firmware.
We are not aware on what is causing this issue. Whether is it daisy as you said, or it is SD Card block write size?
I'm trying to determine if your code fix, is general enough to work in all situations. So if you universally increase the block write size, does not that mean that for the 8 channel case, there is now some unused space in the block?
Probably does not matter much since SD cards are so large anyway. [shrug emoji]
Show the section of (or function) of code you changed. It seems likely that AJ Keller never tested SD card with 16 channels.
Sure, here is the below logic which I have implemented for firmware change.
For 5 min, in firmware count of 11000 is mentioned, but currently this records for only 3 mins, so that means 3 mins represents 11000, so for 1 min it will be 11000/3, and for 5 min it will be (11000/3)*5 = 11000 *(5/3)
5 mins represents 3 mins ----> 11000 ----> then for 1 min ----> (11000/3) ----> for 5 mins ----> (11000/3)5 = (11000)(5/3)
15 mins represents 9 mins ----> 33000 ----> then for 1 min ----> (33000/9) ----> for 15 mins ----> (330000/9)15 = (33000)(5/3)
30 mins represents 18 mins ----> 66000 ----> then for 1 min ----> (66000/18)----> for 30 mins ----> (66000/18)30 = (66000)(5/3)
Same will continue for 15 mins, and other time intervals.
The above image contains the modified part, this is the only place where the firmware is modified, the commented part is the actual OpenBCI firmware, where as the top one without comments is modified by multiplying 5/3 to each time interval.
Hi @Shirley, thanks for your response.
There were packet loss initially so we went ahead with SD Card log, but with proper alignment of dongle (Dongle's antenna) with Cyton board, even the system logs had 0 % packet loss. But still sometimes minute amount of packet drop happens, so to avoid it we are using SD Card logs.
Are you using a usb extension cord? With that you can get the dongle as close as needed to avoid any packet loss. Also note that the radio channel can be changed with the GUI, in case there are other interfering Bluetooth sources. Be aware of other nearby wireless equipment, which may also be contributing.
Yes William, we are using USB extension cord.
Yes, we are using the GUI to set distinct bluetooth channel as the two cyton boards should not communicate on same channel as suggested in OpenBCI docs.
It seems likely that if the dongle is sufficiently close to the Cyton, that packet loss would be zero. Providing that there are not other interfering wireless sources in the area. If this is the case it seems possible for you to drop the requirement of needing the SD card recording.
We are using SD Card for logs because of below constraints:
Hi @Shirley,
Thanks for your support on this. It would be great if you can let us know whether can we get the firmware level fix on this anytime soon?
Also, We would be very keen in knowing the cause for this issue.
Also @wjcroft @retiutut, we wanted to know, is the solution being used by us as of now (Modifying the firmware) is general enough to use in all cases without interrupting any other modules?
Regards,
Pluto
If you have found a firmware fix that works, then that would seem to solve your issue, does it not? Most likely the section of firmware that estimates SD card file size, just needs updating. Your current fix of expanding the size for ALL recording modes is a quick fix that works. The only refinement might be to take into account, whether full sample rate Aux data is being recorded (analog or digital read modes). If not, the recording file length can be the original shorter lengths. If full Aux data mode is in effect, then the longer size is needed.
The only very minor downside to your current fix, is that a small of amount of reserved space is lost, when doing recordings without full Aux data. But since common SD cards are many gigabytes in size, this is very low impact.
The 'cause' of the failures you were seeing, prior to your fix, was likely exceeding the pre-reserved file size. Your fix is fine for all other usage.
I have documented the idea for the firmware fix on https://github.com/OpenBCI/OpenBCI_GUI/issues/893. This addresses my original concern for making this ticket. Would like to replicate @pluto 's firmware change to see if this needs to be applied to the next round of Cytons.
Thanks William and Richard for your response on this.
We will be proceeding ahead with the fix as of now.
Regards,
Pluto