SampleIndex drops and the order of index is not right
km22
Tokyo
I have collected data generated from 8 channel OpenBCI Cyton. I have found an awkward behavior of OpenBCI GUI of how they record sampleIndex jump inside the another jump. An example is the following data.
22 (SampleIndex)
56 Hub: apparent sampleIndex jump from Serial data: 22 to 56. Keeping packet. (1)
54
43
26
52 Hub: apparent sampleIndex jump from Serial data: 26 to 52. Keeping packet. (2)
41
50 Hub: apparent sampleIndex jump from Serial data: 41 to 50. Keeping packet. (3)
30
31
48 Hub: apparent sampleIndex jump from Serial data: 31 to 48. Keeping packet. (4)
The awkward part is that the first jump is from 22 to 56, so sample index from 23 to 55 were expected to be dropped, however, the next index appeared was 54 (which should not exist if the jump happens from 22 to 56) and another jump happens from 26 to 52 (which should be already dropped). The example listed above is a continuous data. The sample Index seems to be randomly labelled when the disconnection between cyton and dongle happens, and I am not able to identify which sampleIndex is actually dropped.
I understand that a solution is to keep dongle and cyton close each other and secure the connections from other interference, but this outcome is awkward and willing to know how jumps are treated in OpenBCI GUI.
Additionally, in the output from console log, it was stated that "Interpolating dropped packets...". As the number of samples were simply reduced due to the jump, I am not sure if the data is interpolated and written to the file as the message indicate? Is there any setting I should make to let those interpolations to be reflected on the recording file.
Comments
Hi, KM22,
Try positioning your Cyton dongle closer to the Cyton mainboard. This could be done with a usb extension cable. You also can setup your Cyton in an area with less interference from other Bluetooth devices, wifi, etc. It is also possible to change the radio channel used by Cyton, to get farther from existing sources.
https://docs.openbci.com/docs/01GettingStarted/01-Boards/CytonGS#5-optional-settings
Expand the arrow in section 5 to see the radio settings.
Regards, William
Also, see this thread for a photo and link to a cardboard box that could fit both Cyton and dongle (on usb extension). Obviously you would need to tape the dongle inside carefully with adequate separation / insulation from the Cyton mainboard circuit traces / pins. If needed, aluminum foil shielding might also be possible to apply to the exterior of the box, to further reduce reception of any outside interfering radio sources.
https://openbci.com/forum/index.php?p=/discussion/2392/safety-housing-for-cyton-resolved
Hi William,
Thank you for your replies. You provided practical solutions for preventing jumps. It was a really helpful tip.
However, what I want to know is how jumps are treated in the program if it happens. From the example in my original post, I do not understand how dropped data is handled in OpenBCI GUI. You can find that jumps/drops happen inside another jump/drops. I want to know that why this weird phenomenon happens in the program, so I can decide how to handle missing data. It was going to be simple if dropped data simply results in reduced data (e.g. jumps from 22 to 56 leads to missing data from 23 to 55th sample index), which is not the case in our example.
Your BEST option is to improve the radio connection to eliminate the drops. The GUI does do interpolation of missing samples. But in the best case that would be handling a single missing sample, and interpolate a single replacement sample midway between the two valid samples.
If you are getting huge jumps in the sample index, it would imply serious radio connectivity issues that CANNOT be solved with any type of interpolation. Mentioning Richard @retiutut, the GUI developer.
This is the most important thing to worry about, not how the packet loss is handled. BDF file format keeps track of whether or not the file is contiguous, so you can also try that. OpenBCI data format is a simple CSV, and during playback goes from one row -> next row.
Dear Retitut, William,
I am running into the same issue - packet losses. They do sometimes occur, even if it's just a single data sample, even with the best possible setup, which is (mostly) clean from wifi, and Dongle/EEG Cyhon Daisy in close proximity. Now I have several questions regarding how they are handled:
Well, these are my main questions
Any explanation would be greatly appreciated!
Thanks and best wishes from Australia
Johan
There are 2 other tidbits probably less appropriate to this thread:
1. I tested the OpenBCI GUI with Arch's Linux's newest kernel, and a long-time support kernel. Since the newest kernel causes the GUI acquisition to crash, I suppose in upcoming ubuntu versions you might run into this issue in the future.
2. I played around with OpenBCI_LSL, but it seems to not be under further development, and it also doesn't include any Wifi sopport. Brainflow is my best bet (and it also gets me the sample ID on channel 0). Are there any plans to support LSL from the command-line? I do realize that LSL support is built-in into the GUI, but I'd rather not use it once any real-time stuff is running. The GUI is a bit too CPU intensive, so I'd rather have the streaming done with a more lighter command-line utility.
Johan, hi. Have you tried eliminating possible interference from other Bluetooth, Wifi, cell phone, etc. devices in the area? Relocate your experiment to a quieter radio environment, room, building? Use a usb extension cable to position the dongle within inches of the Cyton mainboard, etc.?
Did you look at the box option mentioned in previous post on this thread, with possible tin foil shield around the box?
Regards, William
Dear William,
For everyday situations, I can live with a small packet loss if I know how it behaves.
Yes, those things do help quite a bit to take down dropped samples to an acceptable level (i.e. once every minute or so).
I can live with a (as small as possible) loss of packets if the system is to be wireless. If the room is free of big noise, things go well and it happens only once or twice per minute with 1 sample each time. Probably if you're on a grassy hill with no electronics around, or a dedicated shielded EEG room, things go even better. Packet loss should approach 0 if the room is well-tested, and you put the receiver next to the board with a USB extension cable (but this defeats the purpose of wireless). If I test it next to my desktop, power converters, and screens - big drama
With the OpenBCI_LSL script, it's not logged what are the missing samples. Also from the LSL timestamp(s), it's not possible to trace back which sample is which, making interpolation impossible. With the BrainFlow, there is better luck capturing which samples were missed. Having said that if there's not much 50 Hz noise, and if there room is good, the low-power bluetooth-inherent transmission lapses (or interference sensitivity) of the signals happens only seldomly.
Packet loss is only bad if the 50 Hz noise is really high, and the srate is 125 Hz. Then the notch filter goes crazy if one packet is lost, and you see a burst because the on-line filter coefficients need the data.
Thanks for taking the time to answer questions! And thanks for making the openBCI, for research it works well also under lockdown
Best wishes
Johan