How is packet loss handled?
Note: I am not interested in how to avoid packet loss. This has been extensively covered in other threads. I have existing data with lost packets that I need to handle. So I want to know how, precisely, it manifests. Hopefully, someone can help.
For the setup, I'm running a Cyton on Bluetooth (the old Cyton, with batteries, I'd have to look up the revision if that's relevant) plus the OpenBCI_GUI v 6.0.0. beta-1.
So:
- Can I rely on the idea that the first column of the .csv created by the GUI will have skipped values, and the values that are skipped are the lost samples? So that I can e.g. just boost the timebase by an extra n *4 ms (1/250 Hz, right)? to have the correct offset after n dropped samples?
- Or can there be somehow skipped samples that don't show up in the .csv?
- Is there a difference to the .txt-raws, is either better to figure this out?
Knowing the answers to that would be a great help for me. Does anyone know?
Edit:
I'm asking this, because I have an especially corrupted snippet looking like this:
68, 69, 70, 95, 72, 93, 74, 80, 91, 77, 78, 89, 87, 98, 84, 85, 96, 99, 100 etc.
so do I interprete this as lacking e.g. samples 96-71 (going once past 255 to start back at 0), or is there a chance it misses more than one 0-255 chunk there? Is there any way to tell?
Comments
Hi NR,
Yes the sample index column normally cycles through 0 to 255 and back. So missing sample numbers imply missing packets.
Samples occur at 250 Hz, every 4 milliseconds. If you see the sample in the csv, it was recorded; missing samples are not.
The 'time stamp' values in the csv are assigned as the samples are received at the GUI / Brainflow, so reflect latencies in radio packets, usb buffering, OS delays, etc.
It should be possible, using a usb extension cable to get the dongle closer to the Cyton. This can minimize packet loss.
Regards, William
Hi, thanks for your quick reply! That already helps me with the occasional misses (which is relevant, because I'm aligning that data to other data).
Is there also a way to detect if more than 255 samples dropped, like I wrote in the edit? I almost suspect not, given what I now know, since, as an example: 67-66 could either mean 255 samples dropped, or 510 or 765, or ... as at that point, connection was so spotty that his cannot be ruled out -- and the file can't separate that?
As for the suggestion, like I said; it's fine now, it was a source of interference I removed and it has been very fine since. Just I'd be loath to dump the already recorded data, unless absolutely necessary because it cannot be saved (i.e. given proper timebases -- naturally it will have gaps).
Normally it would be rare for more than 255 to be dropped. That would signify an extremely poor connection. The packet widget can inform you of the dropped packets, during a session. There are no other packet counters than the one byte value 0 to 255.
https://docs.openbci.com/Cyton/CytonDataFormat/#binary-format
And the packet widget information isn't saved ... Maybe something for a session log file in a future version of the GUI? Sounds useful, anyway.
Right. I think this clears up all my questions and now I can work on the data. Thanks again for the quick help
I'm experiencing the same issue. In my case, it's not just a simple drop of 255 packets — when I calculate the expected duration based on the missing data, the recording becomes far too long to make sense. So my question is: could it be that if too many packets are lost in a short time, the GUI mixes up the lines when displaying or saving the data, even though the packets themselves are received in the correct order?
Exemple :
Hi @Malou,
I think you are referring back to the comment by @N_R on Feb 28. There are no 'simple drops' of hundreds of packets. That case would indicate huge disruptions in radio link between the OpenBCI board and the laptop. Cyton sends 250 packets per second in 8 channel mode. Ganglion sends 100 packets/sec, each packet containing 2 samples.
Thanks for including that snippet of packet IDs. It actually has many similarities to the list that N_R shared on his opening post. What we see in both lists is not just that numbers are dropped, but that they seem to be rearranged in a bizarre fashion. Skipping forward and back by huge jumps. Mentioning Richard @retiutut (GUI dev) and @Shirley.
No. Unfortunately these packet IDs are assigned at the Cyton or Ganglion as the radio packets go out from the board. So neither the GUI nor Brainflow have any control over how Packet IDs are assigned. This is kind of mysterious.
Have you tried just the simple solutions mentioned previously in this thread? In other words, get the dongle closer to the Cyton. This can easily be done by using a usb extension cord, for example draping the dongle over the top of a nearby monitor. This also gets the dongle receiver away from other conductive objects such as metallic tables, cases, etc.
Hoping that Richard might have some insights.
Regards, William
Also mentioning @Tharun_OpenBCI, a staff hardware, firmware engineer. Tharun, what could cause the Packet IDs to get totally scrambled like this. Not just 'dropped' packets, but rearranged packets. I starting to think this could be some weird thing with Brainflow buffer management? Or the GUI getting confused upon taking packets OUT of the buffer.
Still interested in the details myself as well. I ended up simply shelving most of the corrupted data, as the time and energy to work out what had to go where was not worth the effort as opposed to re-running the experiment ...
@Malou Interesting that the calculation did not work out. When I have some downtime, I will get my data and try it as well. I never did end up going that route, so it's possible my attempts would result in the same.
One observation, or musing perhaps: I doubt that in my case, it had to do with the radio transmission. I was working in a literally interference free room, no AC power, nothing. Just the Cyton, battery powered lamps and the laptop with the dongle, at a distance of perhaps 3 or 4 ft of nothing but air. I've since had a situation, with that setup, that resulted in 90% or so package loss, according to the GUI widget.
On the other hand, the laptop did run at high loads, because it was also processing camera images. This has been subsequently changed, with a dedicated laptop for either, and since then it's been fine, although this was just recently, so the sample size is tiny. It's not as if the package loss happened regularly or even semi-regularly. Either way, though, could the GUI be allergic to not having enough CPU power/bandwidth/whateverItNeeds to process the incoming packets, and then something weird happens?
To compare this with my camera, I know there is the situation where frames can get dropped during transmission, as well as after already receiving them because a buffer isn't cleared fast enough, say because it takes too long to process the previous image(s). Maybe something similar is possible here too, although the data should be positively tiny when compared to images, I imagine.
Yes, your own testing has shown that the GUI needs adequate CPU processing resources to service the incoming packets. There is no 'flow control' in the stream of packets that come from the Cyton. They just come out at a steady rate of 250 per second. So the laptop must have enough resources to process these as they are received, immediately.
It is possible that if you ran your own tiny Brainflow data stream receiving program (samples at link below), that this process would require less resources than the fully featured GUI. And thus could get by with fewer CPU resources. But generally with real-time recording operations it is best to give the real-time process priority.
https://brainflow.readthedocs.io/en/stable/Examples.html#python
This is also the case with other types of EEG signal processing applications, such as real-time neurofeedback or BCI Brain Computer Interfacing.
Hi everyone,
In our case our computer was only recording the EEG, but it was connected to the network via Wifi. And in the recording area, there were metal barriers all around the EEG as well as telephones (which were not in aeroplane mode). It was our first time, so the protocol wasn't very good in terms of interference.
I tried once to run the EEG right next to the computer and I didn't get a drop but I didn't have time to investigate further.
I am attaching an EEG file which contains mixed data if some people want to do tests.
I tried once to run the EEG right next to the computer and I didn't get a drop but I didn't have time to investigate further.
I am attaching an EEG file which contains mixed data if some people want to do tests (this is not the best example but the other files are too big for me to be able to put them on).
Hi everyone,
In our case our computer was only recording the EEG, but it was connected to the network via Wifi. And in the recording area, there were metal barriers all around the EEG as well as telephones (which were not in aeroplane mode). It was our first time, so the protocol wasn't very good in terms of interference.
I tried once to run the EEG right next to the computer and I didn't get a drop but I didn't have time to investigate further.