packets are 31 bytes over the air, then dongle adds header/footer

edited January 2015 in Software
Shouldn't this be 33 bytes instead? or the header and footer are implicit?

streamingData
Device radio expects to get 31 bytes in each data packet from the uC
After 1 second of no transmission, or not getting 31 bytes in time, Device and Host will revert to!streamingData mode
Command characters can be sent from PC following timing protocol above

Comments

  • biomurphbiomurph Brooklyn, NY
    The Gazelle stack allows for maximum of 32 bytes per packet. 
    We reserve the first byte of each packet as a check-sum between the radios. The check-sum byte is added by the Device radio (on the OpenBCI board) and so the maximum number of bytes that can be sent to the Device radio is 31.
  • wjcroftwjcroft Mount Shasta, CA
    Joel, Dhruv, hi.

    Joel, so what you are saying is that our OpenBCI packets are ALWAYS split across two radio packets. Didn't realize that. That likely accounts for the jittery-ness of the plots that @P300_MAN made of the arrival times,

    http://openbci.com/forum/index.php?p=/discussion/comment/915/#Comment_915

    William

  • biomurphbiomurph Brooklyn, NY
    William,
    OpenBCI data samples are 30 bytes long

    8x3 ADS bytes
    3x2 acellerometer or aux bytes

    Then add 1 for the sample counter = 31 bytes that get sent to the RFduino Device every 4mS (250SPS)

    The OpenBCI data packets are NEVER split across radio packets!

    When we send 16 channel data, we have to alternate between the board mounted ADS and the Daisy mounted ADS to get all the data across, and then we are sending averages over air, but still, the entire sample is sent in a single data packet.
  • wjcroftwjcroft Mount Shasta, CA
    edited January 2015
    OK, I think what you're saying is that the start and end bytes are not included, but added to the COM port serial stream by the dongle on reception of the radio packet. Cute trick to squeeze that into 31 bytes.
  • biomurphbiomurph Brooklyn, NY
    yup!
    The prefix and postfix characters are added by the Dongle radio (Host).

    It may be possible to alter the Host code so that it will package the data in a way that other software expects to see it. Seems that drivers are being made to interface with our protocol, but it's an option for software that doesn't have a driver already built up.

  • I notice some sync issues after a stop and start or reset sequence (=>"s" =>"b" or =>"v" =>"b").

    Specifically, after initiating the binary stream I see some header bytes that do not belong to the header of a packet. Thus far I have implemented a pretty dumb serial stream decoder that finds the header byte, reads 32, and then checks the sequence number and footer. This works well enough except for a couple hiccups during initialization of the binary stream.

    What sort of noise might be coming over the line after writing a stop-start or reset sequence to the device? Could it be that the uC sends it's last 31 byte packet to the dongle, and the dongle does not add the header/footer bytes before sending the packet to the serial port?
  • biomurphbiomurph Brooklyn, NY
    I have noticed that when you stop the data stream by sending a 's' there are some packets that will get received after the 's'.
    On starting the data stream with 'b' it is probably best to throw out the first second or so of data, while things settle.

    The uC doesn't really send much at all over the air, except when in streaming Data mode. 
    When it gets the 'b', the uC enables the data stream and starts waiting for the DataReady pin to signal that new data is available.
    the 'b' message is noted by both the Device and Host radios, then they go into streamingData mode and only the 31byte packet is allowed to be sent.


  • image

    Given this
    packet stream, where "s" is written to the uC while P3 is mid-assembly,
    should we expect a partial p3, no p3 or a full p3 to show up in the
    serial device? If partial or full, then before or after writing "b"?
  • wjcroftwjcroft Mount Shasta, CA
    Kevin,

    You wrote:

    > Thus far I have implemented a pretty dumb serial stream decoder that ...

    Have you considered a simple state machine approach, see parse_byte_OPENBCI() function ~ line 750 in,

    My sense is that you just have to assume some partial packets occasionally, especially at off / on transitions. If you use a state machine to reconstruct your packets, you'll always get all the valid packets.

    William
  • Thanks wjcroft. That's nice code, I will switch over my implementation.
  • hmmmm, I implemented the state machine. Alls well on restart, the device returns to a happy state after the s, v, b sequence. However, on stop/start (s, b), I get some very interesting behavior. I see the sequence number consistently jump by 5. It's oddly consistent. Is there any reason to believe that this is expected, or should I peer a little deeper into my code?

    For instance:
    96
    97
    98
    99
    "s"
    "b"
    100
    105
  • wjcroftwjcroft Mount Shasta, CA
    Kevin,

    Joel @biomurph might comment further, but my guess is that this is just evidence of a partial packet (100) being in the pipeline as the stream is stopped. Then that pops out when restarted.

    The skip of 4 packets, again, could be considered normal if there are some internal buffers being cleared or reinitialized somewhere in the pipeline in that s/b transition. Also, the state machine looks for an END / START sequence (0xC0 followed by 0xA0) if it ever loses sync. So that may account for an extra packet loss.

    I would say this kind of minor packet loss at transitions is to be expected. In your code you may want to make available to the user interface some indication of how many sync loss events have occurred (as Brainbay and the ob_eeg.cpp code does.) As long as this is constant, or rarely changing, you are fine.

    William
  • biomurphbiomurph Brooklyn, NY
    Hi Kevin,

    I'm not sure if the 4 packets that you seem to not see are getting dropped by our system or your software.
    In the tests that I've done to verify packets I have not seen this occur after 's' and 'b' commands. The first sample number is not always the same, so that can sometimes throw an error... but I don't see a consistent 4 or so packets dropped at the beginning during tests.

    William, I don't think that your idea of the 100 being 'in the pipeline' is correct. The OpenBCI Board and Dongle don't hold onto any data. If there is anything in the pipeline, it gets pushed through. 

    At the very least, you could just throw away that first group of samples when the system starts up?
  • wjcroftwjcroft Mount Shasta, CA
    edited February 2015
    Joel, Kevin, hi.

    Kevin, one additional step you could do just as you are about to send the 'b', is to clear the state machine 'readstate' variable (or set it to '1' if we assume the new serial stream will immediately begin with a START code.) The BrainBay state machine function is at line 750 here.

    With the state cleared or initialized, then any previous partial packet that had been assembled will be discarded. I still do not think this explains the packet number 100 you saw. Since the restarted serial flow would not have matched up precisely with the END byte expected. Anyway, it's one more tweek you could apply.

    I would also agree with Joel as far as the number of packets lost on stop/start transitions, when I do this with Brainbay I see the sync loss counter increment by 2 usually. Occasionally by 1. This corresponds with the way the state machine re-syncs if sync is lost. And I don't currently reset the state before sending 'b'. I should give that a try.

    Joel, on a 'b', does the packet counter always start off with known sequence number? If so then an additional tweek to the state machine (initializing expected sequence # before 'b' sent) would eliminate that first packet not matching what is 'expected'.

    William
  • biomurphbiomurph Brooklyn, NY
    The sampleCounter variable is initialized as 0 (zero) when the board starts up, but after that, it is merely incremented after each packet is sent to the radio.
    The sampleCounter is a byte, and it rolls over at 256. It is 'owned' by the Board (8bit or 32bit) 

  • Thanks for all the feedback. I went with a sync counter threshold in the state machine as flow control. We do not move data down stream until two sync'ed packets have been seen.
  • sonalisonali ga tech
    edited August 2015
    The raw data files have a header that clearly states the sampling rate is 250 Hz, but the first column of the data loops from 0 to 255, giving a total of 256 samples per second. Can someone help clarify this discrepancy?
  • wjcroftwjcroft Mount Shasta, CA
    @sonali , hi.

    I merged your question into this existing thread. The sample rate IS 250 hz. The counter is just a one byte packet counter that rolls over at 255 and goes back to 0. See Joel's (biomurph) comment above on Feb 16.

    William

Sign In or Register to comment.