Regarding Sampling: Increasing Sampling Rate / Having Known Sampling Instants
Hello all,
I've been discussing with @wjcroft and @pushtheworld for some time now and the focus of our discussion has shifted to hardware related issues, so I decided to reinitialize the discussion under hardware forum as well.
I have 2 questions/aims that I would like to hear about from you:
1) Increasing Sampling Rate. There are lots of topics all around the oBCI forums. Right now there is a bottleneck in the serial communication between the PIC32 and RFduino on the board. Bluetooth 4 seems to support higher throughput, so its not the limiting factor in this case. Since I'm new to the community I do not know much about the previous attempts to overcome this issue. There are alternatives, @Winslow_Strong's usb communication and @yj's SD card only methods eliminate the RFduino and allow for higher sample rates. Is there any other similar work/attempt? What is the current situtation/state of the art with this problem?
I would like to reduce resolution (eliminating noisy bits, 24bits to 16bits if possible) and remove some unused EEG & AUX channels to allow for 500sps. As I'm new, I do not know where exactly the protocol is implemented and which files I need to modify (I do need changes in host and device RFduino firmwares (for protocol changes) and PIC32 firmware (for resolution and protocol changes). I'd be glad if someone showed the files I need to make changes at.
2) Having known sampling instants (with reference to start of stimulus). I want to have my first sample at a known instant for better averaging. Initially I thought interrupting the PIC32 with trigger input to command ADS to start sampling would be a good idea but @wjcroft corrected me by saying that ADS has to run/sample continuously. @pushtheworld gave me a great advice, to start the stimulus at the first sampling instant after "start stimulus" button is pressed on my system (I'm using an FPGA for stimulus generation). I would love to hear others opinions on the matter as well before implementing the mentioned idea.
Thank you for the great product and community!
Yigit
Comments
Hello yttuncel,
If you are ready to modify the pic32 code and the pc code (device and host) then you could compress the 16bit data by a factor of 4, using ADPCM : see "Adaptive Differential Pulse Code Modulation Using PIC® Microcontrollers".
Look for "Microchip application note AN643" on google...
The quality is very good, I verified it under matlab...
I found only one probleme : If some data were lost then the resulting shift (or step) is kept, the error is kept... With adpcm we need an "error free" communication.
This way you will keep some free room (31-2x16/4) bytes , enough to tag and much more, the record with your FPGA-trigger.
y.j.
Yes , "every thing is relative"
If we lost raw datas, the values of the new incoming datas are true.
After ADCPM "uncompression" the new uncompressed values are no longer reliable. One no longer know where we stand.
This makes me think that we could time to time , send some raw data from one chanel... in order to reset the algo combined with "time stamping" the data to know if some miss.
I am not an expert in this area and I do not know if there is a method to account for ADPCM lost data .
If you dont need 8 channels and 24bits/sample , then yes, what you plan to do is the more simple (and the more simple is the best...)
y.j.
2) YES! That's exactly correct, you would need to change the rfduino code to send less bytes per packet. Also remember you could send 3 samples per packet if you only have 10 bytes of data.
- Do you have aproximately 5 x 60 x 250 lines in the last "not empty" 5' recorded txt file?
- What do you get in the *.txt files from the un-modified code?
(Some SD cards that perfectly work on your P.C., will not be usable on the OBCI Card)
- On which code modification does the result become wrong?
I suppose that you already tried these points...
Have you made any progress, gained any further indices?
Your approach is interesting,
keep in mind that in general, but mostly at 2kHz, data will be lost during wait busy time before block writing on the SD...
http://openbci.com/forum/index.php?p=/discussion/546/solution-for-session-freezing-overruns-2khz-sample-rate-with-sd-only/p1
Sincere encouragements,
y.j.
The command 'a' generates à 512 blocks file, a block 512 bytes ; 512*512 = 262.144 = 262.128 (last value of your counter) +16 bytes (content of the last line) as you write. Thus it seems correct.
You should see at the end of this file , an ascii written footer ; 512 bytes , unused filled with zeros.
Do you use the v2 from pushtheworld?
Any way the best way to understand is to look at the code.
In the v1 the function that writes on the SD is :
void writeDataToSDcard(byte sampleNumber)
Strange : in the ascii window of your dump file NGGOG I do not see the commas.
In the v1 txt files were binary hexconverted and comma limited, starting by the sample number, ending, if required, with 6 bytes of auxiliary datas...
In your dump I see mostly binary data...
May be things have been changed in the V2... even if "txt" file extension has be kept.
Sincerely,
y.j.
Pushtheworld is right, first of all, ensure yourself that your SD card works.
These SD are rather ticklish, I got incredible problems with 16gb using the very same ref, one works the other dont...
Have you red : http://docs.openbci.com/Tutorials/09-SD_Card
Currently we use an 8gb SanDisk Ultra.
A bug in the code is unlikely (and would be known) but if you want have a look at the V1 it is in
http://docs.openbci.com/Tutorials/02-Upload_Code_to_OpenBCI_Board copy pasted :
You can find the previous OpenBCI firmware and libraries on our github repository.
https://github.com/OpenBCI/OpenBCI_32bit
https://github.com/OpenBCI/OpenBCI_32bit_Library/tree/maint/1.0.0
- OpenBCI_32bit
- This is the firmware that runs on the OpenBCI 32bit Board
- OBCI_SD
- Supports writing raw data to on-board SD card
- OpenBCI_32_Daisy
- The OpenBCI 32bit Library
y.j.I do not have the V2, but someone from the forum or from OBCI project, should be able to quickly tell you if SD writing is ok or not...
Can you read the footer at the end ? It should be ascii.
y.j.
The last 512 bytes block begins by a short ascii message and is filled with zeros.
If you do not have zeros (at least 256) at the end , it might be a SD card problem.
Try any 8gb SD card ... Some good quality and very fast 16gb cards were not working when I was trying to find a reliable one... But some low price 8gb gave me good results...
And wait for confirmation from other V2 users...
y.j.