Higher sample rates / sample at 2000hz to SDcard?

tskipstskips Denmark
edited July 2017 in Wifi Shield
So... i am aware that the higher sample rate discussion has already been mentioned in a few threads, and i am sorry if this all should be obvious to me.

I do really wonder though, whether it would not be possible to drastically increase the sample rate, if one chose to only use one or two channels?
And if so, how would one go about telling the ADS1299, to only process data from channel 1 and 2 ?


«1

Comments

  • wjcroftwjcroft Mount Shasta, CA
    edited June 2017
    Tskips, hi.

    I moved your question on higher sample rates over to this Category: Hardware/Wifi Shield. AJ Keller @pushtheworld is the firmware dev and is beta testing a wifi shield that allows fast sample rates without dropping channels. I believe these shields will be available in the OpenBCI Store later this year. Maybe AJ can tell us the time frame.

    If you want to modify the Cyton firmware, that is also a possibility. However all the software which receives the samples from the Cyton assumes 8 or 16 channels. So you would need to tweek this piece as well. So overall your work will be much easier just using the shield.

    There are some historical posts related to higher sample rates elsewhere in threads on the forum. Some have modded the firmware to use a hardwired optoisolated usb link, others have mods that allow faster sampling directly to the SDcard. Unfortunately all of these earlier tricks were done using the V1 firmware as a base. V2 firmware is now the version that is shipping. And V3 is the beta version with the support for the shield. So again, your fastest route and most up to date will be with the shield. Hopefully AJ can comment here on when to expect availability.

    William
  • Hey all,

    Sorry for the delay! I have been working so hard on the wifi shield that I have not been on the forum!

    We have 200 OpenBCI Wifi Shields in production right now! We are about two weeks into production so we should have boards back in two weeks. 

    The OpenBCI Wifi shield features will be locked down on June 30th. You can see what I'm working on here: https://github.com/OpenBCI/OpenBCI_WIFI/issues

    I've really been having a lot of fun with this project! Working with marion and cloudbrain to have direct to cloudbrain capability. That's going over the MQTT protocol

    I'm all about these http requests to get information from the OpenBCI board! You can see all the specs for the HTTP REST Api Server that runs on the OpenBCI Wifi Shield https://app.swaggerhub.com/apis/pushtheworld/openbci-wifi-server/1.1.0

    You can GET information like the number of channels (4, 8, 16), connection health status for TCP and MQTT protocols, if you're like me and just want to stream you can now call from any web browser, phone, computer, pretty much anything with an internet connection, to start streaming.

    You can POST information such as board commands like "z011Z" to run impedance tests on channel 1, or POST to establish a new MQTT or TCP (and soon WEBSOCKETS!!) connection and start streaming data.

    We are getting great sample rates with the RAW mode, where the wifi shield sends raw byte stream data over TCP in the same 33byte format as the standard OpenBCI stream protocol for the Cyton. We were able to hook the wifi shield in raw mode up to our node_js driver in about 30 minutes, just by calling the processBytes call from the tcp on data event! It was awesome! In JSON, which is a still work in progress mode we are working through the final implementations and test results will be out soon. In JSON format, we follow LSL stream protocols, for example {"timestamp":12349292929, "data":[992922.3, 392923.0, 29393.0, 233294.0]} it's really fun! For this one we wrote a patch into the Cyton/Ganglion source code to send the channel gains to the Wifi shield right before we started streaming. Using our well-tested protocols for message passing between micro controllers, we were able to start converting the channel data to doubles with the correct scale factors, always. The timestamps come from NTP, or the Network Time Protocol. We are working on more validation but we are optimistic we will be able to get NTP with beyond acceptable precision. 

    The features are really coming together nice! There are a couple more I'll talk about soon! Expect shields soon! Will be on sale for less than $150 for sure! Ok I'm getting back to work! Time to write tests!

    See the code:


  • tskipstskips Denmark
    Hi Both of you!

    Thanks a lot for the detailed reponses. I see now that i should have given a little more detail, as too what i wanted: i am currently trying to look at BAEP's which requires a sampling frequency of at least 2000 Hz (i'd like 3000).
    As the AD12199 has the max sampling frequency of 16000 Hz, this leads to the necessity of reducing the amount of channels no matter what. I'll surely look into the wifi shield and the other possibilities but these will not solely be able to solve the problem.

    But if i for an instances merely wanted to transfer 2 channels to an sd card, could you give me a few hints as to where in the code to look? (I quess i'll avoid interfacing with any software which expects 8 channels in this way ?)
    (I will be looking at the other forum posts regarding the SD card, after posting this comment, but nonetheless i don't think the channel reduction question has been asked before, if so i am sorry)

    Best regards
    Tobias


  • @tskips using the new 3.0.0 code you can increase the sample rate to 4000Hz and write to SD card. You would not be able to see the data over Bluetooth, but it would be writing to the SD card, you could see the data over Wifi though (once the board comes out)
  • tskipstskips Denmark
    edited June 2017
    So i gave it a try.
    First i did as follows:
     - overwritten the SD card using the sd formatter software
     - followed the "Cyton Board Programming Toturial", to upload the 3.0.0 code.

    secondly, as it didn't really work out for me, i also did the following
     - redownloaded the OBCI32_SD
     - downloaded the newest GUI
     - as the "~"-sign is badly accessible in the danish keyboard layout, i've changed the following in the "OpenBCI_32bit_Library_Definitions"
    ----------------------------------------------
    #define OPENBCI_SAMPLE_RATE_SET '~'
    To
    #define OPENBCI_SAMPLE_RATE_SET 'f'
    --------------------------------------------------------------------------------

    Now, the problems i had after both the first and the second time was, not being able to pres "~ ~" or "f f" and get the sample rate. (And neither getting any response when doing "~ x"/"f x" (where x is a number))
    and not being able to save anything useful to the sd card, whenever i press "a", it just sorta freezes up for a second and the EEG graph in the gui is then updated extremely slowly.
    The resulting data whether i let it run for 5 minutes or just stop it right away is shown in the attached screenshot.
    Whenever i try to convert the sd file through the OpenBCI_Gui run with Processing (Choosing Playback(from file) -> Select SD File), Processing delivers the following error "NumberFormatException: For or inputstring: "8���...."

    Would any of you happen to have any advice, as too what to do next ?

    (the screenshot can be found at: https://ibb.co/iSAt7k)
  • tskipstskips Denmark
    edited June 2017
    So, i just decided to test whether i had actually uploaded the new firmware:

    I changed : "#define OPENBCI_CHANNEL_OFF_1 '1'"
    To : "#define OPENBCI_CHANNEL_OFF_1 '9'"

    Now when pressing 9, channel 1 turned off, indicating that the code was uploaded correctly.
    but when clicking "f 2" it merely turned of channel 2...

    have i misunderstood the whole "f + number" thing. (I'm currently pressing and not holding f, then pressing and not holding a number) .
  • Are you placing a space between the two character? Don't do that if so. Let's start there :)
  • tskipstskips Denmark
    No, absolutely not. I had merely misunderstood the whole "sending commands" stuff though. I was thinking that i would be able to use them as hotkeys, much like the numbers do, for disabling the electrodes.
    I got it working now through Arduino IDE.

    The output file on my SDcard doesn't change though, it still seems rather unlike the sample data in the guide.
  • Can you please post the SD card file?

    Please keep in mind you are using beta software that has not been fully tested yet. We are writing the test procedures for these new features and appreciate your time and effort for helping!
  • tskipstskips Denmark
    Would you like it as raw text, here in a comment?

    I can't seem to find any "attach file" feature?

    and again! thanks a lot! I really appreciate your effort!
  • wjcroftwjcroft Mount Shasta, CA
    @tskips, hi.

    DO NOT post large files by inserting as raw text. Instead upload to Google Drive and post the link here.

    Thanks,

  • Why are there so many? Which one am I supposed to look at?
  • tskipstskips Denmark
    edited June 2017
    I tried numerous times, i thought more data would equal a higher chance of identifying something.
    they are all separate failed attempts, where ether the gui froze, or the connection was lost.

    The one that looks the most like what i've commonly seen is this one: 
    https://drive.google.com/open?id=0B90vk1SBDirubVp4ZElneUJTXzA

    If you want me to, i can test it more rigidly another day.
  • This is great. @tskips you have done well. And to confirm you are running v3.0.0-beta2 correct?
  • @biomurph is the resident SD expert. Check this out!
  • tskipstskips Denmark
    i am running v3.0.0, which i downloaded yesterday!
    I do not know where to verify if that is the beta2 version.

    I should mention though that this problem did seem to persist from before i uploaded the v3.0.0 firmware to the board,
    which makes me wonder whether the cause is actually something about the sd-card... (my current one is a 16 gb samsung, class 10, uhs speed 1)

    I'll try testing with another one, as soon as i've found one, on of the next days.
  • tskipstskips Denmark
    Hello!
    I am back and sorry about the long delay. 
    I have now re downloaded the v3.0.0-beta2, uploaded this to the board and gotten myself a brand new Class 10 micro SD, which have been formatted twice with the SDFormatter application before use.

    Using the arduino serial monitor i then firstly sent "~1" to change the sample rate.
    next send "a", to record signal, followed by "j", after some time.
    next i changed the sample rate back to 250Hz by sending "~6"
    and send "a". which resulted in an error, which can be seen in the following log:
    https://drive.google.com/open?id=0B90vk1SBDiruZGdzbGR6RjVKNEU

    The first file was empty/filled with blanks, and so it seems impossible for me to derive anything from this. 

    Should i move this to a new thread?
  • Can you please try beta6?
  • tskipstskips Denmark
    So, after uploading beta6, i applied the a test signal by sending "-".
    i then proceeded the same way as above. 

    I did this four times, the first three were blank, the fourth 8000 Hz was blank, but the 250 Hz recording was this:
    https://docs.google.com/document/d/14-zUbAzOigYmWnGdEaJucfVBhH7BCHIcxk_Xl3E5Pck/edit?usp=sharing
  • Ok this is great. I will try to find an SD card and reproduce and put a patch out for beta7. Thanks! 
  • tskipstskips Denmark
    edited July 2017
    Sounds great!

    But once again i would like to mention that this problem persists from when i was using v2.x.x.
    i just tried a fresh download of: 
    OpenBCI_32bit_Library v2.0.1
    OpenBCI_32bit_SD

    and uploaded these to the board, the results are exactly the same.

     EDIT: 
    I did this, thinking that the 2.0.1 build would be stable and tested, dont hesitate to tell me if my reasoning is off here.
  • v2.0.1 is stable... There may be an issue with your SD card. Nothing really significant changed here and the SD code should work. Has anyone else been able to get SD working with v2.0.1? Or 3.0.0?
  • @tskips were you able to try with a different SD card? 

    The v2.0.1 is what has been flashed to boards for almost a year with minimal SD issues.
  • This is super exciting.   Two quick questions:
    1. Does the OpenBCI_GUI software support a wifi connection yet?  Will it?
    2. Is there a possibility to open a UDP port on the Wifi shield to listen for Markers/Stimulations to add to the data streams?
    Thanks for the great work
  • edited August 2017
    @gvzb19

    1. Yes. Looking to release a 3.0.0 release candidate by end of week which supports wifi. I had to rip out the whole back end of the GUI lol and moved all mission critical board connections to an electron hub. So now a single cross platform sub app serves the data for cyton, ganglion and both over wifi.

    2. It's open source, there are tons of examples of UDP on ESP8266 Arduino (some nice Google terms right there!) But udp is too basic for a reliable connection after testing and testing so I didn't want to focus on it for the 1.0.0 release. There is an open issue for adding udp support on GitHub, you should comment on that too track whenever the feature is added. I must admit, I NEVER thought of using udp as a stimulus channel, that's pretty dang interesting!
  • @pushtheworld,

    Thanks for your response.  I can see from all the posts and excitement for wifi that you are committing a lot of time here.

    I am not dev with superpowers.  However, I am an experienced hobbyist with knowledge of C/C++, Java and Python.  (I have some limited experience with ESP8266 on the arduino) In other words, with some guidance from you, I may be able to do some of the work to make this happen.

    So here are the questions: 
    1. Where can I buy a wifi board?
    2. Where can I find your code for the firmware and GUI?

    Thanks for your help!!!
    Gerrie




  • One more question, since the wifi is so much faster than the serial bluetooth would it be appropriate to add a "marker" channel to the data now - just a simple int.  If we do it now then all the raw data files will have the correct format.  If we don't then we need some way to identify whether a new stream has a marker or not.

    I don't think we need to be backwards compatible with pre-wifi software since bluetooth can't support markers anyway
  • edited August 2017
    @gvzb19

    Thanks so much for your interest! 

    1. In short, you cannot yet. I just flashed and functionally tested 95 boards yesterday, they are working great!! They will be on the store in a couple weeks, trust me, i'm pushing as fast as possible to get these listed for sale haha

    2. You should go to my GitHub. The OpenBCI_WiFi v1.0.1 is stable, the docs are almost done. The GUI, please just wait, for real, it's not stable right now, will be by the end of the week. As for the Cyton and Ganglion, idk which you have, but there are release candidates out for their respective wifi support firmware (3.0.0 for Cyton and 2.0.0 for Ganglion) on their GitHub repositories. I always put out release candidates in the official repos before a merge with master. For the most current wifi info I would recommend following me on twitter @pushtheworld_aj and you can monitor and see the daily progress of the wifi shield. 

    There is a great getting started guide in the works on the docs.openbci.com that you can take a look at. We also need python, matlab, C++, java examples of getting data from the wifi shield.

    In regards to triggers I think you should wire a photo diode to the board and put that on your screen and tag your data stream that way. The 3.0.0 firmware adds support for board mode switching which allows you to do analog reads and digital reads with just a simple ascii command, no reprogramming needed.


  • AJ, 

    Sorry for the trouble - I don't do Twitter.  Any chance I can preorder a shield?  Paypal?

    One quick question: does the current GUI work with the V3.0 firmware? 
    (I have made some changes to the V3.0 firmware that accepts makers and inserts them into the auxData[0] stream)




Sign In or Register to comment.