FFT data parsing in Max/MSP
shaluska
Ohio
Hey OpenBCI community,
I have been successfully transmitting data from the OpenBCI GUI to Max/MSP via OSC with the Band Power and Average Band Power data types. I figured this out without too much trouble and have been creating interesting sonification patches reflective of the different brainwave types and channels. However, I am struggling to transmit the FFT data. I assume it's some syntax error with my OSC objects but I was hoping someone could help guide me through how this works. I know that I will end up with a large amount of data--125 bins per channel but when I transmit this data, I am almost entirely seeing empty bins.
[OSC-route /openbci/fft/ch0]
Should I be able to unpack the 125 bins from channel 0?
[unpack f f f f f f ...]
I might receive one channel of data with the read of something like "/bin124 0.01" in a message box but when I [pak] all of the bins into a message box, the "0.01" moves to the bin 0 string location as opposed to its original bin 124 location. Some channels have a few bins receiving data but again, almost entirely empty. I'm clearly missing something here with FFT data via OSC.
Thanks in advance for your help!
-S
Comments
Hi Shaluska,
I'm assuming you have read the format of the data in this doc:
https://docs.google.com/document/d/e/2PACX-1vR_4DXPTh1nuiOwWKwIZN3NkGP3kRwpP4Hu6fQmy3jRAOaydOuEI1jket6V4V6PG4yIG15H1N7oFfdV/pub
Linked from:
https://docs.openbci.com/Software/OpenBCISoftware/GUIWidgets/#networking
Hi Wjcroft,
Thank you for the reply. Yes, I am familiar with the document that you linked and it helped me understand how to format other data networking options. Am I not able to transfer FFT data with OSC and only through UDP? I would prefer to transfer the FFT data with OSC as this is the preferred protocol for audio applications and for the software receiving the data, Max/MSP. I am able to transfer band power data just fine with OSC from the OpenBCI GUI to Max/MSP without needing to transfer the data with UDP as JSON messages. I feel stumped as to why FFT wouldn't work much the same way.
Ah, apologies for mentioning the UDP section of the document instead of the OSC section. I do think the OSC packets are encapsulated in UDP. Max CAN deal with UDP directly. Here is a page that mentions issues with high data rate incoming packets to Max. And the need to process OUTSIDE the UI thread.
https://www.reddit.com/r/MaxMSP/comments/14tmtzs/is_max_msp_not_powerful_enough/
Are you saying that you are not seeing the format the doc presents:
It's possible this was never fully debugged / tested with a Max/MSP app. These are the code sections:
https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/W_Networking.pde
https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/W_FFT.pde
Richard @retiutut, is it possible there is a glitch in the data packets sent when OSC FFT mode is selected?
William
Hi William,
Thank you, I appreciate your insight. I believe you're right; Max can deal with UDP directly and that OSC is encapsulated in UDP. I'd prefer to stick with OSC because I'm using the data for audio applications and therefore I've built the Max patch to receive OSC but if necessary, I can rebuild it to receive and parse the data as UDP packets instead. I'm not technically sending audio signals so it probably doesn't actually matter but I have more experience with OSC, as I am coming from a music background.
Thanks for the reddit post link. I'll look into more efficient ways to thread FFT data packets in Max. While I'm sure this would help, I don't think it explains what I am seeing. I'm not experiencing lagging that the reddit post was describing nor a spike in CPU as I might expect. You're correct, it's that I'm not seeing the FFT data formatted the way that the doc presents and possibly because of this formatting, I am not receiving the full packet of data. I can follow up with some screen shots of Max when I'm in the lab tomorrow.
Thanks again!
-S
If you can save / print the text data packet that you get from OSC, that might help @retiutut to have some insight what is happening.
Hi William,
Unfortunately, I can't print the data because I am receiving an error message with my OSC-route objects that are associated with the FFT data, "OSC-route: doesn't understand 'float'." Even though some of the data makes it through to the various routing as a float.
What I am trying to do is parse the data coming in from the port, parsing each channel:
[OSC-route /openbci/fft/ch0]
Before unpacking each channel's individual data packet.
When connecting a message box to the output of the OSC-route object, I can receive a data reading but just for what appears to be a sing bin, bin 124. For example a reading for Channel 0 reads as "/bin124 0.013945"
Another thing is that when I unpack each channel's data packet, this data for bin 124 appears in the second bin position. Nearly all other bins are reading as "0." with the third bin reading some integer that doesn't make sense to me, for example Channel 0 has the third bin reading "45"
I have attached some screen shots of my extremely messy, in-progress max patch.
Thanks again!
Hi again, @wjcroft and @retiutut
I'm just following up to see if you have any suggestions for how to get the FFT data into Max. Thanks
Hi @shaluska, I just mentioned you and your Forum thread here on a staff message board. If you don't get a reply, please send an email to (contact at openbci.com) and that will create an official customer support ticket.
Regards, William
I've marked this as saved for later internally for myself to answer. We are currently finishing up a large software release for Galea and there are a few higher priority tickets in front of this. Just checking in that I've put this in my queue.
Great, thank you for the updates. I'll hold tight.
Ok, I've identified an issue with the GUI OSC output for FFT. I've put in a fix along with some Javascript code that can run in a Max
jsobject.https://github.com/OpenBCI/OpenBCI_GUI/pull/1264
Can you please run this branch from source along with the Max and Javascript parts?
I reached for Javascript because this can now be cleaner than using Max objects and subroutines
Hope this helps,
RW
Richard, thanks very much.
@shaluska, you can view the steps for running the GUI in the Processing IDE here:
https://docs.openbci.com/Software/OpenBCISoftware/GUIDocs/#running-the-openbci-gui-from-the-processing-ide
Hello again and thank you for your help from 4 months ago!
Revisiting this again. I have opened the code in Processing and was looking to implement the fix but it looks different than the screenshot Richard sent me. According your screenshot, there should be a NetworkStreamOutOSC.pde tab/in the libraries. I'm not seeing this, however, I am seeing a NetworkStreamOut.pde. There, I have found a section for sendFFTData but the code looks different than your screenshot. I've attached my own screenshot of what I'm seeing in Processing below.
I'm working on a Mac if that makes any difference.
Thanks again!
Mentioning Richard @retiutut, who is an expert on Max / PD. And the developer for the GUI.
Hi @shaluska,
I may have created some confusion by pointing to the 'standard' link for loading the GUI in Processing:
https://docs.openbci.com/Software/OpenBCISoftware/GUIDocs/#running-the-openbci-gui-from-the-processing-ide
That 'standard' instruction does not apply 'exactly' to the fix that Richard did. Because his fix is not yet incorporated into the released GUI source. His fix is what is called a 'pull' request.
https://github.com/OpenBCI/OpenBCI_GUI/pull/1264
I'm not a Git / Github expert, so I don't know the exact commands to force Git to incorporate this pull on your local copy. However a workaround might be to fetch the small number of files affected and manually copy them over the default / standard source that you already have. The list of files affected is here:
https://github.com/OpenBCI/OpenBCI_GUI/pull/1264/files
Green text means 'added' source code. Red text shows 'deleted' source code lines.
Regards,
However when I added the NetworkStreamOutOSC.pde file (from the pull) into OpenBCI_GUI folder/libraries and run the program in Processing, it comes up with error message, "Cannot find a class or type named “NetworkDataType”" in console with several error messages in the errors tab.
Am I adding this additional file in the incorrect location?
Thanks for all the insight.
It replaces the existing file in the main OpenBCI_GUI folder:
https://github.com/OpenBCI/OpenBCI_GUI/tree/master/OpenBCI_GUI
But there isn't an existing file with the name NetworkStreamOutOSC.pde
OK, well seems to be TWO related files, the one that is there, NetworkStreamOut.pde, and the new one NetworkStreamOutOSC.pde
So don't REPLACE, just add the new file.
Correct and this is what I did, which resulted in the error messages when attempting to run it in Processing
Have you tried a 'grep' or string search for the function name NetworkDataType ?
Richard @retiutut, hi.
User @shaluska is trying to use Processing IDE to run your fix pull request. But is getting an error that NetworkDataType is not defined anywhere. I just went to the Github GUI folder and indeed that definition is nowhere in the default GUI source, nor in the pull request source.
Any insights?
Regards, William