BCI-Controlled AI Music Improvisation in Max/MSP

Hello all,

Greeting! I am a composer from Taiwan.

I would like to share my recent project that I have been working since 2025. This project focuses on the interaction between AI music improvisation (based on Somax2 of Max/MSP) and EEG (using Ultracortex). Here is the short demo of my work:

In this experiment, EEG signals are captured through the headset and used to estimate different mental and affective states, including arousal/valence and concentration. The EEG data are first sonified in Max/MSP, transforming brain activity into sound. These resulting sounds then interact with the parameters of Somax2 in real time, creating a continuous musical dialogue between brain activity, sound, and machine improvisation.

I have been experimenting with an Affective State widget for the OpenBCI GUI (v6.0.0-beta.1) and would really appreciate some feedback from the community! As I am not from coding background, so I created Arousal/Valence detection widget with help of Claude and ChatGPT to see if this is possible to work out...I have attached the photo of the widget in this post! [] (https://openbci.com/forum/uploads/editor/1r/hx9zyd3h6cbq.png "")

Comments

  • wjcroftwjcroft Mount Shasta, CA

    Hi Sammaaly,

    Wow, very impressive! Innovative the way you combine the initial sonification with Somax2. I like the jazzy brain-riffs !

    Our GUI developer, Richard @retiutut, is also a Max/MSP user. He may have further comments on your GUI widget, do you have a Github or source code repository?

    I happened to find your bio online, for those interested in your other composing,

    https://archive.ncafroc.org.tw/composer/composer_file?id=4028b7828d0cc097018d216cd7f81311&lang=en

    And your Youtube channel,

    https://www.youtube.com/channel/UC10moIapJ1hxFOWAttDXtIg

    Our 'Community' page is currently getting some maintenance, but you might consider posting there as well once it comes back.

    https://openbci.com/community/
    https://docs.openbci.com/GettingStarted/Community/Community/

    NeuroTechX has a Slack, with thousands of members, that is another place to post BCI projects:

    https://neurotechx.com/ [click on the Community tab, then select Slack]

    Best regards, William

  • Which Somax2 parameter is easiest for a listener to hear changing? For feedback on the mapping, I'd start with just that one parameter and save the widget's control trace during a short performance. Replay it once in its original order and once shifted by several bars, keeping the musical input and any random seed fixed. With the versions unlabelled, can listeners tell which one responds at the moments the performer intended? That would give you a musical test of the timing. Whether the widget identifies arousal or valence accurately is a separate question.

  • Hello everyone,

    Thank you all for your replies! My apologies for the delayed response. This semester has been particularly busy.
    I have now created a GitHub account and uploaded my Affective Widget, together with installation instructions:
    https://github.com/yhc219/arousal-valence-widget-for-OpenBCI-

    The widget was developed with assistance from ChatGPT Codex and Claude AI. At this stage, I treat its outputs as exploratory EEG-derived indicators rather than validated measurements of emotional states. Establishing their relationship to arousal and valence would require further validation against independent measures, such as self-report ratings, potentially supported by complementary physiological measures such as heart rate variability (HRV).

    For now, I am using the widget to explore EEG-driven interaction with Somax2 in sound art creation.

    In my current setup, the arousal-related proxy controls the balance between external and internal influences in Somax2. Lower values give greater weight to external sound sources, while higher values give greater weight to internal influences.

    The valence-related index controls the volume levels of additional Somax2 players, shaping the orchestration and musical texture. The project currently uses four players.I have also uploaded a new demonstration video to YouTube:

    Although the project is still at an early stage, I am exploring whether changes in these EEG-derived indicators may accompany listening to and interacting with Somax2.

    So far, I have conducted exploratory two-minute sessions with five participants. These sessions are part of my sound art research, and I hope to refine and improve the experimental design and methodology as the project develops.

    The attached JPG files show the average band power and derived arousal proxy for one participant. The session consisted of two phases, with participants keeping their eyes closed throughout:

    0:00–1:00: Resting in silence.

    1:00–2:00: Listening to the sound generated by Somax2, and the arousal/valence values are streaming to Max/MSP.

    Across these preliminary sessions, I observed fluctuations in the arousal-related value in most participants. I still think that further study is needed to understand how these fluctuations relate to their listening experiences and interaction with the system. Perhaps I will do, as you suggested, the self-reported ratings from the user.

    I would welcome any thoughts or suggestions, particularly regarding the signal interpretation and experimental methodology.

    Thanks again for your kind response!

  • sammaalysammaaly Taiwan
    edited November 15



    Sorry, I forgot to attach the JPG file of the charts of band power and arousal values.

  • Thanks for sharing the two-minute protocol. Since silence always comes first, the change at one minute could reflect hearing music, the passage of time, or the feedback itself. For a next session, would you consider two listening blocks? In one, the listener’s EEG controls the mix. In the other, they hear a recording from an earlier live block at the same volume. You could alternate which comes first across listeners and ask after each block when they felt the music respond. That gives you a way to study the experience of playing the instrument while leaving the emotional meaning of the EEG signal open.

Sign In or Register to comment.