This tutorial shows how to integrate MindAffectBCI with Unity to make a mind-controlled game like in the videos below.
What you need:
OpenBCI board (Ganglion Board or Cyton Board)
EEG Headband or cap
MindAffectBCI
EEG electrodes + cables
1. OpenBCI, MindAffectBCI and Headband configuration
For OpenBCI and MindAffectBCI configuration, follow steps 1 to 3 at this link.
2. Unity and MindAffectBCI integration
After a successful Unity Hub installation (instruction here) go to MindAffectBCI-Unity repo at this link and clone the project.
At this point you have three options:
1. Experiment with samples from MindAffect repo (go to step 3)
2. Clone a repo with a mind-control game from the videos above (go to step 4)
3. Create your own game (go to step 5)
3. MindAffectBCI samples
After cloning a project from MindAffect you should have three folders in a repository directory:
fake_recogniser – contains a recogniser that you can use for testing without a connection with BCI. Just remember to run startFakeRecogniser.bat before running a Unity project.
Folders fpsshooter and minimal_presentation contain sample projects that you can open in Unity. Open Unity Hub and add a project:
After adding a project you should be able to run a game (fpsshooter from a repo directory in the case below):
If you are not using a fake recogniser please make sure to run mindAffectBCI app and perform a calibration before running a game. Instruction here
4. FPS with a mind-control configuration
Clone a repo from this link. Open Unity Hub and add a project.
Before running a game please make sure you have OpenBCI, MindAffectBCI configuration and Headband with EEG electrodes. If not follow steps at this link. And remember about Calibration!
You can also use a fake recogniser in mindAffect app directory mindAffectBCI/hub if you dont want to use a connected BCI:
After adding a project to Unity Hub and BCI configuration (OpenBCI board, MindAffectBCI calibration) you should be able to run the game:
Game controls:
W, A, S, D – Forward, Left, Backward, Right
Space – Jump (Double press for double jump)
Mouse Left Click – Fire
Please be aware that I am developing this game and there might be frequent updates.
It is still an early stage – a proof of concept.
5. Unity + MindAffectBCI integration
If you successfully followed steps 1 to 3 from this tutorial clone a MindAffect Unity repo from this link.
Create a new project in Unity Hub:
Go to mindAffect Unity repo directory fpsshooter and copy:
Assets\FPS\Scripts\Noisetag
Assets\FPS\Scripts\NoisetagBehaviour.cs
Assets\FPS\Scripts\NoisetagController.cs
and paste to your Unity project Assets/Scripts folder.
Create a new script (you can name it ‘StartNoisetagFlicker’) in your Assets/Scripts directory and add this line to your Start() function (you can change a value 50, it is a max number of predictions):
NoisetagController.Instance.startPrediction(50);
If you run a game in Unity and you get this error
go to Noisetag.cs and in line 841 make sure you have a correct path
In the next step add an empty GameObject in the scene (right click -> Create Empty) and you can name it as you wish (in my case NoiseTagConfig).
Click on it and go to inspector on the right side of a screen and add two components: NoisetagController and StartNoisetagFlicker (that you previously created)
Expand the NoisetagController and add a codebook file from your Assets/Scripts/Noisetag folder:
For a testing purpose to verify if MindAffect works in Unity create an empty GameObject in the scene for example a Sphere.
And add a new component with a script NoisetagBehaviour in the inspector window.
Drag Main Camera from the Scene (a window on the left side of the screen) and drop it on Cam Object.
Click on + in the Selected Event field and drag and drop Mesh Renderer and select MeshRenderer.enabled function
When you run the game a sphere should be flickering until the recogniser responds. If it disappears it means that the recogniser responded, event was triggered and the function disabled the sphere.
I hope these instructions were helpful.
Good luck with your experiments!
Pyotr
Leave a Reply
You must be logged in to post a comment.