EMG Multiclass Classification for Forearm

ThilinaThilina Toronto
in Hardware

Hello,

I am using the Cyton board with 6 channels and IDUN Dryodes to collect data to train a CNN to recognize 4 different forearm positions as I livestream. I used the Brainflow and tensorflow libraries to collect data and build the CNN and then stream data to test. I have trained several models trying various parameters, data collection methods and CNN layers. I am able to classify at most 3 motions accurately but when I use the same model and reapply the dryodes the next day, it no longer works.

Here is a video of what I had last night (does not work now):
https://www.youtube.com/shorts/7bjFltX8CKI

I want to figure out what might be causing this so was wondering if anyone had advice on things I should change. I am attaching images of the 4 positions I would like to classify and of my electrode placements as well as how I set up the board (please excuse the makeshift look):




I have used the plastic feet to screw the board down to the wood block (one screw is not in because it went through angled as I did not bother to pre-drill). I taped the cables and battery pack to the block to prevent as much motion from the cables to the pins as I move my arm. Note that the channels connected are 3 to 8 (6 total).

My methods to collect data were as follows:

  1. Hold one of four positions and stream data form the board
  2. Create a time series sample of 140 every 0.02 seconds using Brainflow get_current_board_data(140) (This allows me to overlap samples but with enough delay that I don't have completely identical samples several times)
  3. Filter the sample with Brainflow perform_bandpass on each channel with an order 4 Butterworth bandpass filter from 25Hz to 240Hz
  4. Rectify the sample
  5. Denoise the sample with Brainflow perform_rolling_filter on each channel with median and period 3
  6. Clip the first 20 elements in the sample resulting in 120 timesteps (I do this because the filters seem to leave a huge spike at the start of each sequence)
  7. Collect all the samples into a 3D array of 5000 samples (I held the position for about 100 seconds, resulting array shape: 5000, 120, 6)
  8. I collected 10000 samples for each of the 4 motions over 30 minutes and scale the combined training set of shape (40000, 120, 6)

I then pass this to the CNN with the following architecture:

After the model is trained and saved, I import the trained model in a separate file to stream data and perform the exact same steps on the sample (collect 140 timesteps, filter, rectify, denoise and clip) before passing it through the neural network.

I know that the IDUN Dryodes mention that the impedence decreases over time. I assume this is because as you sweat, the salt water increases conductivity between skin and electrode? I record the time between once I have finished applying the electrodes and when I start and finish collecting samples every time. Typically I try to wait 1 hour after applying the electrodes and then I collect samples for an hour.

Again, almost always the models are able to classify at least 2 motions (the clenched fist always works) and sometimes 3. So far I have been unable to do all 4 and no matter the model, my test the next day on the same model completely fails.

Also, I have read online and in the forums that EMG data is best filtered from 25-400Hz, which is why I used the 25-240Hz bandpass. I do not have the WiFi Shield to increase the sampling rate and it is not an option for me at the moment.

I would like to know if I am doing things wrong and how to change them. Specifically, for my purposes, is the bandpass filter set appropriately? Are the electrode placements suitable?

I am going to try a few different methods now. I am going to collect 500 samples at a time (hold position for 10 seconds) because I have seen studies that say the muscle fatiguing over time greatly affects the EMG waveform. I am also going to change 1 of the electrode placements from the bicep to the forearm because I think I will do different hand positions. Instead of supination and pronation, I will try wrist flexion and extension.

Thank you for any advice if possible.

Comments

  • wjcroftwjcroft Mount Shasta, CA
    edited May 2023

    Thilina, hi.

    Thanks for your detailed and advanced post on EMG ML applications.

    Just looking at your comments on the Dryodes at the moment. And their mixed performance based on time used. Have you tried just using more typical (disposable) EMG / ECG sensors, which use silver chloride buttons and gel?

    https://shop.openbci.com/products/skintact-f301-pediatric-foam-solid-gel-electrodes-30-pack
    [Shop version]

    Actually you can find even larger packs of these online.

    https://www.3m.com/3M/en_US/medical-us/red-dot-ecg-electrodes/
    [photo at bottom of that page shows the huge size range available]
    https://multimedia.3m.com/mws/media/1415893O/3m-red-dot-electrodes-ecg-electrodes-product-comparison-chart.pdf

    https://www.amazon.com/3M-2570-Electrodes-Foam-Braider-3M/dp/B004Y8CZ4I

  • ThilinaThilina Toronto

    Hello again William,
    Thank you for your encouragement! I have not tried the disposable electrodes because I expected to make several mistakes in the process of this project. I did of course, and had to apply and reapply the Dryodes several times over. Now that I am more confident with the application, I will try your suggestion. I will look to buy the disposable ones from somewhere that either delivers quickly or has them in store as I live in Canada and need them soon.

    I also seem to have greatly improved the model. I did all of the 3 changes I mentioned:
    1. 100 second position holds for sample capture ---> changed to10 seconds
    2. Electrodes readjusted
    3. Median rolling filter ---> changed to mean

    It now recognizes all 4 motions! I will post an updated video later tonight once I have completed tracing new position images for my display function. I'll also post a link to my code once I learn how to use GitHub...

  • ThilinaThilina Toronto

    Sorry for the long delay on my follow-up. I wanted to make sure that I would avoid misguiding others on the forums looking to do similar projects.

    I have been able to recognize the different motions I wanted, however, there are some issues with my project and I would like to resolve them. Here is a video link of what I had as of 2 weeks ago:

    As seen, the script can recognize 4 positions with high accuracy. As in the description of the video, the few moments where incorrect positions are displayed are typically due to either the switching between positions or certain positions being difficult to hold for long periods. This can easily be resolved by only changing the display if the multiclass probability is above a specified threshold.

    My issue is that the data collected can vary significantly between days, the same issue I had before. I have ordered new electrodes (3M 2560) which I will be using today to see if they provide more consistent results compared to the IDUN Dryodes. I also asked on the Brainflow Slack if the order I was performing the filters were correct and for now it seems it is okay. I mentioned there, that on the OpenBCI GUI, the readings always seem consistent. The only time I notice a huge variance in recorded values is when I collect data using Brainflow in code. Andrey also mentioned that the GUI displays stddev.

    I would like to replicate the EXACT conditions from the GUI in my recorded data. Below are the filters I want and the code I used:


    In order these are:
    1. Notch filter on frequencies, 50Hz and 60Hz
    2. Bandpass filter between 25Hz and 110Hz (Butterworth, order 4)
    3. Rectify
    4. Rolling filter (mean, period 3)

    I would like to know exactly what the GUI is doing between the "Time Series" and the "FFT Plot":

    I would appreciate if someone could confirm that I am performing the filters in the correct order, and what order to implement the rolling standard deviation and any other operations on the signal. @wjcroft

  • wjcroftwjcroft Mount Shasta, CA

    Thilina, hi

    Great work on your EMG application.

    Richard @retiutut may comment here regarding the order in which the GUI performs filtering / operations. You can always view the code, for example the individual widget codes:

    https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/W_TimeSeries.pde
    https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/W_EMG.pde
    https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/W_FFT.pde
    https://github.com/OpenBCI/OpenBCI_GUI/blob/master/OpenBCI_GUI/OpenBCI_GUI.pde [main program]

    William

  • retiututretiutut Louisiana, USA
    edited July 2023

    https://github.com/OpenBCI/OpenBCI_GUI/blob/5547183de001c37c525e58f5f1d4d87524f029c2/OpenBCI_GUI/DataProcessing.pde#L111

    I'm not sure that the order-of-operations for the filters would really change the output, though it could be fun/interesting to see the difference using BrainFlow synthetic board. I would not try such experimenting using a live board, or even a live board + signal generator, since feeding a generated sine wave is more reliable.

    Also, I'd be interesting in reviewing any papers on this subject.

Sign In or Register to comment.

© Vanilla Keystone Theme 2024