Subvocal Phoneme Recognition using Facial EMGs | NeurotechSC

This year, NeurotechSC pursued a project which identifies intended phoneme vocalization utilizing facial electromyography (fEMG). This was achieved by training a LTSM/RNN machine learning model on fEMGs collected by various dry surface electrodes placed among different muscle groups that are used in speech production. Using this model, initially only 5 distinct phonemes were able to be recognized, then in time the model was extended to predict all 44 distinct English phonemes. 

The fEMGs were recorded with an OpenBCI Cyton Board along with the OpenBCI GUI. The OpenBCI 8-channel Cyton Board data was collected at an amplification factor of 1170, 16 bits A/D conversion, a resolution of 0.298 microvolt/ bit, with EMG signals sampled at a rate of 250 Hz and a frequency range of 0.9-295 Hz. The board was modified to allow for a push-to-talk interface. Additionally, the OpenBCI board allowed for a continuous stream of fEMG data that could simultaneously be passed through the machine learning model using the python library Brainflow. 

Due to time constraints, we were not able to fully realize a few aspects of the project. In the future, we hope that we are able to implement an electrode mask in order to make the fEMG recordings more efficient, a user interface that would allow for the predictions to be classified as incorrect/correct, and to pass the phoneme predictions to another model to predict text. If these obstacles were to be overcome, we believe that this project would achieve subvocal speech transcription and have the potential to aid those with speech related pathologies, as well as expand upon human-computer interfaces. 

A full video of our project submission may be viewed here:
NeurotechSC Phoneme Recognition Project Submission 2023

NeurotechSC is extremely grateful to accept 1st place in the NeurotechX Student Club competition. We would like to formally thank OpenBCI for the technology and the platform to pursue this project. Additionally, we would like to thank NeurotechX for providing NeurotechSC with an avenue to pursue neurotechnology research at the undergraduate level and a community to support us in this niche field. Lastly, thank you to the members of our club that have persevered through this challenging year. We applaud your patience with our newly found cohort and congratulate you on your victories in the past and the many more to come. 

Our Team (in alphabetical order)

Project Leads: Mathew Sarti, Nivriti Bopparaju

Hardware Team: Ben Cooper, Chris Bjordahl

Machine Learning Team: Dennis Koshta

Software Team: Rico Rodriguez Passanisi