Alpha oscillations (8-14 Hz) are one of the most prominent electrophysiological signals from the human scalp. Previous research demonstrated that alpha activity is modulated by covert spatial attention and these modulations could be used as a control signal for brain-computer interface (BCI). The main goal of this study was to build a classifier, on a trial-by-trial basis, of EEG recordings during both offline and online covert visuospatial attention tasks. Alpha power imbalance measured while the user-oriented attention to left or right hemifield was used as control signal for a closed-loop BCI system based on the EEG signal. The study was divided between an offline data exploration stage and an online implementation stage. In the first stage, I used a dataset from a covert spatial attention task using a Posner paradigm to explore a set of parameters (e.g., electrodes, frequencies, and trial selection strategies) to find which of these parameters were more suited for BCI. The decision was based on the performance of the classifier. In the second stage of the study, I designed a Posner task and used the parameters selected in the first stage to train a classifier for the online task. In the online BCI setup, ongoing brain EEG activity was recorded, classified, and used to provide real-time feedback to subjects about their internal attention state, according to the output of the classifier. Overall, offline and online classification confirmed that it is possible to discern attention shifts to the left and right locations based on modulations of power over the posterior alpha rhythm. Nonetheless, the performance of the classifier was lower than expected and possible modifications are proposed to improve it.
Keywords: covert visuospatial attention, alpha rhythm, lateralization index, brain-computer interface, closed-loop BCI, brain oscillations, real-time EEG analysis.
What is covert visuospatial attention?
Attention facilitates the visual processing of task-relevant stimuli while inhibits processing of task-irrelevant elements, allowing enhanced processing of relevant information (Pashler 1999). Although attention can be directed based on various criteria including spatial locations, features/objects, or sensory modalities, the present study is focused on endogenous covert visuospatial attention (see Pashler (1999) for other types of attention). Covert attention is the act of selecting visual stimuli in space without moving the eyes or the head. One example of a visual task using this strategy is the Posner’s standard cueing paradigm (Posner 1980). In this task, an attentional cue (for instance, an arrow) directs subjects to attend to a particular spatial location, and a subsequent target requiring a response appears either at the cued location or elsewhere. At the behavioral level, higher detection rates, quicker reaction times and higher performance are usually expected for targets appearing at the attended location as compared to those appearing in unattended locations (Posner 1980).
In the following section, I will explain the relationship between attention and neural activity and, more, in particular, the role of alpha activity in covert visuospatial attention.
The functional role of the alpha activity in covert spatial attention
The role of human alpha oscillations (8-14Hz) in attention processes has been studied intensively in the past. Some studies using electrophysiological (EEG) or magnetoencephalography (MEG) measurements have investigated how the synchronous activity of populations of neurons in parieto-occipital regions change with attention. In the case of visuospatial attention deployment, decreases of alpha activity have been observed in the hemisphere contralateral to the attended visual field (Sauseng et al. 2005; Thut et al. 2006), interpreted to reflect enhanced cortical excitability to facilitate future visual processing at the attended position. In addition, alpha increases have been documented in the ipsilateral hemisphere to the attended location (Worden et al. 2000; Yamagishi et al. 2003; Kelly et al. 2006; Rihs et al. 2007), potentially reflecting an active inhibitory process protecting against visual input from task-irrelevant positions. Based on these results, it has been found that the direction of visuospatial attention can be determined by the spatial distribution of alpha activity in parieto-occipital regions (Rihs et al. 2007; van Gerven, Jensen 2009; Bahramisharif et al. 2010). Topographic maps in Figure 1 show the classic pattern of alpha power (i.e., desynchronization in the contralateral hemisphere and synchronization in the ipsilateral hemisphere) in posterior regions when attention is covertly oriented to the left or right visual field. Hence, the differential changes of alpha activity over hemispheres can be expressed through an imbalance in alpha power, incorporating in this way the relative distribution of alpha activity over hemispheres in one value. Some examples are the logarithm of the left hemisphere alpha power divided by the right hemisphere alpha power used by Kelly et al.(2005a), or the lateralization index of alpha by Thut et al.(2006).
Several studies have investigated the dynamics of alpha activity lateralization during covert visuospatial attention tasks. As shown in Figure 2, alpha activity develops attention-related changes at ~400-600 ms following cue onset and holds over the post-cue period to reach maximal values towards the end of the period (Tonin et al. 2012). This timing is in agreement with previous studies (Kelly et al. 2005a; Rihs et al. 2007, 2009; van Gerven et al. 2009). On that note, maximum performance is expected to be reached at the end of the post-cue period since participants have more time to prepare themselves for the forthcoming targets (Worden et al. 2000; Thut et al. 2006; Rihs et al. 2007; Tonin et al. 2012; Tonin et al. 2013). Accordingly to Tonin et al. (2012), performance based on the first ∼500 ms would not exceed chance level, and the best classification intervals occur from ~500 ms onward, specifically towards target onset.
A large extent of interindividual differences in oscillatory power spectra has been found for different frequency bands because of age, neurological diseases, brain volume, task demands, and memory performance (Klimesch et al. 2007). Some studies use the individual alpha frequency (IAF) as an anchor point to adjust the frequency bands individually and, thus, minimize these differences (Klimesch et al. 1998; Klimesch 1999; Thut et al. 2006; Horschig et al. 2014). Accordingly to Klimesch et al., the IAF is defined as the frequency corresponding to the peak in the alpha range of 8-12 Hz, and IAF-band ranges from IAF minus 4 to IAF plus 2 Hz. In general, the IAF has been proved to be stable across sessions and correlated with cognitive tasks (Klimesch et al. 1998), its use prevents interactions with other adjacent frequency bands (Klimesch 1999), and it is a good individual biomarker for BCI applications (Horschig et al. 2014).
In short, the direction of visuospatial attention is indexed by an alpha power imbalance over posterior regions that can be computed using different methods, such as the lateralization index (Thut et al. 2006). In the next section, I will discuss how the alpha power imbalance created by covert visuospatial attention has been extensively used as control signal for BCI.
Alpha activity modulated by attention as a control signal for BCI
A brain-computer interface (BCI) consists of a hardware and software communication system that constitutes a processing pipeline originating from recordings of brain activity and producing data that assist human functioning (e.g., Scherer et al. 2013). This has diverse applications such as communication and control by patients that suffer from severe disabilities, new forms of human-computer interaction, as well as novel methods of data-analysis in cognitive neuroscience (van Gerven, Jensen 2009). A widely used BCI setup enables the recognition of particular patterns in ongoing brain oscillations following the serialized stages: signal acquisition, preprocessing, feature extraction, classification, and interface control (Nicolas-Alonso, Gomez-Gil 2012). Most of non-invasive BCI research relies on recordings of EEG or MEG due to their high temporal resolution. Indeed, EEG represents the most dominant measurement modality and holds most potential to enable true wearable BCIs promptly since it offers an acceptable signal quality, high temporal precision, direct measurement of population-level neural activity in humans, and low-cost, easy-to-use, and accessible equipment (Cohen 2017).
Figure 3. The architecture of a BCI system consisting of five steps: EEG signal acquisition, preprocessing of the signal, extraction of the feature of interest in the signal, classification of the feature, and a control interface (i.e. control signal) to interact with an external device (e.g., a speller, a mouse cursor, a wheelchair, a robotic arm…).
Several studies have used offline analysis to show that alpha power is a robust signal which can be utilized in a BCI setup involving spatial attention. Kelly et al.(2005b; 2005a; 2005c) showed that posterior steady-state evoked response in the alpha-band power could potentially be used as a control signal for a BCI. They found that external stimulus might not be required and covert attention to spatial locations in the visual field alone may be sufficient to drive a BCI. In addition, they were the first to demonstrate that shifts in covert spatial attention between the left and right visual hemifield can be picked up at the single trial level. Their work was later extended by vanGerven et al. (2009) who demonstrated that spontaneous alpha band activity modulated by spatial attention could be decoded as well. A similar approach was used by Tonin et al. (2012), showing that appropriate feature selection in time-frequency and sensor-space can improve classification performance. In 2007, Rihs et al. showed that the distribution of posterior alpha reflects two-dimensional attention mapping, which can successfully be used for continuous two-dimensional control in an offline setting. Rihs et al.(2009) showed that shifting and maintenance of attention generate different patterns of alpha power modulations in oscillatory EEG activity over posterior recording sites. Treder et al.(2011) found that visuospatial attention shifts could be decoded by using alpha-band power and that classification rates correlate with the strength of resting state alpha-power. While these studies do suggest that modulations in alpha power can be used for BCI control, all studies relied on offline analyses after data acquisition. In 2010, Bahramisharif et al. demonstrated that modulations of posterior alpha activity due to the direction of covert attention has potential as a control signal for continuous control in a BCI setting using MEG. Recently, Tonin et al.(2013) showed for the first time that covert attention brain-computer interfacing is possible using EEG in an online setup. In 2014, Horschig et al. provided evidence that posterior alpha-band activity can be employed as a reliable control signal for continuous online brain-computer interfacing.
This collection of studies demonstrates that modulations in alpha activity with covert attention can be used as a control signal for BCI. In fact, its use might have several advantages compared to other control signals currently used, such as motor imagery (Neuper et al. 2006). First, concerns about muscle artifacts driving the BCI can be better controlled. Nevertheless, in the case of alpha, eye movement and blinks have to be removed properly to prevent confounding the signals of interest (Jensen et al. 2011). Second, covert attention provides an alternative for BCI systems in which visually displayed objects have to be controlled (Bahramisharif et al. 2010). One example is the cursor on the screen that moves in the direction to which the subject covertly attends (i.e., a brain-controlled computer mouse). While motor imagery has been proven to work only in a few non-invasive setups (Wolpaw, McFarland 2004; McFarland et al. 2008), it is quite a natural setup to have the cursor move in the attended direction. Finally, little training time is required for the users to attain acceptable results as compared to motor imagery.
In summary, alpha power is a robust signal that can be used in a BCI setup involving visuospatial attention. The distribution of alpha power is linked to behavioral performance, and it should allow for fast and reliable classification stable over time. In addition, little training time is needed, and it is quite natural for the subject to orient one’s attention to a given direction.
From: “A closed-loop BCI system based on EEG alpha activity modulated by covert visuospatial attention“. Master Thesis on Brain and Cognition by Irene Vigué Guix. This post is the first of the Ms Thesis series.
Bahramisharif, Ali; van Gerven, Marcel; Heskes, Tom; Jensen, Ole (2010): Covert attention allows for continuous control of brain-computer interfaces. In The European journal of neuroscience 31 (8), pp. 1501–1508. DOI: 10.1111/j.1460-9568.2010.07174.x.
Brainard, D. H. (1997): The Psychophysics Toolbox. In Spatial vision 10 (4), pp. 433–436.
Cohen, Michael X. (2017): Where Does EEG Come From and What Does It Mean? In Trends in neurosciences 40 (4), pp. 208–218. DOI: 10.1016/j.tins.2017.02.004.
Fu, K. M.; Foxe, J. J.; Murray, M. M.; Higgins, B. A.; Javitt, D. C.; Schroeder, C. E. (2001): Attention-dependent suppression of distracter visual input can be cross-modally cued as indexed by anticipatory parieto-occipital alpha-band oscillations. In Brain research. Cognitive brain research 12 (1), pp. 145–152.
Hanley, J. A.; McNeil, B. J. (1982): The meaning and use of the area under a receiver operating characteristic (ROC) curve. In Radiology 143 (1), pp. 29–36. DOI: 10.1148/radiology.143.1.7063747.
Horschig, Jorn M.; Oosterheert, Wouter; Oostenveld, Robert; Jensen, Ole (2014): Modulation of Posterior Alpha Activity by Spatial Attention Allows for Controlling A Continuous Brain-Computer Interface. In Brain topography 28 (6), pp. 852–864. DOI: 10.1007/s10548-014-0401-7.
Jensen, Ole; Bahramisharif, Ali; Oostenveld, Robert; Klanke, Stefan; Hadjipapas, Avgis; Okazaki, Yuka O.; van Gerven, Marcel A. J. (2011): Using brain-computer interfaces and brain-state dependent stimulation as tools in cognitive neuroscience. In Frontiers in psychology 2, p. 100. DOI: 10.3389/fpsyg.2011.00100.
Kelly, S. P.; Lalor, E.; Reilly, R. B.; Foxe, J. J. (2005a): Independent Brain Computer Interface Control using Visual Spatial Attention-Dependent Modulations of Parieto-occipital Alpha. In : 2nd International IEEE EMBS Conference on Neural Engineering, 2005. Arlington, Virginia, USA, March 16-19, 2005, pp. 667–670.
Kelly, Simon P.; Lalor, Edmund C.; Finucane, Ciarán; McDarby, Gary; Reilly, Richard B. (2005b): Visual spatial attention control in an independent brain-computer interface. In IEEE transactions on bio-medical engineering 52 (9), pp. 1588–1596. DOI: 10.1109/TBME.2005.851510.
Kelly, Simon P.; Lalor, Edmund C.; Reilly, Richard B.; Foxe, John J. (2005c): Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication. In IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society 13 (2), pp. 172–178. DOI: 10.1109/TNSRE.2005.847369.
Kelly, Simon P.; Lalor, Edmund C.; Reilly, Richard B.; Foxe, John J. (2006): Increases in alpha oscillatory power reflect an active retinotopic mechanism for distracter suppression during sustained visuospatial attention. In Journal of neurophysiology 95 (6), pp. 3844–3851. DOI: 10.1152/jn.01234.2005.
Klimesch, W. (1999): EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. In Brain research. Brain research reviews 29 (2-3), pp. 169–195.
Klimesch, W.; Doppelmayr, M.; Russegger, H.; Pachinger, T.; Schwaiger, J. (1998): Induced alpha band power changes in the human EEG and attention. In Neuroscience Letters 244 (2), pp. 73–76. DOI: 10.1016/S0304-3940(98)00122-0.
Klimesch, Wolfgang (2012): α-band oscillations, attention, and controlled access to stored information. In Trends in cognitive sciences 16 (12), pp. 606–617. DOI: 10.1016/j.tics.2012.10.007.
Klimesch, Wolfgang; Sauseng, Paul; Hanslmayr, Simon (2007): EEG alpha oscillations: the inhibition-timing hypothesis. In Brain research reviews 53 (1), pp. 63–88. DOI: 10.1016/j.brainresrev.2006.06.003.
Kriegeskorte, Nikolaus; Simmons, W. Kyle; Bellgowan, Patrick S. F.; Baker, Chris I. (2009): Circular analysis in systems neuroscience: the dangers of double dipping. In Nature neuroscience 12 (5), pp. 535–540. DOI: 10.1038/nn.2303.
MATLAB: R2015b. Natick, Massachusetts, United States: The MathWorks Inc.
McFarland, Dennis J.; Krusienski, Dean J.; Sarnacki, William A.; Wolpaw, Jonathan R. (2008): Emulation of computer mouse control with a noninvasive brain-computer interface. In Journal of neural engineering 5 (2), pp. 101–110. DOI: 10.1088/1741-2560/5/2/001.
Mesulam, M. M. (1999): Spatial attention and neglect: parietal, frontal and cingulate contributions to the mental representation and attentional targeting of salient extrapersonal events. In Philosophical transactions of the Royal Society of London. Series B, Biological sciences 354 (1387), pp. 1325–1346. DOI: 10.1098/rstb.1999.0482.
Neuper, Christa; Müller-Putz, Gernot R.; Scherer, Reinhold; Pfurtscheller, Gert (2006): Motor imagery and EEG-based control of spelling devices and neuroprostheses. In Christa Neuper, Wolfgang Klimesch (Eds.): Event-related dynamics of brain oscillations, vol. 159. Amsterdam: Elsevier (Progress in Brain Research, 159), pp. 393–409.
Nicolas-Alonso, Luis Fernando; Gomez-Gil, Jaime (2012): Brain-computer interfaces, a review. In Sensors (Basel, Switzerland) 12 (2), pp. 1211–1279. DOI: 10.3390/s120201211.
Oostenveld, Robert; Fries, Pascal; Maris, Eric; Schoffelen, Jan-Mathijs (2011): FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. In Computational intelligence and neuroscience 2011, p. 156869. DOI: 10.1155/2011/156869.
Pashler, Harold E. (1999): The psychology of attention. First MIT Press paperback edition. Cambridge, Mass., London: MIT Press (A Bradford book).
Posner, Michael I. (1980): Orienting of attention. In Quarterly Journal of Experimental Psychology 32 (1), pp. 3–25. DOI: 10.1080/00335558008248231.
Rihs, Tonia A.; Michel, Christoph M.; Thut, Gregor (2007): Mechanisms of selective inhibition in visual spatial attention are indexed by alpha-band EEG synchronization. In The European journal of neuroscience 25 (2), pp. 603–610. DOI: 10.1111/j.1460-9568.2007.05278.x.
Rihs, Tonia A.; Michel, Christoph M.; Thut, Gregor (2009): A bias for posterior alpha-band power suppression versus enhancement during shifting versus maintenance of spatial attention. In NeuroImage 44 (1), pp. 190–199. DOI: 10.1016/j.neuroimage.2008.08.022.
Sauseng, P.; Klimesch, W.; Stadler, W.; Schabus, M.; Doppelmayr, M.; Hanslmayr, S. et al. (2005): A shift of visual spatial attention is selectively associated with human EEG alpha activity. In The European journal of neuroscience 22 (11), pp. 2917–2926. DOI: 10.1111/j.1460-9568.2005.04482.x.
Scherer, Reinhold; Faller, Josef; Balderas, David; Friedrich, Elisabeth V. C.; Pröll, Markus; Allison, Brendan; Müller-Putz, Gernot (2013): Brain–computer interfacing. More than the sum of its parts. In Soft Comput 17 (2), pp. 317–331. DOI: 10.1007/s00500-012-0895-4.
Starlab: ENOBIO 8: Neuroelectrics.
Thut, Gregor; Nietzel, Annika; Brandt, Stephan A.; Pascual-Leone, Alvaro (2006): Alpha-band electroencephalographic activity over occipital cortex indexes visuospatial attention bias and predicts visual target detection. In The Journal of neuroscience : the official journal of the Society for Neuroscience 26 (37), pp. 9494–9502. DOI: 10.1523/JNEUROSCI.0875-06.2006.
Tonin, L.; Leeb, R.; Del R Millan, J. (2012): Time-dependent approach for single trial classification of covert visuospatial attention. In Journal of neural engineering 9 (4), p. 45011. DOI: 10.1088/1741-2560/9/4/045011.
Tonin, L.; Leeb, R.; Sobolewski, A.; Millan, J. del R. (2013): An online EEG BCI based on covert visuospatial attention in absence of exogenous stimulation. In Journal of neural engineering 10 (5), p. 56007. DOI: 10.1088/1741-2560/10/5/056007.
Torralba, M., Soto-Faraco, S., Morís, L. and Ruzzoli, M.: Fom brain oscillations to new technological applications: Proof of concept. Poster in European Conference on Visual Perception (ECVP). In : European Conference on Visual Perception (ECVP).
Treder, Matthias S.; Bahramisharif, Ali; Schmidt, Nico M.; van Gerven, Marcel A. J.; Blankertz, Benjamin (2011): Brain-computer interfacing using modulations of alpha activity induced by covert shifts of attention. In Journal of neuroengineering and rehabilitation 8, p. 24. DOI: 10.1186/1743-0003-8-24.
van Gerven, Marcel; Bahramisharif, Ali; Heskes, Tom; Jensen, Ole (2009): Selecting features for BCI control based on a covert spatial attention paradigm. In Neural networks : the official journal of the International Neural Network Society 22 (9), pp. 1271–1277. DOI: 10.1016/j.neunet.2009.06.004.
van Gerven, Marcel; Jensen, Ole (2009): Attention modulations of posterior alpha as a control signal for two-dimensional brain-computer interfaces. In Journal of neuroscience methods 179 (1), pp. 78–84. DOI: 10.1016/j.jneumeth.2009.01.016.
Wolpaw, Jonathan R.; McFarland, Dennis J. (2004): Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. In Proceedings of the National Academy of Sciences of the United States of America 101 (51), pp. 17849–17854. DOI: 10.1073/pnas.0403504101.
Worden, M. S.; Foxe, J. J.; Wang, N.; Simpson, G. V. (2000): Anticipatory biasing of visuospatial attention indexed by retinotopically specific alpha-band electroencephalography increases over occipital cortex. In The Journal of neuroscience: the official journal of the Society for Neuroscience 20 (6), RC63.
Yamagishi, Noriko; Callan, Daniel E.; Goda, Naokazu; Anderson, Stephen J.; Yoshida, Yoshikazu; Kawato, Mitsuo (2003): Attentional modulation of oscillatory activity in human visual cortex. In NeuroImage 20 (1), pp. 98–113. DOI: 10.1016/S1053-8119(03)00341-0.