I. INTRODUCTION
1.1
Evolution of Brain Computer Interfaces
The human
brain consists of about 86 billion neurons [1], and all a person does – from
picking up a glass to formulating a sentence – can be expressed in terms of
neural electrical activities. BCI is an attempt to detect some of those
activities in order to translate them into commands. The most popular
definition of BCI was suggested by Wolpaw et al., according to which a BCI
represents a communication or control channel independent of the brain's
default means of conveying its signals through peripheral nerves and muscles
[2]. This difference for the one whose nerves and muscles are damaged makes a
BCI a very different kind of device when compared to a keyboard or a joystick.
An
electroencephalography (EEG) is the most popular technique for detecting a BCI
signal. The activity of large populations of cortical neurons is registered
using electrodes that are placed on the skull, and the technique features
millisecond temporal resolution. The technology of EEG monitoring, the hospital
staff's experience and the track record of the technique are well known since
hospitals have been employing EEGs for decades in order to monitor seizures and
people's sleep states. In addition, an EEG technique is cheap and mobile when
compared to most other brain imaging techniques.
However,
stroke serves as the best example of this issue. According to the Global Burden
of Disease study conducted in 2019, there were more than 12 million first-time
strokes globally, while stroke is among the leading causes of permanent
disability [3]. Patients often experience a loss of control over one hand and
arm. While conventional rehabilitation requires extensive repetition of
purposeful movements, a person that cannot move his/her hand will not be able
to do any repetitions. Brain-computer interfaces could provide an alternative
solution for this problem as they enable the recording of brain activity when
the patient is trying to make a movement with the paralyzed arm, which allows producing
a certain response in form of a visual or tactile stimulus, such as opening and
closing of a robotic hand orthosis. Such approach with a help of "brain
intention + feedback" is assumed to be promoting brain rewiring [4].
The
present literature review starts with an introduction of the EEG-BCI technology
and its current place within the spectrum of brain activity monitoring
technologies. Signal processing approach is discussed along with the comparison
between the existing post-stroke training protocols and the results achieved
via placebo-controlled trials. The other clinical applications of the
technology are briefly reviewed before moving on to the numerous limitations and
research directions.
1.2
EEG AND BCI LOOP
For
any application, the BCI uses a similar sequence of steps represented in a loop
shown in Fig. 1. The user executes a mental strategy (such as imagining moving
one’s hand). The computer captures and analyzes the brain activity and
determines the intent of the user and generates output accordingly. The
importance of this step is that it helps the user evaluate their mental
strategy through feedback from the output generated. Such a closed-loop design
is essential for a BCI to become an interface rather than just a brain decoder.
Fig. 1. The closed loop of an EEG-based BCI.
Feedback from the output device lets the user adapt their mental strategy.
A.
Methods
of recording and their pros and cons
Recording
of brain activity can be performed using different degrees of invasiveness, and
selection of degree depends on the trade-off between quality of signal and
danger for the patient (Figure 2). The EEG, magnetoencephalography (MEG) and
functional near-infrared spectroscopy (fNIRS) are non-invasive. Even though the
EEG provides signal with high temporal resolution, its spatial resolution is
relatively poor because the skull and scalp distort the electrical field and
allow to determine sources only within a few centimeters [5], while MEG
requires magnetically shielded room, and is not applicable in routine therapy.
In case of fNIRS, several seconds are needed to see changes in blood
oxygenation due to neural activity [6].
Fig. 2. Schematic comparison of recording
methods. Positions are qualitative and meant only to show the general trend,
not measured values.
Electrocorticography (ECoG) is the next step,
with the electrodes being placed on the brain surface underneath the skull. The
information gained by this technique is richer and more precise, but it
involves a neurosurgical procedure. Even further, the microelectrode arrays are
able to monitor single neuron activity. These techniques enabled patients with
tetraplegia to operate a robotic arm [7] and very recently, even to interpret
attempted speech from a paralytic patient at 60 words per minute [8]. These achievements
are certainly far ahead of any current scalp EEG techniques, but they involve
both risks and costs, and in the case of a rehabilitation process, when the
patient might use the technology repeatedly for several weeks, safety and costs
play the decisive role.
B.
Control signals
As a rule, EEGs for
BCI systems do not cover a wide range of information due to the fact that they
involve a small set of signals known to scientists. The most famous BCI speller
system [9] utilized P300 event-related potential (an approximately 300-msec
peak occurring after some rare or attended event) as its main signal.
Steady-state visual-evoked potentials (SSVEP) can be considered a
stimulus-based method. These techniques fit better for spelling tasks but what
if we consider BCIs that would be useful for rehabilitation? We will require an
increase in sensorimotor wave power. When the movement itself, preparation of
the body to make a movement, or even planning of the movement occurs, the mu
rhythm (approximately 8-12 Hz) and beta rhythms (13-30 Hz) of the motor cortex
weaken. The process is called the event-related desynchronization (ERD) whereas
the following increasing of power is called event-related synchronization (ERS)
[10]. As a matter of fact, the phenomena is lateralized and, therefore,
thinking about the movement of the left hand produces different scalp
topography than thinking about the right one [11].
1.3. SIGNAL PROCESSING
CHAIN
The
methodologies described in this section are the ones most frequently employed
within the literature on motor BCI systems. Such methodologies are standard
tools but not standard configuration and are varied by the different studies.
A. Acquisition
Placement
of electrodes happens with the use of the 10-20 electrode placement system. As
concerns motor activity, the optimal place for placement are sensorimotor areas
C3 C4 and Cz, and several electrode arrangements exist with the inclusion of
adjacent channels so that some data can be filtered in space. Wet electrodes
with the use of the gel are most suitable for achieving good contact although
they take more time for placement and create more noise than dry electrodes.
Inexpensive, compact research setups are made possible thanks to the appearance
of cheap hardware; however, decoding of information depends on the quality of
amplifiers and of the contact between electrodes; surface EMG from the forearm
is sometimes used in BCI setups for furtive movement detection.
B. Preprocessing and
artifact removal
EEG
recorded on the scalp is microvolts in magnitude, and it is always contaminated
with unwanted signals. We can simplify these contaminations to give x(t) = s(t)
+ n(t), where s(t) is the particular brain activity we are after, and n(t) is
everything else: eye blinks and eye movements, muscle activity, cable movement,
50 or 60 Hz mains powered signal. A band-pass filter (generally around 1 Hz to
40 Hz) gets rid of slow drift and high frequency muscle activity, and a notch
filter is often used to get rid of the mains frequency.
The
filter output signal is y(t) = x(t) * h(t).
Eye
blinks and movement occur at the frequency of the signal under measurement and
cannot simply be filtered out without corrupting the brain signal. The usual
solution: independent component analysis (ICA) separates the recording into
statistically independent elements so that those for eye or heart activity are
identified and discarded [12]. Artifact subspace reconstruction (ASR) is a more
automated approach, which identifies transient high-variance bursts and
replaces them with good reference activity; a technique promising for home and
real-time applications [13].
C. Feature extraction
The
signal then undergoes transformation in terms of a few numerical values that
indicate the state of the user. When we talk about sensorimotor BCIs, the most
common way of quantification is the estimation of band power in the mu and beta
bands, using a brief interval in time. After the N samples of a band-pass
filtered x[n], P will be,
P =
(1/N) x[n] (1)
and
the ERD% value can be calculated based on the rest baseline R using the
expression ERD% = (A R)/R 100. Here an denotes the power during the task [10].
One of the well-known ways of spatial analysis is known as common spatial
patterns (CSP). Using CSP we can extract a linear combination of the channels
that maximizes the variance of the signals in one of the classes while
minimizing the same for the other class [14], [15].
D. Classification
The
classification component that translates features to the desired movement. Linear
discriminant analysis (LDA) remains a common classifier choice in system
development due to its cost-effectiveness simplicity, ease of training even
from small amounts of data, and capacity for delivering high accuracy in short
calibration sessions. Other approaches include comparison to the support vector
machines, Riemannian-geometry-based algorithms, and recently, to deep networks
[16]. For instance, the EEGNet convolutional neural network extracts relevant
features on its own and could be applied in different BCI architectures [17]. A
sufficiently large dataset could allow a deep network approach to outperform
the classical approaches, but such data is not always available.
For
a therapeutic system, an output of the classifier and a confidence measure is
usually presented, and only when this value passes a certain threshold the
actuator takes action. This prevents incorrect triggering, as it could be
counterproductive, since the misleading feedback would confuse a patient and
break the entire training system altogether. The typical percentage threshold
for binary classification control systems is roughly 70% accuracy, below which
the communication becomes frustrating [18], but in therapy it could be
important to minimize the feedback delay relative to the event being reflected.
1.4. BCI TRAINING
PROTOCOLS AFTER STROKE
A. Rationale
Overall,
the stroke-affected hemisphere generates fewer sensorimotor rhythms when
attempting to move the affected hand. The theoretical basis behind the use of a
BCI lies in its capacity to provide an association between the patient’s motor
commands with a sensor feedback at the moment of their manifestation, for
example, by actuating a robotic orthosis or a functional electrical stimulation
system at the moment of manifestation of ipsilesional ERD. This coupling of the
output signal and input signal is supposed to enhance the pathway due to the
Hebbian type of plasticity [4], [19]. This is a theoretical notion without
empirical basis, which the results presented below could confirm. There are
three main protocols considered.
B. Motor imagery BCIs
Motor
Imagery Brain-Computer Interfaces (MI-BCIs): the user imagines an action, for
example, hand opening, without doing the action. Neural networks used in the
imaging tasks coincide with the neural networks used in motor actions and
generate mu and beta ERD over the scalp EEG. The user gets feedback using an
image of a hand on a screen or a bar and a mobile device. Randomized trials
conducted among subacute stroke patients showed that MI-BCI training together
with regular rehabilitation was more effective for the recovery of the upper
limb and brain connectivity [20].
The
most important advantage of MI-BCIs is that they can be started regardless of
any motor capacity of a patient. However, one of the major drawbacks is that
motor imagery is a trained technique. There are people who cannot perform such
task, and it depends on various factors such as cognitive abilities, neural
state, and feedback method of MI [21]. A considerable number of users, often
mentioned as 15–30% of them, cannot perform motor imagery tasks.
C. Movement-attempt BCIs
MA-BCIs
tell the patient to make the utmost effort to move the paralyzed limb
irrespective of whether the limb actually moves or not. This approach is
predicated upon the notion that a real attempt at moving would stimulate the
motor system in a more natural way than imaginary movement and the device would
provide the movements that the patient fails to generate. Through a
well-designed experiment conducted on stroke patients with chronic hand
paralysis, Ramos-Murguialday et al. demonstrated that training using MA-BCI
feedback along with physical therapy yielded better results in terms of
Fugl-Meyer score than sham-feedback training [22].A preliminary study had
suggested that chronic patients who had no finger movement could acquire the
ability to control sensorimotor rhythms to open and close an orthosis [23].
D. Sensorimotor-rhythm neurofeedback
Another
type of neurofeedback is the sensorimotor rhythm (SMR). In place of a specific
motor task, the patient is shown his/her own mu and beta activity levels and
learns how to regulate them through self-regulation training. The SMR technique
of neurofeedback is widely used beyond stroke [24] despite the fact that the
differences between it and imagery and attempt are somewhat unclear as the
protocols for both measure the same types of brain activity in the same brain
areas; the only difference is in instructions to the patient.
E. What the evidence shows
In
the framework of the meta-analysis of randomized and controlled studies
regarding the impact of the BCI training on stroke patients, positive effects
were reported on the upper limb motor function post-stroke for both subacute
and chronic stroke patients using BCI in combination with FES and robotic
devices [25]. The following review reached similar conclusions regarding the
overall impact of the technology, but it discussed other applications of the
BCI as well [26]. However, in both reviews, small sample sizes and significant
differences in protocols, devices and outcomes measurements have been reported.
The
most commonly used instrument is the Fugl-Meyer Assessment (FMA) for the upper
extremities [27], which can take from 0 to 66 points on the motor impairment
scale. In order to understand the reported results, one should keep in mind
that the minimal clinically important difference of FMA score for the chronic
stroke is approximately 4-7 points [28], and therefore the statistically
significant effect is not necessarily clinically significant. Moreover, the
benefit of the BCI training in addition to the physical therapy, which is
applied in parallel to the BCI training, has remained controversial due to the
design of most studies.
1.5.
OTHER MEDICAL APPLICATIONS
A.
Communication
The
first practical use of BCI technology was in the hope of making spoken language
available to people who could not speak or move. It was demonstrated by Farwell
and Donchin that selection of a letter using a P300 spelling system does not
require any muscle movement [9], and by Birbaumer et al. that paralyzed
individuals can learn how to spell with the help of slow cortical potentials
[29]. Contemporary P300 or SSVEP EEG spellers may be very slow (up to a few
characters per minute up to a few tens), and they need sustained concentration
from the users; however, they are valuable for patients with late stages of
amyotrophic lateral sclerosis or locked-in syndrome where it may be the only
possible way of communication [30]. The mentioned invasive systems are far more
efficient, although still experimental [8].
B. Disorders
of consciousness
There
are some patients who have an unconscious state caused by serious brain damage;
however, some of them are actually conscious although they do not display any
evidence of that. For example, according to Cruse et al., in a group of
patients with a diagnosis of vegetative state, the researchers asked them to
imagine the movement of their hands and toes while their EEG readings were
taken, and some of them had the expected reaction [31].
C.
Diagnosis and monitoring
Various
machine learning algorithms are being developed to analyze EEG data for
diagnostic purposes such as epilepsy seizure detection. While this is clearly
connected with the field of BCI research and uses many of the same techniques,
it cannot be considered BCI research since there is no output device being
controlled by the algorithm. Deep learning techniques have shown great success
in many tasks involving the classification of EEG data; however, they have
proven to be less accurate when using a pre-trained model on one group of
patients with a new group of patients [32].
II. CHALLENGES AND LIMITATIONS
There
are several barriers that stand between lab results and practical application
in clinical practice. First, it is the problem of variability. Signals from
EEGs differ among individuals, change over different days, and also change
depending on whether users get tired or distracted during the course of the
experiment [21]. Thus, non-stationarity in such cases will mean that
classifiers trained in the morning will become useless in the afternoon and
some users will not be able to control the system at all.
The
next barrier is the problem of practicality. Using gel-based electrodes takes
much time, calibration sessions might require much energy which might be useful
for therapy instead, as well as trained personnel to operate the device. For
home usage, the device should be easy enough to apply by the user or caregiver
and should include artifact handling and signal quality detection tools.
Next
comes the problem of quality of evidence. Trials are too small, too
heterogeneous in terms of participants or protocols, and sham controls are hard
to come up with because the patient knows if he or she really has the control
over the feedback or not. Long-term follow-up is rare and although there are
evidences of the positive effect at the beginning, it is unknown how long it
lasts. Besides, there is lack of data on cost-effectiveness. The last but
definitely not the least barrier is the issue of vulnerability of the medium.
The data on the brain are very private and could reveal much more than what has
been actually entered into it, creating serious problems of consent and storage
as BCIs go out of labs [33].
III. FUTURE DIRECTIONS
Hybrid
BCI systems employ more than one source of information, say, two different
forms of brain waves or EEG and EMG, in order to increase robustness in case
information is not sufficient from one particular channel [34]. An example of
such an approach was using ERD and SSVEP in conjunction to obtain
two-dimensional control of the cursor [35]. A second possible development area
is the design of better decoders. Adaptive decoding algorithms, which have the
ability to learn while working, transfer learning that uses the information
about previous patients to save time on calibration and smaller-scale neural
network systems all seek to come nearer to the very much needed decoders that
are easy to configure and robust enough to cope with the variability mentioned
above [16], [17]. The third improvement possible in terms of hardware: dry
electrodes, lights and wireless headsets along with tele-rehabilitation platforms
will allow patients to exercise at home, controlled by their therapist remotely.
IV. CONCLUSION
It
was decades later, however, before EEG BCIs became testable in a clinical
setting and stroke rehabilitation is where most of the results are now emerging.
The principle of motor imagery training, attempts at movement and sensorimotor
rhythm training is identical to what is seen in the laboratory: the brain
activity needs to be coupled with the appropriate feedback, and in controlled
experiments, it can be useful to certain patients in improving their upper limb
functions. There is little evidence thus far, but: the trials have been small,
the outcomes can vary depending on the patient and there has been no conclusive
evidence about the extent of the BCI's contribution to the outcome. It will all
depend on developing user-friendly devices, adaptive decoders and accurate
reporting of the trials. In such a scenario, BCIs could well become a regular
part of rehabilitation and speech therapy.
REFERENCES
[1] S. Herculano-Houzel, “The human brain in
numbers: a linearly scaled-up primate brain,” Front. Hum. Neurosci., vol. 3,
art. 31, 2009.
[2] J. R. Wolpaw, N. Birbaumer, D. J. McFarland,
G. Pfurtscheller, and T. M. Vaughan, “Brain–computer interfaces for communication
and control,” Clin. Neurophysiol., vol. 113, no. 6, pp. 767–791, 2002.
[3] V. L. Feigin et al., “Global, regional, and
national burden of stroke and its risk factors, 1990–2019,” Lancet Neurol.,
vol. 20, no. 10, pp. 795–820, 2021.
[4] J. J. Daly and J. R. Wolpaw, “Brain–computer
interfaces in neurological rehabilitation,” Lancet Neurol., vol. 7, no. 11, pp.
1032–1043, 2008.
[5] P. L. Nunez and R. Srinivasan, Electric
Fields of the Brain: The Neurophysics of EEG, 2nd ed. New York, NY, USA: Oxford
Univ. Press, 2006.
[6] N. Naseer and K.-S. Hong, “fNIRS-based
brain–computer interfaces: a review,” Front. Hum. Neurosci., vol. 9, art. 3,
2015.
[7] L. R. Hochberg et al., “Reach and grasp by
people with tetraplegia using a neurally controlled robotic arm,” Nature, vol.
485, pp. 372–375, 2012.
[8] F. R. Willett et al., “A high-performance
speech neuroprosthesis,” Nature, vol. 620, pp. 1031–1036, 2023.
[9] L. A. Farwell and E. Donchin, “Talking off
the top of your head: toward a mental prosthesis utilizing event-related brain
potentials,” Electroencephalogr. Clin. Neurophysiol., vol. 70, no. 6, pp.
510–523, 1988.
[10] G. Pfurtscheller and F. H. Lopes da Silva,
“Event-related EEG/MEG synchronization and desynchronization: basic
principles,” Clin. Neurophysiol., vol. 110, no. 11, pp. 1842–1857, 1999.
[11] G. Pfurtscheller and C. Neuper, “Motor imagery
and direct brain–computer communication,” Proc. IEEE, vol. 89, no. 7, pp.
1123–1134, 2001.
[12] A. Delorme and S. Makeig, “EEGLAB: an open
source toolbox for analysis of single-trial EEG dynamics including independent
component analysis,” J. Neurosci. Methods, vol. 134, no. 1, pp. 9–21, 2004.
[13] C. A. E. Mullen et al., “Real-time neuroimaging
and cognitive monitoring using wearable dry EEG,” IEEE Trans. Biomed. Eng.,
vol. 62, no. 11, pp. 2553–2567, 2015.
[14] H. Ramoser, J. Müller-Gerking, and G. Pfurtscheller,
“Optimal spatial filtering of single trial EEG during imagined hand movement,”
IEEE Trans. Rehabil. Eng., vol. 8, no. 4, pp. 441–446, 2000.
[15] B. Blankertz, R. Tomioka, S. Lemm, M. Kawanabe,
and K.-R. Müller, “Optimizing spatial filters for robust EEG single-trial
analysis,” IEEE Signal Process. Mag., vol. 25, no. 1, pp. 41–56, 2008.
[16] F. Lotte et al., “A review of classification
algorithms for EEG-based brain–computer interfaces: a 10 year update,” J.
Neural Eng., vol. 15, no. 3, art. 031005, 2018.
[17] V. J. Lawhern et al., “EEGNet: a compact
convolutional neural network for EEG-based brain–computer interfaces,” J.
Neural Eng., vol. 15, no. 5, art. 056013, 2018.
[18] A. Kübler et al., “Brain–computer
communication: unlocking the locked in,” Psychol. Bull., vol. 127, no. 3, pp.
358–375, 2001.
[19] S. R. Soekadar, N. Birbaumer, M. W. Slutzky,
and L. G. Cohen, “Brain–machine interfaces in neurorehabilitation of stroke,”
Neurobiol. Dis., vol. 83, pp. 172–179, 2015.
[20] F. Pichiorri et al., “Brain–computer interface
boosts motor imagery practice during stroke recovery,” Ann. Neurol., vol. 77,
no. 5, pp. 851–865, 2015.
[21] M. Ahn and S. C. Jun, “Performance variation in
motor imagery brain–computer interface: a brief review,” J. Neurosci. Methods,
vol. 243, pp. 103–110, 2015.
[22] A. Ramos-Murguialday et al., “Brain–machine
interface in chronic stroke rehabilitation: a controlled study,” Ann. Neurol.,
vol. 74, no. 1, pp. 100–108, 2013.
[23] E. Buch et al., “Think to move: a neuromagnetic
brain–computer interface (BCI) system for chronic stroke,” Stroke, vol. 39, no.
3, pp. 910–917, 2008.
[24] R. Sitaram et al., “Closed-loop brain training:
the science of neurofeedback,” Nat. Rev. Neurosci., vol. 18, no. 2, pp. 86–100,
2017.
[25] M. A. Cervera et al., “Brain–computer
interfaces for post-stroke motor rehabilitation: a meta-analysis,” Ann. Clin.
Transl. Neurol., vol. 5, no. 5, pp. 651–663, 2018.
[26] R. Mane, T. Chouhan, and C. Guan, “BCI for
stroke rehabilitation: motor and beyond,” J. Neural Eng., vol. 17, no. 4, art.
041001, 2020.
[27] A. R. Fugl-Meyer, L. Jääskö, I. Leyman, S.
Olsson, and S. Steglind, “The post-stroke hemiplegic patient. 1. A method for
evaluation of physical performance,” Scand. J. Rehabil. Med., vol. 7, no. 1,
pp. 13–31, 1975.
[28] S. J. Page, G. D. Fulk, and P. Boyne,
“Clinically important differences for the upper-extremity Fugl-Meyer scale in
people with minimal to moderate impairment due to chronic stroke,” Phys. Ther.,
vol. 92, no. 6, pp. 791–798, 2012.
[29] N. Birbaumer et al., “A spelling device for the
paralysed,” Nature, vol. 398, pp. 297–298, 1999.
[30] I. Lazarou, S. Nikolopoulos, P. C.
Petrantonakis, I. Kompatsiaris, and M. Tsolaki, “EEG-based brain–computer
interfaces for communication and rehabilitation of people with motor
impairment: a novel approach of the 21st century,” Front. Hum. Neurosci., vol.
12, art. 14, 2018.
[31] D. Cruse et al., “Bedside detection of
awareness in the vegetative state: a cohort study,” Lancet, vol. 378, no. 9809,
pp. 2088–2094, 2011.
[32] A. Craik, Y. He, and J. L. Contreras-Vidal,
“Deep learning for electroencephalogram (EEG) classification tasks: a review,”
J. Neural Eng., vol. 16, no. 3, art. 031001, 2019.
[33] U. Chaudhary, N. Birbaumer, and A.
Ramos-Murguialday, “Brain–computer interfaces for communication and
rehabilitation,” Nat. Rev. Neurol., vol. 12, no. 9, pp. 513–525, 2016.
[34] G. Pfurtscheller et al., “The hybrid BCI,”
Front. Neurosci., vol. 4, art. 30, 2010.
[35] B. Z. Allison et al., “A hybrid ERD/SSVEP BCI
for continuous simultaneous two-dimensional cursor control,” J. Neurosci.
Methods, vol. 209, no. 2, pp. 299–307, 2012.