Current Biology 22, 1333–1338, July 24, 2012 ª2012 Elsevier Ltd All rights reserved
DOI 10.1016/j.cub.2012.05.022
Report
A Real-Time fMRI-Based Spelling Device
Immediately Enabling
Robust Motor-Independent Communication
Bettina Sorger,1,2,* Joel Reithler,1,2,3 Brigitte Dahmen,4
and Rainer Goebel1,2,3
1Department of Cognitive Neuroscience, Maastricht
University, Maastricht 6200 MD, The Netherlands
2Maastricht Brain Imaging Center (M-BIC),
Maastricht 6200 MD, The Netherlands
3Department of Neuroimaging and Neuromodeling,
Netherlands Institute for Neuroscience, an Institute of the
Royal Netherlands Academy of Arts and Sciences (KNAW),
Amsterdam 1105 BA, The Netherlands
4Department of Child and Adolescent Psychiatry,
Psychosomatics and Psychotherapy, RWTH Aachen
University, Aachen 52074, Germany
Summary
Human communication entirely depends on the functional
integrity of the neuromuscular system. This is devastatingly
illustrated in clinical conditions such as the so-called
locked-in syndrome (LIS) [1], in which severely motordisabled patients become incapable to communicate naturally—while being fully conscious and awake. For the last
20 years, research on motor-independent communication
has focused on developing brain-computer interfaces
(BCIs) implementing neuroelectric signals for communication (e.g., [2–7]), and BCIs based on electroencephalography
(EEG) have already been applied successfully to concerned
patients [8–11]. However, not all patients achieve proficiency
in EEG-based BCI control [12]. Thus, more recently, hemodynamic brain signals have also been explored for BCI
purposes [13–16]. Here, we introduce the first spelling
device based on fMRI. By exploiting spatiotemporal characteristics of hemodynamic responses, evoked by performing
differently timed mental imagery tasks, our novel letter
encoding technique allows translating any freely chosen
answer (letter-by-letter) into reliable and differentiable
single-trial fMRI signals. Most importantly, automated letter
decoding in real time enables back-and-forth communication within a single scanning session. Because the
suggested spelling device requires only little effort and pretraining, it is immediately operational and possesses high
potential for clinical applications, both in terms of diagnostics and establishing short-term communication with nonresponsive and severely motor-impaired patients.
Results
By employing a novel, robust, and immediately applicable
letter encoding technique and advanced real-time fMRI
decoding methods, we enabled all participants of our proofof-concept study to motor-independently communicate any
given thought within a 1 hr MRI session. In each ‘‘communication experiment,’’ a mini-conversation consisting of two open
questions and answers was held. All participants’ answers
*Correspondence: [email protected]
could be successfully deciphered online (i.e., within the
ongoing MRI session).
A Novel Letter Encoding Technique
In order to represent each letter of the English alphabet and the
blank space by a distinct neural activation pattern, we
conceived a letter encoding technique that requires almost
zero pretraining. The technique exploits spatiotemporal characteristics of single-trial blood oxygenation level-dependent
(BOLD) signals that are associated with the participant’s
performance of different mental tasks and certain temporal
aspects of task execution. That a single cognitive event can
be assessed online has been demonstrated previously by
Posse and colleagues [17]. To intentionally generate 27 distinguishable hemodynamic activation patterns, participants
voluntarily influenced three BOLD signal aspects:
(1) location of the signal source by performing three
different mental tasks (see Movie S1 available online),
(2) signal onset delay by delaying the start of the mental
task for 0 s, 10 s, or 20 s, and
(3) signal duration by varying the mental task duration
between 10 s, 20 s, and 30 s.
Full combination of these parameters results in the generation of 27 unique hemodynamic brain responses, which can be
assigned to 27 characters (Figure 1).
The letter-by-letter translation of any freely chosen answer
into single-trial BOLD signals required only little effort from
participants as the encoding process was facilitated by using
a convenient visual letter encoding display designed according to the coding scheme (Figure 1). For encoding a certain
letter, participants merely had to attend to the desired letter
and to perform the designated mental task as long as
the letter was highlighted in the display (Figure 2; Movie S2).
Automated Letter Decoding in Real Time
Based on a short ‘‘localizer experiment’’ (Supplemental Experimental Procedures), brain regions that were robustly and
differentially engaged during the mental tasks’ execution
were defined as regions of interest (ROIs; two sub-ROIs per
mental task; Table S1). During the ‘‘communication experiment,’’ fMRI activation time courses derived from these
ROIs (see Figure S1 for example data) were forwarded as input
for an automated letter decoder. The two sub-ROIs were
labeled ‘‘shape/pattern’’ and ‘‘pattern only’’ ROI—indicating
their functional role during the decoding process (Figure S2;
Supplemental Experimental Procedures). The developed
automated letter decoder operates in real time and computationally combines a ‘‘shape’’ criterion (obtained by correlating
the ‘‘shape/pattern’’ ROI time courses with a multifilter bank of
ideal response shapes) and an ‘‘amplitude pattern’’ criterion
(extracted from estimated single-trial beta weights within all
six ROIs). Note that for automated letter decoding, individual
reference time courses (RTCs) based on the ROIs’ time course
information of the ‘‘localizer experiment’’ were used (Supplemental Experimental Procedures; Figure S3). This individualized approach takes into account the inter- and intraindividual
Current Biology Vol 22 No 14
1334
offline analyses of pilot data had confirmed that individual
RTCs outperform reference functions derived from standard
hemodynamic impulse functions [20, 21] in terms of decoding
accuracy. The automated letter decoder ranked all 27 characters according to their overall fit with a combined letter decoding criterion (incorporating both ‘‘shape’’ and ‘‘amplitude
pattern’’ information). For each letter encoding trial, the ‘‘top
three’’-letter choices were visualized to the experimenters
(Figure 4; Movie S2).
Figure 1. Letter Coding Scheme
Combining three mental tasks, three task onset delays, and three task durations allows for encoding 3 3 3 3 3 = 27 distinct information units. When
assigning these 27 combinations to 26 letters and the blank space (‘‘–’’),
every character can be represented by a certain cognitive event leading to
a unique dynamic brain response pattern. In order to encode, e.g., the letter
‘‘E,’’ a motor action has to be imagined, starting 10 s after the general trial
onset and lasting for 30 s. Remarks/abbreviations are as follows: RTCs,
reference time courses; ROIs, regions of interest; see Table S1 for mental
tasks and ROIs used in the current study; see Movie S1 for brain activation
patterns evoked by performing different mental tasks; curves shown are
modeled RTCs derived from standard hemodynamic response functions
(note, however, that individual RTCs were used for letter decoding [see Figure S3 for comparison of individual versus standard RTCs]).
variability of the BOLD signal across regions and tasks [18, 19]
that is especially expected in the context of tasks involving
higher cognitive functions, such as mental imagery, which
has to be self-induced by the participant (versus automatically
evoked by external stimulation). For example, a participant
might need more time to fully engage in vivid mental task
performance resulting in a delayed BOLD signal onset. Initial
High Letter Decoding Accuracies Enabling Back-and-Forth
Communication
Averaged across participants, the top-ranked letter choice
was correct in 82% of the cases (chance level: 3.7%; Figure 3).
When considering the first two and three letter choices, the encoded letter was included in 95% and 100% of the cases.
Taking into account the main output of the automated decoder
(‘‘top three’’-letter choices) and word/question context information, the experimenters correctly deciphered all answers
of the participants online (Figure 4), enabling back-and-forth
communication within the ongoing MRI session.
Discussion
Our real-time fMRI study shows that by using easy-to-perform
mental imagery tasks and an appropriate experimental design,
one can hemodynamically encode a remarkable amount of
distinct information units on a single-trial basis, in a reasonable
time frame, and with little effort, pretraining, and preparation.
The results of our study substantially extend recent work
in the field of hemodynamically based BCI research by
increasing the number of discretely encodable information
Figure 2. Visually Guided Letter Encoding Help with Time Line
In order to encode a specific letter, participants did not have to memorize the encoding parameters for a certain letter (i.e., mental task, onset delay, task
duration). They only had to attend to the selected letter and to perform the particular mental task assigned to the respective row (indicated in the first column
of each window) as long as the accordant letter cell was highlighted (for 10 s, 20 s, or 30 s). In order to encode, e.g., the letter ‘‘E,’’ no mental task has to be
performed in the first 10 s following the onset of the letter encoding trial (passive). When the ‘‘E’’ cell gets highlighted, the participant starts performing
‘‘motor imagery’’ (active). As soon as the letter cell is no longer highlighted (after 30 s in this example), the participant stops and stays focusing on ‘‘E’’
for another 10 s (passive) until the whole letter encoding period is finished. During the subsequent resting period, in which none of the letter cells is highlighted, the participant is asked to select (and switch to) the next intended letter—remaining in a sort of ‘‘stand-by’’ mode, selecting the next mental task to
be performed, and awaiting the next active letter encoding phase. Note that the conceptualization of our letter encoding display and the visual display
invented in the context of the development of an EEG-based spelling device via event-related potentials (P300) [2] share common characteristics. Remarks
are as follows: circles were not visible for participants; they are shown here in order to emphasize active (green) and passive (red) answer encoding phases
when encoding the letter ‘‘E’’; a dynamic version of the letter encoding help is provided as part of Movie S2.; note that hemodynamically based letter encoding can also be realized using exclusively auditory instructions (see Figure S4).
A Real-Time fMRI-Based Spelling Device
1335
Figure 3. Real-Time Letter Decoding Group Results
Averaged across all participants, the first letter choice of the automated decoding was correct in 82% of the cases (chance level: 3.7%). Considering
the first two and three letter choices, the encoded letter was included in
95% and 100% of the cases. When regarding the main decoder output
(‘‘top three’’-letter choices) and word/question context information, the
experimenters were able to decipher the answers in 100% of the cases.
Comparing the results obtained from answers to initial (bright colors) versus
follow-up (dark colors) questions revealed that encoding performance remained stable throughout the MRI scanning session. Remarks are as
follows: light gray bar areas represent the letter decoding accuracy based
on prior letter choices; error bars indicate variance across participants
(6SEM); see Figure S2 for overview of the automated letter decoding
procedure.
units from four or less [14–16, 22, 23] to 27—allowing for ‘‘free
letter spelling.’’ To our knowledge, no previous fMRI study has
been published in which brain states were correctly decoded
out of 27 alternatives, let alone on a single-trial level, without
time-consuming training and in real time.
Although the main goal of the present study was to demonstrate the feasibility of immediate and robust motor-independent letter spelling based on the hemodynamic response, it
adds several neurobiological insights and methodological
implications from a more general perspective:
(1) Single-trial BOLD responses can be reliable enough to
robustly decode multiple covert cognitive processes in
real time and without a long preceding training phase.
(2) We introduce an automated decoding procedure that
works in real time and computationally combines ROI
time course multifilter bank correlation and pattern matching of single-trial ROI beta weights that might be used in
other contexts as well.
(3) The use of individual RTCs and multiple ROIs in the
decoding procedure provides a substantial benefit
(Figure S3).
These novel insights into the quality of single-trial fMRI
signal responses and the developed data-analysis strategies
may be inspiring for ‘‘brain reading’’ research.
Our noninvasive, relatively fast-to-apply, and reliable fMRIbased spelling device considerably enriches the spectrum
of currently available, mainly scalp EEG-based communication approaches [24]. Although EEG-based approaches still
constitute the current state-of-the-art in BCI research and its
practical applications, our proposed method offers new possibilities: it provides an alternative brain-based communication
means which may be useful in acute stages of the LIS when
other communication tools are not immediately available.
Moreover, it might be applied in patient populations so far
not benefiting from EEG-based BCI systems, e.g., in complete
LIS patients, who do not have any residual motor abilities left
[25]. For this patient group, successful EEG-based BCI control
has not been demonstrated [24]. However, the hemodynamic
approach might be particularly well-suited for complete LIS
patients as in a recent study employing hemodynamic brain
signals via functional near-infrared spectroscopy (fNIRS) BCI
control to at least some degree (allowing for ‘‘yes’’/‘‘no’’
answers) has been demonstrated in this patient group [23].
Unfortunately, the related article provides rather sparse information about the clinical population that was tested. Nevertheless, the hemodynamic approach seems promising, which is
also suggested by previous fMRI studies with nonresponsive
patients [26–28].
Another possibility offered by our spelling device is that
it can serve as a crucial diagnostic tool to assess preserved
consciousness in nonresponsive patients—an assessment
that is difficult to establish by other means. So far, inferring
consciousness from fMRI data has been done offline [26–28].
With our real-time methods, inferring consciousness would
become possible online, which would constitute a significant advantage. Importantly, if a patient conveyed a single
meaningful message, any potential remaining doubt on a
patient’s state of consciousness [29–31] could be definitely
excluded.
For now, our study constitutes only a proof of concept
working with healthy volunteers. However, by exclusively
allowing mental processes for communication purposes, the
current design mimicked LIS patients’ limited behavioral
capabilities while retaining the possibility to express thoughts
by using preserved mental abilities. The positive results obtained in all tested volunteers highly encourage the conduction
of clinical trials in affected patient groups though it should be
noted that the encoding procedure may be more challenging
for patients due to potential difficulties with attention and
eye movement control. Note, however, that we have successfully performed an additional auditorily guided letter spelling
experiment that reduces the cognitive load to some extent
and, most importantly, does not require any visual capabilities
or eye movement control (Supplemental Experimental Procedures, Figure S4).
The general preparation time for fMRI measurements is relatively short compared to EEG because no time-consuming
placement of electrodes is necessary. Furthermore, although
most recent BCI applications have involved a certain amount
of pretraining (e.g., [8, 11, 32–34]), voluntarily evoking differential hemodynamic brain activation by using our encoding technique requires almost zero pretraining. Any person who can
understand the instructions and perform different mental
tasks (which LIS patients are able to per definitionem) should
be able to communicate letters via fMRI signals in the
described manner (though it might be beneficial to extend
the pretraining in patients). Moreover, most EEG-based BCI
systems require a considerable amount of time to acquire
data for classifier training (e.g., w20 min [7]) to be repeated
in every EEG session. Instead, our procedure includes a relatively short (12:24 min) ‘‘localizer experiment’’ in order to define
ROIs and individual RTCs.
It is important to stress that only intentionally generated
brain activation—originating from active mental task performance alone—was employed for letter decoding. The visual
input was identical across trials and was only used to provide
the letter encoding information. Note that the encoding technique does not necessarily rely on visual input: the same mental
Current Biology Vol 22 No 14
1336
Figure 4. Intrasession Mini-Conversations and Single-Trial Letter Decoding Results
Participants (column 1) were asked an initial question (column 2) and encoded their answers letter-by-letter using the letter encoding technique. The automated letter decoder displayed its letter choices in ranked order (first three lines in each cell of column 3). After the experimenters had decided which word
was most likely encoded (green characters in column 3), a follow-up question—containing this particular word (column 4)—was posed to and answered by
the participant (column 5). Remarks are as follows: blue letters indicate correctly decoded letters; green letters form the word the experimenters decided for
online; red letters represent incorrectly decoded letters; gray letters represent letter choices being less likely correct than the correct letter choice; participant 1 forgot to encode the first blank space in the initial run and added it at the end of that run; participant 5 forgot to encode two blank spaces; see Table S2
for participants’ characteristics; see Figure S1 for single-trial multi-ROI time course data used for decoding the word ‘‘INDIA.’’
operations can be guided by auditory input (Supplemental
Experimental Procedures; Figure S4). This will become important when transferring the encoding technique to patients
with impaired vision or lacking control of gaze movement [33].
We would also like to stress that potential nonneuronal
parameters, such as head motion, heartbeat, and breathing
are unlikely to have driven the classification because any
kind of physiological noise could not explain the (sustained)
differential brain activation that we exploit in our communication procedure (see also Supplemental Experimental Procedures for specific analyses assessing a potential effect of
head motion).
Alternative communication devices relying on hemodynamic brain signals are still in their infancy. The current stage
of our fMRI-based letter spelling device leaves room for
many further developments: The encoding process might be
optimized by considering letter frequency information (e.g.,
assigning short mental task durations to more frequently
occurring letters). Moreover, communication efficiency might
be increased. Currently, the letter encoding time amounts to
50 s corresponding to an information transfer rate [35] of
w5 bits/min, which is considerably longer compared to current EEG-based spelling devices reaching up to 80 bits/min
[24]. Even if roughly 1 min seems quite an expensive
requirement to encode a single letter, time itself should not
be a critical constraint from a clinical perspective: medical
professionals working with LIS patients generally note that
these patients are highly motivated to invest time to re-establish communication. Furthermore, fMRI-based letter spelling
might also be possible when using shorter letter encoding
durations as indicated by a recent study demonstrating
successful decoding of four different brain states employing
encoding durations of 5 s [16]. Finally, it is conceivable that
further fine-tuning of the decoding procedures, e.g., by implementing multivoxel pattern analysis (MVPA) [36], will allow
considerably shortening the required encoding time and at
the same time improving the decoding accuracy—though the
currently obtained decoding accuracy of 82% is more than
sufficient for communication purposes given that spelling
accuracies of 70% are usually regarded as the lower limit for
spelling with an assistant device [37]. Note also that MVPA
techniques usually require a long training phase (e.g., [38]),
especially for multiclass problems, which may significantly
extend a communication session.
In the long run, MRI scanners obviously need to be replaced
by mobile equipment, e.g., by fNIRS—a portable technique
that is also based on the hemodynamic brain response and
which should be sensitive enough to monitor single-trial
A Real-Time fMRI-Based Spelling Device
1337
time-course-related changes as required by our letter encoding technique [39]. Only that would re-enable patients to
communicate in ordinary, daily-life situations.
Until such mobile and less costly communication devices
based on hemodynamic brain signals become available, the
suggested fMRI-based letter spelling device can already now
be of great benefit (even if only employed offline). Because
MRI scanners constitute standard clinical equipment, the suggested communication techniques can be readily applied at
numerous sites. Note that even a single fMRI-based communication session can be of highest importance [40]—giving
patients more confidence (especially in early stages of
a disease) and potentially providing more certainty concerning
ethical issues in patient care and treatment as the patient’s
opinion could be considered.
Experimental Procedures
Participants and Preparation
Six healthy volunteers (average age = 29.0 years, Table S2) gave written
informed consent according to procedures approved by the local research
ethics committee. Participants chose three mental tasks (Table S1),
selected an initial question, and practiced letter encoding for about
10 min outside the MRI scanner.
fMRI Experiments
First, a short ‘‘localizer experiment’’ was performed in order to determine
ROIs showing robust mental task-related BOLD signal changes and to
derive individual RTCs (Supplemental Experimental Procedures).
In the subsequent ‘‘communication experiment,’’ participants first
answered their initial question using the letter encoding help (Figure 2)
and indicating the start and end of each word by encoding a blank space.
At the same time, functional MR images were collected and analyzed in
real time (see below). After the experimenters had deciphered the initial
answer based on the output of the automated letter decoder, participants
were asked a follow-up question containing their first answer (Figure 4)
and again encoded an answer while being scanned.
Data Acquisition and Analysis
Brain data were collected at 3T (Magnetom Allegra, Siemens Healthcare,
Erlangen, Germany) using standard (f)MRI sequences (Supplemental Experimental Procedures). Functional images were reconstructed and written to
the scanner console’s hard disk in real time using a custom-made image
export running on the image reconstruction computer (implemented in
Siemens ICE VA30) [41].
Online data analysis was performed using Turbo-BrainVoyager (v2.6;
Brain Innovation B.V., Maastricht, The Netherlands). Functional time series
of the ‘‘localizer experiment’’ were preprocessed, and for each participant,
six ROIs showing mental task-‘‘preference’’ (two ROIs per mental task) were
functionally defined by performing regression analysis based on the general
linear model using predictors corresponding to the particular mental
task conditions and applying appropriate statistical contrasts (e.g., ‘‘motor
imagery’’ versus ‘‘resting’’). The automated letter decoding procedure, performed in real time, was applied letter-by-letter to appropriate sections of
the online motion-corrected functional time series of the ‘‘communication
experiment.’’ Online word identification by the experimenters was based
on the main output of the automated letter decoder (‘‘top three’’-letter
choices; Figure 4; Movie S2). For further computational details, see Supplemental Experimental Procedures.
Supplemental Information
Supplemental Information includes four figures, two tables, Supplemental
Experimental Procedures, and two movies and can be found with this article
online at doi:10.1016/j.cub.2012.05.022.
Acknowledgments
We thank Lars Riecke and Hans Reithler for helpful comments on the
manuscript. The authors gratefully acknowledge the support of the
BrainGain Smart Mix Programme of The Netherlands Ministry of Economic
Affairs and The Netherlands Ministry of Education, Culture and Science
(grant number: SSM06011). B.S. was funded by The Netherlands Organization for Scientific Research (NWO, Rubicon program, project number: 44609-010).
Received: August 22, 2011
Revised: April 10, 2012
Accepted: May 10, 2012
Published online: June 28, 2012
References
1. Plum, F., and Posner, J.B. (1966). The Diagnosis of Stupor and Coma
(Philadelphia, PA: Davis, F.A.).
2. Farwell, L.A., and Donchin, E. (1988). Talking off the top of your head:
toward a mental prosthesis utilizing event-related brain potentials.
Electroencephalogr. Clin. Neurophysiol. 70, 510–523.
3. Chapin, J.K., Moxon, K.A., Markowitz, R.S., and Nicolelis, M.A. (1999).
Real-time control of a robot arm using simultaneously recorded neurons
in the motor cortex. Nat. Neurosci. 2, 664–670.
4. Scherer, R., Müller, G.R., Neuper, C., Graimann, B., and Pfurtscheller, G.
(2004). An asynchronously controlled EEG-based virtual keyboard:
improvement of the spelling rate. IEEE Trans. Biomed. Eng. 51, 979–984.
5. Leuthardt, E.C., Schalk, G., Wolpaw, J.R., Ojemann, J.G., and Moran,
D.W. (2004). A brain-computer interface using electrocorticographic
signals in humans. J. Neural Eng. 1, 63–71.
6. Mellinger, J., Schalk, G., Braun, C., Preissl, H., Rosenstiel, W.,
Birbaumer, N., and Kübler, A. (2007). An MEG-based brain-computer
interface (BCI). Neuroimage 36, 581–593.
7. Blankertz, B., Dornhege, G., Krauledat, M., Müller, K.R., and Curio, G.
(2007). The non-invasive Berlin Brain-Computer Interface: fast acquisition of effective performance in untrained subjects. Neuroimage 37,
539–550.
8. Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey,
B., Kübler, A., Perelmouter, J., Taub, E., and Flor, H. (1999). A spelling
device for the paralysed. Nature 398, 297–298.
9. Kübler, A., Kotchoubey, B., Hinterberger, T., Ghanayim, N.,
Perelmouter, J., Schauer, M., Fritsch, C., Taub, E., and Birbaumer, N.
(1999). The thought translation device: a neurophysiological approach
to communication in total motor paralysis. Exp. Brain Res. 124,
223–232.
10. Hochberg, L.R., Serruya, M.D., Friehs, G.M., Mukand, J.A., Saleh, M.,
Caplan, A.H., Branner, A., Chen, D., Penn, R.D., and Donoghue, J.P.
(2006). Neuronal ensemble control of prosthetic devices by a human
with tetraplegia. Nature 442, 164–171.
11. Nijboer, F., Sellers, E.W., Mellinger, J., Jordan, M.A., Matuz, T., Furdea,
A., Halder, S., Mochty, U., Krusienski, D.J., Vaughan, T.M., et al. (2008).
A P300-based brain-computer interface for people with amyotrophic
lateral sclerosis. Clin. Neurophysiol. 119, 1909–1916.
12. Wilhelm, B., Jordan, M., and Birbaumer, N. (2006). Communication in
locked-in syndrome: effects of imagery on salivary pH. Neurology 67,
534–535.
13. Weiskopf, N., Veit, R., Erb, M., Mathiak, K., Grodd, W., Goebel, R., and
Birbaumer, N. (2003). Physiological self-regulation of regional brain
activity using real-time functional magnetic resonance imaging (fMRI):
methodology and exemplary data. Neuroimage 19, 577–586.
14. Yoo, S.S., Fairneny, T., Chen, N.K., Choo, S.E., Panych, L.P., Park, H.,
Lee, S.Y., and Jolesz, F.A. (2004). Brain-computer interface using
fMRI: spatial navigation by thoughts. Neuroreport 15, 1591–1595.
15. Lee, J.H., Ryu, J., Jolesz, F.A., Cho, Z.H., and Yoo, S.S. (2009).
Brain-machine interface via real-time fMRI: preliminary study on
thought-controlled robotic arm. Neurosci. Lett. 450, 1–6.
16. Sorger, B., Dahmen, B., Reithler, J., Gosseries, O., Maudoux, A.,
Laureys, S., and Goebel, R. (2009). Another kind of ‘BOLD Response’:
answering multiple-choice questions via online decoded single-trial
brain signals. Prog. Brain Res. 177, 275–292.
17. Posse, S., Binkofski, F., Schneider, F., Gembris, D., Frings, W., Habel,
U., Salloum, J.B., Mathiak, K., Wiese, S., Kiselev, V., et al. (2001). A
new approach to measure single-event related brain activity using
real-time fMRI: feasibility of sensory, motor, and higher cognitive tasks.
Hum. Brain Mapp. 12, 25–41.
18. Handwerker, D.A., Ollinger, J.M., and D’Esposito, M. (2004). Variation of
BOLD hemodynamic responses across subjects and brain regions and
their effects on statistical analyses. Neuroimage 21, 1639–1651.
Current Biology Vol 22 No 14
1338
19. Aguirre, G.K., Zarahn, E., and D’esposito, M. (1998). The variability of
human, BOLD hemodynamic responses. Neuroimage 8, 360–369.
20. Boynton, G.M., Engel, S.A., Glover, G.H., and Heeger, D.J. (1996). Linear
systems analysis of functional magnetic resonance imaging in human
V1. J. Neurosci. 16, 4207–4221.
21. Glover, G.H. (1999). Deconvolution of impulse response in event-related
BOLD fMRI. Neuroimage 9, 416–429.
22. Sitaram, R., Zhang, H., Guan, C., Thulasidas, M., Hoshi, Y., Ishikawa, A.,
Shimizu, K., and Birbaumer, N. (2007). Temporal classification of
multichannel near-infrared spectroscopy signals of motor imagery for
developing a brain-computer interface. Neuroimage 34, 1416–1427.
23. Naito, M., Michioka, Y., Ozawa, K., Ito, Y., Kiguchi, M., and Kanazawa, T.
(2007). A Communication Means for Totally Locked-in ALS Patients
Based on Changes in Cerebral Blood Volume Measured with NearInfrared Light. IEICE Trans Inf & Syst E 90, 1028–1037.
24. Birbaumer, N., Murguialday, A.R., and Cohen, L. (2008). Brain-computer
interface in paralysis. Curr. Opin. Neurol. 21, 634–638.
25. Bauer, G., Gerstenbrand, F., and Rumpl, E. (1979). Varieties of the
locked-in syndrome. J. Neurol. 221, 77–91.
26. Owen, A.M., Coleman, M.R., Boly, M., Davis, M.H., Laureys, S., and
Pickard, J.D. (2006). Detecting awareness in the vegetative state.
Science 313, 1402.
27. Monti, M.M., Vanhaudenhuyse, A., Coleman, M.R., Boly, M., Pickard,
J.D., Tshibanda, L., Owen, A.M., and Laureys, S. (2010). Willful modulation of brain activity in disorders of consciousness. N. Engl. J. Med. 362,
579–589.
28. Aslaksen, P.M., Vangberg, T.R., and Schäfer, C. (2012). Probing
for consciousness after severe brain injury by functional magnetic
resonance imaging (fMRI). J. Neurol. 259, 576–578.
29. Naccache, L. (2006). Psychology. Is she conscious? Science 313, 1395–
1396.
30. Greenberg, D.L. (2007). Comment on ‘‘Detecting awareness in the
vegetative state’’. Science 315, 1221.
31. Nachev, P., and Husain, M. (2007). Comment on ‘‘Detecting awareness
in the vegetative state’’. Science 315, 1221.
32. Weiskopf, N., Scharnowski, F., Veit, R., Goebel, R., Birbaumer, N., and
Mathiak, K. (2004). Self-regulation of local brain activity using realtime functional magnetic resonance imaging (fMRI). J. Physiol. Paris
98, 357–373.
33. Kübler, A., Furdea, A., Halder, S., Hammer, E.M., Nijboer, F., and
Kotchoubey, B. (2009). A brain-computer interface controlled auditory
event-related potential (p300) spelling system for locked-in patients.
Ann. N Y Acad. Sci. 1157, 90–100.
34. Caria, A., Veit, R., Sitaram, R., Lotze, M., Weiskopf, N., Grodd, W., and
Birbaumer, N. (2007). Regulation of anterior insular cortex activity using
real-time fMRI. Neuroimage 35, 1238–1246.
35. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., and
Vaughan, T.M. (2002). Brain-computer interfaces for communication
and control. Clin. Neurophysiol. 113, 767–791.
36. LaConte, S.M., Peltier, S.J., and Hu, X.P. (2007). Real-time fMRI using
brain-state classification. Hum. Brain Mapp. 28, 1033–1044.
37. Sellers, E.W., Krusienski, D.J., McFarland, D.J., Vaughan, T.M., and
Wolpaw, J.R. (2006). A P300 event-related potential brain-computer
interface (BCI): the effects of matrix size and inter stimulus interval on
performance. Biol. Psychol. 73, 242–252.
38. Stokes, M., Thompson, R., Cusack, R., and Duncan, J. (2009). Top-down
activation of shape-specific population codes in visual cortex during
mental imagery. J. Neurosci. 29, 1565–1572.
39. Irani, F., Platek, S.M., Bunce, S., Ruocco, A.C., and Chute, D. (2007).
Functional near infrared spectroscopy (fNIRS): an emerging neuroimaging technology with important applications for the study of brain disorders. Clin. Neuropsychol. 21, 9–37.
40. deCharms, R.C. (2008). Applications of real-time fMRI. Nat. Rev.
Neurosci. 9, 720–729.
41. Weiskopf, N., Mathiak, K., Bock, S.W., Scharnowski, F., Veit, R., Grodd,
W., Goebel, R., and Birbaumer, N. (2004). Principles of a brain-computer
interface (BCI) based on real-time functional magnetic resonance
imaging (fMRI). IEEE Trans. Biomed. Eng. 51, 966–970.