Topical Review
Section Editors: Amanda Thrift, PhD, and Barbara G. Vickrey, MD, MPH
Are Cognitive Screening Tools Sensitive and Specific Enough
for Use After Stroke?
A Systematic Literature Review
Renerus J. Stolwyk, DPsych; Megan H. O’Neill, BPsych(Hons); Adam J.D. McKay, PhD;
Dana K. Wong, PhD
I
Downloaded from http://stroke.ahajournals.org/ by guest on July 31, 2017
t is estimated that up to three quarters of acute and subacute
stroke survivors exhibit cognitive impairment, with many
experiencing ongoing problems.1,2 Cognitive impairment can
significantly compromise functional recovery, quality of life,
and social engagement after stroke.2–4 Encouragingly early
detection and rehabilitation can improve functional recovery
of stroke-related impairments.5 Unfortunately, however, a significant amount of cognitive dysfunction is not detected by
health professionals in acute and subacute settings.6
Comprehensive neuropsychological assessment using reliable and valid tools to measure multiple cognitive domains
is considered the gold standard method of detecting and
characterizing cognitive dysfunction after stroke. However,
neuropsychological assessments are often considered too
expensive and lengthy to be routinely administered to patients
with stroke. In an attempt to improve detection of cognitive
impairments, while managing expense, many national stroke
clinical management guidelines now recommend the use of
screening measures to detect cognitive impairment.7–9 If cognitive difficulties are detected during this screening process,
comprehensive assessment and intervention is then recommended. The Mini-Mental State Examination (MMSE) and
Montreal Cognitive Assessment (MoCA) are 2 screening
tools that are regularly used in clinical practice. Although
these tests are commonly used to detect cognitive impairment
in dementia settings, neither was specifically designed for use
after stroke. The profile of cognitive impairment after stroke is
heterogeneous, and focal impairments such as dysphasia, dyspraxia, unilateral inattention, and agnosia are often observed.
Therefore, we cannot assume that reliability and validity of
cognitive screening tools found in other clinical populations
will be comparable in stroke.
It is acknowledged that numerous reliability and validity
indices are important to consider when evaluating neuropsychological measures. However, when considering the use of
cognitive screening measures, sensitivity, specificity, positive
predictive value (PPV), and negative predictive value (NPV)
are particularly important to ensure patients with cognitive
impairment are not missed, and patients without cognitive
impairment do not undergo comprehensive neuropsychological evaluation unnecessarily. Several studies have investigated the sensitivity and specificity of cognitive screening
tools within stroke populations. However, a range of different methodologies have been used, and results seem to vary
considerably across studies. Thus, the aims of this review
were (1) to systematically review the sensitivity, specificity,
PPV, and NPV of a range of cognitive screening tools used in
stroke and (2) to critically evaluate methodologies used within
these studies. It is intended that findings from this review will
inform clinicians regarding suitability of these screening tools
for clinical use and direct best practice for future research in
this field.
Methods
Search Strategy
This systematic literature review was conducted and reported in line
with the current Preferred Reporting Items for Systematic Reviews
and Meta-Analyses (PRISMA) statement. Articles were identified
through MedLine, PsychInfo, Scopus, PubMed, and CINAHL databases. Keywords included stroke, cerebrovasc*, cognit*, screen*,
sensitivity, and specificity. Common screening measure names were
also used. See Figure I in the online-only Data Supplement for an
example of key words and search strategy. The search was limited to
studies of adult humans published in English. The electronic search
was conducted on December 27, 2013. Reference lists of articles included in this review and other relevant publications were also used to
identify any studies overlooked in the electronic search.
Study Selection
Articles were included in this review if they met 3 key criteria: (1)
male or female participants aged ≥18 years; (2) confirmed ischemic or
hemorrhagic stroke, and (3) analysis of the sensitivity and specificity
of a cognitive screening measure compared with a gold standard neuropsychological assessment. If >1 clinical population was included
Received February 18, 2014; final revision received May 27, 2014; accepted July 1, 2014.
From the School of Psychological Sciences, Monash University, Melbourne, Australia (R.J.S., M.H.O., A.J.D.M., D.K.W.); and Epworth Rehabilitation,
Melbourne, Australia (A.J.D.M.).
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.114.004232/-/DC1.
Correspondence to Renerus J. Stolwyk, DPsych, School of Psychological Sciences, Building 17, Clayton Campus, Monash University, Melbourne,
Victoria, 3800, Australia. E-mail [email protected]
(Stroke. 2014;45:3129-3134.)
© 2014 American Heart Association, Inc.
Stroke is available at http://stroke.ahajournals.org
DOI: 10.1161/STROKEAHA.114.004232
3129
3130 Stroke October 2014
in a study (eg, transient ischemic attack and stroke), stroke-specific
data must have been available. Cognitive screening measures were
included if they were designed to screen for cognitive impairment
or had been used for that purpose and typically took <30 minutes
to administer. Gold standard neuropsychological assessments were
included if they used multiple domain-specific neuropsychological
assessments with established reliability and validity.10 Some studies
identified during the literature search investigated screening tools that
aim to detect just 1 cognitive domain, such as dysphasia or dyspraxia.
However, understandably these studies typically only included 1 cognitive domain within their gold standard assessment and thus did not
meet our eligibility criteria.
In line with PRISMA guidelines, 2 authors (R.J.S. and M.H.O.)
separately reviewed results from the electronic search and identified
potentially relevant titles and abstracts. If the abstract suggested the
article met the inclusion criteria, the full-text article was obtained and
evaluated. Full-text articles were then compared across authors, and
contrasting/ambiguous studies were discussed to determine whether
they met criteria for inclusion. Articles that met the inclusion criteria
were included for subsequent data extraction.
Downloaded from http://stroke.ahajournals.org/ by guest on July 31, 2017
Data Extraction
The following data were extracted from each article: author, year,
title, participant data (sample size, age, sex, education), stroke data
(mechanism, location, hemisphere, severity), recruitment procedures
(inclusion/exclusion criteria, participant attrition), cognitive screening measure used, domains and tests included in gold standard cognitive assessment, time poststroke of screening and gold standard
cognitive assessments, and sensitivity, specificity, PPV, and NPV
results. In some studies, multiple sets of sensitivity, specificity, PPV,
and NPV data were presented at different screening measure cut
points. To limit the amount of data presented, the cut point that resulted in the most favorable sensitivity and specificity results was selected. This was based on commonly used criteria of sensitivity >80%
and specificity >60%.11,12
Results
Search Results
Electronic and additional searching returned 13 201 records;
duplicate ones were removed. Sixty-six records remained
following title and abstract screening. A further 50 records
were excluded during full-text review for the following reasons: review articles (n=2), combined transient ischemic
attack/stroke data (n=3), full neuropsychological battery used
instead of screening tool (n=4), screening measure did not
meet inclusion criteria (n=3), not written in English (n=2), no
sensitivity and specificity data (n=6), nonstroke samples used
(n=11), and gold standard neuropsychological battery did not
meet inclusion criteria (n=19). Sixteen articles were found to
meet our inclusion criteria and were retained for analysis. See
Figure II in the online-only Data Supplement for a summary
of the above.
Study Participants
Summary of study descriptions and evaluations is presented in
Tables I and II in the online-only Data Supplement. All studies adequately reported sample size; however, few justified the
sample size used or reported whether assumptions for statistical analyses were met. Based on key references within this
field,13,14 it seems that many studies did not use sufficient sample
sizes, particularly those that used samples of ≤50 people.15–18
This may have contributed to the large confidence intervals of
sensitivity and specificity results noted across studies.
With regard to demographic variables, all studies provided adequate information regarding age and sex. Four
studies failed to include education information.11,15,16,19 Most
study samples appeared representative of stroke populations.
However, most studies excluded nonnative language speakers. With regard to stroke variables, most studies reported key
information such as stroke mechanism (12 of 16), location (12
of 16), and severity (11 of 16). Stroke mechanism and lesion
location variables across studies appeared generally representative of stroke populations. However, most studies excluded
people with severe stroke. Only 2 studies provided adequate
statistical analysis of whether the final study sample used for
analysis was representative of the wider patient group within
their clinical setting.20,21 There was significant heterogeneity
across studies regarding time since stroke, with mean/median
times ranging from 6 days17 to 29 months18 across studies.
Two studies failed to report this information,15,22 and others
provided only limited information.
Sensitivity and Specificity Methodology
All studies calculated sensitivity and specificity using receiver
operative characteristics curve analysis. Only 9 of 16 studies calculated PPV or NPV data. Gold standard assessments
used to classify the cognitive status of participants differed
across studies. Cognitive domains such as language, visual/
space perception, attention, memory, and executive functions were generally well represented. However, cognitive
functions such as calculic function, praxis, and mental speed
were less well represented, included in ≤4 of the 16 studies.
Most studies used age- and education-based normative data
to interpret gold standard test performances, using a criterion ranging from fifth to tenth percentile as an indicator of
impaired performance. However, studies differed regarding
whether impairment on single or multiple cognitive domains
was required to classify participants as impaired on gold standard assessments. Furthermore, some studies did not use psychometric criteria at all, instead relying on clinician opinion
taken from neuropsychological reports.11,15,20 To reliably and
validly assess sensitivity and specificity, it was expected that
screening and gold standard assessments would be conducted
within a short period of each other. Unfortunately, 5 studies
did not sufficiently report this information,16,18,20,23,24 and 3
reported the mean time interval between assessments as >10
days.11,15,25 Only 6 studies stratified sensitivity or specificity
results according to demographic or stroke variables.20,22,23,25–27
Sensitivity and Specificity Results
As seen in Table, the MMSE and MoCA were the most commonly studied screening measures. With regard to the MMSE,
11 studies investigated the sensitivity and specificity of the measure, with just 3 reporting sensitivity and specificity at or above
commonly regarded acceptable levels (sensitivity >80% and
specificity >60%).23,25,27 All 3 studies obtained these levels using
cut points of either 26/30 or 27/30. Bour et al25 achieved acceptable sensitivity and specificity levels only when the gold standard
assessment impairment criterion was increased to ≥2 cognitive
domains. Of the 3 studies that reported adequate sensitivity and
specificity, PPVs were generally >80%; however, NPVs were
less impressive, ranging from 65% to 73% across studies.
Stolwyk et al Cognitive Screening After Stroke 3131
Table. Results From Included Studies
Study
Screening Measure
20
Agrell et al
MMSE
Blake et al11
MMSE
Boosman et al26
BNI
Bour et al
MMSE
25
Cartoni et al15
Cumming et al27
MEAMS
Screening Measure Cut Point
SE, % (CI)
SP, % (CI)
PPV, % (CI)
NPV, % (CI)
23/24
56
80
<24
62
88
Age <55, <47
80 (49–94)
39 (20–61)
61
93
Age >55, <41
92 (67–99)*
84 (58–96)*
92
96
27/28 ≥1 impaired gold standard
domain
72
71
93
27/28 ≥2 impaired gold standard
domains
80*
70*
86
26/27 ≥4 impaired gold standard
domains
82*
75*
72
≥3 failed subtests
52 (32–71)
100 (29–100)
≥5 failed subtests
26 (11–46)
100 (29–100)
Downloaded from http://stroke.ahajournals.org/ by guest on July 31, 2017
MMSE
26/27†
82*
76*
86
70
MoCA
23/24
92*
67*
84 82 MMSE
≤26
80*
77*
88
65
MoCA
≤23
88*
71*
86
73
MMSEadj
≤27
89*
61*
83
73
MoCAadj
(adjusted scores created using
control group to account for age
and education)
≤23
84*
81*
90 71
MMSE
<27
81
45
Modified MMSE
<86
94
50
Green et al28
RBANS
83/84
84*
90*
98
53
Morris et al
ACE-R
82
80
40
87
28
MMSE
27
80
20
84
16
Cognistat
8/9
81 (68–93)*
67 (22–96)*
SINS
2/3
71 (57–85)
67 (22–96)
Godefroy et al23
Grace et al22
12
Nøkleby et al16
CDT
9/10
63 (49–78)
67 (22–96)
Nys et al17
MMSE
<27
96
40
Salvadori et al21
MoCA
21
91*
76*
80
89
Schweizer et al18
MMSE
<27
MoCA
<26
Domain-specific data provided.
No global sensitivity/
specificity data; MMSE:
sensitivity=0 for all cognitive
domains, specificity=1.00 for
all cognitive domains; MoCA:
sensitivity ranged from 40 to
100 across domains; specificity
ranged from 54 to 70 across
domains
Srikanth et al24
MMSE-S
≤23
14 (4–32)
100 (92–100)
100 (40–100)
63 (50–74)
Wong et al
MoCA
2–4 wk 17/18
19
75 (43–95)
95 (87–99)
75 (41–95)
95 (87–99)
100 (74–100)*
75 (63–85)*
41 (24–61)
100 (93–100)
2–4 wk 23/24
75 (43–95)
90 (80–96)
60 (32–84)
95 (86–99)
1 y 23/24
58 (28–85)
84 (73–92)
39 (17–64)
92 (82–97)
1 y 21/22
MMSE
ACE-R indicates Addenbrooke Cognitive Examination, Revised; AUC, area under the receiver operating curve; BNI, Barrow Neurological Institute (screen for higher
cerebral functions); CDT, clock drawing task; CI, confidence interval; MEAMS, Middlesex Elderly Assessment of Mental State; MMSE, Mini-Mental State Examination;
MMSE-S, MMSE-Standardized; MoCA, Montreal Cognitive Assessment; NPV, negative predictive value; PPV, positive predictive value; RBANS, Repeatable Battery for the
Assessment of Neuropsychological Status; SE, sensitivity; SINS, Screening Instrument for Neuropsychological Impairments in Stroke; and SP, specificity.
*Results reach traditionally acceptable levels of 80% SE and 60% SP.
†Improved AUC for MMSE noted when z score criterion increased to −1.5.
3132 Stroke October 2014
Downloaded from http://stroke.ahajournals.org/ by guest on July 31, 2017
Five studies investigated the sensitivity and specificity of
the MoCA. Three of these reported adequate sensitivity and
specificity21,23,27; 1 further study reported adequate sensitivity and specificity at 1 year poststroke, but not 2 to 4 weeks
poststroke,19 and the final study reported adequate sensitivity and specificity for only 2 of the 13 gold standard tests
included (naming and verbal learning).18 Acceptable sensitivity and specificity were found at different MoCA cut points
across the studies, ranging from 21 to 26. Of the 4 studies that
reported adequate sensitivity and specificity, only 2 studies
also reported PPVs and NPVs >80%.21,27
Four studies directly compared the MMSE and MoCA.
Using area under the receiver operating curve scores, 2 studies reported no significant differences between the MMSE and
MoCA,23,27 whereas another reported higher MoCA area under
the receiver operating curve scores compared with the MMSE
at 1 year poststroke only.19 The final study reported superior
sensitivity of the MoCA compared with the MMSE.18 The
general trend noted across these 4 studies was of somewhat
better sensitivity and slightly poorer specificity of MoCA
compared with the MMSE.
The sensitivity and specificity of other screening measures
have been investigated in only a single study that met our inclusion criteria. Both the Repeatable Battery for the Assessment
of Neuropsychological Status (RBANS) and Cognistat demonstrated typically accepted sensitivity and specificity levels;
however, NPV was only 53% for the RBANS, and no PPV or
NPV data were presented for the Cognistat.16,28 The Barrow
Neurological Institute (BNI) screen for higher cerebral functions returned acceptable sensitivity, specificity, PPV, and
NPV levels in people aged >55 years but not <55 years.26 The
Middlesex Elderly Assessment of Mental State (MEAMS),
Addenbrooke Cognitive Examination, Revised (ACE-R),
Screening Instrument for Neuropsychological Impairments in
Stroke (SINS), and Clock Drawing Test all failed to achieve
adequate levels of sensitivity and specificity.12,15,16
In addition to providing total scores, some cognitive screening measures also provided domain-specific subscores. Four
studies included in this review examined the sensitivity and
specificity of these subscores to detect domain-specific cognitive impairment assessed using gold standard measures.12,15,16,28
This information is presented in Table III in the online-only
Data Supplement. Three of the 5 subscores from the RBANS
achieved acceptable sensitivity and specificity levels (immediate memory, language, visuospatial) with the other 2 subtests
only just under acceptable thresholds (delayed memory, attention).28 Results from the Cognistat, MEAMS, ACE-R, and
SINS were less impressive. The Cognistat and MEAMS both
achieved acceptable results for their naming subscores; however, all other subscores did not reach acceptable levels.15,16
No subscores from ACE-R or SINS reached acceptable levels.12,16 Although memory, language, and visuospatial domains
were regularly assessed, attention, processing speed, praxis,
and executive function were seldom examined.
Few studies specifically investigated whether methodological factors or patient variables impacted on sensitivity
and specificity results. Qualitatively there was no consistent
evidence to suggest time poststroke significantly affected
sensitivity and specificity results. Although Wong et al19
reported acceptable MoCA sensitivity and specificity at 1
year but not 2 to 4 weeks after stroke, other studies reported
favorable MoCA, Cognistat, RBANS, and BNI sensitivity and
specificity ranging from 1 week to 1 year poststroke. With
regard to time interval between screening and gold standard
assessments, almost all studies that reported favorable sensitivity and specificity of the MoCA, Cognistat, RBANS, and
BNI used mean assessment time intervals within ≈1 week.
Studies varied regarding their criteria for impairment on gold
standard assessment. Those who used a multiple cognitive
domain criterion were more likely to report adequate sensitivity and specificity compared with those who used a singledomain criterion. A minority of studies stratified sensitivity
and specificity results according to demographic and stroke
variables. One study reported better sensitivity and specificity results in older stroke participants.26 Lesion hemisphere
effects were equivocal. One study reported no effect,25 whereas
others reported better results in right hemisphere20,27 or left
hemisphere groups.22 No studies specifically investigated the
impact of premorbid cognitive function, stroke severity or
mechanism, or cultural factors on screening measure performance. Few studies directly compared screening measure performance across different gold standard cognitive domains.
The MoCA was shown to be relatively more sensitive to naming and verbal learning difficulties compared with other cognitive domains in 1 study,18 whereas performance was higher
for language and visuospatial impairments in another.19
Discussion
Sixteen articles that investigated the sensitivity and specificity
of cognitive screening tools in stroke met our inclusion criteria. Eleven of these studies investigated the MMSE, and most
reported inadequate sensitivity and specificity. MoCA results
were somewhat better, with 3 of 5 studies reporting consistent
acceptable sensitivity and specificity results. It is not clear why
the MoCA performed better than the MMSE. However, possible reasons include the fact the MoCA contains items assessing executive functions, which are often affected by stroke and
the total score of MoCA can be adjusted for education level,
albeit crudely. Interestingly, 2 relatively more recently developed measures, the RBANS and Cognistat, demonstrated traditionally acceptable levels of sensitivity and specificity. There
is also some preliminary support for the use of BNI within
older stroke populations. Furthermore, analysis of RBANS
subscores highlighted promising sensitivity and specificity
results to detect a range of focal cognitive difficulties, including memory, language, and visuospatial difficulties. However,
it is noted that the RBANS, Cognistat, and BNI were only
investigated in 1 study that met our eligibility criteria, and further research confirming these initial findings is warranted.
The above findings provide some preliminary support for the
use of the MoCA, BNI, Cognistat, and RBANS as screening
measures for stroke. However, these findings should be considered in the context of some key methodological issues. First,
of the 7 studies that reported adequate sensitivity and specificity of MoCA, BNI, Cognistat, and RBANS, 3 either failed to
report PPVs and NPVs16 or reported NPVs <80% (indicating
Stolwyk et al Cognitive Screening After Stroke 3133
Downloaded from http://stroke.ahajournals.org/ by guest on July 31, 2017
≥20% false-negative rates).23,28 Second, adequate sensitivity and specificity of the MoCA were found at different cut
points, making recommendations for clinical practice difficult.
Third, most studies did not include calculation, praxis, and
speed of information processing within gold standard assessments. Thus, capacity for screening measures to detect these
cognitive difficulties remains unknown, which is problematic
considering that impairments of calculic, praxis, and mental
speed functions are not uncommon after stroke and can significantly impact functional recovery.1 Fourth, most studies that
reported adequate sensitivity and specificity used a criterion of
multiple cognitive impairments (≥2 domains) within their gold
standard assessments. Studies have shown higher screening
measure sensitivity for multiple-domain versus single-domain
cognitive impairments.25,29 Thus sensitivity results from studies that used multiple-domain impairment as a gold standard
criterion may have been lower if a single-domain criterion
was used (although equally specificity results may have been
higher). Finally, few studies stratified sensitivity and specificity results according to demographic and stroke variables. This
can be problematic for several reasons. For example, many
screening measures do not account for age, education, or premorbid intelligence. Thus, it is possible that sensitivity of these
screening measures for young and highly intelligent people
may be limited, and specificity may be limited in older people
and those with lower premorbid intelligence. Furthermore,
people with severe stroke and those from culturally and linguistically diverse backgrounds were often excluded from
studies altogether. Additional research is required within these
groups before use of these screening measures is warranted.
See Figure for recommendations for future research.
With regard to more general methodological issues, significant heterogeneity and poor reporting regarding time interval
between screening and gold standard assessments and time
of assessment since stroke were noted across studies. Unless
long-term predictive validity is a specific research aim, we
recommend screening and gold standard assessments be conducted as time-congruent as possible. Cognitive function can
change significantly during the course of stroke recovery,
and results from early screening cannot be assumed to be
Figure. Recommended methodological considerations for future
studies. CALD indicates culturally and linguistically diverse; NPV,
negative predictive value; PPV, positive predictive value; SE, sensitivity; and SP, specificity.
an accurate picture of longer-term cognitive function. Thus,
we recommend further research investigating the potential
impact of time since stroke on the sensitivity and specificity of
screening measures. On another note, although it is important
to report PPV and NPV data, we acknowledge these values
vary according to prevalence of impairment in the population.
Thus, direct comparison of these values across studies is not
valid. Importantly, however, PPVs and NPVs can be calculated based on sensitivity, specificity, and prevalence data.30
Thus, clinicians and researchers alike may choose to use data
presented in this review to estimate PPV and NPV across a
range of stroke populations where prevalence data are known.
Finally, few studies have investigated which specific cognitive
domains are more or less likely to be detected by these screening measures. Further research is warranted.
Many researchers have previously suggested that 80% sensitivity and 60% specificity of cognitive screening measures
is considered adequate for clinical practice. However, the
significant negative impact of cognitive impairment in stroke
survivors has been consistently demonstrated.2–4 As such, 20%
nondetection of patients with cognitive impairment seems
unacceptable for clinical practice. Further research is required
to more comprehensively examine existing screening measures that show initial promise (MoCA, Cognistat, RBANS,
BNI) addressing previous methodological weaknesses noted
above. Further development of more appropriate stroke-specific screening measures may be warranted if future research
does not generate positive results. Furthermore, it is important to evaluate how current recommended guidelines (cognitive screening followed by comprehensive assessment) are
being implemented in clinical practice. There is evidence to
suggest good adherence to cognitive screening protocols, but
limited provision of further comprehensive assessment when
indicated by screening results.31 Further research exploring
potential modifications to screening processes is also warranted. For example, benefits of including patient, close other,
or clinician reports of cognitive difficulties, in conjunction
with screening measures, to improve detection of cognitive
difficulties could be explored. Addition of items not included
in current screening measures, but often affected by stroke
such as calculation, praxis, and mental speed, should also be
considered. It would be particularly helpful for these cognitive measures to be incorporated as standard measures within
stroke trials to ensure ongoing comprehensive investigation
of their utility across research and clinical settings. Although
beyond the scope of this review, it is also important to consider cognitive screening in other cerebrovascular disorders.
For example, some screening protocols have been specifically
developed for small-vessel disease and have demonstrated
encouraging results.32 This may be because of the relatively
more homogeneous neuropathology and associated cognitive
profile seen in this population, compared with the relatively
more heterogeneous cognitive profile across the stroke population, which seems to present as a challenge for some existing
screening measures to accommodate.
In conclusion, a limited number of studies have adequately
investigated the sensitivity and specificity of cognitive screening measures after stroke. Although most studies do not
support the MMSE for clinical use, the MoCA, Cognistat,
3134 Stroke October 2014
RBANS, and BNI show some initial promise. However, further research addressing key methodological considerations
and further discussion regarding what is considered acceptable sensitivity and specificity for clinical practice is required
before use of these screening measures can be fully supported.
Disclosures
None.
References
Downloaded from http://stroke.ahajournals.org/ by guest on July 31, 2017
1.Leśniak M, Bak T, Czepiel W, Seniów J, Członkowska A. Frequency
and prognostic value of cognitive disorders in stroke patients. Dement
Geriatr Cogn Disord. 2008;26:356–363.
2. Nys GM, van Zandvoort MJ, de Kort PL, van der Worp HB, Jansen BP,
Algra A, et al. The prognostic value of domain-specific cognitive abilities in acute first-ever stroke. Neurology. 2005;64:821–827.
3. Hochstenbach JB, Anderson PG, van Limbeek J, Mulder TT. Is there
a relation between neuropsychologic variables and quality of life after
stroke? Arch Phys Med Rehabil. 2001;82:1360–1366.
4. Hommel M, Miguel ST, Naegele B, Gonnet N, Jaillard A. Cognitive determinants of social functioning after a first ever mild to moderate stroke at
vocational age. J Neurol Neurosurg Psychiatry. 2009;80:876–880.
5. Paolucci S, Antonucci G, Grasso MG, Morelli D, Troisi E, Coiro P, et al.
Early versus delayed inpatient stroke rehabilitation: a matched comparison conducted in Italy. Arch Phys Med Rehabil. 2000;81:695–700.
6.Jaillard A, Naegele B, Trabucco-Miguel S, LeBas JF, Hommel M.
Hidden dysfunctioning in subacute stroke. Stroke. 2009;40:2473–2479.
7. Intercollegiate Stroke Working Party. National Clinical Guidelines for
Stroke. 4th ed. London, UK: Royal College of Physicians; 2012.
8. National Stroke Foundation. Clinical Guidelines for Stroke Management
2010. Melbourne, Australia: National Stroke Foundation; 2010.
9. Lindsay MP, Gubitz G, Bayley M, Hill MD, Davies-Schinkel C, Singh
S, Phillips S. Canadian Best Practice Recommendations for Stroke Care
(Update 2010). Ottawa, Canada: Canadian Stroke Network; 2010.
10. Lezak MD, Howieson DB, Bigler ED, Tranel D. Neuropsychological
Assessment. 5th ed. New York, NY: Oxford University Press; 2012.
11. Blake H, McKinney M, Treece K, Lee E, Lincoln NB. An evaluation of
screening measures for cognitive impairment after stroke. Age Ageing.
2002;31:451–456.
12. Morris K, Hacker V, Lincoln NB. The validity of the Addenbrooke’s
Cognitive Examination-Revised (ACE-R) in acute stroke. Disabil
Rehabil. 2012;34:189–195.
13. Carley S, Dosman S, Jones SR, Harrison M. Simple nomograms to calculate sample size in diagnostic studies. Emerg Med J. 2005;22:180–181.
14. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver
operating characteristic (ROC) curve. Radiology. 1982;143:29–36.
15. Cartoni A, Lincoln NB. The sensitivity and specificity of the Middlesex
Elderly Assessment of Mental State (MEAMS) for detecting cognitive
impairment after stroke. Neuropsychol Rehabil. 2005;15:55–67.
16. Nøkleby K, Boland E, Bergersen H, Schanke AK, Farner L, Wagle J, et
al. Screening for cognitive deficits after stroke: a comparison of three
screening tools. Clin Rehabil. 2008;22:1095–1104.
17. Nys GM, van Zandvoort MJ, de Kort PL, Jansen BP, Kappelle LJ, de
Haan EH. Restrictions of the Mini-Mental State Examination in acute
stroke. Arch Clin Neuropsychol. 2005;20:623–629.
18.Schweizer TA, Al-Khindi T, Macdonald RL. Mini-Mental State
Examination versus Montreal Cognitive Assessment: rapid assessment
tools for cognitive and functional outcome after aneurysmal subarachnoid hemorrhage. J Neurol Sci. 2012;316:137–140.
19. Wong GK, Lam SW, Wong A, Ngai K, Poon WS, Mok V. Comparison
of montreal cognitive assessment and mini-mental state examination in
evaluating cognitive domain deficit following aneurysmal subarachnoid
haemorrhage. PLoS One. 2013;8:e59946.
20. Agrell B, Dehlin O. Mini mental state examination in geriatric stroke
patients. Validity, differences between subgroups of patients, and relationships to somatic and mental variables. Aging (Milano). 2000;12:439–444.
21. Salvadori E, Pasi M, Poggesi A, Chiti G, Inzitari D, Pantoni L. Predictive
value of MoCA in the acute phase of stroke on the diagnosis of mid-term
cognitive impairment. J Neurol. 2013;260:2220–2227.
22. Grace J, Nadler JD, White DA, Guilmette TJ, Giuliano AJ, Monsch AU, et
al. Folstein vs modified Mini-Mental State Examination in geriatric stroke.
Stability, validity, and screening utility. Arch Neurol. 1995;52:477–484.
23. Godefroy O, Fickl A, Roussel M, Auribault C, Bugnicourt JM, Lamy C,
et al. Is the Montreal Cognitive Assessment superior to the Mini-Mental
State Examination to detect poststroke cognitive impairment? A study
with neuropsychological evaluation. Stroke. 2011;42:1712–1716.
24. Srikanth V, Thrift AG, Fryer JL, Saling MM, Dewey HM, Sturm JW,
et al. The validity of brief screening cognitive instruments in the diagnosis of cognitive impairment and dementia after first-ever stroke. Int
Psychogeriatr. 2006;18:295–305.
25. Bour A, Rasquin S, Boreas A, Limburg M, Verhey F. How predictive is the
MMSE for cognitive performance after stroke? J Neurol. 2010;257:630–637.
26. Boosman H, Visser-Meily JM, Post MW, Duits A, van Heugten CM.
Validity of the Barrow Neurological Institute (BNI) screen for higher
cerebral functions in stroke patients with good functional outcome. Clin
Neuropsychol. 2013;27:667–680.
27. Cumming TB, Churilov L, Linden T, Bernhardt J. Montreal Cognitive
Assessment and Mini-Mental State Examination are both valid cognitive
tools in stroke. Acta Neurol Scand. 2013;128:122–129.
28. Green S, Sinclair E, Rodger E, Birks E, Lincoln N. The Repeatable Battery
for the Assessment of Neuropsychological Status (RBANS) for post-stroke
cognitive impairment screening. Int J Ther Rehabil. 2013;20: 536–541.
29. Pendlebury ST, Mariz J, Bull L, Mehta Z, Rothwell PM. Impact of different operational definitions on mild cognitive impairment rate and
MMSE and MoCA performance in transient ischaemic attack and stroke.
Cerebrovasc Dis. 2013;36:355–362.
30. Smith JE, Winkler RL, Fryback DG. The first positive: computing positive predictive value at the extremes. Ann Intern Med. 2000;132:804–809.
31. McClure JA, Salter K, Foley N, Mahon H, Teasell R. Adherence to
Canadian Best Practice Recommendations for Stroke Care: vascular cognitive impairment screening and assessment practices in an Ontario inpatient stroke rehabilitation facility. Top Stroke Rehabil. 2012;19:141–148.
32.O’Sullivan M, Morris RG, Markus HS. Brief cognitive assessment
for patients with cerebral small vessel disease. J Neurol Neurosurg
Psychiatry. 2005;76:1140–1145.
Key Words: cognition ◼ neuropsychology ◼ sensitivity and specificity ◼ stroke
Are Cognitive Screening Tools Sensitive and Specific Enough for Use After Stroke?: A
Systematic Literature Review
Renerus J. Stolwyk, Megan H. O'Neill, Adam J.D. McKay and Dana K. Wong
Downloaded from http://stroke.ahajournals.org/ by guest on July 31, 2017
Stroke. 2014;45:3129-3134; originally published online July 29, 2014;
doi: 10.1161/STROKEAHA.114.004232
Stroke is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231
Copyright © 2014 American Heart Association, Inc. All rights reserved.
Print ISSN: 0039-2499. Online ISSN: 1524-4628
The online version of this article, along with updated information and services, is located on the
World Wide Web at:
http://stroke.ahajournals.org/content/45/10/3129
Data Supplement (unedited) at:
http://stroke.ahajournals.org/content/suppl/2014/07/29/STROKEAHA.114.004232.DC1
Permissions: Requests for permissions to reproduce figures, tables, or portions of articles originally published
in Stroke can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Office.
Once the online version of the published article for which permission is being requested is located, click
Request Permissions in the middle column of the Web page under Services. Further information about this
process is available in the Permissions and Rights Question and Answer document.
Reprints: Information about reprints can be found online at:
http://www.lww.com/reprints
Subscriptions: Information about subscribing to Stroke is online at:
http://stroke.ahajournals.org//subscriptions/
SUPPLEMENTAL MATERIAL
Are cognitive screening tools sensitive and specific enough for use following stroke? A systematic literature review.
*Renerus J. Stolwyk, DPsych 1; Megan H. O’Neill, BPsych(Hons) 1; Adam J.D. McKay, PhD 1, 2; Dana K.Wong, PhD 1
1
2
School of Psychological Sciences, Monash University, Melbourne, Australia
Epworth Rehabilitation, Melbourne, Australia
*Corresponding author: Dr Renerus J. Stolwyk, School of Psychological Sciences, Building 17, Clayton Campus, Monash University,
Melbourne, Victoria, 3800, Australia. Phone: +61 3 9902 0099. Fax: +61 3 9905 3948. Email: [email protected]
1
Supplemental Table I. Descriptions of research methodologies
Study
Screening
Measure
Agrell et
al.1
MMSE
Blake et al.2 MMSE
Sample Size
(included in
statistical
analyses)
116
112
Inclusion and
exclusion criteria
Demographic
Information
Stroke Information
Time poststroke
Inclusion: Stroke
patients on geriatric
rehabilitation unit
Age: Female M=77
SD=7; Male M=76
SD=7
Sex: Male N=65,
Female N=51
Education: Primary
school only N=85,
higher-level schooling
N=31
Age: M=70.8,
SD=12.2
Sex: Male N=64,
Women N=48
Education: NR
Mechanism: Infarction
N=84, Haemorrhage
N=10, Unspecified N=22.
Location: NR
Hemisphere: Right N=60,
Left N=39, Infratentorial
N=17
Severity: Barthel Index
M=67, SD=22
Mechanism: NR
Location: NR
Hemisphere: Left
weakness N=50
right weakness N=56,
Bilateral N=1, no signs
N=1, presence of
weakness unknown N=2
Severity: Barthel Index
M=10.5 (SD.5.8)
Mechanism, Location and
Hemisphere:
Ischaemic=46.3%,
Left=44%, Right=44%,
Bilateral=8%.
Unknown=4%.
Haemorrhagic=11.1%,
2-8 weeks post
stroke
Exclusion: Previous
diagnosis of
dementia of
cognitive
impairment
Inclusion: could sit
and co-operate with
an assessment of 30
minutes
Exclusion:
Significant hearing
or sight issues
Boosman et
al.3
BNI
54
Inclusion: Recently
suffered a stroke
(<12 months),
sufficient command
of the Dutch
language, Barthel
Score ≥ 19
Age: M= 53.8,
SD=12.3
Sex Male=64.8%,
Female=35.2%
Education:
High=33.3%,
Low=66.6%
Recruited 4
weeks poststroke
Weeks: M=15.0,
SD=12.8
2
Exclusion: NR
Bour et al.4
Cartoni et
al.5
MMSE
MEAMS
194
30
Inclusion: First ever
hemispheral stroke,
age above 40, native
speaker
Exclusion: Severe
aphasia, pre-stroke
dementia, and other
major neurological
or psychiatric
disorders that could
interfere with
testing.
Inclusion: Stroke
patients within
rehabilitation ward
Exclusion: no other
neurological
impairments, no
mental
Age: M=68.4,
SD=12.5
Sex: Male N=107,
Female N=89
Education:
high/low=86/107
Age: M=75.80,
SD=7.98
Sex: 17 Male, 13
Female
Education: NR
Left=50%,
Right=33.3%,
Bilateral=16.7%.
Subarachnoid
haemorrhage=42.6%,
Anterior circulation
aneurysm=47.8%,
posterior circulation
aneurysm=30.4%,
Nonaneurysmal=21.7%
Severity: NR
Mechanism and location:
Territorial=39.2%,
Lacunar=52.6%,
Haemorrhagic=8.2 %
Hemisphere: Left=
41.6% , Right=56.8%,
Bilateral=1.6%
Severity: Rankin M=3.1,
SD=1.4
Mechanism: NR
Location: TACI N=6,
PACI N=13, POCI N=8,
LACI N=3
Hemisphere: Right N=13,
Left N=14, Bilateral N=3.
Severity: NR
Within 1 month
NR
3
Cumming
et al.6
MMSE
60
MoCA
Godefroy et MMSE
al.7
MoCA
95
confusion/dementia
prior to stroke. No
aphasia, visual
impairment or
auditory impairment
such that they could
not be tested
Inclusion: Ischaemic
or intracerebral
haemorrhagic stroke.
Exclusion: younger
than 18 years of age,
unconscious on
admission to
hospital, required an
interpreter, major
visual/hearing/
language impairment
Inclusion: Recent
infarct or
haemorrhage (< 3
weeks)
MMSEadj
MoCAadj
(Adjusted
scores
created
using
control
Exclusion: Severe
general and
neurological
conditions
precluding
neuropsychological
assessment,
illiteracy, mental
Age: M=72.1,
SD=13.9
Sex: Male N=44,
Female N=16
Education: M=10.5,
SD=3.9
Mechanism: Infarct N=55,
Haemorrhage N=5
Location: TACI N=4,
PACI N=21, POCI N=20,
LACI N=8
Hemisphere: Right N=31,
Left N=20, Bilateral N=2,
Unknown N=7
Severity: NIHSS median =
5
Days: M=98.3
SD=12.0
Age : M=68.2,
SD=13.7
Sex: Male N=60,
Female N=35
Education: primary
N=55, secondary
N=24, high school
N=16
Mechanism: Infarct N=88,
Haemorrhage N=7
Location: ACA N=1
MCA N=65, PCA N=4,
Posterior Fossa N=18.
Haemorrhages: Lobar
N=3, Deep N=4.
Hemisphere: Right N=44,
Left N=39, Bilateral N=12
Severity: NIHSS M=2.7,
SD=3.2
Days: M=6.6,
SD=3.5
4
Grace et
al.8
group to
account
for age
and
education)
MMSE
70
Modified
MMSE
Green et
al.9
RBANS
60
Morris et
al.10
ACE-R
61
MMSE
retardation, nonnative speaker, other
neurological and
psychiatric
conditions
Inclusion: Patients
with stroke
undergoing
treatment on a
medical
rehabilitation unit
Exclusion: NR
Exclusion: unable to
provide consent,
aphasia,
visual/auditory
impairments
precluding
completion of tests,
poor English,
diagnosis of
dementia, older than
80 years
Inclusion: NR.
Recruited from
stroke wards and
identified from
medical notes
Exclusion: History
of psychiatric
Age: M=74.5, SD=7.7
Sex: Male N=27,
Female N=43
Education: M= 10.4,
SD= 2.6.
Mechanism: NR
Location: NR
Hemisphere: Right N=41
Left N=15, bilateral N=8,
Non-focal lesion N=6.
Severity: NR
NR
Age: M=67.7,
SD=13.2
Sex: Male N=35,
Female N=25
Education: M= 10.6,
SD= 3.0
Mechanism: Ischemia:
N=45, Haemorrhage N=6,
Unspecified N=9.
Location: TACS N = 17,
PACS N = 15, POCS = 1,
LACS N =18
Hemisphere: Right N=36,
Left N=20, Bilateral: 2,
Unknown = 2
Severity: NR
Days: M=24,
SD=16.
Age: Median=76.0,
IQR 67.0–82.5
Sex: Male N=31,
Female N=30
Education:
Median=9.0, IQR 911
Mechanism: NR
Location: TACS N=15,
PACS N=10, POCS N=9,
LACS N=21,
Unknown N=6
Hemisphere: Right N=36,
Left N= 22, Unknown N=
3
Days: Median=
18, IQR 9-48-8
5
Nøkleby et
al.11
Cognistat
49
SINS
Clock
Drawing
problems, blind/deaf,
too ill, too drowsy,
poor English,
moderate or severe
dysphasia
Inclusion: Diagnosis
of stroke, age > 18,
stable medical
condition, ability to
give informed
consent
Severity: Barthel Index
Median=9.0, IQR 6-12
Age: Median=62,
IQR=53.5–77
Sex: Male N=32,
Female =17
Education: NR
Exclusion: NR
Nys et al.12
MMSE
Salvadori et MoCA
al.13
34
80
Inclusion: Ischaemic
or haemorrhagic
stroke
Exclusion: High
degree of handicap,
non-native speaker,
severe disturbance in
communication and
consciousness, blind
Inclusion: Diagnosis
of stroke (ischemic
or haemorrhagic),
age 18+ years, native
speaker, informed
consent and
availability for
bedside evaluation
Age: M=64.7,
SD=11.5
Sex: Male=41.2%
Education (scale):
Median=4
Age: M=68.2,
SD=14.6
Sex: Male=66%,
Female=33%
Education: No PCSI
M=10.9, SD=4.3,
PSCI M=7.9, SD=4.1
Mechanism: Infarct N= 40,
Intracerebral Haemorrhage
N=6,
Subarachnoid
Haemorrhage N=3
Location: NR
Hemisphere: Right N=24,
Left N=19, Other N=6
Severity: NR
Mechanism: Infarct N=33,
Haemorrhage N=1
Location: PACI – 5, LACI
– 21, POCI – 6, TACI – 1,
POCH - 1
Hemisphere: Right N=17,
Left N=17
Severity: Modified Rankin
Scale: Median=3
Days: Median
38, IQR 17-89
Mechanism: No PSCI
Ischemic=82%, PSCI
Ischaemic=89%
Location: No PSCI
Supratentorial=85%, PSCI
Supratentorial=85%
Hemisphere: No PSCI
Left=55%,
5-9 days post
admission
Days: M = 6.5,
SD = 2.9
6
between 5th and 9th
day after stroke.
Schweizer
et al.14
MMSE
32
MoCA
Srikanth et
al.15
Wong et
al.16
S-MMSE
MoCA
79
74 at 2-4 week
assessment
Exclusion: Patients
living outside the
Florence area
Inclusion:
aneurysmal
subarachnoid
haemorrhage,
classified as good
outcome or moderate
disability according
to Glasgow Outcome
Scale
Inclusion: First ever
stroke, spoke
English, not severely
dysphasic, score ≥2
on the NIHSS,
adequate vision and
hearing.
PSCI Left=51%
Severity: NIHSS: M=3.6,
SD=4.8
Age : M=55.2,
SD=7.8
Sex : Male N=13,
Female N=19
Education: M=15.8,
SD=3.8
Age: M=69.0,
SD=14.4
Sex: Male N=47,
Female N=32
Exclusion: NR
Education:
Cognitively intact
M=9.7 SD=2.2,
Cognitive impairment
no dementia M=10.0
SD=2.3, Dementia
M=9.9 SD=2.2
Inclusion:
spontaneous
subarachnoid
2-4 weeks
Age: Median=58
IQR=49-66
Mechanism:
Aneurysm=100%
Location: Anterior
circulation N=18, MCA
N=4, Posterior circulation
N=9, Unknown N=1
Hemisphere: NR
Severity: WFNS grade: I
N=22, II N=3, III N= 3, IV
N=2
Mechanism: Ischaemic N=
72, Haemorrhagic N=7
Location: LACI N=26,
PACI N=27, POCI N=17,
TACI N=2
Hemisphere: Right N=39,
Left N=25, Bilateral N=15
Severity: Barthel
Cognitively intact M=19.1
SD=1.7, Cognitive
impairment no dementia
M=19.6 SD=1.0,
Dementia M=17.6 SD=3.0
Mechanism:
Haemorrhage=100%
2-4 weeks
Months:
M=29.3,
SD=17.5
Days: M=381.6,
SD=45.6.
2-4 weeks & 1
year
7
MMSE
80 at 1-year
assessment
haemorrhage,
hospital admission
within 96 hours of
ictus, between 21
and 75 years of age,
native speaker,
willing and able to
provide informed
consent
Exclusion: history of
previous
cerebrovascular or
neurological disease
other than
unruptured
intracranial
aneurysm, a history
of neurosurgery
before ictus, inability
to cooperate with
cognitive
assessments (unable
to obey commands).
Sex: Female=50%
Education: NR
1-year
Age: Median=52
IQR=47-61
Sex: Female=55%
Education: NR
Location: ICA other than
PComA N= 11, PComA
N=13, ACA N=24, MCA
N=18, PCA N=8
Severity: WFNS Grade 1
N= 48, 2 N=15, 3 N=4, 4
N=6, 5 N=1
1-year
Location: ICA other than
PComA N=16, PComA
N=16, ACA N=26, MCA
N=20, PCA N=12
Severity: WFNS grade 1
N=45, 2 N=21, 3 N=1, 4
N=9, 5 N=4
Hemisphere: NR
ACA Anterior Cerebral Artery, ACE-R Addenbrooke’s Cognitive Examination – Revised, AUC Area Under Curve, BNI Barrow Neurological
Institute screen for higher cerebral functions, CDT Clock Drawing Task, ICA IntraCerebal Aneurysm, IQR Interquartile Range, LACI/S
Lacunar Infarct/Syndrome, M Mean, MEAMS Middlesex Elderly Assessment of Mental State, MCA Middle Cerebral Artery, MMSE MiniMental State Examination, MMSE-S Standardised Mini-Mental State Examination, MoCA Montreal Cognitive Assessment, NIHSS National
Institute of Health Stroke Severity, N Sample Size, NR Not reported, PACI/S Partial Anterior Circulation Infarct/Syndrome, PCA Posterior
Cerebral Artery, PComA Posterior Communicating Artery, POCH Posterior Circulation Haemorrhage, POCI/S Posterior Circulation
8
Infarct/Syndrome, PSCI, Post Stroke Cognitive Impairment, RBANS Repeatable Battery for the Assessment of Neuropsychological Status, SD
Standard Deviation, SINS Neuropsychological Impairments in Stroke, TACI/S Total Anterior Circulation Infarct/Syndrome, WFNS World
Federation of Neurological Surgeons
9
Supplemental Table II. Evaluation of Research Methodologies
Study
Sample
Documented
size
recruitment
justificat and attrition
ion
Agrell et
al.1
NR
Blake et
al.2
NR
Yes. No
differences in
sex, education
or diagnosis.
However,
study sample
had less left
sided lesions
and were
younger than
nonparticipants
NR
Boosman
et al.3
NR
NR
Stroke
diagnosis
confirmed
with brain
imaging
Diagnosis was
determined by
CT in 60% of
participants.
The remainder
were
determined by
clinical
investigation
Time between
screening and
gold-standard
assessment
NR
Within 3
months
NR
Same day
NR
Cognitive domains
included in gold
standard assessment
for SE and SP
analyses
Spatial memory
Arithmetic*
Verbal Memory
Spatial Function
Visual reasoning*
Criteria for goldstandard
impairment
Spatial perception
Visual inattention
Memory (verbal and
visual)
Apraxia
Executive functioning
Language
Perception
Language
Memory
Attention
Reasoning
Clinician classified
patient as impaired
on at least one
cognitive domain
NR
Below 5th
percentile or within
the first decile on
one or more
cognitive domains.
Higher SE and
SP in people
aged over 55.
From
neuropsychological
tests an estimate
was made blindly
of whether a
patients was
mentally
unimpaired or
impaired
Stratified
results for
demographic
and stroke
variables
No lesion
hemisphere
effect for SE.
SP higher in
right
hemisphere
group.
10
Executive Functioning
Bour et
al.4
Yes
Cartoni et
al.5
NR
Cumming
et al.6
Yes
Godefroy
et al.7
NR
NR between
participants
and nonparticipants.
Those who did
not complete
follow up were
older, less
educated and
had lower
original
MMSE scores
Yes. However,
no statistical
comparison
between
original and
final sample.
Yes. However,
no statistical
comparison
between
original and
final sample.
NR
Yes
Within 1
month
Memory
Mental Speed
Executive function
Calculation*
Visuospatial*
Orientation*
Attention*
Praxis*
Reasoning*
Language*
Dutch norms used
when possible.
Performance below
10th percentile
compared to
normative group.
Results
comparable
across left and
right lesions
Separate analyses
for 1,2, and 4
impaired domains
NR
Days:
M=20.73, SD=
24.37
Memory
Language
Perception
Apraxia
Executive functioning
Overall clinician
conclusion from
report
NR
NR
Days: M=8.1,
SD 2.4
Visuospatial
Memory
Executive
Language
Attention
Visual neglect
Domain z-scores of
<−1 compared to
age/education
normative data in 2
or more domains
MMSE: No
lesion
hemisphere
effect
NR
Not directly
reported.
Language
Visuoconstructive
Follow up analyses
included more
stringent criterion
of z-score <-1.5.
Impairment of at
least 2 cognitive
MoCA: better
predictive
validity in
right lesion
participants
When
adjusting for
11
Estimated
M=17.5 days
abilities
Working memory
Long term memory
Action speed
Executive function
domains, at a
criterion that
corresponded to the
5% level (whether
compared to age
and education
normative data not
reported)
OR
Grace et
al.8
NR
NR
NR
Within 1week, across
three sessions
Orientation and mental
control
Language
Verbal fluency
Visuospatial ability
Memory
Green et
al. 9
NR
NR
Days: M=4,
SD = 5
Language
Perception
Attention
Memory
Executive abilities
Morris et
NR
Yes. However,
no statistical
comparison
between
original and
final sample
Limited
NR
Days:
Verbal memory
MMSE score lower
than ≤ 23
Performance <2
SDs below
published norms in
two or more
cognitive domains.
Memory
performance was
defined as
impaired when it
fell at least 1.5 SDs
below published
norms
Performance < 5th
percentile on any
subtest published
normative data
Below 5th
age and
education,
MMSE SE
slightly
increased but
SP slightly
decreased.
MoCA SE and
SP showed
opposite trend.
No lesion
hemisphere
effects for the
modified
MMSE.
Higher SE and
SP in left
hemisphere
lesions for
standard
MMSE.
NR
NR
12
al.10
Median=3,
IQR 2−7.
Nøkleby
et al.11
Limited
rationale
provided
Limited
WHO criteria
used
NR
Nys et
al.12
NR
Yes. However,
no statistical
comparison
between
original and
final sample.
Yes
Same day
Salvadori
et al.13
NR
Yes
Months:
M=8.4, SD 2.2
Schweizer
et al.14
NR
Yes. Patients
not included in
analyses had
lower premorbid
cognitive
status, higher
stroke severity
and lower
MoCA scores
Yes. However,
no statistical
comparison
NR
NR
Visual memory
Executive functioning
Attention
Perception
Language
Visuospatial
Function
Attention and neglect
Apraxia
Speed in unaffected
arm
Memory
Abstract reasoning
Verbal memory
Executive function
Visual perception and
construction
Visual memory
Language
Verbal memory
Focalised and
maintained attention
Divided and selective
attention
Language
Attention
Executive function
Verbal learning and
percentile on one
or more cognitive
domains
At least one
cognitive domain
impaired (at least
1.5–2 SD from
estimated level
before stroke)
NR
Performance below NR
-1.65 z-score in at
least one cognitive
domain compared
to matched controls
Abnormal
performance
(below 5th
percentile on age
and education
adjusted norms) on
one test and
borderline
performance on at
least one other test.
2 SD below
age/education
normative data
NR
NR
13
between
original and
final sample.
Srikanth
et al.15
Wong et
al.16
NR
NR
Yes. However,
no statistical
comparison
between
original and
final sample.
NR
memory
Naming
Motor function
Yes
Yes
NR
During same
session or in 2
sessions on
consecutive
days
General intellect
General
intellect/executive
ability
Verbal memory
Everyday memory
Spatial ability
Language/executive
ability
Verbal Memory
Visuospatial skills and
memory
Attention and working
memory
Executive functioning
and psychomotor speed
Naming
Each cognitive
domain
investigated
separately
Scores more
NR
than 1 SD below
age- and educationcorrected norms in
at least two tests
measuring the same
domain.
Participants with
dementia excluded
from ROC analysis
A cognitive domain NR
deficit was defined
as a cognitive
domain z score ≤1.65.
Cognitive
impairment was
defined as two or
more cognitive
domain deficits
* Limited assessment of these domains noted.
IQR Interquartile Range, NR Not reported, M Mean, MMSE Mini-Mental State Examination, MoCA Montreal Cognitive Examination, ROC
Receiver Operating Characteristics, SD Standard Deviation, WHO World Health Organisation
14
Supplemental Table III. Sensitivity and Specificity Data for Focal Cognitive Domains.
Screening Tool
MEAMS
Cartoni et
al. 5
Subtest
Orientation
Remembering
Pictures
Name Learning
Naming
Comprehension
Verbal fluency
Spatial
construction
Fragmented
letter
Unusual views
Usual views
RBANS
Green et
al. 9
ACE-R
Morris et
Motor
perseveration
Immediate
Memory
Delayed
Memory
Language
Visuospatial
Attention
Visuospatial
Fluency
Cut-point
Pass raw
score
Pass raw
score
Pass raw
score
Pass raw
score
Pass raw
score
Pass raw
score
Pass raw
score
Pass raw
score
Pass raw
score
Pass raw
score
Pass raw
score
79/80
79/80
79/80
76/77
83/84
14
8
Gold Standard Assessment Domain
Memory
Immediate Memory
SE
SP
PPV NPV SE
SP
PPV NPV
57
80
NR
NR
28
80
NR
NR
28
86
NR
NR
84
71
56
Delayed Memory
SE SP
PPV
NPV
Language
SE
SP
PPV
NPV
100
69
NR
NR
50
88
NR
NR
75
84
NR
NR
88
91
77
95
91
86
58
65
82
15
al. 10
Cognistat
Nokleby
et al. 11
Memory
Attention and
Orientation
Memory
Attention
Similarities
SINS
Nokleby
et al. 11
Comprehension
Naming
Repetition
Visuoconstruction
Aphasia
Visuocognitive
Apraxia
20
16
85
20
65
43
>65yo 7/8
≤65yo
9/10
5/6
>65y 3/4
≤65y 4/5
4/5
6/7
10/11
>65yo 2/2
≤65yo 3/4
11/12
17/18
20/21
69
52
NR
NR
54
73
NR
NR
60
82
NR
NR
60
80
60
77
77
82
NR
NR
NR
NR
NR
NR
70
85
NR
NR
16
Supplemental Table III. Sensitivity and Specificity Data for Focal Cognitive Domains (continued).
Screening Tool
Subtest
MEAMS
Cartoni
et al. 5
Orientation
Remember
Pictures
Name
Learning
Naming
Comprehen.
Verbal
fluency
Spatial
construction
Fragmented
letter
Unusual
views
Usual views
Cutoff
Pass
raw
score
Pass
raw
score
Pass
raw
score
Pass
raw
score
Pass
raw
score
Pass
raw
score
Pass
raw
score
Pass
raw
score
Pass
raw
score
Pass
raw
score
Gold Standard Assessment Domain
Perception/Visuospatial
Executive Function
SE
SP
PPV NPV SE
SP
PPV
44
100
NR
NR
16
100
NR
NR
61
91
NR
NR
55
75
NR
NR
NPV
Attention/Neglect
SE
SP
PPV
NPV
Speed
SE
SP
PPV
NPV
Praxis
SE
SP
PPV
17
NPV
Motor
persever.
RBANS
Green et
al. 9
ACE-R
Morris et
al. 10
Cognistat
Nokleby
et al. 11
Immediate
Memory
Delayed
Memory
Language
Visuospatial
Attention
Visuospatial
Fluency
Memory
Attention
and
Orientation
Memory
Attention
Similarities
Comprehen.
Naming
Repetition
Visuoconstruction
SINS
Nokleby
et al. 11
Aphasia
Visuocog.
Apraxia
Pass
raw
score
79/80
11
100
NR
NR
79/80
79/80
76/77
83/84
14
8
20
16
>65y
7/8
≤65y
9/10
5/6
Age
depen
dent
4/5
6/7
10/11
>65y
2/3
≤65y
3/4
11/12
17/18
23/24
89
96
97
85
88
47
76
67
81
50
7
79
48
66
65
92
44
29
96
65
64
67
NR
NR
64
58
NR
NR
82
54
NR
NR
72
42
NR
NR
61
71
NR
NR
61
68
NR
NR
80
65
NR
NR
40
65
NR
NR
18
All results presented as percentages. Due to large amount of data Confidence Intervals are not included in this table. Bolded data indicates results
reach traditionally acceptable levels of 80% SE and 60% SP.
MEAMS Middlesex Elderly Assessment of Mental State, NPV Negative Predictive Value, NR Not Reported, PPV Positive Predictive Value,
RBANS Repeatable Battery for the Assessment of Neuropsychological Status, SINS Neuropsychological Impairments in Stroke, SE Sensitivity,
SP Specificity
19
Supplemental Figure I. Search strategy example – OVID MEDLINE
20
Supplemental Figure II. Flowchart of Systematic Literature Review
21
Supplemental References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
Agrell B, Dehlin O. Mini mental state examination in geriatric stroke patients. Validity, differences between subgroups of patients, and
relationships to somatic and mental variables. Aging Clin Exp Res. 2000;12:439-444
Blake H, McKinney M, Treece K, Lee E, Lincoln NB. An evaluation of screening measures for cognitive impairment after stroke. Age
Ageing. 2002;31:451-456
Boosman H, Visser-Meily JM, Post MW, Duits A, van Heugten CM. Validity of the barrow neurological institute (BNI) screen for
higher cerebral functions in stroke patients with good functional outcome. The Clin Neuropsychol. 2013;27:667-680
Bour A, Rasquin S, Boreas A, Limburg M, Verhey F. How predictive is the MMSE for cognitive performance after stroke? J Neurol.
2010;257:630-637
Cartoni A, Lincoln NB. The sensitivity and specificity of the middlesex elderly assessment of mental state (MEAMS) for detecting
cognitive impairment after stroke. Neuropsychol Rehabil. 2005;15:55-67
Cumming TB, Churilov L, Linden T, Bernhardt J. Montreal cognitive assessment and mini-mental state examination are both valid
cognitive tools in stroke. Acta Neurol Scand. 2013;128:122-129
Godefroy O, Fickl A, Roussel M, Auribault C, Bugnicourt JM, Lamy C, et al. Is the montreal cognitive assessment superior to the minimental state examination to detect poststroke cognitive impairment? A study with neuropsychological evaluation. Stroke. 2011;42:17121716
Grace J, Nadler JD, White DA, Guilmette TJ, Giuliano AJ, Monsch AU, et al. Folstein vs modified mini-mental state examination in
geriatric stroke: Stability, validity, and screening utility. Arch Neurol. 1995;52:477-484
Green S, Sinclair E, Rodger E, Birks E, Lincoln N. The Repeatable Battery for the assessment of Neuropsychological Status (RBANS)
for post-stroke cognitive impairment screening. Int J Ther Rehabil. 2013;20: 536-541.
Morris K, Hacker V, Lincoln NB. The validity of the addenbrooke's cognitive examination-revised (ACE-R) in acute stroke. Disabil
Rehabil. 2012;34:189-195
Nokleby K, Boland E, Bergersen H, Schanke AK, Farner L, Wagle J, et al. Screening for cognitive deficits after stroke: A comparison of
three screening tools. Clin Rehabil. 2008;22:1095-1104
Nys GM, van Zandvoort MJ, de Kort PL, Jansen BP, Kappelle LJ, de Haan EH. Restrictions of the mini-mental state examination in
acute stroke. Arch Clin Neuropsychol. 2005;20:623-629
Salvadori E, Pasi M, Poggesi A, Chiti G, Inzitari D, Pantoni L. Predicitver value of the MoCA an in the acute phase of stroke on the
diagnosis of mid-term cognitive impairment. J Neurol. 2013;260:2220-2227
Schweizer TA, Al-Khindi T, Macdonald RL. Mini-mental state examination versus montreal cognitive assessment: Rapid assessment
tools for cognitive and functional outcome after aneurysmal subarachnoid hemorrhage. J Neurol Sci. 2012;316:137-140
22
15.
16.
Srikanth V, Thrift AG, Fryer JL, Saling MM, Dewey HM, Sturm JW, et al. The validity of brief screening cognitive instruments in the
diagnosis of cognitive impairment and dementia after first-ever stroke. Int Psychogeriatr. 2006;18:295-305
Wong GK, Lam SW, Wong A, Ngai K, Poon WS, Mok V. Comparison of montreal cognitive assessment and mini-mental state
examination in evaluating cognitive domain deficit following aneurysmal subarachnoid haemorrhage. PLoS ONE. 2013;8:e59946
23