Brain Injury
NOTICE: This material may be protected
by copyright law (Title 17 U.S. Code)
ISSN: 0269-9052 (Print) 1362-301X (Online) Journal homepage: http://www.tandfonline.com/loi/ibij20
Temporal stability and responsiveness of the
Montreal Cognitive Assessment following acquired
brain injury
Patricia A. Lim, Alison M. McLean, Christiane Kilpatrick, Daniel DeForge,
Grant L. Iverson & Noah D. Silverberg
To cite this article: Patricia A. Lim, Alison M. McLean, Christiane Kilpatrick, Daniel DeForge,
Grant L. Iverson & Noah D. Silverberg (2016) Temporal stability and responsiveness of the
Montreal Cognitive Assessment following acquired brain injury, Brain Injury, 30:1, 29-35, DOI:
10.3109/02699052.2015.1079732
To link to this article: http://dx.doi.org/10.3109/02699052.2015.1079732
Published online: 10 Nov 2015.
Submit your article to this journal
Article views: 251
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=ibij20
Download by: [Bloomsburg University]
Date: 04 January 2017, At: 07:08
http://tandfonline.com/ibij
ISSN: 0269-9052 (print), 1362-301X (electronic)
Brain Inj, 2016; 30(1): 29–35
! 2016 Taylor & Francis Group, LLC. DOI: 10.3109/02699052.2015.1079732
ORIGINAL ARTICLE
Temporal stability and responsiveness of the Montreal Cognitive
Assessment following acquired brain injury
Patricia A. Lim1, Alison M. McLean2,3, Christiane Kilpatrick3, Daniel DeForge1,3, Grant L. Iverson4,5,6, &
Noah D. Silverberg1,3
1
Division of Physical Medicine and Rehabilitation, 2Department of Occupational Science and Occupational Therapy, The University of British
Columbia, Vancouver, BC, Canada, 3G.F. Strong Rehabilitation Centre, Vancouver, BC, Canada, 4Department of Physical Medicine and
Rehabilitation, Harvard Medical School, Boston, MA, USA, 5Spaulding Rehabilitation Hospital, Boston, MA, USA, and 6Red Sox Foundation and
Massachusetts General Hospital Home Base Program, Boston, MA, USA
Abstract
Keywords
Objectives: To evaluate the temporal stability and responsiveness of the Montreal Cognitive
Assessment (MoCA) in acquired brain injury (ABI).
Research design and methods: English-speaking adults with stroke or moderate-to-severe
traumatic brain injury were administered alternate forms of the MoCA (version 1, then 2),
6 weeks apart. Chronic group participants (n ¼ 40) were community-dwelling, at least 1 year
post-ABI (mean ¼ 12.1 years, SD ¼ 9.0), and presumed clinically stable. Sub-acute group
participants (n ¼ 36) were 30.8 days post-ABI (SD ¼ 12.4) and were undergoing intensive
rehabilitation. Individuals with an unstable medical or psychiatric condition or severe receptive
aphasia were not eligible.
Results: The chronic group scored 21.6 (SD ¼ 4.5) initially and 22.7 (SD ¼ 3.8) on the second
administration, demonstrating a small but significant practise effect (p ¼ 0.009). The Pearson
test–re-test correlation coefficient was 0.83. Using reliable change methodology in the chronic
group, the 80% confidence interval (CI) for change across the two administrations was 2 to
+4, adjusting for practise. Applied to the sub-acute group, 39% improved and 0% declined.
Conclusions: The MoCA is a brief standardized tool that appears useful for monitoring cognitive
change after ABI. The findings enable clinicians to detect statistically reliable change across
serial MoCA administrations in individuals with an ABI.
Brain injuries, cognition disorders,
neuropsychological tests,
psychometrics, stroke
Introduction
The Montreal Cognitive Assessment (MoCA) is a brief
cognitive screening tool originally designed to detect mild
cognitive impairment with greater sensitivity than the MiniMental State Examination [1]. It has since accrued an evidence
base and clinical uptake across a variety of disorders, including
Parkinson’s disease, Huntington’s disease, brain cancer and
human immunodeficiency virus [2–6]. The MoCA appears
well-suited to screen for cognitive impairment and monitor
cognitive recovery in an acquired brain injury (ABI) rehabilitation setting. Although there is emerging evidence for its
validity following stroke and traumatic brain injury (TBI)
[7–15], data on serial testing in these clinical groups are
lacking. English alternate forms [16] of the MoCA were
recently developed to facilitate serial administration, but
evaluation of their temporal stability and change score metrics
is needed to guide their clinical use. Given that monitoring
cognitive recovery is a cornerstone of ABI rehabilitation, it is
History
Received 16 January 2015
Revised 14 July 2015
Accepted 1 August 2015
Published online 4 November 2015
essential that interpretation of significant test–re-test change
be informed by research evidence.
The present study aims to (i) estimate the MoCA’s
alternate forms test–re-test reliability in a stable chronic
ABI sample; and (ii) evaluate the responsiveness of the
MoCA to change in a sub-acute ABI sample. Test–re-test data
from the chronic ABI group will be used to establish a reliable
change index, i.e. the upper and lower thresholds for change
on serial MoCAs that could be attributed to measurement
error. The reliable change index will then be applied to the
sub-acute ABI sample to determine responsiveness. It was
hypothesized that individuals with remote ABI would perform
similarly on the MoCA over time, demonstrating its temporal
stability. It was further hypothesized that individuals with
recent ABI would obtain higher MoCA scores over the course
of rehabilitation, supporting the MoCA’s responsiveness.
Methods
Participants
Correspondence: Noah D. Silverberg, PhD, G.F. Strong Rehabilitation
Centre, Rehabilitation Research Centre, 4255 Laurel St., Vancouver, BC
V5Z 2G9, Canada. E-mail: [email protected]
Two convenience samples were recruited between December
2011 and February 2013. The chronic sample (n ¼ 43) was
30
P. A. Lim et al.
recruited through referrals from hospital-based electronic
mail broadcasts, posters and presentations given to community-based ABI support groups and drop-in programmes. The
sub-acute sample (n ¼ 41) was recruited from an urban
Canadian rehabilitation hospital (G.F. Strong Rehabilitation
Centre). Upon admission to the ABI inpatient unit or an early
supported discharge outpatient programme with comparable
therapy intensity, eligible individuals for the sub-acute group
provided verbal consent to be referred to the study researchers
by their most responsible physiatrist or resident. Verbally
consenting individuals were then seen within the first 7 days
after admission.
Individuals were included if they were at least 19 years old,
were proficient in English and had a history of an ABI, which
for the present study was defined as either (i) moderateto-severe TBI or (ii) stroke. Individuals with a TBI qualified if
they had an acute care admission Glasgow Coma Scale score
of less than 13 or post-traumatic amnesia duration of greater
than 60 minutes. Individuals with brainstem strokes were
excluded because their cognition should be unaffected.
Individuals were also excluded if they had an unstable
medical or psychiatric condition or a severe aphasia such that
they were unable to follow one-step verbal commands. Each
of the two study samples had additional group-specific
eligibility criteria: Chronic group participants were required
to (i) be living in the community, (ii) have had their ABI at
least 1 year ago and (iii) not be attending any formal cognitive
rehabilitation therapies or programme. Sub-acute group
participants were required to (i) have had an ABI within the
previous 2 months and (ii) be receiving intensive daily
interdisciplinary rehabilitation.
The target sample size of 35 chronic group participants
was determined using the Bonett [17] corrected formula for a
95% confidence interval (width of 0.2) and an estimated intraclass correlation coefficient of 0.85, which was based on
studies of test–re-test reliability of the original MoCA (in
mostly non-ABI and non-English samples) [1, 18–23] because
no English alternate forms of test–re-test reliability estimates
were available at the time of study enrolment. The target
sample size of 30 sub-acute participants was determined by
feasibility estimates.
Materials
Participants were administered English alternate forms of the
MoCA (version 1 [1] then 2 [16]), 6 weeks apart. For the
sub-acute group, FIM(R) [24] scores were among the data
extracted from a national rehabilitation registry, for the
purpose of characterizing the sample.
Montreal Cognitive Assessment
The MoCA [1] is a brief standardized mental status examination that provides a screening assessment of orientation,
short-term and working memory, visuospatial and executive
function, attention, concentration, language and abstraction.
Scores range from 0–30, with higher scores representing
better cognition. It takes 10 minutes to administer. Version 2
(English) was developed and has undergone initial validation
by Phillips et al. [16]. This alternative form differs from
the original MoCA only in item content. Emphasis on
Brain Inj, 2016; 30(1): 29–35
interpretation of the MoCA total score is supported by
strong internal consistency estimates [1], Rasch analysis [17]
and most factor analytic studies [14, 23, 25].
FIM(R) instrument
The FIM(R) instrument [24] is an 18-item ordinal scale used
by clinicians to document physical and cognitive functioning
in terms of how much assistance is required to perform
activities of daily living, in the areas of self-care, bowel and
bladder control, transfers, locomotion, communication, cognition and social interaction. Scores for each item range from
1 (complete dependence) to 7 (complete independence) and
the total score ranges from 18–126. The FIM(R) was
administered at rehabilitation admission and discharge.
Procedure
Ethical approval was obtained from the University of British
Columbia Behavioural Research Ethics Board and the
Vancouver Coastal Health Research Institute. Participants
met with a study researcher (senior resident in Physical
Medicine and Rehabilitation, PL, or occupational therapist,
AM) for 1 hour on each of two occasions. Meetings took place
either at the G.F. Strong Rehabilitation Centre (both groups)
or at a mutually agreed-upon location off-site (chronic group
only). If a participant’s family member was present, he/she
was instructed not to provide any input or assistance during
testing. For both groups, consent for study participation was
obtained at the first assessment and reviewed at the second
assessment as needed. Modest financial compensation was
provided to participants.
At the first assessment, participants were administered an
oral questionnaire to obtain demographic and injury information. For the chronic group, this study relied on participants’
self-report of their ABI injury type and date. For the subacute group, corroborating demographic and injury details as
well as admission and discharge dates, FIM(R) scores and
prior MoCA exposure were extracted by electronic chart
review and through linkage with a national rehabilitation
registry (managed by the Canadian Institute for Health
Information).
MoCA English versions 1 and 2 were administered to all
participants in the same order, 6 weeks apart, using the
version-specific standardized instructions available online
[26]. Version 2 was employed in an effort to mitigate practise
effects. This test order was selected to most closely approximate clinical practise. That is, in the authors’ experience,
most clinicians administer version 1 (original version) during
an initial assessment and, if a repeat MoCA assessment is
warranted, consider using an alternate form.
Statistical analysis
SPSS version 19 was used for statistical analyses. Frequency
distributions revealed near-normal distributions for all continuous variables, permitting parametric statistical testing.
Independent sample t-tests were utilized for between-group
mean differences and paired sample t-tests for within-group
mean differences on repeated MoCA assessments.
Proportional differences were assessed with the chi-squared
The Montreal Cognitive Assessment following ABI
DOI: 10.3109/02699052.2015.1079732
(2) statistic. Aside from a comparison of baseline demographic characteristics and injury type between individuals
who dropped-out and those who remained enrolled in the
study, all analyses were performed using data from participants who completed both MoCA versions 1 and 2.
Alternate forms test–re-test reliability was calculated using
a Pearson correlation coefficient in the chronic group.
Reliable change confidence intervals (CIs) were obtained by
multiplying the standard error of the difference (SEdiff) by zscores associated with the 70%, 80% and 90% CIs and then
adjusting for the mean practise effect [27]. The steps for
calculating the SEdiff and the confidence intervals are
presented in the Appendix. Repeated measures analysis of
variance was used for the sub-group analyses in the sub-acute
sample.
Results
Eighty-four participants (43 in the chronic group, 41 in the
sub-acute group) were initially enrolled, but one participant
withdrew consent during the initial MoCA and seven
participants dropped out or became ineligible prior to
completing their second MoCA, leaving a final total of 40
in the chronic group and 36 in the sub-acute group.
Participants who did vs did not complete the study were
comparable with respect to age, education, initial MoCA
score and diagnosis (each p40.05).
Baseline characteristics of participants completing both
assessments are reported in Table I. Participant ages ranged
from 19–80 years. The sub-acute group was found to be
younger, have a greater proportion of males and have a
more even proportion of TBI and stroke diagnoses than the
chronic group.
31
test–re-test correlation coefficient was 0.83 and the SEdiff was
2.4. The reliable change intervals associated with various
levels of confidence are presented in Table II. A change score
at or outside the limits of each interval is considered
statistically reliable. Exploratory sub-group analyses showed
that participants with stroke (n ¼ 29) performed similarly to
participants with TBI (n ¼ 11) with regard to mean change
(1.1 vs 1.3) and test–re-test stability (0.81 vs 0.88).
Sub-acute group
Mean MoCA scores were 20.1 (SD ¼ 6.0) and 23.8 (SD ¼ 5.3)
for versions 1 and 2, respectively. Change scores from MoCA
versions 1 to 2 ranged between 1 to +12. The proportion of
the sub-acute sample exhibiting reliable improvement or
decline based on 70%, 80% and 90% CIs is presented in
Table II (based on the SEdiff calculated from the chronic
group). Of note, based on the 80% CI calculated in the chronic
group, 39% of the sub-acute group improved and 0% declined
between MoCA versions 1 and 2. In exploratory analyses,
stroke and TBI sub-groups were compared on the MoCA
across time. The sub-groups performed similarly overall
[main effect: F(1, 34) ¼ 1.59, p ¼ 0.217] and both obtained a
higher score on the second assessment [F(1, 34) ¼ 44.24,
p50.001]. The sub-group time interaction effect was nonsignificant [F(1, 34) ¼ 3.06, p ¼ 0.089]. Average MoCA
change was 2.8 points (SD ¼ 2.8) in the stroke sub-group
and 4.7 points (SD ¼ 3.8) in the TBI sub-group. Because of
this potential difference, the proportion of the sub-acute
sample exhibiting reliable improvement or decline was
reported separately in Table II for participants with stroke
and TBI.
Correspondence with FIM(R)
Chronic group
Mean MoCA scores were 21.6 (SD ¼ 4.5) and 22.7 (SD ¼ 3.8)
for versions 1 and 2, respectively. The one-point gain
was statistically significant (p ¼ 0.009). The alternate forms
FIM(R) scores were missing for one participant (2.8%). The
mean FIM(R) score at admission was 94.9 (SD ¼ 21.2).
Admission to discharge FIM(R) change scores ranged from
0–57 (M ¼ 21.3, SD ¼ 16.6). FIM(R) change scores were
Table I. Demographic and clinical characteristics of the sample*.
Characteristic
Chronic group (n ¼ 40)
Sub-acute group (n ¼ 36)
Statistic
p
Age in years
M ¼ 56.3
SD ¼ 11.6
50
Stroke ¼ 29 (72.5%)
Traumatic brain injury ¼ 11 (27.5%)
M ¼ 42.3
SD ¼ 15.5
77.8
Stroke ¼ 18 (50%)
Ischemic ¼ 9
Haemorrhagic ¼ 9
Traumatic brain injury ¼ 18 (50%)
M ¼ 30.8 days
SD ¼ 12.4 days
M ¼ 13.1
SD ¼ 2.5
75
16.7
M ¼ 42.4
SD ¼ 7.0
M ¼ 94.9
SD ¼ 21.2^
M ¼ 21.3
SD ¼ 16.6^
t
50.001
Sex (% men)
ABI type
Time since ABI
Years of education
Ethnicity (% Caucasian)
English as a second language (%)
Days between assessments
FIM(R) admission
FIM(R) discharge minus admission
M ¼ 12.1 years
SD ¼ 9.0 years
M ¼ 13.0
SD ¼ 2.8
70
27.5
M ¼ 42.1
SD ¼ 2.2
N/A
N/A
*Excluding individuals that dropped out prior to their second assessments.
^Based on n ¼ 35, as FIM(R) scores were missing for one participant.
2
2
0.012
0.059
t
50.001
t
0.827
2
2
t
0.626
0.258
0.785
–
–
–
–
32
P. A. Lim et al.
Brain Inj, 2016; 30(1): 29–35
Table II. Confidence intervals for statistically reliable change derived from the chronic ABI group and applied to the sub-acute group.
Level of confidence
(two-tailed)
Combined sub-acute
sample (n ¼ 36)
Sub-acute Stroke (n ¼ 18)
Subacute TBI (n ¼ 18)
70%
80%
90%
70%
80%
90%
70%
80%
90%
Recommended reliable
change interval based on
chronic ABI sample*
1
2
3
1
2
3
1
2
3
to
to
to
to
to
to
to
to
to
+3
+4
+5
+3
+4
+5
+3
+4
+5
Proportion of sub-acute
sample demonstrating
reliable improvement (%)
Proportion of sub-acute
sample demonstrating
reliable decline (%)
58
39
22
38
28
17
78
50
39
0
0
0
0
0
0
0
0
0
*The reliable change intervals were derived from the chronic ABI group (n ¼ 40) that was assumed to have stable cognitive functioning. The ranges are
asymmetric because they were adjusted for a practise effect of +1 point. A decline of 2 points and an improvement of 4 points constitutes the 80%
confidence interval. That means that 10% or fewer patients are expected to worsen by 3 or more points or improve by 5 or more points on the MoCA
upon re-testing due to test–re-test measurement error. The 70% confidence interval is more sensitive for detecting change, but 15% of ABI patients
who are clinically stable will obtain scores outside this confidence interval in each direction (i.e. false positive estimates of change).
unrelated to MoCA change scores (Pearson r ¼ 0.05,
p ¼ 0.798).
Influence of ceiling effects
Further supplementary analyses were undertaken to determine
if performance gains on the MoCA in the sub-acute group
were limited by ceiling effects. Only one participant (2.8%)
obtained a perfect score on the second MoCA test.
Participants scoring near or at the ceiling (425/30) on their
second MoCA (n ¼ 15, 41.7%) had comparable change scores
to those scoring lower (t(34) ¼ 0.644, p ¼ 0.524) and they
were no less likely to exhibit reliable improvement based on
the 80% CI (4 points) [2 (1) ¼ 0.013, p ¼ 0.908].
Impact of prior MoCA exposure
Based on electronic chart review, 14 sub-acute participants
(38.9%) had the MoCA administered within the 2 months
prior to study enrolment (M ¼ 15.9 days, SD ¼ 10.5). Prior
MoCA administration was significantly associated with
higher study MoCA version 1 scores [t(34) ¼ 2.27,
p ¼ 0.030], but not with lower change scores [t(34) ¼ 1.49,
p ¼ 0.145]. For the participants who took the MoCA prior to
study enrolment, there was a significant gain of 5.1 points
between prior (i.e. pre-study) MoCA (mean total score
¼ 17.6) and study MoCA version 1 (mean total score ¼ 22.8)
scores [t(13) ¼ 2.48, p ¼ 0.028].
Discussion
The present study aimed to evaluate the MoCA’s temporal
stability in a chronic ABI sample and its responsiveness to
change in a sub-acute ABI sample. With regard to the first
objective, the MoCA demonstrated good alternate forms test–
re-test reliability (versions 1 and 2; r ¼ 0.83) in the chronic
ABI sample, with a correlation coefficient comparable to
those found in studies repeating the original version of the
MoCA on two occasions, in various languages, and at
different test–re-test intervals [1, 18–23, 28–30]. The presence of a small (1 point) but significant practise effect
between MoCA versions 1 and 2 is consistent with the notion
that alternate forms often do not completely eliminate practise
effects because they vary the test stimuli but not the
procedures, making it possible for examinees to develop
test-taking strategies [31]. To date, only Costa et al. [32] have
studied the psychometric properties of the MoCA using
alternate forms and they found no practise effect across
alternate forms of the German MoCA administered 60
minutes apart in a counterbalanced manner to older adults
with memory impairment. Further study of the MoCA’s
English alternate forms could clarify the degree of practise
effects and how they can be best adjusted for in serial
assessment.
From the alternate forms test–re-test metrics in the chronic
ABI sample, reliable change CIs were developed (Table II) to
provide clinicians with guidelines for interpreting MoCA
score changes over time in individuals with an ABI. For
example, if a patient scores at least 4 points higher or at least
2 points lower on his/her second MoCA assessment, he/she
can be said to have reliably improved or declined, respectively
(based on the 80% confidence interval). The 80% confidence
interval for change is reasonable and fairly conservative for
clinical use. Note that the vast majority of the chronic sample
(87.5%) had test–re-test difference scores within this
estimated 80% confidence interval. The 70% confidence
interval will be more sensitive to change, with only a modest
increase in false positives (i.e. 15% of patients believed to
be stable will show reliable improvement or decline using this
confidence interval). The present findings support the MoCAs
use in individuals with an ABI, such as to help monitor
cognitive recovery, evaluate efficacy of a cognitively enhancing medication or determine cognitive decline associated
with a medical complication.
With regard to the second objective, the MoCA was
moderately responsive to recovery in a sub-acute ABI sample.
Depending on the confidence level selected (70–90%; see
Table II), 22–58% of the sample reliably improved on this
measure between the first and second assessments.
Establishing reliable improvement at 4 points, 39% of the
sub-acute sample showed improvement in cognitive functioning. Reliable improvement may be more common in patients
with TBI vs stroke over the interval used in this study (1
month to 2.5 months post-injury). During inpatient rehabilitation, it is very unlikely for a patient to reliably worsen on the
MoCA. No patient did so in the present study.
DOI: 10.3109/02699052.2015.1079732
Although ceiling effects seemed not to restrict the MoCA’s
sensitivity to change, two methodological factors suggest that
the MoCA’s responsiveness was under-estimated: the time
from index event to study enrolment and the administration of
a MoCA prior to study enrolment in some participants.
Since the trajectory of cognitive recovery is steepest soon
after TBI or stroke [33–35], this study aimed to enrol
individuals within the early stages of cognitive recovery.
However, the sub-acute participants were only seen an
average of 1 month post-ABI (mean time between index
event and assessment ¼ 30.8 days). This probably made the
assumption that 100% of the sub-acute participants had real
underlying cognitive recovery over the study interval (i.e.
non-statistically reliable improvements on their serial MoCAs
were false-negative detection errors) less tenable, especially
for participants with stroke, who may improve more rapidly
within the first month. MoCA scores would likely have
improved more had the initial MoCA assessment been
administered earlier post-injury.
The other potential confounding factor is that 39% of the
participants had already been administered the MoCA
(version 1) prior to study enrolment, because the MoCA is
widely used in local acute hospitals. The results suggest that
this sub-group may have performed better on the initial study
assessment (with MoCA version 1) due to recovery and/or an
artefact of same-version practise effects. However, the impact
on change scores (on MoCA version 2, 6 weeks later) seemed
less pronounced. Nevertheless, clinicians should interpret
change scores with caution in the circumstance of prior
MoCA exposure.
Change on the MoCA was contrasted with change on the
FIM(R) instrument, one of the most widely used outcome
measures in rehabilitation research and practice. A relationship between change scores on both measures would have
supported the MoCA’s criterion validity for serial assessment.
However, no statistically significant relationship was found.
One possible explanation for the lack of a relationship is that
the two measures were obtained asynchronously. FIM(R)
admission scores were assigned within days of the initial
MoCA, but FIM(R) discharge scores were assigned at the
time of discharge from intensive daily rehabilitation, which
tended to occur at a later (by 2 weeks on average) and
more variable date (SD ¼ 5.4 weeks) than the second MoCA
administration. The correlates of MoCA gains during
rehabilitation warrant more research.
The sensitivity of the MoCA to detect cognitive recovery
during sub-acute rehabilitation may not be uniform across
geographic regions. The admission FIM(R) score and admission-to-discharge FIM(R) change scores in the sub-acute
sample were similar to the Canadian national average [36].
However, in the US, for example, individuals with an ABI are
comparatively admitted to inpatient rehabilitation sooner
(with lower initial FIM(R) scores) and have shorter lengths
of stay but similar admission-to-discharge FIM(R) change
scores [37, 38]. The admission FIM(R) scores in this study
were comparable to discharge FIM(R) scores in the US
Uniform Data System for Medical Rehabilitation [37, 38].
Estimates of the MoCA’s responsiveness in the present study
may, therefore, more closely approximate post-acute outpatient rehabilitation in the US.
The Montreal Cognitive Assessment following ABI
33
To the authors’ knowledge, this is the first study to
examine the MoCA English alternate forms test–re-test
reliability and to investigate the utility of the MoCA
English alternate forms to detect cognitive recovery or
decline with serial administration. With regard to generalizability, the reliable change indices derived from the chronic
ABI sample can likely be applied to individuals with various
ABI types and severities. However, when using different
versions of the MoCA and/or a different order of administration than in the present study or different test–re-test
intervals, these guidelines must be applied with due caution.
The results directly inform the interpretation of change on
serial MoCA administrations when version 2 is administered
after version 1.
Study limitations
There were a number of study limitations. The English
version of the MoCA was administered and 17–28% of the
participants identified that they had learned English as a
second language (ESL). In these individuals, total MoCA
scores may reflect a synergy of pre-injury English proficiency
and cognitive changes after an ABI and their potential for
improvement on an English cognitive assessment measure
may have been affected by their ESL status.
In the chronic group, the inclusion criterion of stroke or
moderate-to-severe TBI diagnosis was based on self-report
and may, therefore, have been inaccurate in some cases. Also,
the method of recruitment may have resulted in a selection
bias where participants who enrolled differed from potentially
eligible participants who did not. Data could not be collected
to evaluate this possibility. It was not feasible to ascertain
whether chronic group participants had any prior exposure to
the MoCA and when. However, given that they were on
average 12.1 years post-injury, recent prior exposure was
considered unlikely.
Another noteworthy limitation is that the present study was
designed to examine statistically reliable change on the
MoCA. Obtaining a MoCA change score outside the reliable
change index may or may not be clinically significant. The
reliable change index is thought to set the lower limit of
‘minimal clinically important difference’ [39]. Further study
combining change metrics based on measurement error with
patient-rated anchors and other complementary techniques is
needed to establish a minimal clinically important difference
for the MoCA [40]. Finally, it is important to recognize that
the alternate forms test–re-test reliability coefficient obtained
in this study (0.83) is a point estimate. Based on the sample
size (n ¼ 40) and a 95% confidence interval, the population
reliability coefficient could be as low as 0.70 and as high as
0.91 and the corresponding reliable change intervals may be
significantly more narrow or wider than reported here.
Conclusions
The MoCA is a brief standardized cognitive assessment tool
that appears suitable to monitor cognitive change in individuals with an ABI. When alternate forms were administered
6 weeks apart, MoCA scores remained quite stable in
participants with a chronic ABI (whom it was expected
would have plateaued in their spontaneous recovery of
34
P. A. Lim et al.
cognitive impairment) and tended to improve in participants
with a sub-acute ABI (whom it was expected would still be
recovering in terms of cognitive impairment). Subject to
replication with a larger sample, the present study provides
rehabilitation clinicians with evidence-based guidelines for
interpreting change on serial MoCA tests.
Brain Inj, 2016; 30(1): 29–35
9.
10.
11.
Acknowledgements
We would like to thank Dr. Jennifer Yao (physiatrist),
Dr. David Koo (physiatrist) and John Tran (research assistant)
for their contributions to this research project.
12.
Declaration of interest
Financial support was gratefully received from the B.C.
Rehab Foundation through its Research and Innovation Fund
(Project Grant #20R67148). GLI has been reimbursed by the
government, professional scientific bodies and commercial
organizations for discussing or presenting research relating to
traumatic brain injury (TBI) and sport-related concussion at
meetings, scientific conferences and symposiums. He has a
clinical practice in forensic neuropsychology involving individuals who have sustained mild TBIs (including athletes). He
has received honorariums for serving on research panels that
provide scientific peer review of programmes. He is a
co-investigator, collaborator or consultant on grants relating
to mild TBI funded by several organizations. He has received
research support from test publishing companies in the past,
including ImPACTÕ Applications Systems (not in the past
5 years). The other authors have no conflicts of interest
to report.
References
1. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S,
Whitehead V, Collin I, Cummings JL, Chertkow H. The Montreal
Cognitive Assessment, MoCA: A brief screening tool for mild
cognitive impairment. Journal of the American Geriatrics Society
2005;53:695–699.
2. McLennan SN, Mathias JL, Brennan LC, Stewart S. Validity of the
montreal cognitive assessment (MoCA) as a screening test for mild
cognitive impairment (MCI) in a cardiovascular population. Journal
of Geriatric Psychiatry and Neurology 2011;24:33–38.
3. Zadikoff C, Fox SH, Tang-Wai DF, Thomsen T, De Bie RMA,
Wadia P, Miyasaki J, Duff-Canning S, Lang AE, Marras C.
A comparison of the mini mental state exam to the Montreal
cognitive assessment in identifying cognitive deficits in Parkinson’s
disease. Movement Disorders 2008;23:297–299.
4. Videnovic A, Bernard B, Jaglin J, Shannon KM. The montreal
cognitive assessment as a screening tool for cognitive dysfunction
in Huntington’s disease. Movement Disorders 2010;25:402–404.
5. Olson RA, Iverson GL, Carolan H, Parkinson M, Brooks BL,
McKenzie M. Prospective comparison of two cognitive screening
tests: Diagnostic accuracy and correlation with community integration and quality of life. Journal of Neuro-Oncology 2011;105:
337–344.
6. Milanini B, Wendelken LA, Esmaeili-Firidouni P, Chartier M,
Crouch P-C, Valcour V. The Montreal Cognitive Assessment
(MoCA) to screen for cognitive impairment in HIV over age 60. J
Acquir Immune Defic Syndr. 2014;67:67–70.
7. De Guise E, Alturki AY, LeBlanc J, Champoux M-C, Couturier C,
Lamoureux J, Desjardins M, Marcoux J, Maleki M, Feyz M. The
montreal cognitive assessment in persons with traumatic brain
injury. Applied neuropsychology. Adult 2014;21:128–135.
8. Wong GKC, Ngai K, Lam SW, Wong A, Mok V, Poon WS. Validity
of the Montreal Cognitive Assessment for traumatic brain injury
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
patients with intracranial haemorrhage. Brain Injury 2013;27:
394–398.
Popović IM, Serić V, Demarin V. Mild cognitive impairment in
symptomatic and asymptomatic cerebrovascular disease. Journal of
the Neurological Sciences 2007;257:185–193.
Martinic-Popovic I, Seric V, Demarin V. Early detection of mild
cognitive impairment in patients with cerebrovascular disease. Acta
Clinica Croatica 2006;45:77–85.
Dong Y, Sharma VK, Chan BPL, Venketasubramanian N, Teoh HL,
Seet RCS, Tanicala S, Chan YH, Chen C. The Montreal Cognitive
Assessment (MoCA) is superior to the Mini-Mental State
Examination (MMSE) for the detection of vascular cognitive
impairment after acute stroke. Journal of the Neurological Sciences
2010;299:15–18.
Pendlebury ST, Cuthbertson FC, Welch SJ V, Mehta Z,
Rothwell PM. Underestimation of cognitive impairment by MiniMental State Examination versus the Montreal Cognitive
Assessment in patients with transient ischemic attack and stroke:
A population-based study. Stroke; 2010;41:1290–1293.
Godefroy O, Fickl A, Roussel M, Auribault C, Bugnicourt JM,
Lamy C, Canaple S, Petitnicolas G. Is the montreal cognitive
assessment superior to the mini-mental state examination to detect
poststroke cognitive impairment?: A study with neuropsychological
evaluation. Stroke 2011;42:1712–1716.
Cumming TB, Bernhardt J, Linden T. The montreal cognitive
assessment: Short cognitive evaluation in a large stroke trial. Stroke
2011;42:2642–2644.
Wong GKC, Lam SW, Wong A, Ngai K, Poon WS, Mok V.
Comparison of Montreal Cognitive Assessment and Mini-Mental
State Examination in evaluating cognitive domain deficit
following aneurysmal subarachnoid haemorrhage. PLoS One
2013;8.
Phillips N, Chertkow H, Nasreddine N, Whitehead V, Bishundayal S,
McHenry C. Validation of Alternate Forms for the Montreal
Cognitive Assessment (MoCA), a Screening Test for Mild
Cognitive
Impairment.
Journal
of
the
International
Neuropsychological Society 2011;44(Suppl 1):291.
Koski L, Xie H, Finch L. Measuring cognition in a geriatric
outpatient clinic: Rasch analysis of the Montreal Cognitive
Assessment. Journal of Geriatric Psychiatry and Neurology 2009;
22:151–160.
Wong A, Xiong YY, Kwan PWL, Chan AYY, Lam WWM, Wang
K, Chu WCW, Nyenhuis DL, Nasreddine Z, Wong LKS, et al. The
validity, reliability and clinical utility of the Hong Kong Montreal
Cognitive Assessment (HK-MoCA) in patients with cerebral small
vessel disease. Dementia and Geriatric Cognitive Disorders 2009;
28:81–87.
Fujiwara Y, Suzuki H, Yasunaga M, Sugiyama M, Ijuin M,
Sakuma N, Inagaki H, Iwasa H, Ura C, Yatomi N, et al. Brief
screening tool for mild cognitive impairment in older Japanese:
Validation of the Japanese version of the Montreal Cognitive
Assessment. Geriatrics and Gerontology International 2010;10:
225–232.
Rahman TTA, El Gaafary MM. Montreal cognitive assessment Arabic version: Reliability and validity prevalence of
mild cognitive impairment among elderly attending geriatric
clubs in Cairo. Geriatrics and Gerontology International 2009;
9:54–61.
Lee J-Y, Lee D-W, Cho S-J, Na DL, Jeon H-J, Kim S-K, Lee YR,
Youn J-H, Kwon M, Lee J-H, et al. Brief screening for mild
cognitive impairment in elderly outpatient clinic: Validation of the
Korean version of the Montreal Cognitive Assessment. Journal of
Geriatric Psychiatry and Neurology 2008;21:104–110.
Zhao S, Guo C, Wang M, Chen W, Wu Y, Tang W, Zhao Y.
A clinical memory battery for screening for amnestic mild
cognitive impairment in an elderly chinese population. Journal of
Clinical Neuroscience: Official Journal of the Neurosurgical
Society of Australasia 2011;18:774–779.
Duro D, Simões MR, Ponciano E, Santana I. Validation studies of
the Portuguese experimental version of the Montreal Cognitive
Assessment (MoCA): Confirmatory factor analysis. Journal of
Neurology 2010;257:728–734.
Keith R, Granger C, Hamilton B, Sherwin F. The functional
independence measure: A new tool for rehabilitation. Advances in
Clinical Rehabilitation 1987;1:6–18.
The Montreal Cognitive Assessment following ABI
DOI: 10.3109/02699052.2015.1079732
25. Freitas S, Simões MR, Marôco J, Alves L, Santana I. Construct
Validity of the Montreal Cognitive Assessment (MoCA). Journal of
the International Neuropsychological Society 2012;18:242–250.
26. Nasreddine ZS. The Montreal Cognitive Assessment. 2014.
Available online at: http://www.mocatest.org/, accessed 16
February 2014.
27. Chelune GJ, Naugle RI, Lüders H, Sedlak J, Awad IJ. Individual
change after epilepsy surgery: Practice effects and base-rate
information. Neuropsychology 1993;7:41–52.
28. Tu Q-Y, Jin H, Ding B-R, Yang X, Lei Z-H, Bai S, Zhang Y-D,
Tang X-Q. Reliability, validity, and optimal cutoff score of the
montreal cognitive assessment (changsha version) in ischemic
cerebrovascular disease patients of hunan province, china.
Dementia and Geriatric Cognitive Disorders Extra 2013;3:25–36.
29. Hu JB, Zhou WH, Hu SH, Huang ML, Wei N, Qi HL, Huang JW,
Xu Y. Cross-cultural difference and validation of the Chinese
version of Montreal Cognitive Assessment in older adults residing
in Eastern China: Preliminary findings. Archives of Gerontology
and Geriatrics 2013;56:38–43.
30. Gõmez F, Zunzunegui MV, Lord C, Alvarado B, Garcı́a A.
Applicability of the MoCA-S test in populations with little
education in Colombia. International Journal of Geriatric
Psychiatry 2013;28:813–820.
31. Benedict RHB. Effects of using same- versus alternate-form
memory tests during short-interval repeated assessments in multiple sclerosis. Journal of the International Neuropsychological
Society 2005;11:727–736.
32. Costa AS, Fimm B, Friesen P, Soundjock H, Rottschy C, Gross T,
Eitner F, Reich A, Schulz JB, Nasreddine ZS, et al. Alternate-form
reliability of the montreal cognitive assessment screening test in a
clinical setting. Dementia and Geriatric Cognitive Disorders 2012;
33:379–384.
33. Christensen BK, Colella B, Inness E, Hebert D, Monette G,
Bayley M, Green RE. Recovery of cognitive function after
34.
35.
36.
37.
38.
39.
40.
35
traumatic brain injury: A multilevel modeling analysis of
Canadian outcomes. Archives of Physical Medicine &
Rehabilitation 2008;89:S3–S15.
Kotila M, Waltimo O, Niemi ML, Laaksonen R, Lempinen M.
The profile of recovery from stroke and factors influencing
outcome. Stroke; a Journal Of Cerebral Circulation 1984;15:
1039–1044.
Douiri A, Rudd AG, Wolfe CDA. Prevalence of poststroke
cognitive impairment: South London stroke register 1995-2010.
Stroke 2013;44:138–145.
Passalent LA, Tyas JE, Jaglal SB, Cott CA. The FIMTM as a
measure of change in function after discharge from inpatient
rehabilitation: A Canadian perspective. Disability and
Rehabilitation 2011;33:579–588.
Granger CV, Markello SJ, Graham JE, Deutsch A, Reistetter TA,
Ottenbacher KJ. The uniform data system for medical rehabilitation: Report of patients with traumatic brain injury discharged
from rehabilitation programs in 2000-2007. American Journal of
physical medicine & rehabilitation/Association of Academic
Physiatrists 2010;89:265–278.
Granger CV, Markello SJ, Graham J, Deutsch A, Ottenbacher KJ.
The uniform data system for medical rehabilitation: Report of
patients with stroke discharged from comprehensive medical
programs in 2000-2007. American Journal of Physical Medicine
& Rehabilitation/Association of Academic Physiatrists 2009;88:
961–972.
Beaton DE, Boers M, Wells GA. Many faces of the minimal
clinically important difference (MCID): A literature review and
directions for future research. Current Opinion in Rheumatology
2002;14:109–114.
Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for
determining responsiveness and minimally important differences
for patient-reported outcomes. Journal of Clinical Epidemiology
2008;61:102–109.
Appendix: Derivation of reliable change confidence intervals
pffiffiffiffiffiffiffiffiffiffiffiffiffiffi
SEM1 ¼ SDp1ffiffiffiffiffiffiffiffiffiffiffiffiffiffi
r12 Standard deviation from time 1 testing multiplied by the square root of 1 minus the test–re-test coefficient.
SEM2 ¼p
SDffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
1 r12 Standard deviation from time 2 testing multiplied by the square root of 1 minus the test–re-test coefficient.
SEdiff ¼ SEM12 þ SEM22 Square root of the sum of the squared SEMs for each testing occasion.
Reliable change confidence intervals ¼ The SEdiff is multiplied by the following z-scores: ±1.04 (70% CI), ±1.28 (80% CI), ±1.64 (90% CI) and
±1.96 (95% CI).
where SEM ¼ standard error of the mean; r12 ¼ Pearson correlation coefficient.
(1)
(2)
(3)
(4)