Figure E1: Summary of Methodology for
Recommending Outcome Measures, after Hospital
Discharge, for Research Purposes
1. Preliminary framework for evaluating post-discharge outcome measures for research purposes
a. SCCM’s Post-Intensive Care Syndrome (PICS) with World Health Organization’s
International Classification of Functioning, Disability, and Health (ICF), including:
1. physical,
2. cognitive,
3. mental health,
4. quality of life,
5. health care utilization,
6. employment/return to normal activities
b. with consideration of modifiers of outcomes (e.g., social economic status, social support)
2. Scoping review of outcome measures used
3. Systematic review, using COSMIN, to evaluate the psychometric rigor of existing measures in our
target population
4. Qualitative studies with patients regarding survivorship experience/outcomes, using initial open ended
questions followed by specific questions for each of the PROMIS outcome domains to help ensure
assessment of a comprehensive set of patient outcomes, rather than just the outcomes in the PICS
framework
a. Qualitative studies with caregivers (for future research)
5. Web-based survey of expert clinicians and researchers regarding important domains of patient
outcomes
6. Phone survey of ARDS survivors and their caregivers regarding important domains of patient
outcomes
7. Empirical analyses of psychometric/clinimetric properties of:
a. existing instruments using existing data sets
b. other existing instruments not yet used in existing studies of acute respiratory failure
survivors (for future research)
c. newly created instruments for acute respiratory failure survivors (for future research)
8. Delphi Process among all relevant stakeholder groups to:
a. reach consensus on domains, measures, and eventually Core Outcome Set
b. identify current limitations and gaps to inform future research agenda
9. Dissemination of results of above process, including resources created for implementation of the
recommended instruments, via custom-made website (www.improveLTO.com), publications,
conferences, and research/dissemination networks.
Table E1: Search terms for the identification of studies eligible for scoping review
Database
Pubmed
11/7/2013
Embase
11/7/2013
CINAHL
11/7/2013
PsycInfo
11/7/2013
Search Terms
(intensive care[tiab] OR "intensive care"[MeSH Terms] OR intensive
therapy[tiab] OR high dependency[tiab] OR critical care[tiab] OR "critical
care"[MeSH Terms] OR intermediate care[tiab] OR step-up care[tiab] OR
step-down care[tiab] OR respiratory distress syndrome[tiab] OR acute lung
injury[tiab]) AND (outcome measure[tiab] OR "outcome assessment (health
care)"[MeSH Terms] OR follow-up[tiab] OR "follow-up studies"[MeSH Terms]
OR health status[tiab] OR "health status"[MeSH Terms] OR functional
status[tiab] OR clinical outcome[tiab]) AND (organ failure[tiab] OR "multiple
organ failure"[MeSH Terms] OR organ dysfunction[tiab] OR sequelae[tiab]
OR quality of life[tiab] OR "quality of life"[MeSH Terms] OR impairment[tiab]
OR morbidity[tiab] OR "morbidity"[MeSH Terms]) NOT (animals[mh] NOT
humans[mh])
Limits: 1970 – present
('intensive care'/exp OR 'intensive care':ti,ab OR 'intensive therapy':ti,ab OR
'high dependency':ti,ab OR 'critical care':ti,ab OR 'critical care':ti,ab OR
'intermediate care':ti,ab OR 'step-up care':ti,ab OR 'step-down care':ti,ab OR
'respiratory distress syndrome':ti,ab OR 'acute lung injury':ti,ab) AND
('outcome measure':ti,ab OR 'outcome assessment'/exp OR 'follow up':ti,ab
OR 'follow up'/exp OR 'health status':ti,ab OR 'health status'/de OR 'functional
status':ti,ab OR 'clinical outcome':ti,ab) AND ('organ failure':ti,ab OR 'multiple
organ failure'/exp OR 'organ dysfunction':ti,ab OR 'multiple organ failure':ti,ab
OR 'sequelae':ti,ab OR 'quality of life'/exp OR 'quality of life':ti,ab OR
'impairment':ti,ab OR 'morbidity':ti,ab OR 'morbidity'/exp) NOT ('animal'/exp
NOT ('animal'/exp AND 'human'/exp))
Limits: 1970 – present
(MH "intensive care units+" OR "intensive care" OR "intensive therapy" OR
"high dependency" OR MH "critical care" OR "critical care" OR "intermediate
care" OR "step-up care" OR "step-down care" OR "respiratory distress
syndrome" OR "acute lung injury") AND ("outcome measure" OR MH
"outcome assessment" OR "follow up" OR MH "prospective studies" OR MH
"health status" OR "health status" OR "functional status" OR "clinical
outcome") AND ("organ failure" OR "multiple organ failure" OR "organ
dysfunction" OR MH "multiple organ dysfunction syndrome" OR "sequelae"
OR "quality of life" OR MH "quality of life" OR "impairment" OR "morbidity"
OR MH "morbidity")
(DE "intensive care" OR "intensive care" OR "intensive therapy" OR "high
dependency" OR "critical care" OR "intermediate care" OR "step-up care" OR
"step-down care" OR "respiratory distress syndrome" OR "acute lung injury")
AND ("outcome measure" OR "outcome assessment" OR "follow up" OR DE
"Followup Studies" OR "health status" OR "functional status" OR "clinical
outcome") AND ("organ failure" OR "multiple organ failure" OR "organ
dysfunction" OR "sequelae" OR "quality of life" OR DE "quality of life" OR
"impairment" OR DE "morbidity" OR "morbidity")
Results
6,391
11,246
1,677
159
Electronic Databases
PubMed
EMBASE
PsycInfo
CINAHL
CENTRAL
6,391
11,246
159
1,677
716
Retrieved
20,189
Duplicates
5,018
Hand Search
Abstract Review
293
15,464
Excluded
14,257
Full Article Review
1,207
Excluded
782
Eligible Articles
425
Figure E2: Flow chart for identifying eligible studies
Abbreviations: CINAHL, Cumulative Index of Nursing and Allied Health Literature; CENTRAL, Cochrane
Controlled Trials Registry
Table E2: Search terms used to identify critical care citations in Figure 2
Database
Pubmed
(3/17/2015)
Search 1
Pubmed
(3/17/2015)
Search 1 +
augmented
version of
the
Cochrane
Highly
Sensitive
Search
Strategy
Search Terms
(intensive care[tiab] OR "intensive care"[MeSH Terms] OR intensive
therapy[tiab] OR high dependency[tiab] OR critical care[tiab] OR "critical
care"[MeSH Terms] OR intermediate care[tiab] OR step-up care[tiab] OR
step-down care[tiab] OR respiratory distress syndrome[tiab] OR acute lung
injury[tiab]) AND (organ failure[tiab] OR "multiple organ failure"[MeSH Terms]
OR organ dysfunction[tiab] OR sequelae[tiab] OR quality of life[tiab] OR
"quality of life"[MeSH Terms] OR impairment[tiab] OR morbidity[tiab] OR
"morbidity"[MeSH Terms]) NOT (animals[mh] NOT humans[mh])
Limits: 1970 – present
(intensive care[tiab] OR "intensive care"[MeSH Terms] OR intensive
therapy[tiab] OR high dependency[tiab] OR critical care[tiab] OR "critical
care"[MeSH Terms] OR intermediate care[tiab] OR step-up care[tiab] OR
step-down care[tiab] OR respiratory distress syndrome[tiab] OR acute lung
injury[tiab]) AND (organ failure[tiab] OR "multiple organ failure"[MeSH Terms]
OR organ dysfunction[tiab] OR sequelae[tiab] OR quality of life[tiab] OR
"quality of life"[MeSH Terms] OR impairment[tiab] OR morbidity[tiab] OR
"morbidity"[MeSH Terms]) AND ("Clinical Trial"[Publication Type] OR "Clinical
Trials as Topic"[Mesh] OR "randomized controlled trials"[TW] OR
"randomised controlled trials"[TW] OR "randomised control trials"[TW] OR
"randomised control trial"[TW] OR "RCTs"[TW] OR "CCTs"[TW] OR
"Longitudinal study"[TW] OR "Longitudinal studies"[TW] OR "Longitudinal
survey"[TW] OR "Longitudinal surveys"[TW] OR "Longitudinal studies"[Mesh]
OR "Clinical Trials as Topic"[Mesh] OR "Clinical Trials, Phase I as
Topic"[Mesh] OR "Clinical Trials, Phase II as Topic"[Mesh] OR "Clinical Trials,
Phase III as Topic"[Mesh] OR "Clinical Trials, Phase IV as Topic"[Mesh] OR
"Controlled Clinical Trials as Topic"[Mesh] OR "Randomized Controlled Trials
as Topic"[Mesh] OR "Multicenter Studies as Topic"[Mesh] OR "Clinical
trial"[TW] OR "Clinical Trials"[TW] OR "randomized controlled trial"[TW] OR
"randomised controlled trial"[TW] OR "randomized control trial"[TW] OR
"randomized control trials"[TW] OR "controlled clinical trial"[TW] OR
"controlled clinical trials"[TW] OR "Phase One"[TW] OR "Phase 1"[TW] OR
"Phase I"[TW] OR "Phase Two"[TW] OR "Phase 2"[TW] OR "Phase II"[TW]
OR "Phase Three"[TW] OR "Phase 3"[TW] OR "Phase III"[TW] OR "Phase
Four"[TW] OR "Phase 4"[TW] OR "Phase IV"[TW] OR "randomized controlled
clinical trial"[TW] OR "randomised controlled clinical trial"[TW] OR
"randomized controlled clinical trials"[TW] OR "randomised controlled clinical
trials"[TW] OR "randomized clinical trial"[TW] OR "randomised clinical
trial"[TW] OR "randomized clinical trials"[TW] OR "randomised clinical
trials"[TW] OR "RCT"[TW] OR "CCT"[TW] OR "multicenter study"[TW] OR
"multicentre study"[TW] OR "multicenter studies"[TW] OR "multicentre
studies"[TW] OR "clinical trials as topic"[TW] OR Randomized[tiab] OR
randomized[tiab] OR placebo[tiab] OR drug therapy[sh] OR randomly[tiab]
OR trial[tiab] OR groups[tiab]) NOT (animals[mh] NOT humans[mh])
Limits: 1970 – present
Results
26,169
9,615
Table E3: Assessments of Physical Activity Limitation from 1970 - 2013* (N = 25)
Assessment
Papers using >1 instrument to assess physical activity limitation
Article to instrument ratio†
4 (16%)
3.1
Instruments used to assess physical activity limitation
6 Minute Walk Test
20 (80%)
Timed Up and Go
3 (12%)
Other tests of physical limitation‡
5 (24%)
* Five (20%) papers were published prior to 2000
† The article to instrument ratio is the quotient of the number of articles to the number of unique
measurement instruments. A higher ratio indicates greater consolidation around a core set of
measures.
‡ The following instruments were used in 1 study each: Locomotion score, Rise from a chair, 50
meter walk test, Incremental Shuttle Walk Test (ISWT), 4 meter walk speed, and standardized
exercise test with ergometer use.
Table E4: Instruments used in the Assessment of Cognitive Activity Limitations from 1970 - 2013* (N = 40)
Papers using more than one instrument
30 (75%)
Article to instrument ratio†
0.66
General Intelligence
37 (93%)
Mini-Mental State Examination (MMSE)
19 (51%)
WAIS-R - entire battery or unspecified
6 (16%)
RBANS - Global score
3 (8%)
Telephone Interview of Cognitive Status (TICS)
3 (8%)
Syndrome Kurz Test - Global score or unspecified
2 (5%)
Raven's Standard Progressive Matrices
1 (3%)
WASI
1 (3%)
Oklahoma Premorbid Intelligence Estimation Method
1 (3%)
Executive function
22 (55%)
Organizational or multitasking ability:
19 (86%)
Trail Making Test Parts A & B - total score
4 (18%)
Hayling Sentence Completion Test
1 (5%)
Modified Six-Elements Test
Reasoning/Concept formation:
2 (9%)
Neurobehavioral cognitive status Examination - Judgement subtest
2 (9%)
WAIS-III - Similarities subtest
Memory
19 (48%)
Immediate Memory (visual and verbal recall and recognition):
7 (37%)
AVLT - Recall Trial 1
7 (37%)
AVLT - Recall Trial 5
6 (32%)
AVLT - Total trials 1-5
5 (26%)
ROCF - Total score
5 (26%)
ROCF - Immediate Recall score
4 (21%)
AVLT - immediate recall subtest
4 (21%)
RBANS - List Learning subtest
4 (21%)
RBANS - List Recall
4 (21%)
RBANS - List Recognition
4 (21%)
RBANS - Story Recall
4 (21%)
RBANS - Story Memory Subtest
4 (21%)
RBANS - Figure Recall
3 (16%)
RBANS - Global score
2 (11%)
WAIS-III - Verbal Paired Associates test
2 (11%)
WAIS-III - Faces
2 (11%)
Syndrome Kurz Test - Immediate recall subtest II
2 (11%)
Syndrome Kurz Test - Recognition memory (subtest IX)
1 (5%)
AVLT - word recognition subtest
Delayed recall (visual and verbal):
6 (32%)
AVLT - Delayed Recall
6 (32%)
WMS-III - Logical Memory II
AVLT - delayed recall subtest
Rey Complex Figure Test - Total score
RBANS - Global score
WMS-R - Delayed Recall Index Score
Syndrome Kurz Test - Delayed recall - subtest VIII
California Verbal Learning Test II - Long-delay free recall
Attention
5 (26%)
5 (26%)
3 (16%)
3 (16%)
2 (11%)
2 (11%)
18 (45%)
Focused attention/Concentration
Digit Span Forward and Backward as part of WMS-R or WAIS-III
RBANS - Digit Span subtest
WAIS-III - Letter/Number sequencing tests
WMS-R - Attention/Concentration Index score
RBANS - Global score
Stroop Color-Word Test- Golden Version - Interface Trial
8 (44%)
4 (22%)
4 (22%)
4 (22%)
3 (17%)
2 (11%)
Divided attention/Interference:
AVLT - True positives, false positives
AVLT - interference subtest
Neuropsychological Assessment - 4th ed. - color word interference
task
Language / verbal fluency / verbal production / naming
Verbal Fluency Test, Thurston Verbal Fluency test, or Controlled
Oral Word Association test
RBANS - Picture Naming
RBANS - Semantic Fluency
Syndrome Kurz Test - Naming objects (subtest I)
Syndrome Kurz Test - Naming Numerals (subtest III)
Visuospatial Construction
RBANS - Global score
ROCF - Total score
ROCF - Copy score
RBANS - Figure Copy subtest
RBANS - Line Orientation subtest
Syndrome Kurz Test - Arranging blocks (subtest IV)
Syndrome Kurz Test - Replacing blocks (subtest V)
Working memory
Digit Span Forward & Backward as part of WAIS-III or WMS-R
WAIS-III - Letter/Number sequencing tests
RBANS - Digit Span subtest
Syndrome Kurz Test - Reversal naming (subtest VII)
Mental Processing
Speed
Trail Making Test Parts A & B - total score
Digit Symbol Subtest score from WAIS-R
6 (33%)
4 (22%)
1 (6%)
17 (43%)
12 (71%)
4 (24%)
3 (18%)
2 (12%)
2 (12%)
16 (40%)
3 (19%)
5 (31%)
10 (63%)
4 (25%)
4 (25%)
2 (13%)
2 (13%)
16 (40%)
8 (50%)
4 (25%)
4 (25%)
2 (13%)
12 (30%)
9 (75%)
6 (50%)
WAIS-R - Digit Symbol Subtest
RBANS - coding subtest
WAIS-III - Digital Symbol Coding
Alertness
6 (50%)
4 (33%)
2 (17%)
2 (5%)
2 (100%)
Syndrome Kurz Test - Counting Symptoms (subtest VI)
Abbreviations: AVLT, Auditory Verbal Learning Test; RBANS, Repeatable Battery for the
Assessment of Neuropsychological status; ROCF, Rey-Osterrieth Complex Figure Test; WAIS-R,
Wechsler Adult Intelligence Scale-Revised; WAIS-III, Wechsler Adult Intelligence Scale-III;
WASI, Wechsler Abbreviated Scale of Intelligence
* Categories are not mutually exclusive
† The article to instrument ratio is the quotient of the number of articles to the number of unique measurement
instruments. A higher ratio indicates greater consolidation around a core set of measures.