Hill B, et al. J Am Med Inform Assoc 2016;23:1136–1142. doi:10.1093/jamia/ocw019, Research and Applications
Automated pictographic illustration of
discharge instructions with Glyph: impact
on patient recall and satisfaction
RECEIVED 19 May 2015
REVISED 7 December 2015
ACCEPTED 23 January 2016
PUBLISHED ONLINE FIRST 27 May 2016
Brent Hill,1 Seneca Perri-Moore,1 Jinqiu Kuang,1 Bruce E Bray,1 Long Ngo,2 Alexa Doig,1 and Qing Zeng-Treitler1
ABSTRACT
....................................................................................................................................................
RESEARCH AND APPLICATIONS
Objectives First, to evaluate the effect of standard vs pictograph-enhanced discharge instructions on patients’ immediate and delayed recall of
and satisfaction with their discharge instructions. Second, to evaluate the effect of automated pictograph enhancement on patient satisfaction with
their discharge instructions.
Materials and Methods Glyph, an automated healthcare informatics system, was used to automatically enhance patient discharge instructions
with pictographs. Glyph was developed at the University of Utah by our research team. Patients in a cardiovascular medical unit were randomized
to receive pictograph-enhanced or standard discharge instructions. Measures of immediate and delayed recall and satisfaction with discharge instructions were compared between two randomized groups: pictograph (n ¼ 71) and standard (n ¼ 73).
Results Study participants who received pictograph-enhanced discharge instructions recalled 35% more of their instructions at discharge than
those who received standard discharge instructions. The ratio of instructions at discharge was: standard ¼ 0.04 6 0.03 and pictographenhanced ¼ 0.06 6 0.03. The ratio of instructions at 1 week post discharge was: standard ¼ 0.04 6 0.02 and pictograph-enhanced 0.04 6 0.02.
Additionally, study participants who received pictograph-enhanced discharge instructions were more satisfied with the understandability of their instructions at 1 week post-discharge than those who received standard discharge instructions.
Discussion Pictograph-enhanced discharge instructions have the potential to increase patient understanding of and satisfaction with discharge
instructions.
Conclusion It is feasible to automatically illustrate discharge instructions and provide them to patients in a timely manner without interfering with
clinical work. Illustrations in discharge instructions were found to improve patients’ short-term recall of discharge instructions and delayed satisfaction (1-week post hospitalization) with the instructions. Therefore, it is likely that patients’ understanding of and interaction with their discharge
instructions is improved by the addition of illustrations.
....................................................................................................................................................
Keywords: informatics, consumer, patient education, health literacy, pictograph
BACKGROUND AND SIGNIFICANCE
The responsibility for the majority of a patient’s care after the patient
is discharged from the hospital is relegated to the patient themselves
and patient’s family.1 For this reason, it is essential that patients and
their caregivers understand critical elements of the patient’s discharge
instructions, such as activity restrictions, dietary guidelines, medication management, wound care, follow-up instructions, signs and
symptoms of potential problems, and emergency contact information,
in order to successfully negotiate the recovery period.2 Discharge instructions are typically given during the hospital discharge process.3
These instructions can be in verbal and/or written format and are prepared and explained to patients/caregivers by a nurse. If successful,
the instructions will give the patients and/or caregivers the knowledge
and skills necessary for the patient’s post-hospitalization care.4 The
patient and their caregivers must be able to understand the patient’s
discharge instructions so that the patient can recall aftercare instructions or recognize that the information they require for their postdischarge care can be found in their discharge instructions.5,6 Patients
who cannot understand health information, and therefore cannot follow instructions regarding their care have higher rates of hospitalization, hospital readmission, costly and unnecessary complications, use
of emergency services, and death.7,8 Should patients leave the hospital with an inadequate understanding of their medical condition and
plan of care, they may not recognize the importance of adhering to
their aftercare instructions.4–6,9
Unfortunately, prior studies have revealed that patients, regardless
of health literacy and education level, commonly have problems understanding and recalling their discharge instructions.8,10,11 Many barriers to a patient’s ability to understand discharge instructions are
inherent in the hospital environment and are difficult to ameliorate.
Examples include side effects of medications, poor sleep quality while
hospitalized, and effects associated with the patient’s morbidity.1 In
order to address patients’ inability to understand and follow discharge
instructions, it is essential to re-evaluate the efficacy of discharge
instructions and employ strategies that increase the comprehensibility
of discharge instruction for patients.12
One intervention that has the potential to improve patients’ comprehension of discharge instructions is the use of illustrations and pictures to depict important health and self-care information.1,7,13–15
Pictographs, such as the Wong-Baker Faces of Pain Scale, have been
found to help patients with low health literacy express their experience
better than numbers, rankings, or potentially unfamiliar words.16,17
Other studies have demonstrated the effectiveness of including pictographs in patient instructions.14,15,18–26 However, illustrating patient
instructions is a time-consuming task.
In our research, we have developed an automated system called
Glyph to enhance hospital discharge instructions. A team of two informaticians, two clinicians, a programmer, and a graphic designer developed Glyph. Glyph is a Java application with a web-based interface that
accepts free text input and generates illustrations for that text. It has a
Correspondence to Dr Brent Hill, 421 Wakara Way, Ste 140 Salt Lake City, Utah 84108, USA. Tel: 801-364-0532. E-mail: [email protected]
Published by Oxford University Press on behalf of the American Medical Informatics Association 2015. This work is written by US Government employees and is in
the public domain in the United States. For numbered affiliations see end of article.
1136
Hill B, et al. J Am Med Inform Assoc 2016;23:1136–1142. doi:10.1093/jamia/ocw019, Research and Applications
few components: several preprocessing and processing modules that
parse and annotate the instruction text, a look-up module that maps
the annotated text to images in a library we created for the project, a
rule engine that combines images based on semantic grammar, and a
rendering module that integrates images with text.27 In preliminary
studies, Glyph has demonstrated the ability to illustrate discharge instructions.28 In this study, we deployed and tested the Glyph system in
a real-life clinical setting in a randomized controlled trial.
OBJECTIVES
MATERIALS AND METHODS
Glyph System
The Glyph system illustrates patient discharge instructions via the following steps: (1) preprocessing free text; (2) annotating illustratable
terms from the text; (3) removing nonillustratable text from further processing; (4) composing images based on the illustratable terms and
grammar patterns; and (5) rendering images for the corresponding
text.27 In a prior study, we used Glyph to illustrate 49 patient instructions representing 10 different discharge templates from the University
of Utah Cardiology Service.28 Healthy participants were asked to review and then recall a set of discharge instructions that contained five
pictograph-enhanced and five nonpictograph-enhanced items. The
presence of Glyph pictographs in the instructions improved patients’
discharge instruction recall (P < .001).28 Table 1 shows examples of
Glyph-illustrated instructions.
Standard Discharge Instruction Processes
The study site, the Cardiovascular Medical Unit (CVMU), is within the
University of Utah Health Care System which is a large academic medical center that serves Utah and five surrounding states.29 The CVMU
has 35 beds and treats approximately 2382 patients annually. The division of cardiology has 45 primary faculty and 20 advanced-practice
clinicians in general cardiology, interventional cardiology, heart failure/
transplant, electrophysiology, and genetic cardiology.30 UUHC CVMU
patients receive discharge instructions that are composed of two documents. One is a document created using a standard set of
Sample and Setting
Study participants were recruited from the UUHC CVMU. This site was
selected because the clinical members of our research team include a
UUHC cardiologist and a cardiology research nurse. Additionally, the
UUHC CVMU has a dedicated team of nurses that provide discharge
teaching and instructions to patients, which facilitated the implementation of the study. Participants that met all the inclusion criteria and
no exclusion criteria were approached by study nurses in their hospital
room for consent to participate in the study. Criteria for inclusion in
the study were as follows: ability to speak, read, and write in English;
being discharged to home; 21 years old; and having a discharge
plan that included complete discharge instructions. Criteria for exclusion from the study were as follows: having cognitive or physical impairments that prevented participation in the study; being employed as
a nurse, physician, or pharmacist or works with discharge instructions; and being on the heart transplant or left ventricular assist device
waiting list or being a recipient of either of these (because discharge
instructions for these patients are more detailed, complex, and are
closely monitored by their clinical team, unlike most patients). The diagnosis or procedure received by study participants included ablation,
atrial fibrillation ablation, angiogram, atrial septal defect/patent foramen ovale, coronary artery disease/stent/heart attack, electrophysiology device, heart failure, heart surgery, and minimally invasive heart
surgery.
All the patients who met the inclusion criteria and no exclusion criteria were approached for study participation until 144 patients consented and completed all the study procedures. Data was not
collected for those patients who declined to participate in the study.
Data collection was completed between September 10, 2013 and
Table 1: Discharge Instructions and Their Associated Illustrations
Instruction
Illustration
Do not drive for 4
weeks or while you are
taking narcotics.
Weigh yourself each
morning after going to
the bathroom but before
eating or drinking.
1137
RESEARCH AND APPLICATIONS
The aims of this study were twofold. The first was to evaluate the effect of automated pictograph enhancement on patients’ immediate
and delayed recall of discharge instructions. The second was to evaluate the effect of automated pictograph enhancement on patients’ satisfaction with discharge instructions.
diagnosis-/procedure-specific templates for atrial fibrillation ablation,
heart surgery, heart failure, minimally invasive chest surgery, electrophysiology device, and post-catheterization (Appendix A). A discharge
nurse sometimes modifies the templates. For example, the nurse
would remove information about a defibrillator from the electrophysiology device template if the patient had a pacemaker implant. Another
document, the Patient Care Summary, is created by the patient’s physician and is combined with the diagnosis-/procedure-specific template and provided to the patient by the discharging nurse immediately
prior to the patient’s discharge and in conjunction with discharge
teaching. The patient care summary often includes personalized instructions about diet and activity as well as activity restrictions.
Hill B, et al. J Am Med Inform Assoc 2016;23:1136–1142. doi:10.1093/jamia/ocw019, Research and Applications
January 22, 2014. The sample size calculation was conducted using
G*Power.31 Baseline immediate recall rate estimates of 0.44 for standard discharge instructions and 0.54 for pictograph-enhanced discharge instructions were based on Dr Zeng-Treitler’s (2009)32
preliminary research; she also estimated the standard deviation to be
0.20. Using the means for each group, 0.44 and 0.54, and a standard
deviation (SD) of 0.20 yielded a Cohen’s d effect size of 0.5 with a
two-tailed test, an a ¼ 0.05 and power ¼ 0.80, a sample size of 128
would have been sufficient to detect the large effect between groups,
with 64 participants randomized to the control group and 64 participants randomized to the intervention group. The study was overenrolled to account for possible attrition.
RESEARCH AND APPLICATIONS
Instruments and Measures
Paivio’s additivity hypothesis posits that text paired with images is additive and results in superior recall rates than text alone.33–37 This
foundational research on verbal and imaginal processing was applied
in this study for instrument development. Additionally, recall has been
used to measure a patient’s memory and understanding of his or her
discharge instructions; some of these studies also used illustrations or
cartoons paired with discharge and other health instruction
text.5,6,14,16,25,38–43 The measure of patients’ free recall of instructions
for this study was collected using prompt questions developed by the
study investigators (Appendix B). The questions were based on key elements of discharge instructions, eg, “What wound care instructions
were listed on your discharge instructions?,” and “What activity restrictions were given in your instructions?” A question about the patient’s medications was initially included in this instrument but was
excluded because the CVMU protocol for medications was in transition
to become a separate discharge task conducted by a clinical pharmacist. Recall rates were calculated by dividing the number of words a
study participant remembered by the total word count of the document.5,44 Given that each patient’s discharge instructions differed in
length and content based on the patient’s diagnosis or procedure and
the amount of personalization, this calculation allowed for a comparison of the recall of different instructions through the creation of a ratio
of words the patient remembered.
Patient satisfaction is strongly correlated with satisfaction with
their hospital care, and discharge preparation has previously been
rated lowest among all aspects of hospital care.45–48 Most publications that describe the measurement of patient satisfaction in the clinical environment do not publish their tools because they are
proprietary, such as Krames On-Demand and the Press Ganey
Inpatient Survey.49,50 The study team used simple patient satisfaction
questions with a Likert-type response that ranged from 1 (completely
dissatisfied) to 7 (completely satisfied) (Appendix C), similar to those
used in the proprietary satisfaction surveys.
Data Collection Procedures
Prospective study participants’ names were provided to our study nurses
by CVMU nurses after a daily CVMU discharge planning meeting, and
study nurses were apprised of patients who received discharge orders
throughout the day. Patients were screened, and those who qualified
based on the inclusion/exclusion criteria were approached for consent.
Patients who consented to participate in the study were randomized to
receive standard vs pictograph-enhanced discharge instructions.
Intervention group discharge instructions were prepared by study nurses,
who obtained an electronic copy of the participant’s Patient Care
Summary from the electronic health record and processed it with the
Glyph system to illustrate diet, activity, and activity restriction sections.
Study nurses reviewed the newly illustrated material for accuracy,
1138
readability, and patient safety, and no issues related to the automated insertion of pictograph illustrations were identified. The Patient Care
Summary was then appended to include a diagnosis-/procedure-specific
template that was illustrated with Glyph and then modified to include personalization alterations that were made to the templates by CVMU preventive cardiology nurses. Control group discharge instructions were
prepared by study nurses, who printed an electronic copy of the participant’s Patient Care Summary from the electronic health record and appended it with a copy of the diagnosis-/procedure-specific template that
included personalization alterations, if any. Modifications to the instructions by the discharge nurse for both pictograph-enhanced and standard
discharge instructions were handled the same way, by either inclusion or
exclusion of instructions. Study nurses were not blinded for the intervention, because they prepared the study documents and assessed recall in
study participants.
Study procedures for data collection began after the patient received all standard hospital discharge teaching by their nurse, using
standard discharge instructions. After completing the traditional final
discharge teaching, study nurses were notified that the patient was
ready to begin the study. The study nurse then went to the patient’s
room and presented the participant with the version of study discharge
instructions that they were randomized to receive and advised the participants to review the document for up to 15 min. When the participant indicated that they were finished reviewing their discharge
instructions, they were asked a series of free recall questions
(Appendix B) to assess their immediate recall of the instructions.
Study nurses used a nonillustrated copy of each patient’s instructions
and highlighted each word, or a close approximation, which was verbalized by each patient. Additionally, participants were asked two
questions about their satisfaction with their discharge instructions
(Appendix C). In order to assess delayed recall, participants were
called 1 week after discharge and asked the same free recall and satisfaction questions by a member of the study team who was blinded
to participant group assignment. Study data were collected and managed using Research Electronic Data Capture electronic data capture
tools hosted at the University of University of Utah (CTSA
5UL1RR025764-02).51,52
Data Analysis
First, we examined the distributions of participant demographics and
baseline clinical information to assess the performance of the randomization procedure. We expected that these distributions would be similar between the two groups (ie, those participants who received
standard discharge instructions and those who received pictographenhanced discharge instructions). There were 11 variables used for
this analysis. For those variables, which were categorical, we used the
chi-square test or Fisher’s exact test, when the expected count for at
least 25% of the cells in the contingency table fell below 5. For continuous variables, we used the two sample t-test when the distributions
were nearly normally distributed; otherwise, we used the non-parametric Wilcoxon test. Variables that were statistically significant in this
analysis were then used as confounding variables in the multivariate
models. We used the type-I error of 0.05 for declaring a statistically
significant difference between the two groups.
Second, we obtained the estimate of the primary endpoint of this
study – the estimated adjusted mean difference of normalized prerecall illustrated ratio and the normalized post-recall illustrated ratio
between the two groups. Recall ratios were normalized by dividing the
number of words a patient remembered by the total number of words
in the patient’s discharge instructions. This was necessary because
the lengths of individual discharge instructions varied based on the
Hill B, et al. J Am Med Inform Assoc 2016;23:1136–1142. doi:10.1093/jamia/ocw019, Research and Applications
procedure or diagnosis the patient had received and on any personalization that was included in the instructions by the patient’s doctor or
nurse. We assessed the distributions of the normalized recall ratios and
found them to be reasonably symmetric and nearly normally distributed.
We structured the data so that each subject had the pre-recall and
post-recall set up for longitudinal data analysis using the linear mixed-
Table 2: Distribution of Demographics and Baseline Clinical
Information
Characteristic
Standard
P-Value*
0.771
Gender
Male
71%
69%
Female
29%
31%
White
92%
99%
African American
5%
1%
Others
3%
0%
Below 12th grade
37%
18%
Above 12th grade
63%
82%
English
93%
97%
0.442
Confidence managing
health
88%
97%
0.056
Age (years) (mean 6 SD,
median)
59 6 13 (62)
61 6 13 (63)
0.199
Length of hospital stay
(days) (mean 6 SD,
median)
3.7 6 4.0 (2)
4.6 6 5.3 (2)
0.577
Charlson Comorbidity Index
(mean 6 SD, median)
2.9 6 3.1 (2)
2.9 6 2.7 (2)
0.763
Number of hospitalizations
in previous year (mean
6 SD, median)
1 6 0 (1)
2 6 0 (2)
0.553
Number of medications
from patient care summary
(mean 6 SD, median)
11 6 5 (10)
12 6 5 (12)
0.142
Race
0.171
Education
0.012
Language
SD, standard deviation. *P-value obtained from Chi-square, Fisher’s
exact test, two-sample t-test, or Wilcoxon Rank Sum test.
RESULTS
Of the 11 variables we examined for the randomization procedure
(Table 2), we found that education (initially 4th grade, 5–8th grade,
9–12th grade, and >12th grade, but, because there were only five
participants who reported having below 9th grade education, we used
binary categories – 12th grade and >12th grade) to be the only variable that is a potential confounder (P ¼ .012). The pictograph group
included a substantially larger proportion of subjects who had above a
12th grade education (82%) compared with the standard group (63%).
Our total sample included more males than females (70% vs 30%),
and was largely white (Caucasian) (92% in the standard group and
99% in the pictograph group). The average age of the participants
was about 60 years old, and the average number of medications the
participants were taking was between 11 and 12. Both groups had a
median length of hospital stay of 2 days and a median Charlson
Comorbidity Index score of 2.0. The Charlson Comorbidity Index is a
method of categorizing comorbidities based on International
Classification of Disease (ICD) diagnosis codes, individually weighted
based on the risk of mortality or resource use and then summed.
For the primary outcome of interest, the normalized recall ratio
(Table 3), we examined both the pre- and post-normalized recall ratio
and found that there was a statistically significant difference for the
normalized pre-recall illustrated ratio (P ¼ .001) with an adjusted estimated mean difference of 0.014 (95% CI, 0.006-0.023) in favor of the
pictograph-enhanced instructions. For the normalized post-recall ratio,
we found no difference between the two groups (P ¼ .852; estimated
mean difference of 0.004; 95% CI, -0.008-0.009).
For the satisfaction items regarding understanding the discharge
instructions (Table 4), we found that all four items (two for pre-recall
and two for post-recall) yielded a consistent trend in favor of the pictograph-enhanced instructions, both in the unadjusted proportions, and
Table 3: Distribution of Pre-Recall and Post-Recall Illustrated Ratio for Each Treatment Group
Ratio
Standard (S, 95% CI)
Pictograph-Enhanced (PE, 95% CI)
Difference Between
Treatments [PE S]* (95% CI)*
P-Value*
Pre-Recall Ratio
0.04 6 0.03 (0.04, 0.01–0.12)
0.06 6 0.03 (0.05, 0.01–0.14)
0.014 (0.006–0.023)
0.001
Post-Recall Ratio
0.04 6 0.02 (0.03, 0.0–0.12)
0.04 6 0.02 (0.03, 0.0–0.12)
0.004 (-0.008, 0.009)
0.852
*The estimated mean differences and the P-values were obtained from the linear mixed effects model, which includes the interaction between time
and treatment group controlling for education, with variance-covariance type compound symmetry.
1139
RESEARCH AND APPLICATIONS
PictographEnhanced
effects models, which could capture the within-subject correlation of the
two recall ratios. We tested different variance-covariance structures and
found compound-symmetry (CS) to yield the lowest Akaike Information
Criterion (AIC). Therefore, we used CS structure in the linear mixed effects model. In this model, we treated education level (below or above a
12th grade level) as a confounder and included time (pre- and postrecall), group, as well as the interaction between time and group. We
then used linear contrasts to estimate the adjusted mean difference between the two groups (pictograph and standard) and also reported the
linear contrast P-values.
Third, we examined the effect of the intervention on the patient satisfaction items (ie, understanding the discharge instructions, having
enough information to manage health after leaving the hospital). We
used a generalized linear model with log link and binary error to obtain
the estimated relative risks, 95% confidence interval (CI), and P-values.
We used the SAS software version 9.3 (SAS Institute, Inc.) for all of
the analyses.
Hill B, et al. J Am Med Inform Assoc 2016;23:1136–1142. doi:10.1093/jamia/ocw019, Research and Applications
Table 4: Distribution of Pre-Recall and Post-Recall
Satisfaction Assessment for Each Treatment Group
RESEARCH AND APPLICATIONS
Pre/Post
Assessments
Standard
(S) (%)
PictographEnhanced
(PE) (%)
Risk Ratio
Between
Treatments
[PE/S]* (95% CI)*
P-Value*
Pre-recall
understanding
discharge
89
90
1.02 (0.91, 1.14)
0.757
Pre-recall have
enough
information
95
97
1.04 (-0.95, 1.28)
0.711
Post-recall
understanding
discharge
85
99
1.13 (0.99, 1.29)
0.07
Post-recall have
enough
information
92
97
1.06 (-0.98, 1.15)
0.142
*The risk ratios and P-values were obtained from the generalized linear models with log link and binary error controlling for education.
the adjusted relative risk estimates (all point estimates were larger
than 1.0) from the generalized linear models adjusting for education.
The biggest difference came from the post-recall understanding discharge instructions model, with a relative risk of 1.13 and a 95% CI of
0.99-1.29 (P ¼ .070).
DISCUSSION
The results of this study demonstrate that an informatics intervention
to improve patient recall of and satisfaction with discharge instructions
through automatic illustration of discharge instructions shows promise. Specifically, pictographic enhancement of the discharge instructions improved patient’s immediate recall of the instructions by 35%,
which is clinically significant. However, the increase in immediate recall of pictograph-enhanced vs standard discharge instructions did not
carry over to patients’ delayed recall 1-week post-discharge.
According to Houts et al., (2006), most patients read written instructions once and then rely on memory to take health actions based on
those instructions. Therefore, theoretically, the patients who were
given pictograph-enhanced discharge instructions had more information to draw on when taking health actions. Although it is not clear
whether patients that received pictograph-enhanced discharge instructions had better outcomes, the results hold sufficient promise to
continue with research to explore the impact of illustrated discharge
instructions on patient outcomes.
Patients were asked to recall the content of their discharge instructions to the best of their ability; however, normalized recall rates appeared to be extremely low. The highest normalized recall rate was
15%, and the lowest recall rate was about 1%. This value should
not be interpreted as the percentage of information that patients recall.
Our recall questions prompted patients to recall key sections of their
discharge instructions, but did not require the patients to recall everything from their instructions. This is due, in part, to the nature of the
content covered by discharge instructions; not all the content is critical
to patients’ immediate post-discharge care (eg, statements such as
“You are the most important person in your in care.”). As such, our
questions focused mostly on the content that was most relevant for
patients’ immediate post-discharge care. In an effort to estimate the
1140
percentage of content covered by the recall questions, a cardiologist
on the study team reviewed a random sample of 10 discharge instructions used in this study and highlighted the content he considered to
be immediate post-hospitalization instructions vs general health education. This analysis revealed that approximately 10–18% of the
Patient Care Summary template content and 20–51% of the preventive cardiology template content are immediate post-hospitalization instructions. This finding leads to several implications for both the
hospital discharge process as well as the metric used for this study.
Patients receive an overwhelming amount of information at discharge
and might benefit from simplified discharge instructions.1,11,14,53,54 It
may be beneficial to limit inpatient discharge instructions to aftercare
needs, such as symptom identification and wound care, and address
health education information, such as dietary recommendations, in a
separate document and educational setting.
Another implication is that, should a patient lose his or her discharge instructions or otherwise not be able to identify them, they
may not have retained sufficient knowledge to adequately adhere to
the post-hospitalization instructions. To mitigate this situation, the
care team could provide an electronic copy of the discharge instructions to the patient and/or their caregivers via e-mail or the patient’s
personal health record. Finally, free recall may not be the best metric
to assess patients’ comprehension of discharge instructions. Reading
for comprehension is a different process than reading for memorization. Therefore, exploring different methods of measuring a patient’s
comprehension of the activities and concepts in discharge instructions
is incumbent for future research in this area. Currently, structured interviews, questionnaires, measures of patient health literacy, and patient self-reporting are used to assess patient comprehension of
discharge instructions.1,2,7,11,12,53,54,55 However, because we sought
to measure the impact of illustrations paired with text on patient recall
of discharge instructions, we sought to reveal patient comprehension
at the instruction level rather than with concept identification.
Patient satisfaction with the ease of use of their discharge instructions was similar at discharge for both groups, but, after 1 week, during which the study participants had the opportunity to review their
instructions at home, those who received the pictograph-enhanced instructions reported greater levels of satisfaction with the ease of use
of their instructions than those that received standard instructions.
Illustrated discharge instructions were considerably longer than the
standard instructions, due to the amount of space pictographs occupy
on the page. It is intriguing that patients found the instruction sets that
included illustrations easier to use than standard instructions, even
though the illustrated instructions were approximately three times longer. Previous research on the use of pictographs in health communication provides some insight into how pictographs increase
patient satisfaction with ease of use. Pictures paired with text helps
readers visualize concepts in the text, which can increase comprehension of the material; this may translate to increased satisfaction
with ease of use because text instructions alone require more cognitive resources to make a mental model of the information presented in
the text.14,25 For example, a study on asthma inhaler instructions
found that patients who received picture-enhanced instructions
performed more instruction steps faster and more accurately while
expressing fewer doubts than those who received text-only
instructions.25
Limitations of this study were that the sample was mostly white
(Caucasian), and the vast majority had completed a high school education and beyond, which matches the demographics of Utah, where
the study was conducted.56 This lack of diversity limits the generalizability of the results to other populations. Additional research with
Hill B, et al. J Am Med Inform Assoc 2016;23:1136–1142. doi:10.1093/jamia/ocw019, Research and Applications
CONCLUSION
The inclusion of pictographs is one method to improve patients’ recall
of and satisfaction with hospital discharge instructions, but is not a solution unto itself. Discharge instructions for patients hospitalized with
cardiovascular diseases are already complex and extensive, and,
when enhanced by pictographs, become considerably longer. Other
methods of providing information to patients should be explored to
capitalize on several avenues of communication, because it is unlikely
that one intervention, however efficacious, is sufficient to address the
multifactorial issues that impact a patient’s ability to understand and
follow his or her hospital discharge instructions. Some possible avenues of providing discharge instructions include via a tethered, electronic personal health record, via e-mail, or even via an infographic
that can be placed in a strategic location in the patient’s home.
This study also demonstrated the feasibility of the Glyph system
for use in a busy clinical environment to automatically enhance patient
education materials. This process was completed in a very busy cardiovascular medical unit and did not interrupt the flow of patient treatment and care. The addition of pictographs to discharge instructions
has the potential to increase the efficiency of patient education, because the use of pictographs in health communication focuses patients’ attention to the materials, aids in their comprehension and
performance of instructions, is not time consuming, and does not interfere with the work of the clinical environment.14,25
CONTRIBUTORS
B.H. participated in the study implementation design, led the data collection,
contributed to the data analysis and interpretation, and drafted and revised the
article. S.P. participated in the study implementation design, the data collection,
and drafting and design of the article. J.K. participated in the study implementation design, the data collection, and the data analysis. B.B. contributed to the
study implementation design. L.G. led the statistical analysis. A.D. participated
in the data interpretation and the drafting and revision of the article. Q.Z. was
the principal investigator of the study, designed the study, led the study implementation design, oversaw the data collection, and contributed to the data analysis and drafting and revision of the article.
FUNDING
This work was supported by the National Institutes of Health (R01 LM07222).
COMPETING INTERESTS
None.
ACKNOWLEDGEMENTS
We would like to express our gratitude for the efforts of Duy Bui, Rebecca
Morris, Katherine Doyon and Carrie Christensen for the many hours of work
they put into this project.
SUPPLEMENTARY MATERIAL
Supplementary material is available online at http://jamia.oxfordjournals.org/.
REFERENCES
1. Chugh A, Williams MV, Grigsby J, Coleman EA. Better transitions: improving
comprehension of discharge instructions. Frontiers Health Services Manag.
2009;25(3):11–32.
2. Jacott WE. JACHO Advocates Improved Communication During Discharge.
The Hospitalist. 2007; Accessed January 14, 2013. http://www.the-hospitalist.org/details/article/234091/New_Moves.html.
3. Pfuntner AW, Wier LM, Elixhauser A. Overview of hospital stays in the United
States, 2010. Agency for Healthcare Research and Quality. Location:
Rockville, MD; 2012.
4. Clark KD, Duco J, Lattimer C. Transitions of care: the need for a more effective approach to continuing patient care. The Joint Commission. Location:
Oak Brook, IL; 2012.
5. Linn AJ, van Dijk L, Smit EG, Jansen J, van Weert JCM. May you never forget what is worth remembering: the relation between recall of medical information and medication adherence in patients with inflammatory bowel
disease. J Crohn’s Colitis. 2013;7(11):e543–e550.
6. Kessels RPC. Patients’ memory for medical information. J Royal Soc Med.
2003;96(5):219–222.
7. Boodman SG. Helping patients understand their medical treatment. 2011.
http://www.kaiserhealthnews.org/stories/2011/march/01/health-literacyunderstanding-medical-treatment.aspx. Accessed June 2, 2013.
8. Institute of Medicine. Health Literacy: a Prescription to End Confusion.
Institute of Medicine, Washington D.C.: National Academies Press; 2004.
9. Pavlik V, Brown AE, Nash S, Gossey JT. Association of patient recall, satisfaction, and adherence to content of an electronic health record (EHR)generated after visit summary: a randomized clinical trial. J Am Board Fam
Med. 2014;27(2):209–218.
10. Sawyer TG, McBroom K, Granger B, Bride W, Harper M. 28. Making the
difference: a shared position to address patients understanding of discharge instructions and post discharge adherence. Heart Lung. 2011;
40(4):386.
11. Engel KG, Buckley BA, Forth VE, et al. Patient understanding of emergency
department discharge instructions: where are knowledge deficits greatest?
Acad Emerg Med. 2012;19(9):E1035–E1044
12. Dartmouth Atlas Project. The Revolving Door: A report on U.S. hospital readmissions. Healthcare Purchasing News. 2013;37(4):10.
13. Choi J. Development and pilot test of pictograph-enhanced breast healthcare instructions for community-residing immigrant women. Int J Nursing
Practice. 2012;18(4):373–378.
14. Houts PS, Doak CC, Doak LG, Loscalzo MJ. The role of pictures in improving
health communication: a review of research on attention, comprehension,
recall, and adherence. Patient Educ Counsel. 2006;61(2):173–190.
15. Jeungok C. Pictograph-based discharge instructions for low-literate older
adults after hip replacement surgery. J Gerontol Nursing. 2011;37(11):47–56.
16. Mayer GG, Villaire M. Health Literacy in Primary Care: A Clinician’s Guide.
New York, NY, USA: Springer Publishing Company; 2007.
17. Wong-Baker Faces Foundation. 1983. http://www.wongbakerfaces.org.
Accessed April 26, 2013.
18. Austin PE, Matlack R, Dunn KA, Kesler C, Brown CK. Discharge instructions:
do illustrations help our patients understand them? Ann Emer Med.
1995;25(3):317–320.
1141
RESEARCH AND APPLICATIONS
diverse patient populations would reveal more about the utility of
pictograph-enhanced discharge instructions for patients with less education and lower health literacy as well as individuals from racial and
ethnic minority groups, who may interpret pictographs in dramatically
different ways. Another limitation is that, because comprehension is
the translation of information into meaningful ideas, recall is not
equivalent to comprehension. Similarly, comprehension does not
always translate to adherence or satisfaction. On the other hand, although perfect recall of instructions is not necessary for adherence to
those instructions, a very low level of recall is likely to hinder
adherence.
Even with the inclusion of illustrations, most patients’ immediate
recall rate of their hospital discharge instructions was relatively low.
As mentioned previously, in hospital discharge instructions, information that is of immediate clinical significance is mixed with advice
about long-term behavior changes and general patient education. To
make discharge instructions more comprehensible, memorable, and
usable, we must improve the content, presentation, formatting, and
delivery of discharge instructions.
Hill B, et al. J Am Med Inform Assoc 2016;23:1136–1142. doi:10.1093/jamia/ocw019, Research and Applications
RESEARCH AND APPLICATIONS
19. Cherry KE, Dokey DK, Reese CM, Brigman S. Pictorial illustrations enhance
memory for sentences in younger and older adults. Exp Aging Res.
2003;29(3):353–370.
20. Choi J. Literature review: using pictographs in discharge instructions for
older adults with low-literacy skills. J Clin Nursing. 2011;20(21-22):
2984–2996.
21. Choi J. Improving discharge education using pictographs. Rehabilitation
Nursing. 2013;38(5):240–246.
22. Dowse R, Ehlers M. Pictograms for conveying medicine instructions: comprehension in various South African language groups. South African J Sci.
2004;(100):687–693.
23. Hickman MS, White WL, White WA. Illustrations as a patient education tool
to improve recall of postoperative cataract medication regimens in the developing world. Hawaii Med J. 2010;69(9):212–215.
24. Katz MG, Kripalani S, Weiss BD. Use of pictorial aids in medication instructions: a review of the literature. Am J Health-System Pharm. 2006;63(23):
2391–2397.
25. Kools M, van de Wiel MWJ, Ruiter RAC, Kok G. Pictures and text in instructions for medical devices: Effects on recall and actual performance. Patient
Educ Counsel. 2006;64(1–3):104–111.
26. Zikmund-Fisher BJ, Witteman HO, Dickson M, et al. Blocks, ovals, or
people? Icon type affects risk perceptions and recall of pictographs. Med
Decis Mak. 2013;34:443–453.
27. Bui D, Nakamura C, Bray BE, Zeng-Treitler Q. Automated illustration of patients instructions. AMIA Annual Symposium Proceedings/AMIA Symposium
AMIA Symposium. 2012;2012:1158-1167.
28. Zeng-Treitler Q, Perri S, Nakamura C, et al. Evaluation of a pictograph enhancement system for patient instruction: a recall study. J Am Med Inform
Assoc. 2014;21(6):1026–1031.
29. University of Utah Healthcare, ND. http://healthcare.utah.edu/about.
Accessed September 3, 2015.
30. University of Utah School of Medicine Division of Cardiovascular Medicine, ND.
http://medicine.utah.edu/internalmedicine/cardiology/index.php.
Accessed
September 3, 2015.
31. Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: a flexible statistical
power analysis program for the social, behavioral, and biomedical sciences.
Behav Res Methods. 2007;41:1149–1160.
32. Kim H, Nakamura C, Zeng-Treitler Q. Assessment of Pictographs Developed
Through a Participatory Design Process Using an Online Survey Tool.
Eysenbach G, ed. Journal of Medical Internet Research. 2009;11(1): e5.
33. Paivio A, Rogers TB, Smythe PC. Why are pictures easier to recall than
words? Psychonomic Sci. 1968;11(4):137–138.
34. Paivio A. Mental Representations: A Dual Coding Approach. New York:
Oxford University Press; 1986.
35. Paivio A, Csapo K. Picture superiority in free recall: imagery or dual coding?
Cognitive Psychol. 1973;5(2):176–206.
36. Paivio A. Dual coding theory: retrospect and current status. Canadian
J Psychology/Revue canadienne de psychologie. 1991;45(3):255–287.
37. Thomas N. 2014. http://plato.stanford.edu/entries/mental-imagery/theories-memory.html. Accessed April 27, 2013.
38. Sanderson BK, Thompson J, Brown TM, Tucker MJ, Bittner V. Assessing patient recall of discharge instructions for acute myocardial infarction.
J Healthcare Qual. 2009;31(6):25–34.
39. Delp C, Jones J. Communicating information to patients: the use of cartoon
illustrations to improve comprehension of instructions. Acad Emerg Med.
1996;3(3):264–270.
40. Houts PS, Bachrach R, Witmer JT, et al. Using pictographs to enhance recall
of spoken medical instructions. Patient Educ Counsel. 1998;35(2):83–88.
41. Houts PS, Witmer JT, Egeth HE, Loscalzo MJ, Zabora JR. Using pictographs
to enhance recall of spoken medical instructions II. Patient Educ Counsel.
2001;43(3):231–242.
42. Jolly BT, Scott JL, Sanford SM. Simplification of emergency department discharge instructions improves patient comprehension. Ann Emerg Med.
1995;26(4):443–446.
43. Zeng-Treitler Q, Kim H, Hunter M. Improving patient comprehension and recall of discharge instructions by supplementing free texts with pictographs.
AMIA Annual Symposium Proceedings/AMIA Symposium AMIA Symposium.
2008:849–853.
44. Jansen J, van Weert J, van der Meulen N, van Dulmen S, Heeren T, Bensing
J. Recall in older cancer patients: measuring memory for medical information. Gerontologist. 2008;48(2):149–157.
45. Clarke C, Friedman SM, Shi K, Arenovich T, Monzon J, Culligan C.
Emergency department discharge instructions comprehension and compliance study. Can J Emerg Med. 2005;7(1):5–11.
46. Perez-Carceles MD, Gironda JL, Osuna E, Falcon M, Luna A. Is the right to
information fulfilled in an emergency department? Patients’ perceptions of
the care provided. J Eval Clin Pract. 2010;16(3):456–463.
47. Topacoglu H, Karcioglu O, Ozucelik N, et al. Analysis of factors affecting satisfaction in the emergency department: a survey of 1019 patients. Adv Ther.
2004;21(6):380–388.
48. Huang J-A, Lai C-S, Tsai W-C, Weng R-H, Hu W-H, Yang D-Y. Determining
factors of patient satisfaction for frequent users of emergency services in a
medical center. J Chinese Med Assoc. 2004;67(8):403–410.
49. Lo S, Stuenkel DL, Rodriguez L. The impact of diagnosis-specific discharge
instructions on patient satisfaction. J PeriAnesthesia Nurs. 2009;24(3):
156–162.
50. Clark PA, Drain M, Gesell SB, et al. Patient perceptions of quality in discharge instruction. Patient Educ Counsel. 2005;59(1):56–68.
51. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research
electronic data capture (REDCap) – a metadata-driven methodology and
workflow process for providing translational research informatics support.
J Biomed Inform. 2009;42(2):377–381.
52. McCray AT. Promoting health literacy. J Am Med Inform Assoc.
2005;12(2):152–163.
53. Engel KG, Heisler M, Smith DM, Robinson CH, Forman JH, Ubel PA. Patient
comprehension of emergency department care and instructions: are patients aware of when they do not understand? Ann Emerg Med.
2009;53(4):454–461.e15.
54. Kadakia RJ, Tsahakis JM, Issar NM, et al. Health literacy in an orthopedic
trauma patient population: a cross-sectional survey of patient comprehension. J Orthopaedic Trauma. 2013;27(8):467–471.
55. Ginde AA, Weiner SG, Pallin DJ, Camargo CA Jr. Multicenter study of limited
health literacy in emergency department patients. Acad Emerg Med
2008;15(6):577–580.
56. Bureau USC. State and County Quick Facts. 2015. http://quickfacts.census.
gov/qfd/states/49000.html. Accessed September 8, 2015.
AUTHOR AFFILIATIONS
....................................................................................................................................................
1
Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah,
USA
2
Harvard School of Medicine, Harvard University, Boston, MA, USA
1142