International Journal for Quality in Health Care 2012; Volume 24, Number 1: pp. 55 –64
Advance Access Publication: 13 December 2011
10.1093/intqhc/mzr072
Impact of format and content of visual
display of data on comprehension, choice
and preference: a systematic review
ZOE HILDON, DOMINIQUE ALLWOOD AND NICK BLACK
Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK
Address reprint requests to: Zoe Hildon, Department of Health Services Research and Policy, London School of Hygiene and Tropical
Medicine, 15 –17 Tavistock Place, London WC1H 9SH, UK. Tel: þ44-20 7927 2151; Mob: þ7722360021; Fax: þ44-20 7927 2701.
E-mail: [email protected]
Accepted for publication 3 November 2011
Abstract
Purpose. Displays comparing the performance of healthcare providers are largely based on commonsense. To review the
literature on the impact of compositional format and content of quantitative data displays on people’s comprehension, choice
and preference.
Data sources. Ovid databases, expert recommendations and snowballing techniques.
Study selection. Evaluations of the impact of different formats (bar charts, tables and pictographs) and content (ordering,
explanatory visual cues, etc.) of quantitative data displays meeting defined quality criteria.
Data extraction. Type of decision; decision-making domains; audiences; formats; content; methodology; findings.
Results of data synthesis. Most of the 30 studies used quantitative (n ¼ 26) methods with patients or public groups (n ¼ 28)
rather than with professionals (n ¼ 2). Bar charts were the most frequent format, followed by pictographs and tables.
As regards format, tables and pictographs appeared better understood than bar charts despite the latter being preferred.
Although accessible to less numerate and older populations, pictographs tended to lead to more risk avoidance. Tables
appeared accessible to all. Aspects of content enhancing the impact of data displays included giving visual explanatory cues
and contextual information while still attempting simplicity (‘less is more’); ordering data; consistency. Icons rather than
numbers were more user-friendly but could lead to over-estimation of risk. Uncertainty was not widely understood, nor well
represented.
Conclusions. Though heterogeneous and limited in scope, there is sufficient research evidence to inform the presentation of
quantitative data that compares the performance of healthcare providers. The impact of new formats, such as funnel plots,
needs to be evaluated.
Keywords: quality improvement, patient– provider communication/information, decision analysis
Purpose
Data displays of quality and risk in health care are being
adopted largely on the basis of commonsense rather than on
research on presentational methods. Yet, information presentation has been examined within several fields including psychology, market research and information systems/management.
However, studies have mostly focused on framing effects, such
as relative vs. absolute risk [1–3]. These have shown that
people will be persuaded more readily by relative rather than
absolute risk information [1, 3, 4] and by positively framed outcomes (e.g. survival rather than mortality data) or loss framing
(e.g. years lost rather than gained) [1, 2, 4].
Another focus of presentational methods has been examining verbal (e.g. ‘probably’) compared with numeric (80%)
terms for risk; it is widely acknowledged that words may
better capture intuitive perceptions of risk, but can suffer
from inconsistency in interpretation [5]. They may also lead
to overestimations of risk [6, 7] or more risk-avoidant
choices [7, 8].
Communication tools have also been examined in the
context of health interventions, which have found that
certain types of information seem to improve decisionmaking, e.g. individualized health risk scores [9]; as can data
summation, e.g. verbal risk summaries [10] and the use of
charts in the consultation process [11]. Studies on
International Journal for Quality in Health Care vol. 24 no. 1
# The Author 2011. Published by Oxford University Press in association with the International Society for Quality in Health Care;
all rights reserved
55
Hildon et al.
information systems [12 – 14] comparing tables and graphs
have proved inconclusive and seem to depend on the complexity of the task (graphs favoured in complex tasks).
An area that has received less attention and is the focus of
this review is making data more meaningful and reducing the
mental computational load so that automatic visual perception helps decision-making [14, 15]. This requires a distinction to be made between compositional formats and content.
Format refers to the way data are housed and includes
options such as bar charts, tables (numeric and non-numeric)
and pictographs. Content includes aspects such as whether
to rank order data, or whether to use explanatory visual cues
(such as þ or 2 symbols) to enhance statistics [15, 16].
When considering the impact of data displays, three
decision-making domains have been identified in the literature [14]: people’s ‘comprehension’ (or interpretation) of the
data; the way displays affect’s their ‘choice’ (either hypothetically or their behaviour in practice) and people’s ‘preference’
(or liking) for one display over another. Our aim was to
review the research evidence on the impact of compositional
format and content of quantitative data displays across these
three domains.
Methods
Data sources
Eight databases hosted by the Ovid interface were searched:
Cochrane, EMBASE, Medline, Global Health, Health
Management Information Consortium, ERIC, PsychInfo and
PsychExtra. Recommendations for grey literature or for literature currently in press were requested from experts.
Snowballing [17, 18] was used to ensure that literature which
might have been missed by the electronic searches was captured; this process included following-up reference lists of
review articles that met our inclusion criteria. We checked to
ensure that in-text citations had reached saturation.
The search string combined general terms for metrics
(MESH or thesaurus terms) or specific terms for metrics of
interest (such as performance indicators, patient-reported
outcomes) with decision-making terms; these were in turn
joined to (MESH or thesaurus and text) terms for data
display methods. The search string was: (metrics OR
decision-making terms) AND (data display terms).
Study selection
Screening was carried out on titles, abstracts and full-texts by
Z.H. The key inclusion criterion was that studies examined
quantitative data displays and the impact of different compositional format and/or content.
The following exclusion criteria were applied at all stages
of screening: selection and testing of specific metrics, including testing framing effects; examining variations in verbal vs.
numerical expressions; studies of decision-support software,
for which an existing review [12] was used; exploring
complex decision tasks, i.e. interpretation of statistical
56
predictions, rather than (implicit or explicit) simple binary or
categorical decisions, e.g. treatment.
Data extraction
The quality of studies was appraised using a tool based on
two reviews of methods for rating quality across quantitative,
qualitative and mixed methods [19, 20]. The Quality
Appraisal across Study Methods (QASM) had five domains:
reporting; appropriateness of study design; validity of data
and rigour of analyses; reliability of findings; use of methodspecific quality enhancing procedures (Supplementary material, Appendix S1). The domains were tailored for the four
main methodologies: qualitative; quantitative descriptive;
quantitative comparative and mixed methods. Each study
was scored on a 10-point scale with results categorized as
weak* (0 – 4), moderate** (5 – 7) and strong*** (8 – 10).
For each study, the following information was extracted by
team members (Z.H., D.A.) using a template: the decision
that the visual displays were intended to inform; the population groups they were tested on; compositional formats and
content; study method; impacts considered (comprehension,
choice, preference) and findings (results and conclusions).
Discrepancies around interpretation of study findings were
discussed and resolved.
The results of studies were synthesized according to a
framework that considered separately the impact of compositional format and of content on comprehension, choice and
preference. For quantitative studies, findings were considered
if they reached statistical significance at the level of P , 0.05.
When the level of significance was not reported, pertinent
findings were extracted. For qualitative and mixed methods
analyses, reported themes and study findings were also
extracted.
Results of data synthesis
Search and screening results
Searching identified 1462 texts (after manual removal of
duplicates) for screening (Fig. 1). Screening yielded 486 inclusions at title screen, 82 at abstract screen and 33 at full-text
screen.
The majority of the studies (n ¼ 28) used quantitative
methods (25 comparative and 3 descriptive) and the rest
used either qualitative (n ¼ 2) or mixed methods (n ¼ 3).
Most studies were of moderate quality (58%), a third (33%)
were strong, and three (9%) were weak (which were excluded
from the synthesis). Of the studies rated as strong, seven
used quantitative comparative designs, two used mixed
methods and two were qualitative.
Description of included studies
Topics. The 30 studies considered a wide range of
decisions. All were hypothetical but used realistic data. Only
two studies focused on decision-making by professionals:
Impact of format and content †
Figure 1 Screening data flow.
Public reporting, quality improvement
one asked clinicians whether to stop a trial early or not [21],
and one required commissioners to identify poorly
performing hospitals [22]. Two studies considered business
strategy—selection of a restaurant site [23, 24]. A few asked
the public or patients about selection of a healthcare plan,
hospital or nursing home (n ¼ 7); or other consumer
decisions such as which tyres, toothpaste or cereal to chose
(n ¼ 6). But mostly they examined individual treatment and
risk-related decisions, e.g. whether to take drug A or B, or
consent to treatment, etc. (n ¼ 13).
Audiences. Most studies tested displays on the public (n ¼ 15).
The rest were on convenience samples of students (n ¼ 9),
patients (n ¼ 4), clinicians (n ¼ 1) or commissioners (n ¼ 1).
Some studies considered the socio-demographic characteristics
of their audiences: age [25–31], sex [32–35] and levels of
numeracy [30, 31, 36].
Compositional formats. There were three principal formats
considered: bar charts (n ¼ 19), pictographs (n ¼ 13) and
(numerical and non-numerical) tables (n ¼ 9). Consistent
with the literature, a matrix-housing icons such as star ratings
instead of or alongside a numerical statistic is categorized as
a table while pictographs denote a visual representation of a
proportion, e.g. 25 smiley faces to denote 25%. These
formats incorporated either single or multiple outcomes. In
graphs, multiple outcomes can be grouped (along the x axis
of a bar chart) or panelled (two charts displayed side by side;
Figs 2 and 3). Tables list either multiple indicators (referred
to as scorecards) or only one (Fig. 4).
A few studies examined other formats: line graphs (n ¼ 5),
pie charts (n ¼ 5) and scatter plots (n ¼ 1) but they were
rarely compared with other formats. In addition, comparisons
were also made with narrative reports using statistics (n ¼ 9).
Content. Five aspects of content were identified: ordering
(n ¼ 6); visual explanatory cues (n ¼ 3); amount of content
(n ¼ 6); instructive aids (n ¼ 4) and representation of
uncertainty (n ¼ 4).
Impact domains. Comprehension (n ¼ 19) had been studied
in terms of accuracy of understanding, numeracy, response
Figure 2 Bar charts (either vertical or horizontal).
57
Hildon et al.
Figure 3 Pictographs housed a range of icons, e.g. simple
blocks, stick figures, etc.
Figure 4 Tables listed cases and indicators either in columns
or rows. Contained numerical and non-numerical data.
time and memory. Preference (n ¼ 14) had been tested in
terms of usability and formatting. Choice (n ¼ 19) was tested
in terms of confidence in choice, influences on choice,
choice processes and making the (statistically) accurate
choice. This last way of examining choice, clearly overlaps
with, and is a mechanism for, testing understanding.
Impact of compositional format
Literature examining compositional format is summarized in
Table 1. Tables and pictographs seemed generally better
understood than bar charts. The study on clinician’s decisionmaking found pictographs were the best understood; 82%
made correct decisions with pictographs compared with 68%
when using tables and only 43% for bar charts [21].
Nevertheless tables were (62%) preferred over the bar chart
(23%) with no one liking the pictograph and some clinicians
actually contemptuous of that format [21]. Despite preferring
bar charts, commissioners using this format were more likely
to make incorrect choices when selecting supposedly underperforming hospitals than with scatter plots [22].
58
A study of the public’s use of quality information for
selecting a nursing home also found that bar charts were less
likely to be understood; 86% of participants correctly interpreted non-numeric tables (housing icons) compared with
47% for bar charts; 46% preferred tables and 22% preferred
graphs [37].
In business strategizing, it was found that bar charts and
line graphs were harder to interpret for people with poorer
perception ability compared with tables or narrative reporting
[23]. Only one study [26] found that vertical bar charts narrowly outperformed pictographs (same error rate of 1.1% but
slower to process) and were better understood than pie charts
and horizontal bar charts, but this study did not test tables.
Comprehension of pictographs was found to be generally
good [27, 28, 31, 38]. For instance, in a task for estimating
chance of survival using this format 86% of responses were
correct [38]. In older people correct responses ranged from
70 to 98% using pictographs, higher than numeric statements (38– 78%) [27]. Apart from simplifying data for older
or less numerate populations, preference for pictographs was
also found to be shaped by numeracy [33, 34]. However, one
study found both pictographs and tables were liked—rated
as effective, trustworthy and scientific—irrespective of level
of numeracy [31].
Pictographs have also been found to increase people’s perception of risk. Younger, less educated people perceived
their life-time risk of breast cancer to be higher when conveyed with pictographs (human figures) than bar charts [34];
only 28% perceived their risk to be low with bar charts compared with 49% with pictographs [35].
Similarly, several studies found that displaying risk reduction using a pictograph rather than by numbers significantly
increased risk-avoidant choices [39, 40, 41]. Although it has
been suggested this effect might be attributable to framing
[39], a later study found the effect to be independent of
framing [42].
Even though bar charts seemed the least understood,
several studies have found people tend to prefer them to
other formats; one study explicitly identified this paradox
[22]. Bar charts seemed preferred particularly for health risk
information. One such study found 60% of people liked the
bar chart compared with 20% the pictographs [33]. In
another, bar charts were favoured over pictographs for
multiple-risk estimates (22 vs. 60%), although the opposite
was true when showing single estimates (46 vs. 28%) [35]. A
study of diabetes treatment options reported even greater
support for bar charts (97%) [43].
Impact of content
Ordering. Literature examining content derivatives is
summarized in Table 2. In presenting consumer product
choices, comparing by rank order improves comprehension
and choice. Ranking health plans in tables improved
comprehension, particularly in older groups, made options
easier to identify and significantly reduced poor decisions from
46 to 30% [15]. Twice as many non-quality maximizing choices
Impact of format and content †
Public reporting, quality improvement
Table 1 Impact of compositional format on comprehension, preference and choice
Comprehension
Choice
Preference
.............................................................................................................................................................................
Bar chart
Pictograph
Bar charts not so easy to understand
Bar charts could lead to
inaccurate choices
Preference was for bar
charts above other formats
Brown [23]**
Elting et al. [21]***
Gerteis et al. [37]***
Marshall et al. [22]**
Elting et al. [21]***
Marshall et al. [22]**
Edwards et al. [43]***
Fortin et al [33]***
Marshall et al. [22]**
Vertical bar charts led to best
understanding compared with
horizontal or pie charts
Preference for bars when
showing multiple risk
Feldman-Stewart et al. [26]**
Schapira et al. [35]**
Pictographs easy to understand
Pictographs led to more
risk-avoidant choices compared
with numbers
People favourably rated
pictographs
Elting et al. [21]***
Fuller et al. [27]**, [28]**
Hawley et al. [31]***†
Price et al. [38]**
Chua et al. [42]**
Schirillo and Stone [41]**
Stone et al. [39]***
Stone et al. [40]**
Hawley et al. [31]***
Pictographs could lead to
overestimations of harm compared
with bar charts
Preference for pictographs
was shaped by numeracy
Schapira et al. [34]***
Schapira et al. [35]**
Fortin et al. [33]***
Schapira et al. [34]***
Preference for pictographs
when showing single risk
Schapira et al. [35]**
Table (numeric –
non-numeric)
Tables housing icons, words or
numbers were sometimes easier to
understand than bar charts
Tables led to more accurate
choices than graphs for people
with poorer perception ability
People reacted favourably to
tables, or preferred them
over other formats
Brown [23]**
Gerteis et al. [37]***
Elting et al. [21]***
Brown [23]**
Elting et al. [21]***
Gerteis et al. [37]***
Hawley et al. [ 31]***
Quality rating: Weak* (0 –4), Moderate** (5 –7) and Strong*** (8 –10).
occurred in an unordered matrix as with ranked information
[16].
As regards acquisition of information, one study noted
that people acquire information in line with the way it is
arranged and adopt an approach to processing it that minimizes cognitive effort and errors, aiding decision quality [24].
In tables, attention seems affected by the importance
assigned to different factors, whereas in bar charts it is by
the height of the bar. However, one study concluded that
consumers (of breakfast cereals) appeared to process information in the fashion which is visually easiest from the table
they were given [32]. Another noted that choice (of
59
Comprehension
Choice
Preference
............................................................................................................................................................................................................................................
Ordering
Visual explanatory
cues
Information is acquired in sequence, either by rank or by
relevancy of information, which seems to make ordered
information easier/faster to understand
Ordering influences choice, helping
Not examined
people to make ‘right’ choices, or can be
used to ‘nudge’ a selection
Bettman and Kakkar [32]**
Hibbard et al. [15]**
Hibbard et al. [16]**
Jarvenpaa et al. [24]**
Hibbard et al. [15]**, Hibbard et al.
[16]**
Jarvenpaa [24]**
Although recall is brand led even if information is
presented by attribute
Biehal et al. [44]**
Mazur and Merz [45]**
Fasolo et al. [30]***
Using visual cues (icons, words) helped to clarify
the meaning of data
Such visual cues also helped towards
making the ‘right’ choice
People liked being helped with the process
of bringing information together; and liked
consistency
Hibbard et al. [16]**
Gerteis et al. [37]***
Hibbard et al. [16]**
Gerteis et al. [37]***
Fasolo et al. [30]***
Gerteis et al. [37]***
Amount of content Simpler graphics were more understandable
Instructive aids
Simpler graphics encouraged the accurate Preference was for simpler graphics; but
choice
also showing information in a variety of
ways, to match knowledge/use
Edwards et al. [43]***
Feldman-Stewart et al. [25]**
Peters et al. [36]***
Uhrig et al. [29]***
Peters et al. [36]***
Uhrig et al. [29]***
Edwards et al. [43]***
Fasolo et al. [30]***
Schapira et al. [34]***
Instructive aids such as practice exercises or worksheets
helped comprehension
Not examined
Examples of risk in everyday life to
illustrate risk statistics unwanted and
patronizing
Armstrong et al. [47]**
Rakow et al. [46]**
Uhrig et al. [29]***
Edwards et al. [43]***
Hildon et al.
60
Table 2 Impact of content on comprehension, preference and choice
Quality rating: Weak* (0– 4), Moderate** (5 –7) and Strong*** (8– 10).
Fasolo et al. [30]***
Gerteis et al. [37]***
Marshall et al. [22]**
Marshall et al. [22]**
Mazor et al. [48]***
Missing data was not understood and negatively interpreted
Using confidence intervals did not
influence making the accurate choices
Representation of
uncertainty
Confidence intervals did not increase understanding of
uncertainty
Not examined
Impact of format and content †
Public reporting, quality improvement
toothpaste) seems to be brand, rather than attribute led; but
that information was remembered by brand even if it had
been presented by attribute [44].
Nevertheless, order was also found to influence choice.
For instance, the order in which a series of survival
curves was shown was a significant predictor of patients
‘decision to select treatment’ (varying from 21 to 47%)
[45]. It was also found that swapping the order of indicators in a hospital scorecard could sway participants to
give more value to indicators that were listed first, or to
‘nudge’ patients to choose particular aspects of quality
over others [30].
Using visual (non-numeric) explanatory cues. Several methods
have been used to clarify data presentation including the
addition of icons (often stars) or word labels (e.g. good,
average); categorizing data according to a colour code (e.g.
traffic lights) and or adding symbols (þ or – signs) to
indicate the direction of a trend. One study found that 75%
compared with 61% of participants made a more accurate
choice if there were additional symbols [16]. Tables
containing explanatory cues were more often interpreted
correctly compared with bar charts without any [37]. People
not only liked being helped with the process of bringing the
data together but also welcomed consistency [30, 37].
Amount of content. Too much content can be confusing
resulting in ‘information overload’ [43] and, although
accuracy of choice was not necessarily affected, information
processing was slower [25]. Selecting the highest quality
product available favours the simplest format: (62% chose a
high-quality hospital when only quality information was
shown, suggesting ‘less is more in presenting quality
information to consumers’ [36]. Similarly when health plan
information was simplified, understanding was significantly
increased as was the accuracy of choice [29].
In terms of preferences, although simplicity is appreciated,
people recognize different levels of information should be
available [43]. Similarly, people acknowledge that more
complex displays are justified by the need for relaying more
complex information [34]. However, presenting many indicators side-by-side was not welcomed by consumers [30].
Instructive aids. Giving instruction can improve
comprehension [46, 47]. Understanding of survival curves was
improved from 74 to 83% by the use of a practice exercise,
although worked examples only helped with simpler graphics
[47]. In another study, simplified health plans that came with a
worksheet outshone the comparator [29]. However, it is not
always the case that instructive aids are welcomed. Offering
examples of risk in everyday life to illustrate risk statistics was
unwanted and seen as patronizing [43].
Representation of uncertainty. Confidence intervals are often
poorly understood not only by the public but by some
professionals. A study of the public’s understanding of hospital
quality demonstrated the inclusion confidence intervals to a bar
chart had little effect on understanding of performance—23%
perceived two hospitals as ‘about the same’ compared with
21% when no confidence interval was provided [48]. The study
on commissioners also suggested a lack of understanding of
the role of confidence intervals in a bar chart [22]. A related
61
Hildon et al.
issue, missing data, rather than being viewed as an ‘unknown’,
tended to be negatively interpreted [30, 37].
Conclusion
Main findings
As regards compositional format, tables and pictographs
were judged to be more accurate decision aids than bar
charts. It was also noted that over-estimation and avoidance
of risk can occur with the use of icons. This data display
method may be more appropriate for older and less numerate populations. Tables appeared accessible to all though for
complex data, bar charts may be more appropriate.
Studies of decision-support software have indicated that
numerical tables are more effective than graphs unless the
task demand is high [49] but this debate has not been
resolved [12, 13, 50]. When the decision is a simple binary or
categorical choice, formats other than bar charts may be the
most appropriate. One reason for the bar chart’s underperformance may be due to a lack of understanding and poor
representation of uncertainty (confidence intervals) [22].
With bar charts, choice may be driven by rank ordering
[15, 16, 24, 32], rather than whether options are within or
outside of normal variation. No studies have tested the interaction between representing uncertainty and ordering, and
only one study has considered ordering effects in bar charts
[24]. Because of this potential caveat, other graphical formats
such as funnel or scatter plots with clearly identified control
limits are likely to be more appropriate for comprehension
and decision accuracy in simple decision tasks.
As regards content several ways of improving displays
emerged. Including consistency in symbols, colours and
metrics; careful selection of visual cues to help clarify the
meaning of data; yet avoidance of over-elaboration; clear
instructions or even practice tasks, on how to interpret displays ( particularly for less familiar formats); ordering by rank
and relevance of information; clear accounts of uncertainty,
particularly with bar charts—especially if data are contingent
on significant associations.
Strengths and limitations
While there were a substantial number of studies of moderate
or strong methodological quality, the heterogeneity of
included studies precluded quantitative synthesis. Instead we
sought to identify important themes in a narrative review. We
were limited by the tendency to test popular formats, which
ignored other displays (e.g. funnel and caterpillar plots)
useful for comparing performance of providers. All the
studies tested hypothetical decisions, which are known to
differ from actual behaviour change.
Implications
Despite the heterogeneity of the studies, we were able to synthesize research on a wide range of topics and using various
62
methodologies. Common findings emerged with considerable
consistency. This suggests that the nature of data processing
is less dependent on the type of information displayed and
more dependent on ‘how’ it is displayed.
In terms of the framework that was applied (comprehension,
choice, preference), there is some overlap between the first two
constructs. All three are multi-faceted, hinging on specific contexts, ability and frame of reference. Comprehension and
choice should be prioritized over preference. Whilst this may
not favour the most popular displays at the outset, people are
willing to alter their preference if given a good reason [51].
Future research to improve our understanding of how
best to convey data that compares performance should
include less used formats, such as funnel or scatter plots;
and should examine visual displays both with public and professional audiences, something that none of the studies that
met our selection criteria attempted.
Supplementary material
Supplementary material is available at INTQHC Journal online.
Acknowledgements
We thank: the Department of Health for funding; Judith
Green and Andrew Hutchings for help in developing and
refining QASM; library staff at the London School of
Hygiene and Tropical Medicine for their support with sourcing literature; Jenny Neuberger and Jan van der Meulen for
comments; Elena Kombou for database maintenance and
contributions to extractions and Rebecca Wright and Tim
Rakow for their comments and recommendations.
Funding
This study was funded by the Department of Health for
England.
References
1. Edwards A, Elwyn G, Covey J et al. Presenting risk information—a review of effects of framing and other manipulations
on patients outcomes. J Health Commun 2001;6:61 –82.
2. Kühberger A. The influence of framing on risky decisions: a
meta-analysis. Organ Behav Hum Decis Process 1998;75:23–55.
3. McGettigan P, Sly K, O’Connell D et al. The effect of information framing on practices of physicians. J Gen Intern Med
1999;14:633–42.
4. Edwards A, Elwyn G, Mulley A. Explaining risks: turning numerical data into meaningful pictures. BMJ 2002;324:827 –30.
5. Lipkus I. Numeric, verbal and visual formats of conveying
health risks: suggested best practices and future recommendations. Med Decis Making 2007;27:696– 713.
Impact of format and content †
6. Hilton NZ, Carter AM, Harris GT et al. Does using nonnumerical terms to describe risk aid violence risk communication? Clinician agreement and decision making. J Interpers
Violence 2008;23:171 –88.
7. Knapp P, Raynor DK, Berry DC. Comparison of two methods
of presenting risk information to patients about the side effects
of medicines. Qual Saf Health Care 2004;13:176 –80.
8. Mazur DJ, Hickam DH, Mazur MD. How patients’ preferences
for risk information influence treatment choice in a case of
high risk and high therapeutic uncertainty: asymptomatic localized prostate cancer. Med Decis Making 1999;19:394 –8.
9. Edwards A, Hood K, Matthews E et al. The effectiveness of
one-to-one risk-communication interventions in health care: a
systematic review. Med Decis Making 2000;20:290– 7.
10. Fetting JH, Siminoff LA, Piantadosi S et al. Effects of patients’
expectations of benefit with standard breast cancer adjunctive
chemotherapy on participation in randomised clinical trial: a
clinical vignette study. J Clin Oncol 1990;8:1476 – 82.
11. Edwards A, Elwyn G, Gwyn R. General practice registrar response to use of different risk communication tools in simulated consultation: a focus group study. BMJ 1999;319:749–52.
12. Schaubroeck J, Muralidhar A. A meta-analysis of the relative
effects of tabular and graphic display formats on decisionmaking performance. Human Perform 1991;4:127– 45.
13. Montazemi AR. Comments on ‘a meta-analysis of the relative
effects of tabular and graphic display formats on decisionmaking performance’. Human Perform 1991;4:147 –53.
14. Ancker JS, Senathirajah Y, Kukafka R et al. Design features of
graphs in health risk communication: a systematic review. J Am
Med Inform Assoc 2006;13:608 –18.
15. Hibbard JH, Peters E, Slovic P et al. Making health care quality
reports easier to use. Jt Comm J Qual Improv 2001;27:591 –604.
16. Hibbard JH, Slovic P, Peters E et al. Strategies for reporting
health plan performance information to consumers: evidence
from controlled studies. HSR 2002;37:291–313.
17. Greenhalgh T, Peacock R. Information in practise: effectiveness
and efficiency of search methods in systematic reviews of
complex evidence: audit of primary sources. BMJ
2005;331:1064 –5.
18. Garrard J. Health Sciences Literature Review Made Easy: The Matrix
Method. London, Mississauga: Joes & Bartlett, 2007.
19. Pluye P, Gagnonb M-P, Griffithsc F et al. A Scoring system for
appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies
in mixed studies reviews. Int J Nurs Stud 2009;46:529– 46.
Public reporting, quality improvement
23. Brown PE. The relationship between graphic aids in a business
report and decision-making and cognitive style of a report
reader. Delta Pi Epsilon J 1992;34:63–76.
24. Jarvenpaa SL. The effects of task demands and graphical
format on information-processing strategies. Manag Sci
1989;35:285 –303.
25. Feldman-Stewart D, Kocovski N, McConnell BA et al.
Perception of quantitative information for treatment decisions.
Med Decis Making 2000;20:228 –38.
26. Feldman-Stewart D, Brundage MD, Zotov V. Further insight
into the perception of quantitative information: judgments of
gist in treatment decisions. Med Decis Making 2007;27:34– 43.
27. Fuller R, Dudley N, Blacktop J. Risk communication and older
people-understanding of probability and risk information by
medical inpatients aged 75 years and older. Age Ageing
2001;30:473 –6.
28. Fuller R, Dudley N, Blacktop J. How informed is consent?
Understanding of pictorial and verbal probability information
by medical inpatients. Postgrad Med J 2002;78:543 –4.
29. Uhrig JD, Harris-Kojetin L, Bann C et al. Do content and
format affect older consumers’ use of comparative information
in a medicare health plan choice? Results from a controlled experiment. Med Care Res Rev 2006;63:701– 18.
30. Fasolo B, Reutskaja E, Dixon A et al. Helping patient’s choose:
how to improve the design of comparative scorecards of hospital quality. Patient Educ Couns 2010;78:344– 9.
31. Hawley ST, Zikmund-Fisher B, Ubel P et al. The impact of
the format of graphical presentation on health-related
knowledge and treatment choices. Patient Educ Couns
2008;73:448 –55.
32. Bettman JR, Kakkar P. Effects of information presentation
format on consumer information acquisition strategies.
J Consum Res 1977;3:233 –40.
33. Fortin J, Hirota L, Bond B et al. Identifying patient preferences
for communicating risk estimates: a descriptive pilot study.
BMC Med Inform Decis Mak 2001;1:2.
34. Schapira MM, Nattinger AB, McHorney CA. Frequency or
probability? A qualitative study of risk communication formats
used in health care. Med Decis Making 2001;21:459– 67.
35. Schapira MM, Nattinger AB, McAuliffe TL. The influence of
graphic format on breast cancer risk communication. J Health
Commun 2006;11:569 –82.
36. Peters E, Dieckmann N, Dixon A et al. Less is more in presenting quality information to consumers. Med Care Res Rev
2007;64:169 –90.
20. Katrak P, Bialocerkowski AE, Massy-Westropp N et al. A systematic review of the content of critical appraisal tools. BMC
Med Res Methodol 2004;6:100–1.
37. Gerteis M, Gerteis JS, Newman D et al. Testing consumers’
comprehension of quality measures using alternative reporting
formats. Health Care Financ R 2007;8:31–45.
21. Elting LS, Martin CG, Cantor SB et al. Influence of data display
formats on physician investigators’ decisions to stop clinical
trials: prospective trial with repeated measures. BMJ
1999;318:1527 –31.
38. Price M, Cameron R, Butow P. Communicating risk information: the influence of graphical display format on quantitative
information perception—accuracy, comprehension and preferences. Patient Educ Couns 2007;69:121–8.
22. Marshall T, Mohammed MA, Rouse A. A randomized controlled trial of league tables and control charts as aids to health
service decision-making. Int J Qual Health Care 2004;16:309–15.
39. Stone ER, Sieck WR, Bull BE et al. Foreground: background
salience: explaining the effects of graphical displays on risk
avoidance. Organ Behav Hum Decis Process 2003;90:19 –36.
63
Hildon et al.
40. Stone ER, Yates JF, Parker AM. Effects of numerical and
graphical displays on professed risk-taking behavior. J Exp
Psychol Appl 1997;3:243–56.
41. Schirillo JA, Stone ER. The greater ability of graphical versus
numerical displays to increase risk avoidance involves a
common mechanism. Risk Anal 2005;25:555 –66.
42. Chua HF, Yates JF, Shah P. Risk avoidance: graphs versus
numbers. Mem Cognit 2006;34:399–410.
46. Rakow T, Wright R, Bull K et al. Informing parents of the risks of
pediatric surgery. SPUDM-22, Rovoreto, Italy. 23–27 August 2009.
47. Armstrong K, FitzGerald G, Schwartz JS et al. Using survival
curve comparisons to inform patient decision making—can a
practice exercise improve understanding? J Gen Intern Med
2001;16:482–5.
48. Mazor KM, Dodd KS, Kunches L. Communicating hospital infection data to the public: a study of consumer responses and
preferences. Am J Med Qual 2009;24:108–15.
43. Edwards A, Thomas R, Williams R et al. Presenting risk
information to people with diabetes: evaluating effects and
preferences for different formats by a web-based randomised
controlled trial. Patient Educ Couns 2006;63:336–49.
49. Montazemi AR, Wang S. The effects of modes of information
presentation on decision making: a review and meta-analysis.
J Man Info Sys 1989;5:101 –26.
44. Biehal G, Chakravarti D. Information-presentation format and
learning goals as determinants of consumers’ memory retrieval
and choice processes. J Con Res 1982;4:431–41.
50. Speier C. The influence of information presentation formats on
complex task decision-making performance. Int J Hum Comput
Stud 2006;64:1115 –31.
45. Mazur DJ, Merz JF. How the manner of presentation of data
influences older patients in determining their treatment preferences. J Am Geriatr Soc 1993;41:223 –8.
51. Wallsten TS, Budescu DV, Zwick R et al. Preferences and
reasons for communicating probabilistic information in verbal
or numerical terms. Bull Psychon Soc 1993;31:135–8.
64