PROBABILITY OF
PRECIPITATION
Assessment and Enhancement
of End-User Understanding
by
Susan Joslyn, Limor Nadav-Greenberg, and Rebecca M. Nichols
Three psychological studies show that many people misunderstand traditional probability
of precipitation forecasts and icons, although adding phrases specifying the chance
of “no precipitation” reduces misunderstanding.
P
robability of precipitation (PoP) has appeared
in public forecasts in the United States from the
late 1960s (National Research Council 2006).
Since then however, very few additional probabilistic
forecasts have been made publicly available. This is
in part due to lingering questions about how well the
general public will understand them. Recent evidence
suggests that many people do not understand PoP,
after decades of exposure (Gigerenzer et al. 2005).
The main source of misunderstanding appears to
be the class of events to which the probability refers
(Gigerenzer et al. 2005). Take, for example, a forecast
for “80% chance of precipitation.” According to a
AFFILIATIONS: Joslyn , Nadav-Greenberg ,
and N ichols —University of Washington, Seattle, Washington
CORRESPONDING AUTHOR: Susan Joslyn, Department of
Psychology, University of Washington, Box 351525, Seattle, WA
98195
E-mail: [email protected]
The abstract for this article can be found in this issue, following the
table of contents.
DOI:10.1175/2008BAMS2509.1
In final form 17 June 2008
©2009 American Meteorological Society
AMERICAN METEOROLOGICAL SOCIETY
recent survey, approximately 35% of New Yorkers and
larger percentages of Europeans do not understand
of what it is 80% (Gigerenzer et al. 2005). It is not
surprising that this issue is difficult for the general
public, given that it is debated even within the scientific community (DeElia and Laprise 2005). Some
propose a “frequentist” interpretation: there will be
at least a minimum amount of rain on 80% of days
with weather conditions like they are today (DeElia
and Laprise 2005; Gigerenzer et al. 2005). Although
preferred by many scientists, this explanation may be
particularly difficult for the general public to grasp
because it requires regarding tomorrow as a class of
events, a group of potential tomorrows. From the
perspective of the forecast user, however, tomorrow
will happen only once. A perhaps less abstract interpretation is that PoP reflects the degree of confidence
that the forecaster has that it will rain (DeElia and
Laprise 2005). In other words, an 80% chance of rain
means that the forecaster strongly believes that there
will be at least a minimum amount of rain tomorrow.
The problem, from the perspective of the general
public, is that when PoP is forecasted, none of these
interpretations is specified.
There are clearly some interpretations that are
not correct. The percentage expressed in PoP neither
february 2009
| 185
refers directly to the percent of area over which
precipitation will fall nor does it refer directly to the
percent of time precipitation will be observed on the
forecast day. Although both interpretations are clearly
wrong, there is evidence that the general public holds
them to varying degrees (Gigerenzer et al. 2005).
Such misunderstandings are critical because they
may affect the decisions that people make. If people
misinterpret the forecast as precent time or percent
area, they may be more inclined to take precautionary
action than are those who have the correct probabilistic interpretation, because they think that it will
rain somewhere or some time tomorrow (Gigerenzer
et al. 2005). The negative impact of such misunderstandings on decision making, both in terms of unnecessary precautions as well as erosion in user trust,
could well eliminate any potential benefit of adding
uncertainty information to the forecast.
It is important to understand the psychological
basis of these misunderstandings and how to overcome them because these issues may be crucial to
successful communication of forecast uncertainty
in general. That is the purpose of the experimental
work reported here. The research reported here asks
whether people are indeed missing the probabilistic aspect of the forecast. It also tests whether PoP
forecasts can be improved to clarify the essential
uncertainty involved and contradict the erroneous
proportional interpretations.
One potential enhancement that was tested is
visual imagery. Increasingly, PoP in television and
Web forecasts is accompanied by visual representations, icons with rain and sun imagery. However, as
far as we know, peoples’ interpretation of such icons
Fig. 1. Icons used in experiment 1. (from left to right)
Question mark icon, pie icon, bar icon.
186 |
february 2009
has never been tested systematically. In other words
we do not know whether this imagery actually helps.
The three experiments reported here were designed
to determine whether enhancements including icons
using visual imagery (see Fig. 1) and explicit verbal explanations promote a more accurate understanding of
PoP forecasts. Each experiment approached the question in a slightly different way. Experiment 1 probed
participants’ understanding of the reference class with
multiple-choice questions. Experiment 2 investigated
understanding of PoP by asking participants to provide an explanation in their own words. Experiment
3 compared the icon that led to the fewest errors in
experiment 1 to two verbal descriptions for probability
of precipitation. In addition, all three experiments
also investigated the effect of the forecast on a binary
decision. All three experiments were conducted in
Seattle, Washington, among participants presumably
very familiar with rain forecasts. The overarching
question for these experiments was as follows: How
can we express probability of precipitation to enhance
understanding and to improve decision making?
Experiment 1. Experiment 1 investigated three
PoP icons, each showing 25% chance of precipitation
and 75% chance of precipitation (see Fig. 1). Like most
icons in use today, they included rain and/or cloud
imagery. The icons were based on icons currently in
use, but simplified and standardized for the purpose
of the experiment, to neutralize extraneous factors
(e.g., color, size, text). The icon in the leftmost column
of Fig. 1 was based on a cartographic convention in
which greater fuzziness or fogginess indicates more
uncertainty (MacEachren 1992). This icon showed a
cloud with raindrops and a question mark that was
pale gray in the 25% condition when precipitation
was unlikely and darker gray in the 75% condition
in which precipitation was more likely. The question
mark disappeared when precipitation was highly
likely, a condition not tested here. This imagery may
appeal to the “confidence that it will rain” interpretation described above because haziness and the question mark indicate lack of confidence (i.e., the rain
pictured here is questionable).
The other two icons represented probability as
a proportion of a whole: the portion filled by rain
imagery corresponded to the percent chance of rain.
We tested both a bar icon and a pie icon (see similar
imagery at www.weather.com). Notice that this imagery may suggest a frequentist interpretation, rain
during a proportion of days with similar weather
conditions. The proportion of the icon with no raindrops may indicate that a “no-rain” outcome is also
a possibility. The question for this experiment was
whether any of these icons, when viewed casually,
as they would be when shown on a TV or Web site
weather forecast, do a better job of clarifying PoP
than the others?
Method. One hundred and eighty-three psychology
undergraduates participated in this study as part of
a course requirement, filling out a one-page anonymous questionnaire during a mass-testing session.
One of the three experimental icons was shown in
the upper-left-hand corner of each questionnaire.
The numerical chance of rain was written above the
icon. The likely amount of rain, 0.0 in. in the 25%
condition and 0.03 in. in the 75% condition, was
written below it. The ensemble forecast from which
the probability was derived also produced a likely
amount that constituted an average over all possible
outcomes for the probability. The instructions in the
upper-right-hand corner read, “The picture to the
left displays the rain forecast for the Seattle–Tacoma
Airport for today. Please use it to answer the following
questions.” Two of the questions were designed to
uncover the reference class misunderstandings of
PoP. One asked, “Over approximately what area of
the Puget Sound region will it likely rain today?” The
next question asked, “How much of the time will it
likely rain today?” For both questions there were four
check-box answer options, “None of the [area/time],”
“Less than half of the [area/time],” “More than half
of the [area/time],” and the correct answer, “Can’t
tell from this forecast.” Another question, designed
to determine the impact of the forecast on decision
making, asked whether participants would take an
umbrella or wear a hooded jacket. Each participant
saw only one icon and forecast and answered this set
of questions only once.
Results. The correct answer to both of the reference
class questions was “can’t tell from this forecast” because no information about percent of area or percent
time was provided. Only 43% (i.e., 79/183) selected the
correct response to both questions.
Approximately 32% of participants instead
selected one of the incorrect area-related answers.1
Although the pie icon led to the fewest errors, in a
logistic regression analysis there were no statistically significant differences by icon type. Table 1
Table 1. Expt 1: Percent of reference class errors
by icon type.
Reference class errors
Icon type
% area errors
% time errors
Pie
19/63 (30%)
21/65 (32%)
Bar
21/59 (36%)
24/59 (41%)
Question mark
18/59 (31%)
25/59 (42%)
Total
58/181 (32%)
70/183 (38%)
lists the mean number of error responses in each
icon type.2 We predicted that if participants thought
the percentage referred directly to area they would
select “less than half the area” in the 25% condition
and “more than half the area” in the 75% condition.
Indeed, the majority of wrong answers fell into these
categories (i.e., 21/34, 62% and 14/24, 58%, respectively). Moreover, the pattern of error responses was
significantly different in the 25% condition compared
to the 75% condition [i.e., χ2 (2, N = 58) = 17.25,
p < 0.001)]. Overall, these results suggest that quite a
few participants believed that PoP had something to
do with percentage area.
A substantial proportion of participants (38%)
selected incorrect responses in answer to the percent
time question as well. A logistic regression analysis
revealed no significant differences due to icon type.
However, as Table 1 shows, the pie icon again yielded
slightly fewer error responses overall. We expected
those who held the “percent time” misconception
to select “less than half the time” in the 25% condition and indeed, 56% of errors (22/39) fell into this
category. In the 75% condition the percent time
misconception would lead participants to select
“more than half the time,” and 39% (12/31) of error
responses fell into this category. Again, the pattern of
error responses was significantly different at the two
levels of uncertainty, χ2 (2, N = 70) = 19.13, p < 0.0001,
suggesting that quite a few participants believed that
PoP had something to do with percent of time.
It is important to note that the group of participants who selected wrong answers to the time question in the predicted categories (i.e., “less than half”
for 25% and “more than half” for 75%) was largely
independent of the group who selected wrong answers
in the predicted categories to the area question. In all,
59 people, 32% of the sample, selected an answer to
Two participants failed to answer this question so the n for these analyses is 181.
Because “exactly half” was not an option, we do not know whether some participants might have chosen it. Also, “none” is
logically included in the category “less than half.” Some who thought it meant none might have selected that category instead
of none.
1
2
AMERICAN METEOROLOGICAL SOCIETY
february 2009
| 187
either the time or area questions that suggested that
they thought the percent provided referred directly
to either time or area.
Examination of the umbrella–hooded jacket decision revealed that, as one might expect, a smaller
proportion (i.e., 28%, 26/94) of participants chose to
take an umbrella in the 25% chance condition than
did those in the 75% chance condition (i.e., 72%,
65/88). Interestingly, however, participants’ decisions
were related to their interpretation of PoP. Pairwise
comparisons in a logistic regression analysis revealed
that the odds of choosing to take an umbrella or wear
a hooded jacket were significantly smaller among participants who chose the correct answer “can’t tell” for
both the time and area question (β = 0.15, p = 0.001)
as compared to those who chose “more than half of
the time” or “more than half of the area.” The same
was true of participants who chose “none” or “less
than half” (i.e., β = 0.08, p < 0.001). This suggests that
misinterpreting the forecast as more than half the
area or time affects decision making by increasing
the tendency to take precautionary action.
Discussion of experiment 1. A surprising percentage of
participants misinterpreted PoP despite the visualizations that accompanied the numeric probabilities.
The pattern of responses suggested that a substantial
proportion of participants thought PoP included
information about percentage of area or percentage
of time.3 Participants choose “less than half” more
often in the 25% condition and “more than half” more
often in the 75% condition for both the area and the
time questions. Those who selected percent-aligned
wrong answers, suggesting that they thought that the
percentage referred directly to time or area rather
than to the chance of precipitation, comprised a third
of participants overall. Furthermore, misunderstandings affected decision making, increasing precautionary action among those with the “more than half”
misinterpretation. This combined evidence suggests
that at least some were indeed interpreting PoP as a
deterministic forecast for rain.
An alternative explanation for the substantial
percentage of incorrect answers in experiment 1
concerns the multiple-choice procedure used to probe
reference class understanding. The correct answer
In fact, these data may underestimate the proportion of
respondents with erroneous understanding because it is
possible that some may have selected the correct answer to
the area question simply because of the mismatch between
the point forecast provided and the area in the question, not
because they knew the correct reference class.
3
188 |
february 2009
to both the area and time questions was “can’t tell
from this forecast” because the information was not
specified in the forecast. Some participants may have
avoided this answer because they thought that experimenters would not construct questions for which this
was correct answer. Thus, some responses may have
been partly an artifact of the experimental procedure
(Orne 1962). Indeed, participants would only resort
to this strategy if they did not understand the forecast
in the first place. Moreover, we would not have seen
the systematic impact on the umbrella decision if
all participants’ error responses had been the result
of such an artifact. Nonetheless, it is important to
rule out this explanation. One way to overcome this
problem is to ask participants to explain PoP in their
own words.
Experiment 2. Experiment 2 probed participants understanding of PoP using open-ended
questions. Due to the minimal differences between
icon types observed in experiment 1, only the question mark icon and the pie icon were used in this study
because the imagery in these icons may give rise to
different interpretations of uncertainty and hence,
might lead to different explanations of PoP. They were
compared to a “no icon” condition.
Methods. One hundred and sixty-nine psychology
undergraduates participated in this study as part of
a course requirement. Participants filled out a onepage anonymous questionnaire during a 30-min
testing session. The instructions indicated that the
questionnaire provided the “rain forecast for the
Seattle–Tacoma Airport for tomorrow.” Two levels
of uncertainty were expressed: 25% and 75%. In the
icon conditions, the forecasts were identical to those
used in experiments 1. In the no-icon condition only
the probability of precipitation phrase and the likely
amount were presented. Below the forecast were four
open-ended questions. The first three questions were
designed to prove participants’ understanding of the
reference class. Pilot testing had revealed that participants were reluctant to specify a reference class, so
three different questions were included in an attempt
to encourage specificity. The first question asked
participants to “Explain the meaning of the forecast.”
The second question asked participants to “Explain the
forecast to someone who did not understand the concept of probability of precipitation.” The third question
asked, “Of what is it 25% (or 75%)?” A final question
asked whether participants would take an umbrella or
wear a hooded jacket. Each participant saw only one
forecast and answered only one set of questions.
Results. We sorted the 428 answers participants gave
to the three open-ended questions into give general
categories (see the appendix). Intercoder reliability
using this system was 87%.4 The first two categories
included answers that directly addressed the issue of
interpretation. One included responses that specified
a reference class (i.e., 6%, 23/428). There was also a
small group who provided a “confidence in the forecast” explanation (i.e., 1%, 5/428). A larger category
(i.e., 25%, 107/428) included answers that provided
clarification to the numeric aspect of the probability
phrase (e.g., 25% of 100%, 1 in 4). A fourth category
included a small proportion (i.e., 3%, 13/428) of
answers that added information about other parameters such as cloud cover or temperature. The fourth
and largest category of responses (i.e., 65%, 279/428)
included those that simply repeated the probability
phrase presented or summarized it verbally, without
adding in any information or clarification.
Then, we summarized responses by participant,
giving each individual credit for the most specific
response given. If participants gave the reference
class or confidence in the forecast explanations they
were put into an “interpretation” category (i.e., 17%,
28/169). If participants did not mention a reference
class or confidence explanation but gave a numeric
clarification they were put into that category (i.e., 47%,
80/169). If they did not give any of the first three explanations but added information about other parameters
they were put into that category (i.e., 2%, 3/169). If
participants gave none of the other explanations but
merely repeated or summarized the probability phase,
they became part of that category (i.e., 34%, 58/169).
Figure 2 shows these proportions. We will focus here
on the group that provided interpretation explanations
(i.e., reference class or confidence).
The majority (i.e., 52%, 12/23) of participants
who addressed the reference class issue provided
an answer, such as “days with atmospheric conditions like today,” which was considered correct. If
we included those who provided a “confidence in
the forecast” explanation in the category of correct
responses, then 61% (17/28) of those providing an explicit interpretation were correct. Of the 39% (11/28)
error responses, the majority were answers explaining
that the forecast indicated the percent of time that
precipitation would fall (i.e., 91%, 10/11). Although
the number of participants who gave interpretation
explanations was small, the proportion of correct
Intercoder reliability was calculated from the percentage
of matching codes from a randomly selected sample of 10
questionnaires by 3 independent coders.
4
AMERICAN METEOROLOGICAL SOCIETY
Fig. 2. Proportions of responses per response category
in experiment 2.
to error responses was similar to that observed in
experiment 1.
The decisions about whether to take an umbrella
or wear a hooded jacket were examined to determine
whether they were related to erroneous reference
class interpretations. Indeed, a greater proportion of
those in experiment 2 who held erroneous reference
class beliefs decided to take an umbrella (i.e., 72%,
8/11) than did those who provided an appropriate
interpretation (i.e., 65%, 11/17). However, due to the
small number of participants who provided explicit
interpretations, this difference did not reach statistical significance. There were no significant differences
due to icon type (e.g., question mark icon, pie icon, or
no icon) for the category of explanation, error rates,
or decisions.
Discussion of experiment 2. Remarkably, when asked
to explain PoP in their own words, few participants
provided information about the reference class,
despite the pointed and probing questions that were
asked. There are at least two possible explanations
for this omission. Participants may not have believed this aspect required explanation. On the other
hand they may not have known what the explanation was and did not want to expose this deficit in
understanding.
The important thing to note, however, is that
among those who attempted to explain the reference class (perhaps those most confident in their
understanding), the error proportion was similar to
that observed in experiment 1. This suggests that the
results of experiments 1 were not simply an artifact
of how the question was asked, but were instead an
accurate reflection of participants misunderstanding
of the reference class issue.
february 2009
| 189
As with experiment 1, the icon visualizations
provided here did not reduce errors and did not influence participants’ explanation of the forecast. Perhaps
some people’s interpretation of PoP is now well
entrenched, developed, and maintained over many
years of exposure. If their entrenched interpretation
involves a misunderstanding, it may be particularly
resistant to correction. These users may expect the
proportional imagery in the icons to be compatible
with their misunderstanding (i.e., as representing
percent time or percent area).
Experiment 3. The combined evidence from
experiments 1 and 2 established that a proportion of
college students mistake probability of precipitation
for information about the percent of time or area.
This is true regardless of whether they are asked in
multiple-choice or open-ended format. Moreover,
the association between incorrect choices and the
tendency to take precautionary action suggests that
at least some of them are converting the probabilistic
forecast into a deterministic forecast for precipitation
with additional information about percent time or
area. Hence, the solution may be to contradict this
notion explicitly in the forecast. Including a statement
of the probability of no precipitation may make it clear
that PoP is a forecast indicating that it might not rain
at all. This solution was tested in experiment 3.
Method. One hundred and two psychology undergraduates participated as part of a course requirement. The
procedure, instructions, and questions were similar to
that described in experiment 1. The forecast displayed
in the upper-left-hand corner of the questionnaire
included a phrase describing the chance of rain. Two
conditions included only this verbal–numeric information. In one, the chance of no rain appeared beneath the chance of rain (e.g.; “Chance of rain: 25%”;
“Chance of no Rain: 75%”); in the other it did not. In
the third condition the pie icon appeared beneath the
chance of rain statement.
Results. We combined the percent area and percent
time answers in experiment 3 to create a single reference class variable because this experiment included
only about half the number of participants as in
experiment 1. As with experiment 1, approximately
half of participants made one or both reference class
errors (i.e., 48%, 49/102). As Table 2 shows, a smaller
proportion of participants answered these questions
incorrectly when the chance of no rain was explicitly
expressed in the forecast. Pairwise comparisons in
a logistic regression analysis revealed that the odds
190 |
february 2009
Table 2. Expt 3: Percentage of error responses to
reference class questions by visualization type.
Visualization type
% reference class errors
“Chance of rain”
21/33 (64%)
“Chance of rain” and
“Chance of no rain”
12/33 (36%)
Pie icon and
“Chance of rain”
16/36 (44%)
Total
49/102 (48%)
of making an error were significantly lower if the
chance of no rain was included as compared to the
condition in which only the chance of rain was provided (i.e., β = 0.33, p < 0.03). The odds of making an
error in the pie condition were also lower than in the
control condition and approached significance (i.e.,
β = 0.46, p = 0.11). No other significant differences
were detected.
Discussion of experiment 3. Again, a large proportion
of participants indicated that they thought that the
PoP forecast included information about percent
time or percent area. In this study for the first time,
however, we have evidence that this misconception
can be significantly attenuated if the forecast makes
explicit the fact that there is also a chance that no
rain will be observed. Including the chance of no
rain along with the chance of rain cuts the reference
class error in half. It apparently clarifies for some that
the percentage expressed in the forecast refers to the
chance of precipitation rather than to the proportion
of area or time. There is also evidence that the pie
icon provided some improvement in understanding,
although the difference observed here did not reach
significance.
General Conclusions. These three
studies demonstrate that probability of precipitation, which has been included in public forecasts for
decades, is still confusing to people. This is strongest
evidence to date that misunderstandings arise from
confusion about the reference class. Errors effected
subsequent decisions and were reduced only when the
chance of no rain was made explicit. Taken together
these results suggest a tendency to construe PoP as a
deterministic forecast that indicates something about
proportion of time or area. If such deep-seated misunderstanding is evident among this college-educated
sample in the rain-experienced Pacific Northwest, we
can assume that it exists in similar or larger proportions among the general public.
Importantly, this research also demonstrates that
targeted enhancements can overcome this misunderstanding among many end users. A reduction in
errors with the pie icon was demonstrated in two of
the three studies reported here. Although none of
these differences reached significance, this icon may
have some effect because, in addition to a proportion
that includes rain, it illustrates a proportion that
includes no rain, clarifying that this too is a possible
outcome. Given the widespread use of icons in the
media, this is an issue that demands further research.
Visualization such as this may be an important tool
for successful communication of forecast uncertainty
if psychological research is part of the design process.
In the research reported here, the biggest reduction
in errors was observed when the phrase describing
the chance of no rain was provided. These combined
results suggest that understanding of PoP forecasts
can be improved with targeted enhancements: those
that describe or illustrate both possible outcomes.
These results have important implications for
extending probabilistic forecasts to other weather
parameters. Although information about forecast
uncertainty could be useful to the general public for
weather-related decision making, misinterpretations
like those reported here could reduce or even eliminate the potential benefit. While the consequences of
carrying around an unnecessary umbrella for a day
may be small, for other weather-related decisions the
cumulative costs of unnecessary school closures, road
treatments, or crop-protection procedures could be
substantial.
In addition, there is a potential cost to user trust.
If the user misinterprets the probabilistic forecast
as deterministic and no precipitation is observed, it
could be regarded as a false alarm, reducing trust in
subsequent forecasts. Although the consequences of
reduced user trust in everyday precipitation forecasts
are not great, the same effect extended to severe
weather warnings could be extremely costly leading
to a reduction in compliance and a threat to safety.
Much additional research is needed in this area. The
results reported here identify some of key psychological issues to be explored.
Why do people misinterpret PoP as a deterministic forecast? This may be part of a general human
tendency to avoid the complications of incorporating
uncertainty in the decision process by ignoring it
or turning it into certainty (Slovic 2007). People
have shown the same tendency when being told
about the side effects of Prozac, thinking that side
effects occur in a percentage of sexual encounters
of every individual who takes it (Gigerenzer 2002).
AMERICAN METEOROLOGICAL SOCIETY
In addition, evidence suggests that those living in
flood planes appeal to perceived cyclical patterns to
predict the size of the next flood (Burton and Kates
1964) rather than regarding it as uncertain. Moreover,
weather forecasters, finding that a numerical model
prediction is not reliable, tend to make corrections
and assume that they have eliminated the element of
uncertainty (Joslyn and Jones 2008). Granted, each
of these strategies is slightly different. Some of them
may have deliberate components. For others the shift
from regarding the situation as uncertain to certain
may be unconscious. However, all of these strategies
strive to reduce or eliminate uncertainty, perhaps
because it is easier than incorporating it.
Incorporating uncertainty directly into the
decision process requires the reasoner to simultaneously consider several hypothetical outcomes,
their corresponding levels of uncertainty, and
their consequences. This is particularly difficult
because many weather-related decisions are made
quickly, relying solely on what psychologists refer
to as “working memory,” roughly synonymous with
consciousness (Baddeley 1987). A major undisputed
conclusion of decades of psychological research is
the severe limitation that human information processing has in this regard (Miller 1956). There are
only so many things we can hold and manipulate in
consciousness simultaneously.
Because uncertainty constitutes an additional and
complex piece of information it may simply be easier
to reduce the cognitive load by committing oneself
to a single outcome and proceeding as though the
uncertainty has been resolved. In many cases, people
may not be aware of making this simplification. As
in the case with PoP, if there is an interpretation that
does not involve uncertainty (e.g., percent refers to
time or area) it may seem preferable to people, without
their understanding why.
As a result, it is crucial when providing the general public with uncertainty information to make
the uncertain aspect of the forecast crystal clear. It is
also important to provide a forecast in the simplest
possible terms to avoid further increase in the cognitive load. In experiment 3 reported here, uncertainty
was made explicit by providing both the chance of
rain and the chance of no rain. The important thing
to realize about this solution is that although “25%
chance of rain” implies “75% chance of no rain” to
those with the correct interpretation of this phrase,
the same implication is NOT clear to those who are
misinterpreting the phrase as percent area or time. In
other words, some people are simply not interpreting
it as an expression of uncertainty.
february 2009
| 191
Given the potentially serious consequences, it is
important for communicators of forecast information to understand the psychology of the end user, to
consider not only what the forecast means, but also
how the user is “hearing” it. We have demonstrated
one kind of misunderstanding here. However, other
expressions (e.g., frequency, predicative intervals,
etc.) and visualizations of uncertainty may give rise
to different misinterpretations and call for different
kinds of clarification. Until the research is done that
specifically targets these expressions, the details of
their psychological impact will remain unknown. The
bottom line is that people can understand and make
good use of forecast uncertainty if it is communicated
in a manner that takes into account the psychological
characteristics of the human information processing
system. Armed with this knowledge, forecast providers will be able to communicate uncertainty in a
manner that will be understandable and beneficial to
a wide range of end users.
APPENDIX. RESPONSE CATEGORIES WITH EXAMPLE RESPONSES
Category
Example response
Reference class
Percent/proportion days like today
“. . . based on the current conditions, 75 days out of 100
days with these conditions will see rain…”
Percent/proportion agreement between forecasters
“. . . it’s like 3 weathermen say it will rain while 1 say it
won’t . . .”
Percent/proportion time it will rain tomorrow
“. . . 25% of the day (tomorrow) would rain . . .”
Percent/proportion area over which it will rain
“75% of the entire area mentioned will have rain . . .”
Confidence in forecast
Degree of confidence of forecaster
“…that there will probably be some rain, but they don’t
know for sure . . .”
Numeric clarification
Switching to frequency format
“. . . 3 times in 4 it will rain . . .”
Switching to ratio format
“There is a 1/4 chance of precipitation . . .”
Describing 25%/75% as a part of 100%
“It is 25% out of 100% that it will rain tomorrow . . .”
Confusing the amount of rain with the chance of rain “If it does precipitate there is a very low chance of rain ~0.0”
Adding information
Adding information
“It will be cold tomorrow.”
Repeat or summarize
Repeating the original probability phrase
“There is a 75% chance of rainfall and a predicted amount
of 0.03 in. of rain.”
Correct verbal summary of the information
“There is a very good chance that it will rain . . .”
Wrong verbal summary of the information
In the 25% condition: “It is more likely to rain.”
Vague or general explanation
“Probability of precipitation is how likely it is that it is
going to rain . . .”
REFERENCES
Baddeley, A. D., 1987: Working Memory. Clarendon
Press, 289 pp.
Burton, I., and R. W. Kates, 1964: The perception of
natural hazards in resource management. Nat.
Resour. J., 3, 412–414.
DeElia, R., and R. Laprise, 2005: Diversity in interpretations of probability: Implications for weather
forecasting. Mon. Wea. Rev., 133, 1129–1143.
192 |
february 2009
Gigerenzer, G., 2002: Calculated Risks: How to Know
When Numbers Deceive You. Simon & Schuster,
310 pp.
—, R. Hertwig, E. van den Broek, F. Fasolo, and K. V.
Katsikopoulos, 2005: “A 30% chance of rain tomorrow”: How does the public understand probabilistic
weather forecasts? Risk Anal., 25, 623–630.
Joslyn, S., and D. Jones, 2008: Strategies in naturalistic
decision-making: A cognitive task analysis of naval
weather forecasting. Naturalistic Decision Making
and Macrocognition, J. M. Schraagen, Ed., Ashgate
Publishing, in press.
MacEachren, A. M., 1992: Visualizing uncertain information. Cartogr. Perspect., 13, 10–19.
Miller, G. A., 1956: The magical number seven, plus or
minus two: Some limits on our capacity for processing information. Psychol. Rev., 63, 81–97.
National Research Council, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty
for Better Decisions Using Weather and Climate Forecasts. The National Academies Press, 191 pp.
Orne, M. T., 1962: On the social psychology of the psychological experiment: With particular reference to
demand characteristics and their implications. Amer.
Psychol., 17, 776–783.
Slovic, P., 2007: Perception of risk from asteroid impact.
Comet/Asteroid Impacts and Human Society, P. Bobrowsky and H. Rickman, Eds., Springer–Verlag,
369–382.
AMERICAN METEOROLOGICAL SOCIETY
february 2009
| 193