Cognitive Biases 2
Incomplete and Unrepresentative
Data
“I know horoscopes can predict the future,
because I’ve seen it happen.”
“Positive thinking can cure cancer, because I
know someone who used it, and got better.”
“Everyone knows you do worse in your second
year in college, you see it all the time.”
Necessary and Sufficient Evidence
If claims like these are true, for example, if it’s
true that horoscopes predict the future, then
the evidence in question is necessary.
If horoscopes predict the future, then there
must be cases where a horoscope predicted
something, and then it happened.
Necessary and Sufficient Evidence
But such evidence is not sufficient for
establishing the truth of these claims.
When a horoscope predicts that X will happen,
and then X happens, that doesn’t prove
anything. Were there times horoscopes
predicted things that didn’t happen? Were there
things that horoscopes should have predicted,
but never happened?
Contingency Tables
Contingency Tables
A contingency table is used to plot predictions of
the form “If A happens, then B will happen.”
In the upper left corner of the table (Prediction
= Yes, Observation = Yes) are cases where A
happens and B happens. These are “true
positives” and they confirm the prediction.
Contingency Tables
A contingency table is used to plot predictions of
the form “If A happens, then B will happen.”
In the lower right corner (Prediction = No,
Observation = No) are cases where A does not
happen, and B does not happen. These are “true
negatives” and also confirm the claim in
question.
Contingency Tables
A contingency table is used to plot predictions of
the form “If A happens, then B will happen.”
Two sorts of cases disconfirm the prediction.
First there are false positives in the upper right
corner, where the predicted outcome
(Prediction = Yes) does not match the observed
outcome (Observation = No).
Contingency Tables
A contingency table is used to plot predictions of
the form “If A happens, then B will happen.”
Second, there are “false negative” cases. These
are also outcomes where the prediction
(Prediction = No) fails to match the observation
(Observation = Yes).
Perfect Correlation Claims
For “absolute” claims (If A happens, B will
always happen, A and B are perfectly positively
correlated), all cases must fall in the upper left
or lower right corners.
The prediction must always match the outcome.
Imperfect Correlation Claims
For probabilistic claims (A and B are imperfectly
positively correlated), you must compare the
proportion of true positives to predicted
positives to the proportion of false positives to
predicted negatives.
(How you should compare them depends on the
base rates…)
Prediction = Yes & Observation = Yes
÷
Prediction = Yes
Compared to:
Prediction = No & Observation = Yes
÷
Prediction = No
For example…
Suppose I claim that you’re more likely to get an
A on the final if you take notes in class than if
you take no notes.
Taking Notes
In order to assess that claim for truth, we need
to consider the ratio of students who take notes
& get an A to the students who take notes:
(Take notes & get an A) ÷ Take notes
Suppose that 50% of note takers get A’s and 50%
get other grades. Does that mean note takers do
better or worse than non-note-takers?
Not Taking Notes
To figure that out we need to know what
proportion of non-not-takers got A’s:
(No notes & got an A) ÷ No notes
If 60% of the people who took no notes got A’s
and 40% got other grades, then it’s false that
you’re more likely to get an A if you take notes.
Confirmation Bias
Even though evaluating predictions requires
looking at both the rates of true positives among
predicted positives, and the rates of false
positives among predicted negatives, human
beings have a tendency to only consider true
positives (and to a lesser extent, true negatives)
when evaluating predictions or other claims of
imperfect correlation.
Wason Selection Task
Around ½ of people studied say “D” and “5”.
About 1/3 say just “D”.
Only about 1/20 get the right answer: “D” and
“2”!
Searching for Confirmation
People have a preference for positive answers
that confirm their theories, even though
negative answers that disconfirm their theories
might give the same amount of information.
For example, suppose A picks a number
between one and ten, and you’re supposed to
guess what it is. I suggest that the number is 3.
The psychological research shows that you’d be
more likely to ask “is the number odd?” than “is
the number even?”– even though both answers
are equally informative.
In this case, the preference for confirmation
does not matter:
Asking the positive question and the negative
question give you the same information. If
people prefer the positive question, that doesn’t
harm them at all. But the preference can harm
them if the positive question gives less info.
A Strange Example
Americans were asked:
Which of these pairs of countries are more
similar to one another?
1. West Germany, East Germany
2. Sri Lanka, Nepal
They said (1), West Germany and East Germany.
Others (Americans) were also asked:
Which of these pairs of countries are more
different from one another?
1. West Germany, East Germany
2. Sri Lanka, Nepal
They also said (1).
Americans thought that West Germany and East
Germany were both more similar to each other
than Sri Lanka and Nepal and less similar to each
other than Sri Lanka and Nepal.
How is that possible?
First, when considering the question ‘which are
more similar?’ the subjects looked for all the
positive evidence that West Germany and East
Germany were similar, and all the positive
evidence that Sri Lanka and Nepal were similar.
Since Americans know nothing about Asian
countries, they had no positive reason to think
Sri Lanka and Nepal were similar.
Similarly, when asked ‘which are more
different?’ the subjects considered the positive
evidence that West Germany and East Germany
were different and the positive evidence that Sri
Lanka and Nepal were different.
Again, having no knowledge of Sri Lanka or
Nepal, Americans chose (1), because of all the
positive evidence in its favor.
But it cannot be true that East Germany and
West Germany are both more similar and more
different than Sri Lanka and Nepal.
What the subjects did not do is consider the
relevant negative evidence that would
disconfirm their hypotheses.
The Problem of Absent Data
Sometimes it’s not just that we only look for or
evaluate the positive evidence, but that there is
no negative evidence. This can lead us to think
we have very well-confirmed beliefs when we do
not.
Hiring Job Applicants
Suppose you’re hiring job applicants for your
shoe company. You think people who haven’t
studied a musical instrument would not be good
employees.
Who do you hire? The people who have studied
music, of course! And if they’re successful at
your company, do you have good reason to
believe that you were right?
No! You have positive evidence– people you
predicted would be successful, who are
successful– but you have no negative evidence.
What about all the people you didn’t hire, the
ones who didn’t study music? They might have
been successful too. They could’ve been more
successful. You don’t know.
Absent evidence is all around us.
Suppose you decide to major in accounting
instead of philosophy. You find that you are very
happy studying accounting. Did you make the
right choice? You can’t know. You could have
been more happy studying philosophy. There’s
just no evidence.
Self-Fulfilling Prophecies
The Prisoner’s Dilemma
There are two general strategies to playing the
prisoner’s dilemma. You can view the game as
one where the goal is for everyone to do well,
and thus play “cooperatively” or you can view
the game as one where the goal is for you to do
better than your opponent, and thus play
“competitively”.
The Prisoner’s Dilemma
Believing that the goal is selfish, to win more for
yourself, is a self-fulfilling prophecy.
If you play against other selfish players, they will of
course play competitively.
But even if you play against cooperators, they will
have to play competitively, or get 0 every round. So
it will seem as if there are no cooperators.
Another Example
Suppose you think I’m not a very nice person, so
you avoid me.
If you avoid me, then you’ll never have a chance
to correct your initial impression.
So if I’m a nice person, you’ll never find out if
you start out thinking I’m not.