The Goldilocks Fallacy
A Challenge to Good Decision-Making
Presentation to IUP
Pete Vanden Bosch
22 October 2014
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Surveillance Example
• U-2 orbits near Korean DMZ to gather photo intelligence.
o Quality of intel degrades with distance (terrain).
o Risk to life, etc., decreases with distance (SAMs).
2
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Best Orbit?
Notional Only
Option A: 5 miles from border. Best coverage, but most dangerous
Option B: 15 miles from border. Poor coverage, but fairly safe from SAMs
Option C: 10 miles from border. Moderate coverage, moderate safety
intelligence
close in
0.80
intermediate
0.68
far out
0.65
3
An FFRDC operated by Analytic Services Inc. on behalf of DHS
survival
0.60
0.62
0.80
Linear Weighting
Intel twice as important as security
wi=2, ws=1
close in
intermediate
far out
Values
close
mid
far
intel
0.80
0.68
0.65
surv
0.60
0.62
0.80
2.20
1.98
2.10
Security twice as important as intel
wi=1, ws=2
close in
intermediate
far out
An FFRDC operated by Analytic Services Inc. on behalf of DHS
2.00
1.92
2.25
Logistic Function
Slope of m at inflection
Y2
1.0
0.8
0.6
0.4
0.2
0.0
-1.0
0.0
1.0
2.0
b
𝑌2 − 𝑌1
𝑌=
+ 𝑌1
−4𝑚
𝑥−𝑏
1+𝑒
5
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Y1
Intel Twice as Valuable as Survival?
A
C
B
6
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Survival Twice as Valuable as Intel?
B
A
C
7
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Concentration vs. Dispersal
• Consider placement of a set of
chemical storage containers.
o Too close together  higher catastrophic risk
o Too far apart  higher logistics & support costs
• Independent variable: “density”
o Could be tons of chemical per acre
o Could be closest inter-container distance
• Dependent variables: risk and cost
o Over most of the domain, both variables are
fairly constant, changing significantly only near
some inflection point.
Situation from Charles Perrow, The Next Catastrophe: Reducing Our Vulnerabilities to Natural,
Industrial, and Terrorist Disasters (2011).
8
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Key to Shape Is Inflection Points
br
bc
br
bc
density
bc
br
density
Lower is better on all four of these plots.
9
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Detector Cascade
• Consider Customs and Border Protection (CBP) mission at border crossing:
o Goal 1: Detect (and interdict) illegal materials
o Goal 2: Expedite legal commerce
• Common approach is a sensor cascade.
o
o
o
o
o
A sensor is anything that provides a binary decision: “OK” or “possibly not OK.”
Fast primary scan, focused on detection
Referral to secondary scan if decision is “possibly not OK”
Slower, refined secondary scan if needed
Ubiquitous approach because it supports both goals
10
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Sensor Cascade
Primary
ILLEGAL
T1
Secondary
1-Pd1
Admit (error)
Pd1
T2
LEGAL
T1
1-Pd2
Pd2
1-Pfa1
Adverse (correct)
Admit (correct)
Pfa1
T2
1-Pfa2
Pfa2
Adverse (error)
Measure 1: Interdiction Probability = Pd1*Pd2
Measure 2: Throughput rate = 1/[T1+Pfa1*(1-Pfa2)*T2]
…but Pd1,Pd2, Pfa1, and Pfa2 are all dependent on T1 and T2
11
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Functional Relationships
Notional values
12
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Sample Interdiction Dependence
T2
T1\T2
T1
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
2.00
2.20
2.40
2.60
2.80
3.00
3.20
0.20
0.07
0.08
0.11
0.17
0.22
0.25
0.27
0.27
0.27
0.27
0.27
0.27
0.27
0.27
0.27
0.27
0.40
0.07
0.08
0.11
0.17
0.23
0.26
0.27
0.27
0.27
0.27
0.27
0.27
0.27
0.27
0.27
0.27
0.60
0.07
0.08
0.12
0.18
0.24
0.27
0.29
0.29
0.29
0.29
0.29
0.29
0.29
0.29
0.29
0.29
0.80
0.09
0.11
0.15
0.23
0.30
0.34
0.36
0.37
0.37
0.37
0.37
0.37
0.37
0.37
0.37
0.37
1.00
0.14
0.16
0.23
0.34
0.46
0.53
0.55
0.56
0.56
0.56
0.56
0.56
0.56
0.56
0.56
0.56
1.20
0.18
0.22
0.30
0.46
0.62
0.71
0.74
0.75
0.75
0.75
0.75
0.75
0.75
0.75
0.75
0.75
1.40
0.20
0.24
0.33
0.51
0.68
0.78
0.81
0.83
0.83
0.83
0.83
0.83
0.83
0.83
0.83
0.83
1.60
0.21
0.24
0.34
0.52
0.70
0.80
0.83
0.84
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
1.80
0.21
0.24
0.34
0.52
0.70
0.80
0.84
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
2.00
0.21
0.24
0.34
0.52
0.70
0.80
0.84
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
2.20
0.21
0.24
0.34
0.52
0.70
0.80
0.84
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
2.40
0.21
0.24
0.34
0.52
0.70
0.80
0.84
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
2.60
0.21
0.24
0.34
0.52
0.70
0.80
0.84
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
2.80
0.21
0.24
0.34
0.52
0.70
0.80
0.84
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
3.00
0.21
0.24
0.34
0.52
0.70
0.80
0.84
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
3.20
0.21
0.24
0.34
0.52
0.70
0.80
0.84
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
0.85
Insight: Interdiction is dependent on T1 and T2 both exceeding certain thresholds.
13
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Sample Throughput Dependence
T2
T1\T2
T1
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
2.00
2.20
2.40
2.60
2.80
3.00
3.20
0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40 2.60 2.80 3.00 3.20
4.2 3.6 3.2 2.8 2.5 2.3 2.1 1.9 1.8 1.7 1.6 1.5 1.4 1.3 1.3 1.2
2.3 2.1 2.0 1.8 1.7 1.6 1.5 1.4 1.3 1.3 1.2 1.2 1.1 1.1 1.0 1.0
1.6 1.5 1.4 1.4 1.3 1.2 1.2 1.1 1.1 1.0 1.0 1.0 0.9 0.9 0.9 0.8
1.2 1.2 1.1 1.1 1.1 1.1 1.0 1.0 1.0 1.0 0.9 0.9 0.9 0.9 0.9 0.8
1.0 1.0 1.0 1.0 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.9 0.8 0.8 0.8
0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.7
0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6
0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6
0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5
0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5
0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4
0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4
0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4
0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3
0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3
0.31 0.31 0.31 0.31 0.31 0.31 0.31 0.30 0.30 0.30 0.30 0.30 0.30 0.30 0.30 0.30
Insight: Throughput is relatively insensitive to secondary parameters.
14
An FFRDC operated by Analytic Services Inc. on behalf of DHS
One Combination of Measures
15
An FFRDC operated by Analytic Services Inc. on behalf of DHS
One Combination of Measures
T1\T2
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
2.00
2.20
2.40
2.60
2.80
3.00
3.20
0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40 2.60 2.80 3.00 3.20
0.76 0.66 0.59 0.55 0.54 0.54 0.52 0.50 0.47 0.45 0.44 0.42 0.41 0.39 0.38 0.37
0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4
0.4 0.3 0.3 0.3 0.4 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4
0.3 0.3 0.3 0.4 0.5 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6
0.3 0.4 0.4 0.4 0.5 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7
0.3 0.4 0.4 0.4 0.6 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.4 0.4 0.6 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.3 0.4 0.6 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.3 0.4 0.6 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.3 0.4 0.5 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.3 0.4 0.5 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.3 0.4 0.5 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.3 0.4 0.5 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.3 0.4 0.5 0.7 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.3 0.4 0.5 0.7 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
0.3 0.3 0.3 0.4 0.5 0.7 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8
Insight: You either want T1 and T2 to be as high as feasible or as low
as feasible – there is no in-between solution, no happy medium.
16
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Goldilocks Heuristic Defined
It is the tendency of the decision maker to choose the
intermediate option of several on a basis other than the
analysis.
It doesn’t mean there aren’t other types of situations for
which intermediate solutions are almost always best.
It isn’t related to the Goldilocks Principle/Effect in
astronomy and evolution, or the Goldilocks Principle/Effect in
cognitive psychology.
17
An FFRDC operated by Analytic Services Inc. on behalf of DHS
We All Seek Context
Classic experiment by Simonson and Tversky:
Group 1
Group 2
choices:
Emerson
$109.99
Panasonic 1 $179.99
choices:
Emerson
$109.99
Panasonic 1 $179.99
Panasonic 2 $199.99
outcome:
Emerson 57%
Panasonic 43%
outcome:
Emerson
27%
Panasonic 1 60%
Panasonic 2 13%
Just adding a high alternative – all other factors equal! – increased
Panasonic’s “market share” dramatically. It is standard in marketing to
offer three choices, expecting the middle to be the dominant choice.
An FFRDC operated by Analytic Services Inc. on behalf of DHS
18
Loss Aversion
 …but reference point (context) doesn’t matter if
classical theory (von Neumann & Morganstern) is
correct that people weight losses and gains equally.
 Experiments by Kahneman and by Knetsch show that
isn’t close to true in most cases.
 Best estimate: losses are weighted 2x gains.
loss
An FFRDC operated by Analytic Services Inc. on behalf of DHS
gain
Objective With and Without Loss Aversion
Example
Unbiased function
(down is good)
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Same function weighted to avoid losses.
Losses are calculated from the dotted
lines (context) and counted 2x gains.
Goldilocks Heuristic: Drivers
• Loss aversion
o Context is huge and almost impossible to avoid.
o Psychologically, losses are usually weighted more heavily than gains.
• Extremeness aversion
o
o
o
o
The extremes may be politically unacceptable, even if within stated trade space.
Decision makers are prone to compromise, to keep peace or expedite agenda.
Picking an extreme opens potential for criticism in hindsight.
Herding instinct.
• Cultural signals
o From Aristotle on, we’ve been taught that there’s a golden mean.
o Briefers are trained to provide three options, the middle being the desired one.
• False analogy
o “Wisdom of crowds” situations.
o We see more physical processes that are convergent – others are often invisible.
An FFRDC operated by Analytic Services Inc. on behalf of DHS
21
Other Common Heuristics
Any decision maker has built up a set of heuristics
(mental shortcuts) over years, but those years may
have honed preferences that are ill-suited for the
current job. A sampling:
•
•
•
•
•
•
•
•
•
Decisiveness vs. deliberation
Standardization (massing) vs. individualization (dispersal)
Deference to experts vs. reliance on own expertise/intuition
Preference for quantitative thinking vs. “the fear of all sums”
Preference for the status quo vs. innovation/change
Trust in one’s own “tribe” vs. willingness to trust “outsiders”
Preference for certainty vs. acceptance of uncertainty
Framing problems as local vs. global
Acceptance of anecdotal information vs. more structured data collection
Heuristics are essential, but they entail potential for bias.
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Observations
• When the solution is anti-Goldilocks, gird yourself for
analytical battle; these are never easy sells.
• When an analysis is going to provide counterintuitive
results, give the decision maker fair warning.
• Hardest is when you don’t have solid data but know
there’s the potential for the decision to be poor.
• Decision-making biases often affect analysts as well!
“It’s difficult to imagine how my team’s analytical
acumen combined with your operator intuition wouldn’t
be an improvement over your intuition alone.”
- Lt Gen Glenn A. Kent
An FFRDC operated by Analytic Services Inc. on behalf of DHS
23
References
Dan Ariely, Predictably Irrational; The Hidden Forces that Shape Our Decisions (2008).
Robert Burton, On Being Certain; Believing You Are Right Even When You’re Not (2008).
Alexander Chernev, “Extremeness Aversion and Attribute-Balance Effects in Choice,” Journal
of Consumer Research, 31: 249-263 (2004).
David Gill and Victoria Prowse, “A Structural Analysis of Disappointment Aversion in a Real
Effort Competition,” American Economic Review, 102(1): 469–503 (2012).
Daniel Kahneman, Thinking, Fast and Slow (2011).
Glenn A. Kent, Thinking About America’s Defense (2008).
Charles Perrow, The Next Catastrophe (2007).
Itamar Simonsen, “Choice Based on Reasons: The Case of Attraction and Compromise
Effects,” Journal of Consumer Research, 16(2) (September 1989).
Itamar Simonsen and Amos Tversky, “Choice in Context: Contrast and Extremeness Aversion,”
Journal of Marketing Research, 29(3): 281-95 (1992).
Amos Tversky, Preference, Belief, and Similarity (2004).
24
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Even bad decision making has its
place…
By Oliver Gaspirtz, used with permission
25
An FFRDC operated by Analytic Services Inc. on behalf of DHS
Loss Aversion Experiment (N=150)
Decision #1: Choose between:
A.
B.
A sure gain of $240 [84%]
25% chance to gain $1000 and 75% chance to gain nothing [16%]
C.
D.
A sure loss of $750 [13%]
75% chance to lose $1000 and 25% chance to lose nothing [87%]
Decision #3: Choose between:
A&D: 25% chance to win $240 and 75% chance to lose $760 [0%]
B&C: 25% chance to win $250 and 75% chance to lose $750 [100%]
Complete
turnaround!!
Decision #2: Choose between:
Observation #1: Loss aversion is greater than desire for gains.
Observation #2: Framing isn’t a small concern.
26
An FFRDC operated by Analytic Services Inc. on behalf of DHS
From Tversky, Preference, Belief, and Similarity