Methods in Perception
Methods in Perception Research
Chapter 1: Fundamentals of Sensory Perception
• Qualitative: Getting the big picture.
• Quantitative: Understanding the details
• Threshold-seeking methods
• Magnitude estimation
others, often similar to those used in
• Many
other areas of psych.
1
2
Qualitative Observation
Qualitative Methods in
Perception
•
•
“Thatcher Illusion”
•
Much quantitative work
now trying to find out why
it happens & what it means
Qualitative observation
uncovered the phenomenon
Schwaninger, A., Carbon, C.C., & Leder, H. (2003). Expert face processing: Specialization and constraints.
In G. Schwarzer & H. Leder (Eds.), Development of Face Processing, pp. 81-97. Göttingen: Hogrefe.
3
•
Also called phenomenological or naturalistic
observation methods
•
Relatively non-systematic observation of a given
perceptual phenomenon or environment (e.g., an
illusion)
•
Yields a verbal description of one’s observations,
(possibly with some simple numerical assessment)
•
First step in any study of any perceptual
phenomenon. Gives the “big picture”
4
Qualitative Methods in
Perception
•
Example: Famous perception researcher Jan
Purkinje noticed that his flower bed looked light
red/dark green during the day but dark red/light
green at twilight
•
This phenomenological observation led to the
hypothesis of 2 visual systems, & ultimately to an
understanding of the functions of rods and cones
5
6
Quantitative Methods
Quantitative Methods in
Perception
seeking methods measure a
• Threshold
physical quantity representing a limit of
perceptual ability (i.e., a threshold)
in physical units (meters, decibels,
• Measured
parts-per-million, candelas of light, etc.)
7
•
Absolute threshold - smallest detectable physical
quantity (e.g., 2 dB, 3.57 grams...)
•
Difference threshold - smallest detectable
difference between two physical quantities
8
Quantitative Methods in
Perception
are defined for a given level of
• Thresholds
response accuracy
• Typically we speak of the “50% threshold”,
meaning the physical quantity detectable
(absolute threshold) or the physical
difference detectable (difference threshold)
50% of the time
a threshold can be defined for any level
• But
of accuracy (e.g., 75%, 83.6%...)
Threshold-seeking
Methods
• Classic methods (Fechner, 1850’s)
• Method of adjustment: Quick and dirty
• Method of limits: Easy on observer, fairly
fast and accurate
• Method of constant stimuli:Very slow but
very accurate
• Adaptive methods: Fast, very accurate, but
can be difficult for untrained observers
9
10
Method of Adjustment
Method of Adjustment
• Stimulus intensity is adjusted (usually by the
Instructions: Adjust the intensity of the light
using the slider until you can just barely see it
observer) continuously until observer says he
can just detect it
• Threshold is point to which observer adjusts
the intensity
• Repeated trials averaged for threshold
but not always accurate, due to
• Fast,
inherently subjective nature of adjustment
11
Photometer Reading: 0.9
0.1 cd/m2
0.2
0.3
0.4
0.5
0.6
0.7
0.8
12
Method of Limits
Method of Limits
of different intensities presented in
• Stimuli
ascending and descending order
(descending sequence)
Instructions: For each light intensity, indicate
whether you can detect it.
• Observer responds to whether she
perceived the stimulus
• Cross-over point (between “yes, I see it” and
“no, I don’t”) is the threshold for a sequence
• Average of cross-over points from several
ascending and descending sequences is taken
to obtain final threshold.
Photometer Reading: 0.9
0.2 cd/m2
0.3
0.4
0.5
0.6
0.7
0.8
13
14
Method of Limits
(ascending sequence)
Instructions: For each light intensity, indicate
whether you can detect it.
Example Data From
Method of Limits
•
Why ascending
and descending
sequences?
•
Why different
starting points?
Photometer Reading: 0.9
0.2 cd/m2
0.3
0.4
0.5
0.6
0.7
0.8
15
16
Method of Constant
Stimuli
• 5 to 9 stimuli of different intensities are
presented many times each, in random order
Method of
Constant Stimuli
Photometer
Readings
Instructions: For each light intensity,
indicate whether you can detect it.
• The intensities must span the threshold, so
must know approx. where it is a priori.
trials (often 100’s) of each intensity
• Multiple
are presented
is the intensity that results in
• Threshold
detection in 50% of trials
17
18
Question
Here are some data from a participant in the MoCS.
Can you estimate her 50% threshold?
Stimulus Intensity
cd/m2
% of stims detected
0.2
0
0.3
0
0.4
0.1
0.5
0.4
0.6
0.7
0.7
0.8
0.8
0.9
0.9
0.9
19
0.7
0.5
0.2
0.9
0.4
cd/m2
The Psychometric
Function
.2 .3 .4 .5 .6 .7 .8 .9
(Stimulus Intensity cd/m2)
20
0.4
0.7
0.4
0.9
0.4
0.2
0.5
0.5
0.5
.
.
.
•
To calculate thresholds we use
curve-fitting techniques to fit a
sigmoidal (=s-shaped) function
to the data (green dots).
•
This is called a psychometric
function (grey line). It links
physical stimulus intensity to
performance
•
Calculating 50%
Threshold Intensity
Math Review: Logs
• Remember exponents? 2 = 4
• Logarithms are the opposite log (4) = 2
exponent will turn the base--usually 2,
• “What
10 or e--into the operand?”
2
2
Debate exists over which kind
of function--Cumulative
Normal, Weibull, Logistic, etc.-is theoretically best, but in
practice differences are minor
• What is the log (100)? log (8)?
• Khan Academy: http://tinyurl.com/7zy5wu7
10
2
21
22
Math Review: e
(or “Euler’s Number”)
Math Review: Inverse
Functions
• A mysterious constant that just pops up
everywhere in nature. e ≈ 2.71828...
• log
is called the “natural logarithm”, often
symbolized as ln
e
• One also sees
ex, where
x can be quite a
complex expression. “exp(x)” is also used.
to use your calculator to do logarithms
• Learn
and work with e
23
• If a function F takes input X and returns Y
then inverse F takes input Y and returns X
• Example:Y = 2X inverts to X = ½Y
y = 1.0 - exp(-(x/b) )
• The Weibull:
• The inverse Weibull x = b(-ln(1-y))
a
(1/a)
24
Math Review: Free
Parameters
Math Review: Free
Parameters
y = 10 +1x
•
Free parameter: Part of a
math function that is adjusted
so that the function fits data
•
Example: The intercept (a)
and slope (b) of a line are
free parameters when fitting
a regression line:
y = a + bx;
y = 0 +2x
y = 0 +1x
y = 1.0 - exp(-(x/5.0)1.0)
•
In the Weibull (and its
inverse) a and b are free
parameters.
•
a is the “offset” (in this
example a = 1 in all cases)
•
b is the “slope” (here varying
from 0.5 to 5.0)
y = 1.0 - exp(-(x/1.5)1.0)
y = 1.0 - exp(-(x/1.0)1.0)
y = 1.0 - exp(-(x/0.5)1.0)
25
The Weibull Function
y = 1.0 - exp(-(x/b)a)
•
•
•
•
x is stimulus intensity (in some positive physical unit)
y is predicted probability of stimulus detection (from 0 to 1)
a is the “offset” and b is the “slope”
a and b are free parameters whose values are chosen so that
the curve best fits the data. These values are determined by
curve-fitting algorithms whose details are beyond the scope
of the course.
27
26
Calculating 50% Threshold Intensity
Via Inverse Weibull
x = b(-ln(1-y))(1/a)
• To figure out the threshold, we need to figure
out what the right values of a and b are for the
Weibull that best fits our data.
will then enter the percentage threshold
• We
we are seeking (e.g., .5 for 50% threshold) into
the inverse Weibull (above) to determine the
associated stimulus intensity x
28
Inverse Weibull
step-by-step
Calculating 50% Threshold Intensity
Open the excel file "WeibullFit.xlsx"
•
Click the Tools menu and select "Solver…". Enter your data in two columns:
X (stimulus intensity) in column A
Y (proportion detected) in column B
x = .603 (-ln (1-0.5) )(1/4.08)
Note that you may have to first activate the solver. See the document on the class website called
“Loading MS Solver Add”.
•
In the solver window, click “Options”, then enter “10” in “max time”,
then click “Okay”.
•
Click "Solve" in the window that appears. After a moment, the Fit
Parameters will appear in cells G2 (slope, or A) and G3 (offset, or B)
•
0.9"
0.8"
0.7"
x = .603 (.6931)(.245)
x = .603 (.9141)
0.6"
0.5"
"Data"
0.4"
"Fit"
0.3"
0.2"
0.1"
So for our data, the inverse Weibull is
x = .603 (-ln (1-y) )(1/4.08)
x = .5512
0"
0.00"
0.20"
0.40"
0.60"
0.80"
1.00"
X"
Plug in the desired probability of 0.5 and to a threshold of 0.5512 cd/m2
29
30
Inverse Weibull
step-by-step
Example for Self-Test
Here are some data from a participant in the MoCS. What
is her 50% threshold? What about her 83% threshold?
Stimulus Intensity
x = .603 (-ln (1-0.5) )(1/4.08)
1"
x = .603
(.6931)(1/4.08)
0.9"
0.8"
Y"(Data)"
0.7"
x = .603 (.6931)(.245)
x = .603 (.9141)
0.6"
0.5"
"Data"
0.4"
"Fit"
0.3"
0.2"
0.1"
x = .5512
0"
0.00"
0.20"
0.40"
0.60"
X"
31
0.80"
1.00"
% of stims detected
0
0
1
0.05
2
0.15
3
0.45
4
0.75
5
0.85
6
0.95
7
0.95
32
50% =c.bj
83% = d.fc
•
1"
x = .603 (.6931)(1/4.08)
Y"(Data)"
•
•
Adaptive Methods
Questions
• Examples: Staircase, QUEST, PEST, etc.
• Stimulus first presented at an arbitrary level
• If observer perceives it, intensity is
reduced by a predetermined step-size
• What is a “free parameter”?
• What is log (10000)?
• If observer does not perceive stimulus,
intensity is increased
10
• This is repeated until several reversals are
obtained.
• Threshold is average of reversal points.
33
34
Stimulus Intensity
Reversal Points
700
600
700
500
600
400
500
300
400
200
300
100
0
200
N N
N N
N N
Y
Y
Y
N N
N
Y
Y
Y
N
N
Y
Y
N
Observer Response “Do you see it?”
Threshold = (700 + 400 + 700 + 400 + 600 + 400) / 6
= 533.33
35
100
0
N N
N N
N N
Y
Y
Y
N N
N
Y
Y
Y
N
N
Y
But why is it called the “Staircase Method”?
36
N N
Stimulus Intensity
Example for Self-test
Adaptive Methods
700
600
500
•
Similar to method of limits, but we don’t run the same
series of levels each time, instead “adapting” each series
based on previous performance.
•
A number of specific procedures exist, which modify how
the adaptation is done (size of steps, etc.): Staircase,
QUEST, PEST, etc.
•
Very efficient, because almost all trials are close to the
threshold and therefore informative.
•
But best used with trained psychophysical observers (can
be frustrating for untrained Ss)
400
300
200
100
0
N N
N N
N
Y
Y
Y
N
N N
N
Y
Y
Y
N
N
Y
N
Y
Observer Response “Do you see it?”
Calculate the threshold for this run by this observer.
ebh.f
37
38
Forced Choice
Variations
X-Alternative Forced
Choice Variations
• Some threshold-seeking methods can be
hampered by response bias.
• Some participants have a lax criterion and
tend to say “yes, I see it” a lot.
participants have a strict criterion and
• Some
tend to say “no, I don’t see it” a lot.
way to mitigate this problem is to use
• One
forced-choice methods.
39
a forced-choice task, a participant is given
• Inseveral
options to choose from: One
contains the stimulus and the others don’t.
• The participant is asked, for example:
“Which of the two boxes has a light in it?”
• Note that a forced choice method is a
modification of (addition to) the thresholdseeking methods we’ve looked at so far.
40
X-Alternative Forced
Choice Variations
number of alternatives can be offered,
• Any
but usually 2 to 8 are given.
2AFC Method of Limits
(descending sequence)
Instructions: For each light intensity, indicate
which side it’s on.
• A “2AFC” is a “two-alternative forced
choice” procedure, for example.
• The number of alternatives affects the
“chance level performance” (= 100% / A)
Photometer Reading: 0.9
0.2 cd/m2
0.3
0.4
0.5
0.6
0.7
0.8
41
42
Threshold-finding With
Other Senses
Questions
same methods can be used with a wide
• The
variety of sensory qualities.
• Different physical units are used depending
on the modality:
• Sound amplitude (decibels)
• Touch pressure (pascals)
• Smell/Taste intensity (parts-per-billion)
43
• What is an absolute threshold?
• Name several methods for measuring
absolute thresholds.
• What is an xAFC procedure? Why use one?
generally how the method of limits
• Describe
works.
44
Difference Threshold
• Smallest intensity difference between two
stimuli a person can detect
• Produces a “Just Noticeable Difference” Weber’s Law
•
(JND) in degree of subjective sensation
•
Same psychophysical methods are used as
for absolute threshold
•
As magnitude of stimulus increases, so does
difference threshold (∆I)
45
Weber’s Law describes the relationship between
stimulus intensity (I) and difference threshold (∆I) as
follows
∆I / I = K or ∆I = I × K
•
This holds true for many senses and many physical
quantities across a wide range of moderate intensity
levels (but see Steven’s Power Law)
46
Weber’s Law
47
48
2AFC Method of Limits
for ∆I
Method of Adjustment for
Difference Threshold
Instructions: Adjust the intensity of the test light using the slider until
you can just barely see the difference between it and the standard light
Standard Light:
Photometer
Readings
0.6 cd/m2
Test Light:
0.1 cd/m2
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Instructions: For each set of lights, indicate
which pair (right or left) differ.
49
Standard Light
(Constant)
50
Test Light
(Adjustable)
20
•
It
Q: If your Weber fraction (k) is .05, and the standard
light’s intensity (Is) is 20, what level will the test light
intensity (It)have to be raised to in order for it to be
just noticeably different from the standard?
Standard Light
(Constant)
Test Light
(Adjustable)
50
It
A: It will have to be higher by the ∆I; That is,
It = Is + ∆I
where ∆I = Is × k
∴ It = Is + (Is × k)
It = 20 + (20 × .05) = 21
51
•
Q: If your Weber fraction (k) is .20, and the standard
light’s intensity (Is) is 50, what level will the test light
intensity (It)have to be raised to in order for it to be
just noticeably different from the standard?
52
fj.j
•
Example for Self-Test
Questions
Beyond the Threshold…
• A participant can just tell the difference
between lights of 100 cd/m2 and 112 cd/m2.
Therefore, she should be able to just tell
the difference between 200 cd/m2 and
____ cd/m2.
• What is this participant’s Weber fraction
for light intensity?
53
Beyond Thresholds
Fechner’s Law
• What do we know about the relationship
•
• Classic work done by Fechner, who derived
From Weber’s findings, Fechner derived the idea
that subjective sensation (S) related to stimulus
intensity according to:
S = k × ln(I/I0)
•
•
•
•
k = empirically-determined free parameter
between subjective sensation and objective
intensity at levels above threshold?
Fechner’s Law from Weber’s work.
has since been largely supplanted by
• This
Steven’s Power Law.
55
Recall that “ln” means “log to base e”
I = stimulus intensity
I0 = stimulus intensity at absolute threshold
56
Fechner’s Assumption
Fechner’s Assumption
assumed that each time you went
• Fechner
up by one difference threshold (or,
subjectively, one JND), that that related to a
an equal jump in subjective intensity.
• Intuitively appealing, but turns out to be
wrong, as Stevens later showed.
58
Fechner’s Assumption
Fechner’s Law
Subjective Sensation (S)
57
• Works fairly well for some modalities
(loudness, weight), but not others (electric
shock) that show response expansion
• More closely models the response of
individual neurones than people.
∆I
∆I
∆I
your music player’s volume control is
• Still,
modelled after Weber’s and Fecher’s laws
Objective Stimulus Intensity (I)
59
60
Magnitude Estimation
•
•
•
•
Technique pioneered by Stevens to examine the
relationship between subjective perception and
objective stimulus intensity at easily perceived levels
•
•
All stimuli are well above threshold
•
Observer assigns numbers to the test stimuli relative
to the standard (e.g., “that looks about twice as
bright, I’ll call it a ‘10’. ”);
• Relationship between intensity and
perception usually shows either:
• Response compression: As intensity
increases, the perceived magnitude
increases more slowly than the intensity
Observer is given a standard stimulus and a value for
its intensity (e.g., “see this light? this is a ‘5’.” )
expansion: As intensity increases,
• Response
the perceived magnitude increases more
quickly than the intensity
61
62
Magnitude Estimation
Magnitude Estimation
Relationship between
intensity and perceived
magnitude is a power
function
Stevens’ Power Law
•
Magnitude Estimation
S = k × Ib
(amazingly simple!)
Note how for b < 1, we
get an approximation of
Fechner’s law (red line)
63
•
•
•
•
S = k × Ib
•
Note that k is not the
Weber Fraction
S = perceived magnitude
I = physical intensity
k and b are empiricallydetermined free
parameters
64
The 3 Laws of
Psychophysics
•
Weber’s law relates two physical units: Standard
stimulus intensity and difference threshold
•
Fechner’s law relates a physical unit (stimulus
intensity) with subjective sensation (or so he thought).
•
•
The above two derive from one another directly,
and are sometimes called the Weber-Fechner law
Steven’s Power Law expands on Fechner’s Law,
covering stimuli that show response expansion as well
as more closely modelling human responses.
65
Sensitivity & Signal
Detection Theory
67
Questions
Steven’s Power Law: S = k × Ib Given that for bitterness b=2 and k=2, what
will sensory magnitude (S) be for stimulus
intensity (I) of 2 ppm?
What if I is doubled to 4 ppm? What does
this mean about the relationship between
stimulus intensity and sensory magnitude in
this case?
66
Sensitivity & Signal
Detection Theory
• Sensitivity (d’)
• Criterion (c)
spend some time on this, as it is used in
• Will
many fields (though with different jargon):
• Diagnostics
• Inferential statistics
• Human factors engineering
68
Absolute Thresholds
Ain’t So Absolute
•
Thresholds shift for many reasons unrelated to actual
sensitivity.
•
An especially problematic factor is criterion (tendency
to say “yes” a lot or “no” a lot).
•
For example, an individual doing Method of Limits
could just say “yes” to all stimuli (lax criterion) and
look like he’s incredibly sensitive!
•
xAFC methods are one way to deal with this, but
another is to measure Sensitivity instead of threshold.
Sensitivity
• Sensitivity is symbolized d' (“dee-prime”).
• Measure of one’s ability to detect a given signal
(= a stimulus), usually at low intensity.
• Arises from Signal Detection Theory (more later)
• How might we measure sensitivity?
69
70
(a bad way of)
(a good way of)
Measuring Sensitivity
Measuring Sensitivity
• Present stimulus 100 times and note % of
times participant says he detects it?
P who says “yes” all the time will
• Problem:
do very well. (the very problem we are trying to avoid!)
• Need measure that reflects ability to
discriminate between “signal present” and
“signal absent”
71
•
Present stimulus (signal) on only half of the trials,
(test trials).
•
Present no stimulus (noise) on other half of trials,
(catch trials).
•
Note that unlike threshold experiments, we
present only one stimulus at one intensity.
•
Many trials are presented. For each, the participant
says “yes, the stimulus is there” or “no, it isn’t”.
72
Four Possible Results on Each Trial
Also Known As...
Present
Absent
Yes, I see it
Hit
False Alarm
No, I don’t
Miss
Correct
Rejection
The difference is really...
Statistical test says...
Participant says...
The stimulus is really...
Yes, it’s there
No, it’s not
Present
Absent
True Positive
False Positive
(type I error)
False Negative
(type II error) True Negative
73
74
Some Example Results
No, I don’t
Absent
(n=100)
90
20
10
Benny
Participant says...
Yes, I see it
Present
(n =100)
80
Present
(n =100)
Absent
(n=100)
Yes, I see it
80
10
No, I don’t
20
The stimulus is really...
Claire
Participant says...
Participant says...
Albert
Sensitivity
The stimulus is really...
The stimulus is really...
Present (n =100)
Absent
(n=100)
Yes, I see it
90
30
No, I don’t
10
70
75
results of such an experiment yield:
• The
proportion of hits (P = N / N
)
h
90
hits
testtrials
proportion of FAs (Pfa= Nfa / Ncatchtrials)
example, for Albert: • For
P = 90 / 100 = .9
h
Pfa= 20 / 100 = .2
• Note that we could calc proportions of
misses and correct rejections too, but these
are redundant (Pm = 1-Ph; Pcr = 1-Pfa)
76
Questions
Sensitivity
• Perfect participant: P = 1, P = 0
• Participant just guessing: P = .5, P = .5
• Worst possible participant (perfectly
h
• What is Benny’s proportion of hits (P )?
• What is his proportion of FA’s (P )?
• How do sensitivity experiments differ in
h
fa
method from threshold-seeking methods?
h
fa
backwards) : Ph = 0, Pfa= 1
• In calculating sensitivity, we want to reward
hits and punish FAs, so we could just use
“Basic Sensitivity”: BS = Ph - Pfa
77
78
we calculate
• Instead
d' = z(P ) - z(P )
Sensitivity
•
•
•
•
•
fa
h
FA
Careful! This means function z,
do NOT multiply z by PFA)
the proportions of hits and FAs
• Converting
to z-scores yields a more valid result.
BS equals:
Perfect participant: Ph of 1 - PFA of 0
=1
Participant guessing: Ph of .5 - PFA of .5 = 0
Backward participant: Ph of 0 - PFA of 1 = -1
So BS seems to work. However, for obscure
statistical reasons, BS is, well, B.S.
79
• d’ is measured in standard deviation units.
• How to calculate the z scores?
• In Excel, use norminv(P, 0, 1)
• Table A5.1 from MacMillan & Creelman
• Or the “unit normal” table from any stats
textbook.
80
Normal distribution for finding d'
(based on Macmillan & Creelman (2005), Signal Detection: A User’s Guide)
Sensitivity
•
•
d' = z(Ph) - z(Pfa)
•
Ben: Ph = .8 ∴ z = 0.84; Pfa = .1 ∴ z = -1.28
∴ d’ = 0.84- (-1.28) = 2.12
•
Claire: Ph = .9 ∴ z = 1.28; Pfa = .3 ∴ z = -0.52
∴ d’ = 1.28 - (-0.52) = 1.8
Albert: Ph = .9 ∴ z = 1.28; Pfa = .2 ∴ z = -0.84
∴ d’ = 1.28 - (-0.84) = 2.12
81
(MacMillan & Creelman, 2005, chp. 1). For
our purposes, just substitute values of 0.01
and 0.99, respectively.
83
The stimulus is really...
Present
(n =100)
Absent
(n=100)
Yes, I see it
0
0
No, I don’t
100
100
Flawless
Fran
Participant says...
Doubting
Dan
Participant says...
• What to do with proportions of 0 and 1?
• These yield z scores of –∞ and +∞
• There are many ways of getting around this
82
Participant says...
Zero and One
Link back to data
(#91)
Easy
Eddie
Present
(n =100)
Absent
(n=100)
Yes, I see it
100
100
No, I don’t
0
0
The stimulus is really...
Present (n =100)
Absent
(n=100)
Yes, I see it
100
0
No, I don’t
0
100
84
The stimulus is really...
Sensitivity
•
Eddie: Ph = 1 ∴ z = 2.33; PFA = 1 ∴ z = 2.33
∴ d’ = 2.33 - 2.33 = 0 (usual minimum)
•
Dan: Ph = 0 ∴ z = -2.33; PFA = 0 ∴ z = -2.33
∴ d’ = -2.33 - (-2.33) = 0 (usual minimum)
Fran: Ph = 1 ∴ z = 2.33; PFA = 0 ∴ z = -2.33
∴ d’ = 2.33 - (-2.33) = 4.66 (conventional max.)
for self-test: Zack does a
• Example
sensitivity experiment with 20 test trials
and 20 catch trials. He gets 10 hits and 5
false alarms.
• What are his P
• What is is d’?
h
85
and Pfa values?
d’ = .fg
d' = z(Ph) - z(PFA)
Ph = .ej
Pfa= .be
•
•
Questions
86
Criterion
Criterion
Link back to data
(#91)
• The flip side of sensitivity is criterion or
response bias.
• How do we measure a person’s tendency to
say yes or no? There are several measures,
but the simplest is:
c = [z(Ph) + z(Pfa)] / -2
• The lower c, the more a person’s tendency
to say “yes”. When c<0, yes > no. When c>0, yes < no.
87
•
•
c = [z(Ph) + z(PFA)] / -2
•
Ben: Ph = .8 ∴ z = 0.84; PFA = .1 ∴ z = -1.28
∴ c = (0.84 -1.28) / -2 = .22 [tends to “no”]
•
Claire: Ph = .9 ∴ z = 1.28; PFA = .3 ∴ z = -0.52
∴ c = (1.28 - 0.52) / -2 = -.38 [tends to “yes”]
Albert: Ph = .9 ∴ z = 1.28; PFA = .2 ∴ z = -0.84
∴ c = (1.28 - 0.84) / -2 = -.22 [tends to “yes”]
88
Isosensitivity Curves
(a.k.a. “ROC curves”)
A range of Ph and Pfa values can produce the same d’.
Pfa
z(Ph) z(Pfa)
0.8
0.4
0.84 -0.25 1.09 -0.295
0.6
0.2
d’
Plotting results for different criterion levels at the same
sensitivity yields an isosensitivity curve (aka, ROC curve)
c
0.25 -0.84 1.09 0.295
1
ROC
Curve
d'
=
1.09
0.8
d’ = 1.09, criterion = -0.30
0.6
Hit
Rate
Ph
Isosensitivity Curves
d’ = 1.09, criterion = 0.30
0.4
0.35 0.07 -0.39 -1.48 1.09 0.935
This occurs when sensitivity remains the same, but
criterion shifts
d’ = 1.09, criterion = 0.94
0.2
0
0
0
Hit Rate (Ph)
0
.5
0
0.5
False Alarm Rate (Pfa)
91
0.8
1
Why Does
Criterion Shift?
0.0
1.0
1.0
Miss Rate (Pm)
.5
0.6
90
Correct Rejection Rate (Pcr)
0.5
0.4
False
Alarm
Rate
89
1.0
1.0
0.2
• Many reasons, but an important one is the
payoff matrix.
• If the cost of a false alarm, or reward for a
correct rejection, is great, one will tend to
make one’s criterion stricter (more “no”)
• If the cost of a miss, or reward for a hit, is
raised, one will tend to make one’s criterion
laxer (more “yes”)
92
Example of a Pay-off Matrix
Leading to a Lax Criterion
The stimulus is really...
Participant says...
Neutral
Cancer is...
Yes, I see it
Patient Saved
Unnecessary Additional
Testing
No, I don’t
Patient
Dies
Yes, I see it
+10$
-$10
No, I don’t
-$10
+10$
The stimulus is really...
The stimulus is really...
Lax
Patient
Goes Home
Absent
(n=100)
Present
(n =100)
Absent
(n=100)
Yes, I see it
+100$
-$10
No, I don’t
-$10
+10$
Strict
Participant says...
Absent
(n=100)
Participant says...
Radiologist
Says...
Present
(n =100)
Present (n =100)
Present
(n =100)
Absent
(n=100)
Yes, I see it
+10$
-$10
No, I don’t
-$10
+100$
93
94
Questions
Questions
Dr. X tests a new field diagnostic procedure
by applying it to 50 individuals known to have
an illness. He finds that it correctly labels all
50 of them as ill, whereas his old procedure
labeled only 40 of them as such. He
concludes that the new procedure is better.
Is his conclusion sound? Why or why not?
95
would one be interested in measuring
• Why
changes in criterion?
• True or False: As one travels up and to the
right along an isosensitivity curve, criterion
rises.
• True or False: A higher criterion value
indicates a stronger tendency to say “yes,
the stimulus is present”
96
Signal Detection Theory
• The concepts of sensitivity and criterion arise
out of Signal Detection Theory (SDT)
• SDT suggests that any attempt at detecting a
signal (stimulus) has to contend with
competing noise
• Noise in this sense is random variations from
the environment (e.g., literal noise) or from
within the detector (e.g., neuron chatter)
SDT
•
•
•
Example:You’re in the shower and expecting a call.
•
If the phone isn’t ringing, you have just noise. If the
phone is ringing, you have signal+noise.
•
d’ is essentially a measure of how similar the signal is to
the noise for you subjectively.
Signal: Phone ringing
Noise: Sound of shower (plus all other sources of
sound), as well as your own internal neuron chatter.
97
98
Probability Distributions of Noise and Signal+Noise
Probability That a Given Perceptual
Effect is Due to N or S+N
noise
only
not at all
phone +
noise
Probability
phone +
noise
Probability
noise
only
a bit
kinda
a lot
unmistakably
How much it sounds subjectively like a phone
not at all
a bit
kinda
completely
How much it sounds subjectively like a phone
Link back to
matrix
99
a lot
100
Different Criteria One Can Adopt
not at all
a bit
kinda
a lot
noise
only
completely
not at all
How much it sounds subjectively like a phone
phone +
noise
Strict
a bit
kinda
Decreasing d’ means decreasing the distance between
the probability distributions. Note how, with a smaller
d’, many misses and/or false alarms result, regardless of
where the criterion is placed.
phone +
noise
d’ (small)
Probability
Probability
d’ (large)
a bit
kinda
a lot
How much it sounds subjectively like a phone
103
completely
102
Increasing d’ means increasing the distance betwen the
probability distributions. Note how, with a greater d’, a
criterion somewhere in the neutral area will produce
very few misses or false alarms.
not at all
a lot
How much it sounds subjectively like a phone
101
noise
only
Result:
Pfa = .05
Ph = .50
d' = 1.64
Probability
phone +
noise
Probability
noise
only
Lax
Different Criteria One Can Adopt
Result:
Pfa = .50
Ph = .95
d' = 1.64
completely
noise
only
not at all
a bit
phone +
noise
kinda
a lot
How much it sounds subjectively like a phone
104
completely
Questions
terms of the SDT, what would be the
• Ineffect
of turning up the phone volume on
the probability curves? What effect would
this have on d’? What about turning down
the shower?
• A guard is watching the woods for visible
signs of intruders. What are some likely
sources of “noise” in his situation?
105
For More Info...
A nice primer:
Stanislaw, H., & Todorov, N. (1999). Calculation of signal
detection theory measures. Behavioral Research
Instruments, Methods and Computers, 31, 137-149.
The bible of SDT:
Macmillan, N. A., & Creelman, C. D. (2005). Detection
Theory: A User's Guide (2nd ed.). Mahwah, N.J.:
Lawrence Erlbaum Associates.
106