The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
An Introduction to Gaussian Processes in
Psychology
www.wright.edu/∼joseph.houpt/GP
Joe Houpt
1
Greg Cox
1 Wright
2
Rich Shiffrin
State University
2 Indiana
University
MathPsych 2014
2
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Outline
1
The Basics
2
Gaussian Process Regression
3
Analysis of Mouse Movements
4
Psychometric Functions
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Outline
1
The Basics
2
Gaussian Process Regression
3
Analysis of Mouse Movements
4
Psychometric Functions
Psychometric Functions
The Basics
Gaussian Process Regression
What is a Gaussian Process?
0.0
0.1
0.2
0.3
0.4
Univariate Normal
−4
−2
0
2
x
N (µ, σ 2 )
4
Analysis of Mouse Movements
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
What is a Gaussian Process?
Bivariate Normal
y
0.0
0.1
0.2
0.3
0.4
Univariate Normal
−4
−2
0
2
4
x
x
N (µ, σ 2 )
N (µ, Σ)
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
What is a Gaussian Process?
Gaussian Process
Bivariate Normal
f(x*)
y
0.0
0.1
0.2
0.3
0.4
Univariate Normal
−4
−2
0
2
4
x
x
N (µ, σ 2 )
N (µ, Σ)
x*
N (µ(x), cov (x1 , x2 ))
The Basics
Gaussian Process Regression
Familiar Gaussian Processes
... at least to the Math Psych audience
f(x*)
White Noise
0
−2
−4
f(x*)
1
2
3
x*
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
Analysis of Mouse Movements
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Familiar Gaussian Processes
... at least to the Math Psych audience
Brownian Motion
f(x*)
f(x*)
White Noise
x*
0.0
f(x*)
0
−1.0
−2
−4
f(x*)
1
2
0.5
3
x*
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Familiar Gaussian Processes
... at least to the Math Psych audience
f(x*)
Fractional Brownian Motion
f(x*)
Brownian Motion
f(x*)
White Noise
x*
x*
0.5
1.0
1.5
x*
2.0
2.5
3.0
2
1
f(x*)
−1
−2
−1.0
0.0
0
0.0
f(x*)
0
−2
−4
f(x*)
1
2
0.5
3
3
x*
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Formal Definition
Uncountably infinite index set T
Often T refers to time, i.e., T = R+
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Formal Definition
Uncountably infinite index set T
Often T refers to time, i.e., T = R+
Definition
{Xt ; t ∈ T } is a Gaussian process iff for any finite subset
t1 , t2 , . . . tn ∈ T ,
(Xt1 , Xt2 , . . . Xtn ) ∼ Multivariate Gaussian.
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Covariance Functions (Part I)
a.k.a. Kernels
White Noise
Brownian Motion
µ(t) = 0

 σ 2 if s = t
cov (s, t) =
 0 if s 6= t
µ(t) = 0
cov (s, t) = min(s, t)
Fractional Brownian Motion
µ(t) = 0
cov (s, t) =
1
2
|s|2H + |t|2H − |t − s|2H
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Some Useful Properties
X (t), Y (t) are independent,
X (t) ∼ GP (µX (t), cov X (t))
Y (t) ∼ GP (µY (t), cov Y (t)) .
Then,
aX (t)+bY (t) ∼ GP aµX (t) + bµX (t), a2 cov X (t) + b 2 cov Y (t) .
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Some Useful Properties
The derivative* of a Gaussian process is also a Gaussian process
with,
∂
cov (xm , xn )
cov fn , ω j,m =
∂xj
cov ω i,n , ω j,m =
∂2
cov (xm , xn ) .
∂xi ∂xj
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Outline
1
The Basics
2
Gaussian Process Regression
3
Analysis of Mouse Movements
4
Psychometric Functions
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
GP Interpolation
i.e., noise free observations
We have observed values of a function x at a set of times
t1 , t2 , . . . , tn , a.k.a t.
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
GP Interpolation
i.e., noise free observations
We have observed values of a function x at a set of times
t1 , t2 , . . . , tn , a.k.a t.
We want function x at a set of other times t1∗ , t2∗ , . . . , tn∗ , a.k.a
t∗ .
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
GP Interpolation
i.e., noise free observations
We have observed values of a function x at a set of times
t1 , t2 , . . . , tn , a.k.a t.
We want function x at a set of other times t1∗ , t2∗ , . . . , tn∗ , a.k.a
t∗ .


cov (s1 , t1 ) · · · cov (s1 , tm )


..
..
..
cov (s, t) = 

.
.
.
cov (sn , 11 ) · · · cov (sn , tm )
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
GP Interpolation
i.e., noise free observations
We have observed values of a function x at a set of times
t1 , t2 , . . . , tn , a.k.a t.
We want function x at a set of other times t1∗ , t2∗ , . . . , tn∗ , a.k.a
t∗ .


cov (s1 , t1 ) · · · cov (s1 , tm )


..
..
..
cov (s, t) = 

.
.
.
cov (sn , 11 ) · · · cov (sn , tm )
Gaussian Process Model:
x(t)
cov (t, t) cov (t, t∗ )
∼ N 0,
x(t∗ )
cov (t∗ , t) cov (t∗ , t∗ )
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
GP Interpolation
i.e., noise free observations
Then, x(t∗ )|x(t) has a multivariate Gaussian distribution with,
µ = cov (t, t∗ )cov (t∗ , t∗ )−1 x(t)
Σ = cov (t, t) − cov (t, t∗ )cov (t∗ , t∗ )−1 cov (t∗ , t)
Gaussian Process Interpolation
3
Gaussian Process Prior
0
−1
f(x*)
+
+
−3
−2
2
1
0
−3 −2 −1
f(x*)
+
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
GP Regression
i.e., noisy observations
xobs (t) = x(t) + i ∼ N(0, σi2 )
For notational convenience, assume σi = σ for all i.
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
GP Regression
i.e., noisy observations
xobs (t) = x(t) + i ∼ N(0, σi2 )
For notational convenience, assume σi = σ for all i.
x(t)
cov (t, t) + σ 2 I cov (t, t∗ )
∼ N 0,
x(t∗ )
cov (t∗ , t)
cov (t∗ , t∗ )
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
GP Regression
i.e., noise free observations
Then, x(t∗ )|x(t) has a multivariate Gaussian distribution with,
µ = cov (t, t∗ )cov t∗ , t∗ + σ 2 I
∗
−1
x(t)
∗
Σ = cov (t, t) − cov (t, t )cov t , t∗ + σ 2 I
cov (t∗ , t)
Gaussian Process Regression
2
1
Gaussian Process Prior
−1
−1
f(x*)
0
+
−2
−1
+
−3
−2
f(x*)
1
0
+
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Bayesian Linear Regression
x(t) = w0 + w1 t
iid
w0 , w1 ∼ N(0, 1)
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Bayesian Linear Regression
x(t) = w0 + w1 t
iid
w0 , w1 ∼ N(0, 1)
µ(t) = E [w0 + w1 t]
= E [w0 ] + E [w1 ] t = 0
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Bayesian Linear Regression
x(t) = w0 + w1 t
iid
w0 , w1 ∼ N(0, 1)
µ(t) = E [w0 + w1 t]
= E [w0 ] + E [w1 ] t = 0
cov (s, t) = E [(w0 + w1 s)(w0 + w1 t)]
= E (w02 + w0 w1 (s + t) + w12 st
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Bayesian Linear Regression
x(t) = w0 + w1 t
iid
w0 , w1 ∼ N(0, 1)
µ(t) = E [w0 + w1 t]
= E [w0 ] + E [w1 ] t = 0
cov (s, t) = E [(w0 + w1 s)(w0 + w1 t)]
= E (w02 + w0 w1 (s + t) + w12 st
E [wi2 ] = 1, E [w0 w1 ] = 0
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Bayesian Linear Regression
x(t) = w0 + w1 t
iid
w0 , w1 ∼ N(0, 1)
µ(t) = E [w0 + w1 t]
= E [w0 ] + E [w1 ] t = 0
cov (s, t) = E [(w0 + w1 s)(w0 + w1 t)]
= E (w02 + w0 w1 (s + t) + w12 st
E [wi2 ] = 1, E [w0 w1 ] = 0
= 1 + st
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Bayesian Linear Regression
Generalizes to linear combinations of functions of t (where t is a
vector),
x(t) =
m
X
wi φi (t) = wT φ(t)
i=1
w ∼ N(0, Σ)
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Bayesian Linear Regression
Generalizes to linear combinations of functions of t (where t is a
vector),
x(t) =
m
X
wi φi (t) = wT φ(t)
i=1
w ∼ N(0, Σ)
h i
µ(t) = E wT φ(t) = 0
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Bayesian Linear Regression
Generalizes to linear combinations of functions of t (where t is a
vector),
x(t) =
m
X
wi φi (t) = wT φ(t)
i=1
w ∼ N(0, Σ)
h i
µ(t) = E wT φ(t) = 0
i
h
cov (s, t) = φT (t)E wwT φ(s)
= φT (t)Σφ(s)
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Bayesian Linear Regression
µ(t) = 0 cov (s, t) = 1 + st σnoise = 0.01
Posterior Samples
Point−wise Posterior Uncertainty
f(x*)
f(x*)
1.0
+
0.5
+ +
0.5
0.0
+
−0.5
0.0
f(x*)
0
−5
1.0
5
1.5
1.5
2.0
Prior Samples
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Return to Covariance Functions
Gaussian processes are determined by their mean and
covariance functions, so it is clear that the choice of
covariance function is critical.
Not all functions T n → R+ are valid covariance functions.
Any t, s ∈ T n , cov (t, s) must produce a valid covariance
matrix.
A valid covariance matrix must be positive semi-definite, hence
cov (t, s) must be a positive semi-definite function.
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Return to Covariance Functions
Gaussian processes are determined by their mean and
covariance functions, so it is clear that the choice of
covariance function is critical.
Not all functions T n → R+ are valid covariance functions.
Any t, s ∈ T n , cov (t, s) must produce a valid covariance
matrix.
A valid covariance matrix must be positive semi-definite, hence
cov (t, s) must be a positive semi-definite function.
Properties of covariance functions
Stationarity: cov (s, t) is a function of s − t (i.e., translations
across time do not change the covariance structure).
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Return to Covariance Functions
Gaussian processes are determined by their mean and
covariance functions, so it is clear that the choice of
covariance function is critical.
Not all functions T n → R+ are valid covariance functions.
Any t, s ∈ T n , cov (t, s) must produce a valid covariance
matrix.
A valid covariance matrix must be positive semi-definite, hence
cov (t, s) must be a positive semi-definite function.
Properties of covariance functions
Stationarity: cov (s, t) is a function of s − t (i.e., translations
across time do not change the covariance structure).
Isotropy: cov (s, t) is a function of |s − t|.
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Return to Covariance Functions
Gaussian processes are determined by their mean and
covariance functions, so it is clear that the choice of
covariance function is critical.
Not all functions T n → R+ are valid covariance functions.
Any t, s ∈ T n , cov (t, s) must produce a valid covariance
matrix.
A valid covariance matrix must be positive semi-definite, hence
cov (t, s) must be a positive semi-definite function.
Properties of covariance functions
Stationarity: cov (s, t) is a function of s − t (i.e., translations
across time do not change the covariance structure).
Isotropy: cov (s, t) is a function of |s − t|.
Rotational invariance: E.g., dot-product covariance, cov (s, t)
is a function of s · t.
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Periodic
µ(t) = 0 cov (s, t) = exp −
l2
Posterior Samples
Point−wise Posterior Uncertainty
0.5
2
1.0
Prior Samples
!
|s−t|
2 sin2 π p
−1.0
−0.5
f(x*)
−1
f(x*)
+
0
f(x*)
1
0.0
0
+
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
−1.5
−2.5
−2
−3
−2.0
−1
−2
+
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Squared Exponential
a.k.a. Radial Basis Function
2
µ(t) = 0 cov (s, t) = τ 2 exp − (s−t)
2
2l
Posterior Samples
Point−wise Posterior Uncertainty
0
2
1
3
Prior Samples
−2
−1
f(x*)
−2
f(x*)
−1
−1
0
+
+
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
−3
−3
−2
−3
f(x*)
1
0
+
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Rational Quadratic
µ(t) = 0 cov (s, t) = 1 +
−α
Posterior Samples
Point−wise Posterior Uncertainty
0.0
3
Prior Samples
(s−t)2
2αl 2
f(x*)
−2.5
−2.0
+
−3.0
−2.5
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
−1.5
−2.0
−1.5
f(x*)
−1.0
+
−3.0
−2
−3
f(x*)
0
−1
−1.0
1
−0.5
−0.5
2
0.0
+
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Matérn Class
µ(t) = 0 cov (s, t) =
21−ν
Γ(ν)
Prior Samples
√
2ν(s−t)
l
ν
Kν
√
Posterior Samples
Point−wise Posterior Uncertainty
−1.0
f(x*)
−2.0
f(x*)
−1.5
+
−3.0
−3.0
−2.5
−2.5
−2.0
0
−1
−2
f(x*)
−1.0
+
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
−1.5
1
−0.5
−0.5
2
0.0
+
0.0
2ν(s−t)
l
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
0.0
0.5
1.0
1.5
x*
2.0
2.5
3.0
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Making New Kernels
If cov 1 (s, t) and cov 2 (s, t) are valid covariance funcitons, the
so are
cov 1 (s, t) + cov 2 (s, t)
cov 1 (s, t)cov 2 (s, t).
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Making New Kernels
If cov 1 (s, t) and cov 2 (s, t) are valid covariance funcitons, the
so are
cov 1 (s, t) + cov 2 (s, t)
cov 1 (s, t)cov 2 (s, t).
(Blur) If h(s, t) is a fixed kernel, then the folowing is a valid
covariance funciton,
R
h(s, z)cov (z, z 0 )h(t, z 0 ) dz dz 0 .
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Outline
1
The Basics
2
Gaussian Process Regression
3
Analysis of Mouse Movements
4
Psychometric Functions
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
A hierarchical Bayesian approach
Hyperparameters of covariance kernel (a, l)
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
A hierarchical Bayesian approach
2
3
Hyperparameters of covariance kernel (a, l)
↓
●
●
1
●
y
−2
−1
0
●
−3
●
−3
−2
−1
0
1
2
3
x
Latent distribution over functions f (x) ∼ GP(0, k(x, x 0 ))
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
A hierarchical Bayesian approach
2
3
Hyperparameters of covariance kernel (a, l)
↓
●
●
1
●
y
−2
−1
0
●
−3
●
−3
−2
−1
0
1
2
3
x
Latent distribution over functions f (x) ∼ GP(0, k(x, x 0 ))
↓
Observed data x ∼ N (0, K (X , X )).
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Spivey, Grosjean, & Knoblich, 2005
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Spivey, Grosjean, & Knoblich, 2005
Psychometric Functions
The Basics
Gaussian Process Regression
Model structure
Analysis of Mouse Movements
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Model structure
Hierarchical models can be constructed by a summation of kernels representing
different sources of variability.
Measurement noise
“white noise”:
δijσ2
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Model structure
Hierarchical models can be constructed by a summation of kernels representing
different sources of variability.
Measurement noise
Single-trial:
kT(ti, tj) + δijσ2
“white noise”:
δijσ2
within-trial variability:
kT(ti, tj)
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Model structure
Hierarchical models can be constructed by a summation of kernels representing
different sources of variability.
Measurement noise
Single-trial:
kT(ti, tj) + δijσ2
“white noise”:
δijσ2
within-trial variability:
kT(ti, tj)
Multiple trials in a condition (sum of
within- and between-trial variability):
kH(ti, tj) + kT(ti, tj) + δijσ2
between-trial variability (longer length-scale):
kH(ti, tj)
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
The GP model
Each trial: (ti , xi , yi )
"
2 #
1 ti − tj
Covariance kernels: kh (xi , xj ) = kh (yi , yj ) = ah exp −
2
lh
|
{z
}
depends on time
"
#
2
1 ti − tj
kt (xi , xj ) = kt (yi , yj ) = at exp −
2
lt
{z
}
|
depends on time
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
The GP model
Each trial: (ti , xi , yi )
"
2 #
1 ti − tj
Covariance kernels: kh (xi , xj ) = kh (yi , yj ) = ah exp −
2
lh
|
{z
}
depends on time
"
#
2
1 ti − tj
kt (xi , xj ) = kt (yi , yj ) = at exp −
2
lt
{z
}
|
depends on time
For simplicity, we assume no direct covariance between conditions
(although hyperparameters are shared), and that the trajectory
structure is determined solely by time, not space (i.e., the x and y
coordinates are conditionally independent, given time).
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Results
Control vs. cohort, overall
−300
100 200
X
−100
−100
x
Cohort−Left
Control−Left
Cohort−Right
Control−Right
0
100
300
X trajectories
0.0
0.4
0.8
Time (s)
1.2
0.0
0.4
0.8
Time (s)
1.2
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Results
Control vs. cohort
position contrast
−300
100
X
−100
−100
x
Cohort−Left
Control−Left
Cohort−Right
Control−Right
0
100
200
300
X trajectories
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Time (s)
0.0
0.2
0.4
0.6
0.8
1.0
∂
1.2
Time (s)
∂x
∂2
Control vs. cohort
acceleration contrast
2000
∂2
0
∂x 2
∂x∂x 0
∂4
−4000
X accel.
The covariance between a
process and its derivatives
means that we can estimate
derivatives by differentiating
the kernel.
∂x 2 ∂x 02
0.0
0.2
0.4
0.6
0.8
Time (s)
1.0
0
k(x, x 0 ) 0
x −x
l2
"
!2
k(x, x 0 )
x − x0
k(x, x ) = −
0
k(x, x ) =
0
k(x, x ) =
0
l2
l
k(x, x 0 )
x − x0
"
l2
k(x, x ) = −
1.2
+3
!2
"
x − x0
l4
!2
l
x − x0
l
!2
#
−3
#
+1
l
k(x, x 0 )
#
−1
3−
x − x0
l
!2 !
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
Results
Control vs. cohort
position contrast
−300
100
X
−100
−100
x
Cohort−Left
Control−Left
Cohort−Right
Control−Right
0
100
200
300
X trajectories
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Time (s)
0.0
0.2
0.4
0.6
0.8
1.0
∂
1.2
Time (s)
∂x
∂2
Control vs. cohort
acceleration contrast
2000
∂2
0
∂x 2
∂x∂x 0
∂4
−4000
X accel.
The covariance between a
process and its derivatives
means that we can estimate
derivatives by differentiating
the kernel.
∂x 2 ∂x 02
0.0
0.2
0.4
0.6
0.8
Time (s)
1.0
0
k(x, x 0 ) 0
x −x
l2
"
!2
k(x, x 0 )
x − x0
k(x, x ) = −
0
k(x, x ) =
0
k(x, x ) =
0
l2
l
k(x, x 0 )
x − x0
"
l2
k(x, x ) = −
1.2
+3
!2
"
x − x0
l4
!2
l
x − x0
l
!2
#
+1
l
k(x, x 0 )
#
−1
3−
x − x0
#
−3
Inflection points can indicate important changes in cognitive processing.
l
!2 !
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Outline
1
The Basics
2
Gaussian Process Regression
3
Analysis of Mouse Movements
4
Psychometric Functions
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
The Data
Goal: Estimate the probability of an observer responding
correctly at a range of signal intensities.
Standard Approach: Fit a parametric function (e.g., Weibull)
to data.
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
The Data
Goal: Estimate the probability of an observer responding
correctly at a range of signal intensities.
Standard Approach: Fit a parametric function (e.g., Weibull)
to data.
Experiment
Method of constant stimuli: present a set of intensities each a
fixed number of times
2AFC: Must choose which of two options contains the signal
(implies chance is 50%)
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
The Data
1.0
True Psychometric Function
0.8
0.6
0.7
+
Chance
+ +
0.5
P(correct)
0.9
+ +
Lapse Rate
+ +
0
1
+
2
3
Intensity
4
5
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
The Model I
Using RBF + White noise kernel
0.9
0.8
0.7
0.6
0.5
0.5
0.6
0.7
0.8
0.9
1.0
Optimized Parameters
1.0
Default Parameters
0
1
2
3
4
5
0
1
2
3
4
5
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
The Model II
0.5
0.6
0.7
0.8
0.9
1.0
Assuming the probability of correct when the signal intensity
is zero is chance (.5).
0
1
2
3
4
5
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
The Model III
1.0
Set the prior to linear between chance at 0 intensity and
perfect at highest intensity.
+
+
+
0.8
0.9
+
0.6
0.7
+
0.5
+
+
0
+
+
1
2
3
4
5
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Psychometric Functions
The Model IV
1.0
Allow uncertainty of point estimates to vary across the range.
+
+
+
0.8
0.6
0.7
+
+
+
0.5
P(Correct)
0.9
+
0
+
1
2
3
Intensity
4
5
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
The Model V
1.0
Assume the slope at the highest intensities is 0.
+
+
+
0.8
0.6
0.7
+
+
+
0.5
P(Correct)
0.9
+
0
+
1
2
3
Intensity
4
5
Psychometric Functions
The Basics
Gaussian Process Regression
Analysis of Mouse Movements
Resources
www.gaussianprocess.org
Gaussian Processes for Machine Learning
Carl Rasmussen and Christopher Williams
In R
gptk: Gaussian Process Took-Kit
GPfit: Gaussian Process Modeling
kernlab: Kernel-based Machine Learning Lab
In Python
pygp: Gaussian process regression in python
scikit-learn: Machine learning in python
Psychometric Functions