Perfect simulation of repulsive point
processes
Mark Huber
Department of Mathematical Sciences, Claremont McKenna College
29 November, 2011
Mark Huber, CMC
Perfect simulation of repulsive point processes
1/56
Some repulsive things
Spanish towns
Mark Huber, CMC
Pine trees
Perfect simulation of repulsive point processes
2/56
Two data sets
Locations: Spanish towns
Mark Huber, CMC
Locations: Swedish pines
Perfect simulation of repulsive point processes
3/56
Points farther apart than under uniformity
Locations: Spanish towns
Mark Huber, CMC
Locations: Uniform placement
Perfect simulation of repulsive point processes
4/56
Modeling repulsion
Mark Huber, CMC
Perfect simulation of repulsive point processes
5/56
Modeling repulsion
Two common modeling approaches
1
Give story about how process developed (time evolution)
2
Family of densities with respect to Poisson point process
(density)
Both based on Poisson point process
# of points has Poisson distribution
Given the # of points, place points uniformly independently
Mean of Poisson is λ times size of region
Mark Huber, CMC
Perfect simulation of repulsive point processes
6/56
Today’s talk
Matérn type III process
Time evolution
Density
Strauss process
Density
Mark Huber, CMC
Time evolution
Perfect simulation of repulsive point processes
7/56
Strauss process uses density
Penalize PPP as follows
Distance parameter R
Penalty parameter γ ∈ (0, 1)
Intensity parameter λ
For points x, let r (x) = # of pairs less than R apart
For points x, density f (x) = γ r (x) λ#x /Z (γ, λ)
Parameter effects
Big R spaces points farther apart
Small γ means fewer points violate R distance
Higher λ means more points
Mark Huber, CMC
Perfect simulation of repulsive point processes
8/56
Example of Strauss
R = .02
f (x) = γ 6 λ55 / Z
Mark Huber, CMC
Perfect simulation of repulsive point processes
9/56
Basic perfect simulation for Strauss
Acceptance rejection
Draw a PPP X with intensity λ
Find r (X )
Accept X as Strauss draw with probability γ r (X )
Otherwise, reject and start over
Takes too long unless λ small or γ big
Mark Huber, CMC
Perfect simulation of repulsive point processes
10/56
Matérn Type I
Matérn is a storyteller
Introduced three ways of explaining the repulsion
Type I, II, and III
Simulate a Matérn Type I as follows
First simulate a Poisson point process (PPP)
Remove any point within R distance of another point
Mark Huber, CMC
Perfect simulation of repulsive point processes
11/56
Matérn Type I example
'$
'$
bt
at
'$
c
&%
&%t
Circles of radius R/2
Circles touch =
points eliminated
Points a, b, c eliminated
&%
'$
'$
dt
et
&%
&%
Mark Huber, CMC
Perfect simulation of repulsive point processes
12/56
Type I: After removal
Call a, b, c ghost points
ad
bd
Call d, e seen points
Ghost points exert
invisible pressure
cd
et
dt
Mark Huber, CMC
Perfect simulation of repulsive point processes
13/56
Type I: Comments
Problems
Too many eliminations
As λ → ∞, # of points → 0
Need method that preserves some points
Mark Huber, CMC
Perfect simulation of repulsive point processes
14/56
Matérn Type II: Older points rule!
Simulate a Matérn Type II as follows
First simulate a Poisson point process (PPP)
Assign each point a birthday in [0, ∞)
Remove any point within R distance of an older point
Mark Huber, CMC
Perfect simulation of repulsive point processes
15/56
Type II: Picture
'$
'$
bt
at
'$
.39
.27 &%
c
&%t
Circles of radius R/2
Point a eliminates b
Point b eliminates c
.84
&%
'$
'$
dt
et
.52
&%
.01
&%
Mark Huber, CMC
Perfect simulation of repulsive point processes
16/56
Type II: After thinning
Call b, c ghost points
at
bd
Call a, d, e seen points
Ghost points exert
invisible pressure
cd
et
dt
Mark Huber, CMC
Perfect simulation of repulsive point processes
17/56
Type II: Comments
Better, but not perfect
First points survive
Higher number of points than Type I
Why should b take out c if already killed by a?
Mark Huber, CMC
Perfect simulation of repulsive point processes
18/56
Matérn Type III
Simulate a Matérn Type III as follows
First simulate a Poisson point process (PPP)
Assign each point a birthday in [0, ∞)
Run time forward
Only allow point birth if not within R of older born point
Mark Huber, CMC
Perfect simulation of repulsive point processes
19/56
Type III: Picture
'$
'$
bt
at
'$
Circles of radius R/2
Point a eliminates b
.39
.27 &%
c
&%t
.84
&%
'$
'$
dt
et
.52
&%
.01
&%
Mark Huber, CMC
Perfect simulation of repulsive point processes
20/56
Type III: After thinning
Call c ghost point
at
bd
Call a, c, d, e seen points
Ghost points exert
invisible pressure
ct
et
dt
Mark Huber, CMC
Perfect simulation of repulsive point processes
21/56
Type III: Comments
The good
Very natural story
Try to add towns/trees
If too close to existing town/tree, dies off
The bad
Density not in closed form–nasty high dimensional integral
Makes it difficult to do maximum likelihood
estimate/posterior
Mark Huber, CMC
Perfect simulation of repulsive point processes
22/56
Using Matérn Type III for inference
The plan
First turn story of Matérn into density (as with Strauss)
Build Markov chain for density
Build perfect sampler around Markov chain
Build product estimator to utilize samples effectively
(Those last two are my area of research)
Mark Huber, CMC
Perfect simulation of repulsive point processes
23/56
Matérn Type III: Time evolution to density
Mark Huber, CMC
Perfect simulation of repulsive point processes
24/56
Casting shadows
The seen points cast a shadow across time
time
1
R = 1.5
space
0
10
Any ghost points must lie in shaded region
More shadow = more space for ghost points
Mark Huber, CMC
Perfect simulation of repulsive point processes
25/56
From shadows to density
Poisson process in spacetime
Looks like PPP conditioned so that no points lie in shadow
Parameters θ = (λ, R), region S
Let Aθ (x, t) be the area of the shadow in spacetime
For seen points x with no shadow violations:
fseen points (x, t|θ) = Cλ#x exp(λAθ (x, t))
Mark Huber, CMC
Perfect simulation of repulsive point processes
26/56
Larger shadow means more likely
Alternate way of drawing Matérn Type III process
Fix points in x
Draw new PPP y to add to x
If all of y lies in shadow of x, accept x as Matérn Type III
Otherwise draw new y and repeat
Mark Huber, CMC
Perfect simulation of repulsive point processes
27/56
Acceptance/rejection leads to density
What is chance of accepting a given y ?
Let |S| be size of region, Aθ (x, t) size of shadow
Probability no points outside of shadow
exp(−λ(|S| − Aθ (x, t))) = exp(λAθ (x, t)) exp(−λ|S|)
Mark Huber, CMC
Perfect simulation of repulsive point processes
28/56
The problem of unknown time of birth
Big problem
We do not see the t values!
To get the density for just x integrate out t:
Z
g(x|θ) = Cλ#x
exp(λAθ (x, t))
t∈[0,1]#x
Doing this integral directly extremely nasty
Even calculating Aθ (x, t) hard
Mark Huber, CMC
Perfect simulation of repulsive point processes
29/56
Monte Carlo methods to the rescue!
Given seen points x, want to approximate g(x|θ)
Monte Carlo approach: treat t values as auxiliary variables
Given x: randomly choose t from f (x, t|θ, x)
Allows use to estimate g(x|θ)
Mark Huber, CMC
Perfect simulation of repulsive point processes
30/56
How to draw time stamps given locations?
Markov chain Monte Carlo (MCMC)
1
Build chain so stationary distribution = target distribution
2
Under mild conditions (φ-irreducibility, aperiodicity) limiting
distribution will equal stationary distribution
3
Run chain “for a long time” to mix, then get samples
Mark Huber, CMC
Perfect simulation of repulsive point processes
31/56
Metropolis protocol for building Markov chain
One step of Metropolis
1
Propose moving from (x, t) to (x, t 0 )
2
Accept move with probability
f (x, t 0 |θ)
min
,1
f (x, t|θ)
3
Otherwise stay where you are
Mark Huber, CMC
Perfect simulation of repulsive point processes
32/56
Even better...
Perfect simulation techniques
For some Markov chains, possible to do better
Can draw exactly from stationary distribution
Without worrying about mixing time
Perfect sampling protocol
Coupling from the past (Propp & Wilson 1994)
CFTP converts Markov chains to perfect samplers
A good property of Markov chain: monotonicity
Monotonicity not necessary for CFTP, is sufficient
Mark Huber, CMC
Perfect simulation of repulsive point processes
33/56
Is Metropolis for Matérn type III monotonic?
Propose changing time stamp for one point in x
How does the shadow change?
time
1
R = 1.5
space
0
10
Probability accept move = exp(-area of shadow change)
Mark Huber, CMC
Perfect simulation of repulsive point processes
34/56
How to flip an exp(−µ) coin
Fun facts about Poisson point processes
For X ∼ PPP(B), P(#X = 0) = exp(−µ(B))
For regions A ⊂ B, X ∼ PPP(B) then X ∩ A ∼ PPP(A)
To check if move in Metropolis
Draw PPP(largest possible change in shadow)
If PPP restricted to actual change in shadow empty, move
Otherwise, stay at current time stamp
Using PPP to flip exponential coins:
First appears in Beskos, Papaspiliopoulos, Roberts (2006)
Perfect simulation of diffusions
Mark Huber, CMC
Perfect simulation of repulsive point processes
35/56
Example
time
1
R = 1.5
space
0
Mark Huber, CMC
10
Perfect simulation of repulsive point processes
36/56
Example
time
1
R = 1.5
space
0
Mark Huber, CMC
10
Perfect simulation of repulsive point processes
36/56
Example
time
1
R = 1.5
space
0
Mark Huber, CMC
10
Perfect simulation of repulsive point processes
36/56
Montonicity of Markov chain step
For t1 ≤ t2 :
Run one step of Markov chain for t1 and t2
Use same auxiliary PPP in change of shadow
Then after step, still have t1 ≤ t2
Immediately gives us perfect sampling!
Mark Huber, CMC
Perfect simulation of repulsive point processes
37/56
Example of Monotonicity
time
1
R = 1.5
t2
t2
t2
t1
t1
0
Mark Huber, CMC
t1
space
10
Perfect simulation of repulsive point processes
38/56
Example of Monotonicity
time
1
R = 1.5
t2
t2
t2
t1
t1
0
Mark Huber, CMC
t1
space
10
Perfect simulation of repulsive point processes
38/56
Example of Monotonicity
time
1
R = 1.5
t2
t2
t2
t1
t1
0
t1
space
10
t2 accepts move
Mark Huber, CMC
Perfect simulation of repulsive point processes
38/56
Example of Monotonicity
time
1
R = 1.5
t2
t2
t2
t1
t1
0
t1
space
10
t2 accepts move
t1 rejects move
Mark Huber, CMC
Perfect simulation of repulsive point processes
38/56
Monotonic CFTP flowchart
CFTP(k )
Output t1
YES
t1 ← all zeros
t2 ← all ones
save RNG seed
Run Markov
chain for k steps
Does
t1 = t2 ?
NO
t1 ← CFTP(2k )
t2 ← t1
reset RNG seed
RNG = Random
Number
Generator
Mark Huber, CMC
Perfect simulation of repulsive point processes
39/56
Monotonic CFTP details
Running CFTP(k )
1
Set t1 to all 0’s, t2 to all 1’s
2
Save seed to random number generator
3
Take k steps in the Markov chain
4
If t1 = t2 , then set Tk to be this common value
Else
5
1
2
3
6
Mark Huber, CMC
Get T0 by calling CFTP(2k ) recursively
Reset seed to random number generator to what it was in
step 2
Get Tk by taking k steps in the Markov chain
Output Tk
Perfect simulation of repulsive point processes
40/56
What’s the point?
Why do we need samples?
Ability to sample gives approximation of nasty integral
Use TPA or IS+TPA to go from samples to integral
Once you have that integral
Gives density of data under Matérn model
Basis for maximum likelihood
...or posterior analysis
Mark Huber, CMC
Perfect simulation of repulsive point processes
41/56
Results: towns
Mark Huber, CMC
Perfect simulation of repulsive point processes
42/56
Results: trees
Mark Huber, CMC
Perfect simulation of repulsive point processes
43/56
Strauss: Density to time evolution
Mark Huber, CMC
Perfect simulation of repulsive point processes
44/56
Preston (1977) Birth-Death Chains
Adding time dimension to Poisson point process
time
Gray bar =
node lifespan
length bar ∼ exp(1)
birth
birth
birth
0
space
birth
birth
time between births
∼ exp(λµ(S))
birth
Mark Huber, CMC
Perfect simulation of repulsive point processes
45/56
Preston Birth-Death Chains
Points are born, and later die
Rate of births is λ times area of region
Each point dies at rate 1
(Rate is parameter of exponential random variable)
Points alive at time 0 form PPP
Mark Huber, CMC
Perfect simulation of repulsive point processes
46/56
Preston for Strauss
Recall Strauss density
Penalty γ for pair of points within distance R
When point is born...
If point within distance R of point already born...
...point only born with probability γ
Mark Huber, CMC
Perfect simulation of repulsive point processes
47/56
Picture for Strauss γ = 0
time
birth
birth
birth
0
space
birth
birth
birth
Mark Huber, CMC
Perfect simulation of repulsive point processes
48/56
Observations
Red points
Strauss process subset of earlier points
Kendall and J. Møller (2000): can find red points by looking
backwards in time
Mark Huber, CMC
Perfect simulation of repulsive point processes
49/56
Adding a Swap move
Swap move
Broder (1986): swap move for perfect matchings
Dyer and Greenhill (2000): swap move for independent
sets of graphs
Adding Swaps to Preston
Huber (2011)
When point not born, give chance to swap
If only blocked by one point...
...remove blocker, allow birth
Mark Huber, CMC
Perfect simulation of repulsive point processes
50/56
Picture with swap
time
birth
birth
birth
0
space
birth
birth
birth
Mark Huber, CMC
Perfect simulation of repulsive point processes
51/56
Swap move helps chain mix better
Easier to find red points
Verified experimentally on plane
Points affect fewer points in future
Need to be blocked by at least two points to affect future
So effect of point on later points cut in half
Mark Huber, CMC
Perfect simulation of repulsive point processes
52/56
Running times with swap
Strauss model on S = [0, 1]2 , γ = 0.5, R = .05
6
3.5
Running time for swap and no swap chains
x 10
Average number of events needed per sample
3
2.5
No swapping
Always swap when possible
2
1.5
1
0.5
0
0
50
100
β1
150
200
250
λ
Mark Huber, CMC
Perfect simulation of repulsive point processes
53/56
Running times with swap
Time for no swap divided by time for swap
No swap times divided by swap times
5
Average no swap divided by average swap
4.5
4
3.5
3
2.5
2
1.5
1
0
50
100
β1
150
200
250
λ
Mark Huber, CMC
Perfect simulation of repulsive point processes
54/56
Conclusions
For Matérn type III models
Built a density (not in closed form)
Can approximate density using perfect MCMC methods
Allows MLE or posterior analysis
For Strauss process
Already had a density (but not in closed form)
Added new type of Markov chain move
Seems to speed up chain in practice
Mark Huber, CMC
Perfect simulation of repulsive point processes
55/56
References
H UBER , M.L. (2011).
Spatial Birth-Death-Swap Chains.
Bernoulli (forthcoming paper, available online)
H UBER , M.L. AND R.L. W OLPERT (2009).
Likelihood based inference for Matérn type III repulsive
point processes.
Advances in Applied Probability 41, 958–977.
M ATÉRN , B. (1986).
Spatial Variation, vol. 36 of Lecture Notes in Statistics.
New York, NY: Springer-Verlag, 2nd ed.
(first edition published 1960 by Statens
Skogsforsningsinstitut, Stockholm).
Mark Huber, CMC
Perfect simulation of repulsive point processes
56/56