Sampling algorithms
and Markov chains
László Lovász
Microsoft Research
One Microsoft Way, Redmond, WA 98052
[email protected]
Sampling: a general algorithmic task
Applications:
- statistics
- simulation
- counting
- numerical integration
- optimization
-…
L: a language in NP, with presentation
L  {x : (y) A( x, y)}
certificate
polynomial time algorithm
Given: x
Find:
- a certificate
- an optimal certificate
- the number of certificates
- a random certificate
(uniform, or given distribution)
One general method for sampling: Markov chains
(+rejection sampling, lifting,…)
Want: sample from distribution p on set V
Construct ergodic Markov chain with
states: V
stationary distribution: p
Simulate (run) the chain for T steps
Output the final state
????????????
mixing time
5
4
5
4
2
3
3
2
1
1
Given: poset
State: compatible
linear order
Transition:
- pick randomly label i<n;
- interchange i and i+1
if possible
Mixing time
 t : distribution after t steps
Roughly: min{t : d ( t , p )  1/10}
Bipartite graph?!
1
t
 t  ( 0  ...   t 1 )
T  max 0 min{t : d (t , p )  1/10}
0
(enough to consider   s )
Conductance
S
V \S
frequency of stepping from S to K\S
in Markov chain: Q( S , V \ S )
in sequence of independent samples: p ( S )p (V \ S )
Q( S , V \ S )
,
conductance: ( S ) 
p ( S )p (V \ S )
  min S ( S )
( x)  min{( S ) : p ( S )  x}
1
1
 T  log(1/ p 0 ) 2


Jerrum - Sinclair
In typical sampling application: log(1/ p 0 ) polynomial
polynomiality of T

1
polynomiality of

But in finer analysis?
Key lemma:
y1  y2  ...  yk  ...  yl  ...  yn
Q ( l ,  k )
1
yl  yk 
Q ( l ,  k )
Proof for
l=k+1
E(# steps from {l ,...n})   y j Q( j,  l )  yl Q( l ,  l )
j l
E(# steps to {l ,...n})   y j Q( j,  l )  yl 1Q( l ,  l )
j l
1
 ( yl  yl 1 )Q( l ,  l )
Simple isoperimetric inequality:
2
voln 1 ( F )  vol( S )
D
L – Simonovits
Dyer – Frieze
Improved isoperimetric inequality:
1
vol( K )
voln 1 ( F )  vol( S ) ln
D
vol( S )


Kannan-L
 

1
 ( x)  min 
ln
,1
 D n x(1  x) 
D2
T 
n 2
After appropriate preprocessing,
T  n3
Lifting Markov chains
T n
T  n2
Diaconis – Holmes – Neal