As one example of the central limit theorem, consider the familiar Bernoulli distribution for n successes in m + n trials:
p(n, m | q) =
(n + m)! n
q (1 − q)m
n! m!
in the limit of large m, n. Recall Stirling’s formula (see next page) that for large N ,
√
N ! ∼ e−N N N 2πN , so we can write
√
n 1 − q m
(n + m)! n
n + m (n + m)n+m n
1 q
m
m
√
q (1−q) ∼ √
q
(1−q)
=
.
n! m!
nn mm
m/(m + n)
2πnm
σ 2π n/(m + n)
To simplify this, we set p = n/(n + m), so that 1 − p = m/(n + m), and let σ =
p
(n + m)p(1 − p), giving
1−q
(n + m) p ln pq + (1 − p) ln 1−p
1 q n 1 − q m
1
p(n, m | q) ∼ √
= √
e
1−p
σ 2π p
σ 2π
.
(1)
The function on the right is strongly peaked near its maximum p = p0 , where the
exponent (n + m)f (p) can be approximated in terms of the Taylor series f (p) ≈ f (p0 ) +
f 0 (p)|p=p0 + 12 f 00 (p)|p=p0 . The point p0 where the maximum occurs is determined by the
vanishing of the first derivative f 0 (p)|p=p0 = 0. The first derivative satisfies
f 0 (p) =
d
d
q
1 − q
q
1−q
f (p) =
p ln + (1 − p) ln
= ln − ln
,
dp
dp
p
1−p
p
1−p
and thus vanishes at p = q. We see that the value at the maximum as well vanishes,
f (q) = 0, so the function is approximated by f (p) ≈ 12 f 00 (p)|p=q near the maximum.
Taking the second derivative gives
d 0
d q
1 − q
1
1
1
f (p) =
f (p) =
ln − ln
=− −
=−
,
dp
dp
p
1−p
p 1−p
p(1 − p)
00
Inserting this into the exponent of (1) and changing variables from p back to n shows that
the Bernoulli form tends to a normal distribution in this large n, m limit:
(n−µ)
1
1
− n+m (p − q)2
p(n, m | q) ∼ √
e 2p(1−p)
= √
e− 2σ2
σ 2π
σ 2π
where as expected µ = (n + m)q and σ =
2
(2)
p
(n + m)q(1 − q).
1
INFO2950 14 Nov 13
√
To prove Stirling’s formula, N ! ∼ 2πN N N e−N for large N , we first recall the method
of steepest descent. An integral of the form,
Z
b
I=
eN f (x) dx ,
a
when N is large, is dominated by the “critical points” x0 of f , at which f 0 (x0 ) = 0. At
any such point, we can approximate in the near vicinity of x0 ,
1
f (x) ≈ f (x0 ) − |f 00 (x0 )|(x − x0 )2 .
2
(1)
Assuming only a single critical point, we can then approximate
I ≈ eN f (x0 )
Z
s
∞
e
− 21 N |f 00 (x0 )|x2
dx = eN f (x0 )
−∞
2π
N |f 00 (x0 )|
(where we have also extended the limits of integration from −∞ to ∞, assuming that only
p
R∞
2
the region near x0 matters anyway, and then used −∞ dx e−ax = π/a † ).
To apply to the factorial function, we first need an integral representation. The
function
Z ∞
e−x xN dx
Γ(N + 1) =
0
satisfies the recursion relation
Γ(N + 1) = −e
−x
Z
∞
x +N
∞
N
0
e−x xN −1 dx = N Γ(N )
0
(using integration by parts). Together with the boundary condition Γ(1) =
we see that Γ(N + 1) = N ! for integer N .
So we write
Z ∞
N ! = Γ(N + 1) = N N +1
R∞
0
e−x dx = 1,
eN (ln z−z) dz
0
where z = x/N . Hence f = ln z − z, f 0 = 1/z − 1, f 00 = −1/z 2 , and z0 = 1. Eqn. (1) gives
r
N! ≈ N
N +1 −N
e
2π √
= 2πN N N e−N .
N
R∞
2
If we write J = −∞ dx e−x , then using r, θ cylindrical coordinates its square can be
R∞
R∞
R 2π R ∞
2
2
2
2 ∞
written J 2 = −∞ dx −∞ dy e−x −y = 0 dθ 0 dr r e−r = 2π − 12 e−r = π . Thus
0
p
R∞
√
2
J = π and −∞ dx e−ax = π/a.
†
2
INFO2950 14 Nov 13