Chapter 4: Numerical Integration: Deterministic and Monte Carlo Methods
JDEP 384H: Numerical Methods in Business
Instructor: Thomas Shores
Department of Mathematics
Lecture 21, April 3, 2007
110 Kaufmann Center
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
Chapter 4: Numerical Integration: Deterministic and Monte Carlo Methods
Outline
1
Chapter 4: Numerical Integration: Deterministic and Monte
Carlo Methods
BT 4.1: Numerical Integration
BT 4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
BTMethods
4.2: Monte Carlo Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BT 4.3: Generating Pseudorandom Variates
Outline
1
Chapter 4: Numerical Integration: Deterministic and Monte
Carlo Methods
BT 4.1: Numerical Integration
BT 4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Outline
1
Chapter 4: Numerical Integration: Deterministic and Monte
Carlo Methods
BT 4.1: Numerical Integration
BT 4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
The Basic Idea
Monte Carlo Simulation:
Create a quantitative model of a process.
Treat the events that constitute the process as random.
Generate random variables to simulate the events.
Use these values to compute the outcome of the process.
A Guiding Example is Monte Carlo Integration:
We want to approximate
Rb
a
g (x ) dx .
For convenience, assume
gR (x ) ≥ 0, so that this integral represents (positive) area.
1 x
0 e dx = e − 1 ≈ 1.7183 as a test case.
Instructor: Thomas Shores Department of Mathematics
Let's use
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
The Basic Idea
Monte Carlo Simulation:
Create a quantitative model of a process.
Treat the events that constitute the process as random.
Generate random variables to simulate the events.
Use these values to compute the outcome of the process.
A Guiding Example is Monte Carlo Integration:
We want to approximate
Rb
a
g (x ) dx .
For convenience, assume
gR (x ) ≥ 0, so that this integral represents (positive) area.
1 x
0 e dx = e − 1 ≈ 1.7183 as a test case.
Instructor: Thomas Shores Department of Mathematics
Let's use
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
The Basic Idea
Monte Carlo Simulation:
Create a quantitative model of a process.
Treat the events that constitute the process as random.
Generate random variables to simulate the events.
Use these values to compute the outcome of the process.
A Guiding Example is Monte Carlo Integration:
We want to approximate
Rb
a
g (x ) dx .
For convenience, assume
gR (x ) ≥ 0, so that this integral represents (positive) area.
1 x
0 e dx = e − 1 ≈ 1.7183 as a test case.
Instructor: Thomas Shores Department of Mathematics
Let's use
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
The Basic Idea
Monte Carlo Simulation:
Create a quantitative model of a process.
Treat the events that constitute the process as random.
Generate random variables to simulate the events.
Use these values to compute the outcome of the process.
A Guiding Example is Monte Carlo Integration:
We want to approximate
Rb
a
g (x ) dx .
For convenience, assume
gR (x ) ≥ 0, so that this integral represents (positive) area.
1 x
0 e dx = e − 1 ≈ 1.7183 as a test case.
Instructor: Thomas Shores Department of Mathematics
Let's use
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
The Basic Idea
Monte Carlo Simulation:
Create a quantitative model of a process.
Treat the events that constitute the process as random.
Generate random variables to simulate the events.
Use these values to compute the outcome of the process.
A Guiding Example is Monte Carlo Integration:
We want to approximate
Rb
a
g (x ) dx .
For convenience, assume
gR (x ) ≥ 0, so that this integral represents (positive) area.
1 x
0 e dx = e − 1 ≈ 1.7183 as a test case.
Instructor: Thomas Shores Department of Mathematics
Let's use
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
The Basic Idea
Monte Carlo Simulation:
Create a quantitative model of a process.
Treat the events that constitute the process as random.
Generate random variables to simulate the events.
Use these values to compute the outcome of the process.
A Guiding Example is Monte Carlo Integration:
We want to approximate
Rb
a
g (x ) dx .
For convenience, assume
gR (x ) ≥ 0, so that this integral represents (positive) area.
1 x
0 e dx = e − 1 ≈ 1.7183 as a test case.
Instructor: Thomas Shores Department of Mathematics
Let's use
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Monte Carlo Integration
Hit or Miss Monte Carlo Method:
Enclose the graph in a box of known area
0
≤ x ≤ 1,
Throw
and
y
N
0
≤ y ≤ 3,
so
A = 3.)
random darts at the area, uniformly distributed in
x
directions. Note: the event of a dart throw is
represented by a random pair
Count up the number
which
A (in our test case,
Yi
≤ g (Xi ).
Rb
NH
of independent r.v.'s.
of darts that fall in the area, i.e., for
g (x ) dx
A
Z b
NH
g (x ) dx ≈ A .
N
a
Proportionately,
(Xi , Yi )
a
Instructor: Thomas Shores Department of Mathematics
≈
NH
, so we have
N
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Monte Carlo Integration
Hit or Miss Monte Carlo Method:
Enclose the graph in a box of known area
0
≤ x ≤ 1,
Throw
and
y
N
0
≤ y ≤ 3,
so
A = 3.)
random darts at the area, uniformly distributed in
x
directions. Note: the event of a dart throw is
represented by a random pair
Count up the number
which
A (in our test case,
Yi
≤ g (Xi ).
Rb
NH
of independent r.v.'s.
of darts that fall in the area, i.e., for
g (x ) dx
A
Z b
NH
g (x ) dx ≈ A .
N
a
Proportionately,
(Xi , Yi )
a
Instructor: Thomas Shores Department of Mathematics
≈
NH
, so we have
N
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Monte Carlo Integration
Hit or Miss Monte Carlo Method:
Enclose the graph in a box of known area
0
≤ x ≤ 1,
Throw
and
y
N
0
≤ y ≤ 3,
so
A = 3.)
random darts at the area, uniformly distributed in
x
directions. Note: the event of a dart throw is
represented by a random pair
Count up the number
which
A (in our test case,
Yi
≤ g (Xi ).
Rb
NH
of independent r.v.'s.
of darts that fall in the area, i.e., for
g (x ) dx
A
Z b
NH
g (x ) dx ≈ A .
N
a
Proportionately,
(Xi , Yi )
a
Instructor: Thomas Shores Department of Mathematics
≈
NH
, so we have
N
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Monte Carlo Integration
Hit or Miss Monte Carlo Method:
Enclose the graph in a box of known area
0
≤ x ≤ 1,
Throw
and
y
N
0
≤ y ≤ 3,
so
A = 3.)
random darts at the area, uniformly distributed in
x
directions. Note: the event of a dart throw is
represented by a random pair
Count up the number
which
A (in our test case,
Yi
≤ g (Xi ).
Rb
NH
of independent r.v.'s.
of darts that fall in the area, i.e., for
g (x ) dx
A
Z b
NH
g (x ) dx ≈ A .
N
a
Proportionately,
(Xi , Yi )
a
Instructor: Thomas Shores Department of Mathematics
≈
NH
, so we have
N
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Monte Carlo Integration
Hit or Miss Monte Carlo Method:
Enclose the graph in a box of known area
0
≤ x ≤ 1,
Throw
and
y
N
0
≤ y ≤ 3,
so
A = 3.)
random darts at the area, uniformly distributed in
x
directions. Note: the event of a dart throw is
represented by a random pair
Count up the number
which
A (in our test case,
Yi
≤ g (Xi ).
Rb
NH
of independent r.v.'s.
of darts that fall in the area, i.e., for
g (x ) dx
A
Z b
NH
g (x ) dx ≈ A .
N
a
Proportionately,
(Xi , Yi )
a
Instructor: Thomas Shores Department of Mathematics
≈
NH
, so we have
N
JDEP 384H: Numerical Methods in Business
Example Calculation
Carry out the following steps in Matlab
>
>
>
>
>
>
>
>
>
>
help rand
format
rand('seed',0)
A = 3
N = 10
X = rand(N,1);
Y = (3-0)*rand(N,1);
hits = sum(Y <= exp(X))
area = A*(hits/N)
Itrue = exp(1)-1 % now try to improve accuracy
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Monte Carlo Integration
Sample Mean Monte Carlo Method:
Z b
Z b
g (x )
f (x ) dx where f (x )
a
a f (x )
is known positive p.d.f. which vanishes outside [a, b ].
Z b
g (X )
Interpret
g (x ) dx = E
where X is r.v. with p.d.f.
f (X )
a
f (x ).
Take N independent samples of X and deduce
Z b
N
1 X g (Xi )
g (x ) dx ≈
.
N i =1 f (Xi )
a
Write integral as
g (x ) dx =
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Monte Carlo Integration
Sample Mean Monte Carlo Method:
Z b
Z b
g (x )
f (x ) dx where f (x )
a
a f (x )
is known positive p.d.f. which vanishes outside [a, b ].
Z b
g (X )
Interpret
g (x ) dx = E
where X is r.v. with p.d.f.
f (X )
a
f (x ).
Take N independent samples of X and deduce
Z b
N
1 X g (Xi )
g (x ) dx ≈
.
N i =1 f (Xi )
a
Write integral as
g (x ) dx =
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Monte Carlo Integration
Sample Mean Monte Carlo Method:
Z b
Z b
g (x )
f (x ) dx where f (x )
a
a f (x )
is known positive p.d.f. which vanishes outside [a, b ].
Z b
g (X )
Interpret
g (x ) dx = E
where X is r.v. with p.d.f.
f (X )
a
f (x ).
Take N independent samples of X and deduce
Z b
N
1 X g (Xi )
g (x ) dx ≈
.
N i =1 f (Xi )
a
Write integral as
g (x ) dx =
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Monte Carlo Integration
Sample Mean Monte Carlo Method:
Z b
Z b
g (x )
f (x ) dx where f (x )
a
a f (x )
is known positive p.d.f. which vanishes outside [a, b ].
Z b
g (X )
Interpret
g (x ) dx = E
where X is r.v. with p.d.f.
f (X )
a
f (x ).
Take N independent samples of X and deduce
Z b
N
1 X g (Xi )
g (x ) dx ≈
.
N i =1 f (Xi )
a
Write integral as
g (x ) dx =
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
Example Calculation
For our example, take
f (x ) = 1.
Carry out the following steps in
Matlab
>
>
>
>
>
rand('state',0)
N = 10
X = rand(N,1);
expectI = sum(exp(X))/N
Itrue = exp(1)-1 % now try to improve accuracy
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Outline
1
Chapter 4: Numerical Integration: Deterministic and Monte
Carlo Methods
BT 4.1: Numerical Integration
BT 4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Pseudo-Random Variables
About random numbers:
They aren't really random. That's why we call them
pseudo-random. Moreover,
They are completely deterministic, given an initial seed
number, a uniform distribution is usually generated by a
congruential formula (in Matlab,
rand
function.)
As such, they repeat values according to a period which is to
be avoided, as that repeating the numbers introduces
discernable bias (in Matlab, old
has period
19937
2
rand
has period 2
31
− 2,
new
− 1 /2.
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Pseudo-Random Variables
About random numbers:
They aren't really random. That's why we call them
pseudo-random. Moreover,
They are completely deterministic, given an initial seed
number, a uniform distribution is usually generated by a
congruential formula (in Matlab,
rand
function.)
As such, they repeat values according to a period which is to
be avoided, as that repeating the numbers introduces
discernable bias (in Matlab, old
has period
19937
2
rand
has period 2
31
− 2,
new
− 1 /2.
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Pseudo-Random Variables
About random numbers:
They aren't really random. That's why we call them
pseudo-random. Moreover,
They are completely deterministic, given an initial seed
number, a uniform distribution is usually generated by a
congruential formula (in Matlab,
rand
function.)
As such, they repeat values according to a period which is to
be avoided, as that repeating the numbers introduces
discernable bias (in Matlab, old
has period
19937
2
rand
has period 2
31
− 2,
new
− 1 /2.
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Pseudo-Random Variables
About random numbers (continued):
They aren't really random. That's why we call them
pseudo-random. Moreover,
Most other distributions are simulated by various tricks applied
to uniformly generated r.v.'s.
A common method is inverse transform, which uses the fact
that if
F
F (X ) is the c.d.f. for r.v. X , and the inverse function
U F X
X F U
−1 can be found, then
= ( ) is uniformly distributed on
[0, 1], and = −1 ( ) (verify this and use it in an
exponential example.)
Also used are an acceptance-rejection method and
Box-Mueller for normal distributions (in Matlab,
randn
function) both depend on uniform r.v.'s.
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Pseudo-Random Variables
About random numbers (continued):
They aren't really random. That's why we call them
pseudo-random. Moreover,
Most other distributions are simulated by various tricks applied
to uniformly generated r.v.'s.
A common method is inverse transform, which uses the fact
that if
F
F (X ) is the c.d.f. for r.v. X , and the inverse function
U F X
X F U
−1 can be found, then
= ( ) is uniformly distributed on
[0, 1], and = −1 ( ) (verify this and use it in an
exponential example.)
Also used are an acceptance-rejection method and
Box-Mueller for normal distributions (in Matlab,
randn
function) both depend on uniform r.v.'s.
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Pseudo-Random Variables
About random numbers (continued):
They aren't really random. That's why we call them
pseudo-random. Moreover,
Most other distributions are simulated by various tricks applied
to uniformly generated r.v.'s.
A common method is inverse transform, which uses the fact
that if
F
F (X ) is the c.d.f. for r.v. X , and the inverse function
U F X
X F U
−1 can be found, then
= ( ) is uniformly distributed on
[0, 1], and = −1 ( ) (verify this and use it in an
exponential example.)
Also used are an acceptance-rejection method and
Box-Mueller for normal distributions (in Matlab,
randn
function) both depend on uniform r.v.'s.
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business
BT 4.1: Numerical Integration
Chapter 4: Numerical Integration: Deterministic and Monte Carlo
BTMethods
4.2: Monte Carlo Integration
BT 4.3: Generating Pseudorandom Variates
Pseudo-Random Variables
About random numbers (continued):
They aren't really random. That's why we call them
pseudo-random. Moreover,
Most other distributions are simulated by various tricks applied
to uniformly generated r.v.'s.
A common method is inverse transform, which uses the fact
that if
F
F (X ) is the c.d.f. for r.v. X , and the inverse function
U F X
X F U
−1 can be found, then
= ( ) is uniformly distributed on
[0, 1], and = −1 ( ) (verify this and use it in an
exponential example.)
Also used are an acceptance-rejection method and
Box-Mueller for normal distributions (in Matlab,
randn
function) both depend on uniform r.v.'s.
Instructor: Thomas Shores Department of Mathematics
JDEP 384H: Numerical Methods in Business