01/30/07 Conditional probability, conditional expectation
Reference: Grigoriu Sec. 2.11, Bertsekas Sec. 3.5
Office hours: Wednesdays and Fridays 4-5 PM
For random vectors X and Y which have a joint
probability density, one can use a simpler formula for the
conditional probability density:
Conditional expectation:
Simple way to handle this:
This is an example of a general principle that you
can apply the usual (unconditional) rules for
probability and expectation also in the conditional
case provided that all probability distributions and
expectations in the formula are conditioned on the
same event.
Note also the above formula works when B is an
event involving another random vector Y, for
example,
There is a more abstract formulation of conditional
expectation which you can find in mathematical
sdenotes013007b Page 1
There is a more abstract formulation of conditional
expectation which you can find in mathematical
texts, and we'll come back to it later.
Some useful formulas for calculations involving
multiple random variables:
computing Marginal probability density function
from joint probability density function:
More generally:
Sums of random variables:
sdenotes013007b Page 2
There is no assumption here that Y and
X are independent. The point is that
when we condition on the random
variable X, it behaves deterministically
with respect to that conditional
expectation.
Important class of jointly distributed random variables is the jointly
Gaussian (jointly normal) distribution.
Joint Gaussian (normal) probability density:
One important property of jointly Gaussian
distributed random variables is that if their
sdenotes013007b Page 3
covariance is 0, then they are independent (and
vice versa).
Lots of known formulas for Gaussian
distributions: conditional probability,
conditional expectation, …
Another important property of jointly Gaussian
random variables is that:
ƒ zero covariance implies independence (and
vice versa)
ƒ linear combinations of Gaussian random
vectors are Gaussian random vectors
ƒ conditional distributions and marginal
distributions resulting from jointly Gaussian
distributions are also Gaussian
Simple calculation example: We will derive the formula for the variance of a
sum of (general) random variables from the formula for expectation of linear
combination of r.v.s
sdenotes013007b Page 4
Simulating Random Variables numerically
Online readings will be posted
Inverse Transform Method (always works but can be rather
impractical to program and/or expensive to compute)
Given a one-dimensional random variable X on a state space
S which is a subset of the real line.
sdenotes013007b Page 5
What does it mean for the computer to
simulate a "pseudorandom" number?
Pseudorandom numbers are generated by
deterministic algorithms that are designed to
behave as much as possible like random
numbers do. If you want to be really careful
about pseudorandom number generators, then
L'Ecuyer has up-to-date high quality
algorithms.
sdenotes013007b Page 6
sdenotes013007b Page 7
What about Gaussian random variables?
Inverse transform method is not very efficient
because the CDF for a Gaussian is erf, need to
invert that. So instead, one uses typically one
of two clever algorithms that are specially
designed for Gaussian random variables:
Box Muller Method
Polar-Marsagla Method
sdenotes013007b Page 8
In Cartesian coordinates:
When I change variables (such as going to
polar coordinates) or more generally, map
one set of random variables to another,
then how do we relate the joint probability
density of the original set of random
variables to the pdf for the new set of
variables?
sdenotes013007b Page 9
Change of
because the mapping
is one-to-one (except for singularity at origin, but
this has zero measures so no problem.)
More generally, if the mapping
has full rank but may have multiple inverse images, the above
argument must be modified by starting with:
and then repeating the argument, we arrive at
the more general formula:
See Grigoriu Sec. 2.11
sdenotes013007b Page 10