Expectation
Conditional expectation
Expectations
Consider a probability space (Ω, F, P) and a random variable X .
How to define the expectation of a random variable X within the
framework given in the previous sections, without violating the
intuitive view on expectation from previous courses?
RMeasure theoretical arguments give that E(X ) could be defined as
ω∈Ω X (ω)P(dω), where the integral is the Lebesgue integral.
This integral is more general than the Riemann integral.
Probability III
Expectation
Conditional expectation
Step 1: simple X
X is a simple random variable if there exists a finite partition
P = {A1 , · · · An } ⊂ F of Ω, s.t. for ω ∈ Ω and x1 , · · · , xn ∈ R,
X
X (ω) =
xi1(ω
1 ∈ Ai ).
i=1
That is, X is an F-measurable simple random variable if it is
F-measurable and takes only a finite number of values.
Recall from the previous lecture:
Proposition
Let the σ-algebra A be generated by a finite partition
P = {A1 , · · · An }. Then the function P
X is A measurable if and
only if X may be written as X (ω) = ni=1 xi1(ω
1 ∈ A1 ) for some
constants x1 , x2 , · · · , xn
Probability III
Expectation
Conditional expectation
Definition
If X is a simple random variable, such that for the finite partition
P = {A1 , · · · An } ⊂ F of Ω, ω ∈ Ω and x1 , · · · , xn ∈ R we have,
X
X (ω) =
xi1(ω
1 ∈ Ai ),
i=1
then the expected value of X is defined by
E(X ) =
n
X
i=1
Probability III
xi P(Ai )
Expectation
Conditional expectation
Lemma
If X , Y are simple random variables and a, b ∈ R, then
(i) E(aX + bY ) = aE(X ) + bE(Y )
(linearity)
(ii) E(X ) ≤ E(Y ) if X ≤ Y a.s.
(monotonicity)
Pn
1 ∈ Ai ) and
Proof: (i)
P Assume X (ω) = i=1 xi1(ω
Y (ω) = m
y
1(ω
1
∈
B
).
We
have
j
j=1 j
Pn Pm
(aX + bY )(ω) = i=1 j=1 (axi + byj )1
1(ω ∈ Ai ∩ Bj )
n X
m
X
E(aX + bY ) =
(axi + byj )P(Ai ∩ Bj ) =
i=1 j=1
n X
m
X
axi P(Ai ∩ Bj ) +
i=1 j=1
n
X
i=1
Probability III
axi P(Ai ) +
n X
m
X
i=1 j=1
m
X
byj P(Ai ∩ Bj ) =
byj P(Bj ) = aE(X ) + bE(Y )
j=1
Expectation
Conditional expectation
Lemma
If X , Y are simple random variables and a, b ∈ R, then
(i) E(aX + bY ) = aE(X ) + bE(Y )
(linearity)
(ii) E(X ) ≤ E(Y ) if X ≤ Y a.s.
(monotonicity)
Proof: (ii) The random variable Y − X is simple and Ω− , the set
on which it is negative, has measure 0.
E(Y − X ) =
n X
m
X
(yj − xi )P(Ai ∩ Bj ) =
i=1 j=1
n X
m
X
i=1 j=1
n X
m
X
i=1 j=1
Probability III
n X
m
X
(yj −xi )P(Ai ∩Bj ∩(Ω\Ω− ))+
(yj −xi )P(Ai ∩Bj ∩Ω− ) ≥
i=1 j=1
n X
m
X
(yj − xi )P(Ai ∩ Bj ∩ (Ω \ Ω− )) +
(yj − xi )P(Ω− ) ≥ 0
i=1 j=1
Expectation
Conditional expectation
Step 2: bounded X
Let X be bounded, say |X | < M.
Let X 0 and X 00 be the classes of simple random variables,
which satisfy X 0 ∈ X 0 ⇒ X 0 ≤ X and X 00 ∈ X 00 ⇒ X 00 ≥ X .
We prove that X − = sup E(X 0 ) = 00inf 00 E(X 00 ) = X + , and
X 0 ∈X 0
X ∈X
call this quantity E(X ).
X − ≤ X + is trivial, so we only have to prove that X + ≤ X − .
, iM
Ai := Ai (n) := {ω; X (ω) ∈ ( (i−1)M
n
n ]} for i = −n, · · · , n
n
n
X (i − 1)M
X
iM
Xn0 =
1(ω
1 ∈ Ai ) and Xn00 =
1(ω
1 ∈ Ai )
n
n
i=−n
E(Xn0
+ M/n)
and therefore,
X− ≥ X+
Probability III
i=−n
00
=
so E(Xn ) − E(Xn0 ) = M/n for
supX 0 ∈X 0 E(X 0 ) ≥ inf X 00 ∈X 00 E(X 00 ) and
E(Xn00 ),
all n
Expectation
Conditional expectation
Lemma
If X , Y are bounded random variables and a, b ∈ R, then
(i) E(aX + bY ) = aE(X ) + bE(Y )
(linearity)
(ii) E(X ) ≤ E(Y ) if X ≤ Y a.s.
(monotonicity)
Proof: (i) First let a, b > 0, for negative a or b the proof is
similar.
sup E(Z 0 ) ≥
sup
E(aX 0 + bY 0 )
Z 0 ≤aX +bY
0
X , Y 0 , Z 0 simple,
X 0 ≤X ,Y 0 ≤Y
for
since the supremum on the right-hand-side
(rhs) is taken over a smaller set of functions. It follows by linearity
of simple functions that
E(aX + bY ) ≥ a sup E(X 0 ) + b sup E(Y 0 ) = aE(X ) + bE(Y )
X 0 ≤X
Y 0 ≤Y
The inverse inequality follows by considering −X and −Y ,
(ii) is proved as is done for simple functions
Probability III
Expectation
Conditional expectation
Step 3: non-negative X
Now s
Define E(X ) := supX 0 ≤X E(X 0 ), where X 0 is bounded
Let X ≥ 0 and define Xn0 = min(X , n) and X 0 ≤ X a given
bounded random variable
Xn0 is increasing in n, therefore E(Xn0 ) is increasing in n and
has a limit (which might be infinite)
Since X 0 is given and bounded, and X 0 ≤ X it holds that
Xn0 ≥ X 0 for large enough n, and therefore, E(Xn0 ) ≥ E(X 0 ) for
large enough n
Similarly any given Xn0 is bounded. Therefore,
E(Xn0 ) ≤ supX 0 ≤X E(X 0 ). So, E(X ) = limn→∞ E(Xn0 )
Probability III
Expectation
Conditional expectation
Similar to the previous slides it is straightforward to prove
Lemma
If X , Y are non-negative random variables and a, b ∈ R, then
(i) E(aX + bY ) = aE(X ) + bE(Y )
(linearity)
(ii) E(X ) ≤ E(Y ) if X ≤ Y a.s.
(monotonicity)
Probability III
Expectation
Conditional expectation
Step 4: a bit more than integrable X
Define X + (ω) = max(X (ω), 0) and X − (ω) = max(−X (ω), 0)
X + and X − are non-negative and X = X + − X −
Assume that min(E(X + ), E(X − )) < ∞
E(X ) = E(X + ) − E(X − )
It follows immediately that
Lemma
If X , Y satisfy min(E(X + ), E(X − )) < ∞, then
(i) E(aX + bY ) = aE(X ) + bE(Y )
(linearity)
(ii) E(X ) ≤ E(Y ) if X ≤ Y a.s.
(monotonicity)
Probability III
Expectation
Conditional expectation
Let X be a random variable and let g : R → R be such that
g (X ) is also a random variable
Assume min(E(g (X )+ ), E(g (X )− )) < ∞
Using the arguments from the lemmas above (with
increasingly complicated g ) we obtain
P
If X is discrete, then E(g (X )) = xRg (x)P(X = x)
If X is continuous, then E(g (X )) = R g (x)fX (x)dx
Probability III
Expectation
Conditional expectation
Conditional expectation
Consider a probability space (Ω, F, P)
Let P = {A1 , · · · , An } be a finite partition, which generates
the σ-algebra A ⊂ F,
Let X be a F-measurable random variable, which is not
necessarily A-measurable
E(X |Ai ) = E(X1(ω
1 ∈ Ai ))/P(Ai )
P
Definition: E(X |A)(ω) = ni=1 E(X |Ai )1
1(ω ∈ Ai ) for ω ∈ Ω
If P = {Ω}, then E(X |A) = E(X )
E(X |A) is a function from P to R.
Note that E(X |A) is a A measurable random variable.
Probability III
Expectation
Conditional expectation
Lemma
If X , Y are integrable random variables, a, b ∈ R and
P = {A1 , · · · , An } is a finite partition, then
(i) E(aX + bY |A) = aE(X |A) + bE(Y |A)
(linearity)
(ii) E(X |A) ≤ E(Y |A) a.s. if X ≤ Y a.s.
(monotonicity)
Proof:(i) If ω ∈ Ai , then
E(aX + bY |A)(ω) = E(aX + bY |Ai ),
which by linearity of ordinary expectation is equal to
aE(X |Ai ) + bE(Y |Ai ) = aE(X |A)(ω) + bE(Y |A)(ω)
(2) follows similarly
Probability III
Expectation
Conditional expectation
Lemma
Conditional expectations have the following properties
(i) E(X |A) is A-measurable
(ii) if Y is bounded and A-measurable then E(XY |A) = Y E(X |A)
(iii) E(X ) = E(E(X |A))
(iv ) If A ⊂ B, then E(E(X |A)|B) = E(X |A) and
E(E(X |B)|A) = E(X |A) i.e. the smallest σ-algebra wins
Proof: (i), (ii) see AD, (iv ) exercise 5.20
E(E(X |A)) = E(
(iii)
n
X
E(X |Ai )1
1(ω ∈ Ai )) =
i=1
n
n
X
X
E(X1(ω
1 ∈ Ai ))
E(X1(ω
1 ∈ Ai ))
1(ω
1 ∈ Ai )) =
P(Ai )
E(
P(Ai )
P(Ai )
i=1
Probability III
i=1
Expectation
Conditional expectation
Jensen’s inequality
φ : R → R is convex if for λ ∈ (0, 1) and x, y ∈ R, we have
φ(λx + (1 − λ)y ) ≤ λφ(x) + (1 − λ)φ(y )
We have E(φ(X )) ≥ φ(E(X )).
To prove this, let c = E(X ) and l(x) = ax + b where a and b
are chosen such that l(c) = φ(c) and l(x) ≤ φ(x) for all
x ∈R
E(φ(X )) ≥ E(l(X )) = aE(X ) + b = ac + b = l(c) = φ(c)
in a similar way we can prove that if A ⊂ F is a σ-algebra
then E(φ(X )|A) ≥ φ(E(X |A)).
Probability III