Proof T : C → C
1
Let T be the following mapping:
T ϕ = max {u (x, a) + β Eϕ [f (x, a, ε)]}
a∈Γ(x)
(1)
and C be the space of continuous and bounded real-valued functions endowed
with the sup-norm1 .
Proposition 1 T maps continuous and bounded real-valued functions into the
space of continuous and bounded real-valued functions i.e. T : C → C
Proof. Assumptions:
(i) x ∈ X ⊆ Rl , X is a convex set.
(ii) u (·) ∈ C
(iii) β ∈ (0, 1)
(iv) Γ (x) is a non-empty, continuous (u.h.c and l.h.c.), and compact-valued
correspondence.
(v) f (x, a, ε) is bounded and continuous in all its arguments2 .
in addition to ϕ ∈ C.
Since u (x, a) is bounded and ϕ (·) is bounded, the sum of two bounded
functions is also bounded. The maximum of a bounded function is also bounded.
Therefore, T ϕ is bounded.
To deal with continuity, let us consider the Theorem of the Maximum. First
of all, we need to show that
u (x, a) + β Eϕ [f (x, a, ε)]
(2)
is continuous. As f (·) and ϕ are continuous functions, the composite function
(ϕ ◦ f ) is also continuous and, moreover, as the expectation is a linear operator,
E [ϕ ◦ f ] is a continuous function. Therefore, (2) is continuous and, given the
assumptions on the constraint correspondence Γ (x), we can invoke the Theorem
of the Maximum3 to ensure that T ϕ is continuous.
1C
is a complete metric space.
(·) being continuous implies that the stochastic structure satisfies the Feller property.
The Feller property states that
Z
ϕ [f (x, a, ε)] dF (ε|x, a)
E [ϕ (xt+1 ) | xt = x, at = a] =
2f
3 Theorem
of the Maximum. Let X ⊆ Rl , Y ⊆ Rm ,
h (x) = max f (x, y)
y∈Γ(x)
and
G (x) = {y ∈ Γ (x) : f (x, y) = h (x)}
such that f : X × Y → R is a continuous function and let Γ : X → Y be a compactvalued and continuous correspondence. Then the function h : X → R is continuous, and the
correspondence G : X → Y is non-empty, compact-valued, and u.h.c.
1
ALTERNATIVE PROOF:
Since the space of continuous and bounded real-valued functions endowed
with the sup-norm is a complete metric space and T is a contraction, we can use
the Contraction Mapping Theorem to ensure that there exists a unique fixed
point i.e.
∃! ϕ ∈ C s.t. ϕ = T ϕ
Therefore, by the previous proof we have that T ϕ ∈ C, then, as there exists a
unique fixed point, ϕ must also belong to C.
2
Proving T maps INCREASING functions into
increasing functions
In particular, we want to show T : D → D where D is the space of increasing,
continuous, and bounded real-valued functions (endowed with the sup-norm i.e.
D is a complete metric space. Thus. we could use the alternative way of proving
the statement).
Proof. Standard assumptions:
(i) x ∈ X ⊆ Rl , X is a convex set.
(ii) u (·) ∈ C
(iii) β ∈ (0, 1)
(iv) Γ (x) is a non-empty, continuous, and compact-valued correspondence.
(v) f (x, a, ε) is bounded and continuous in all its arguments
In order to show that T maps increasing functions into increasing functions,
we need the following additional assumptions:
(vi) u (·) is increasing
(vii) Γ (x) is increasing in the following sense
∀ x, x0 ∈ X s.t. x0 ≥ x =⇒ Γ (x0 ) ⊇ Γ (x)
(viii) f (x, a, ε) is increasing in x.
Let ϕ be a continuous, bounded, and increasing real-valued function. Given
the standard assumptions, we can invoke either the Theorem of the Maximum
or the Extreme Value Theorem, to argue the existence of an optimal solution
to the optimization problem stated in (1) for any x ∈ X.
Let x, x0 ∈ X s.t. x0 ≥ x, and a, a0 be the corresponding optimal solution
when the state is given by x and x0 .
By assumption (vii) , we have
a ∈ Γ (x) ⊆ Γ (x0 ) =⇒ a ∈ Γ (x0 )
By assumptions (vi) and (viii), u (·) and f (·) are increasing functions, therefore
u (x, a) + β E ϕ [f (x, a, ε)] ≤ u (x0 , a) + β E ϕ [f (x0 , a, ε)]
(3)
2
where
u (x0 , a) + β E ϕ [f (x0 , a, ε)] ≤ u (x0 , a0 ) + β E ϕ [f (x0 , a0 , ε)]
(4)
Hence, by (3) and (4) , we can conclude that
u (x, a) + β E ϕ [f (x, a, ε)] ≤ u (x0 , a0 ) + β E ϕ [f (x0 , a0 , ε)]
where
u (x, a) + β E ϕ [f (x, a, ε)] = (T ϕ) (x)
u (x0 , a0 ) + β E ϕ [f (x0 , a0 , ε)] = (T ϕ) (x0 )
So, x ≤ x0 =⇒ T ϕ (x) ≤ T ϕ (x0 ) .
3
Proving T maps STRICLY INCREASING functions into strictly increasing functions
Note that the space of continuous, bounded, and strictly increasing real-valued
functions endowed with the sup-norm is not a complete metric space. Here,
the proof has to be done by using a two steps procedure.
Proof. We need an extra assumption:
(ix) u (·) is strictly increasing
First step: show T maps increasing functions into increasing functions.
Second step: As u (·) is strictly increasing and β E ϕ [f (x, a, ε)] is increasing,
u (x, a) + β E ϕ [f (x, a, ε)] is stricly increasing since the sum of an increasing
and a strictly increasing function is a strictly increasing function (CHECK!!!!!).
The max operator preserves such a property, therefore T ϕ is strictly increasing.
Since we already have ∃ ϕ ∈ D (D is the space of continuous, increasing, anb
bounded real-valued functions) s.t. ϕ = T ϕ, and that T ϕ is strictly increasing,
it directly follows that ϕ is strictly increasing.
4
Proving T maps CONCAVE functions into concave functions
Proof. Let ϕ be a concave, continuous, and bounded real-valued function. Let
x0 > x and a0 , a be the corresponding optimal solutions.
Standard assumptions:
(i) x ∈ X ⊆ Rl , X is a convex set.
(ii) u (·) ∈ C
(iii) β ∈ (0, 1)
(iv) Γ (x) is a non-empty, continuous, and compact-valued correspondence.
(v) f (x, a, ε) is bounded and continuous in all its arguments
Additional assumptions:
3
* for concavity:
(x) u (·) is concave in (x, a)
(xi) Γ (x) is convex in the following sense
∀θ
θ a + (1 − θ ) a0
∈ [0, 1] , ∀a ∈ Γ (x) , ∀a0 ∈ Γ (x0 )
∈ Γ (θ x + (1 − θ ) x0 )
(xii) f (a, x, ε) is concave in (x, a)
(xiii) ϕ (·) is increasing (in addition to concave, continuous, and bounded)
We want to show the following
T ϕ [θ x + (1 − θ ) x0 ] ≥ θ T ϕ ( x) + (1 − θ ) T ϕ (x0 )
By assumption (xi) , [θ a + (1 − θ ) a0 ] is feasible but not necessarily optimal
T ϕ [θ x + (1 − θ ) x0 ] ≥ u [θ x + (1 − θ ) x0 , θ a + (1 − θ ) a0 ] +
+β E ϕ [f (θ x + (1 − θ ) x0 , θ a + (1 − θ ) a0 , ε)]
By concavity of u (·)
(5)
T ϕ [θ x + (1 − θ ) x0 ] ≥ θ u (x, a) + (1 − θ) u (x0 , a0 ) +
+β E ϕ [f (θ x + (1 − θ ) x0 , θ a + (1 − θ ) a0 , ε)]
By concavity of f (·) and increasigness of ϕ (·) we have
β E ϕ [f (θ x + (1 − θ ) x0 , θ a + (1 − θ ) a0 , ε)] ≥ β E ϕ [θ f (x, a, ε) + (1 − θ) f (x0 , a0 , ε)]
≥ β [θ E ϕ [f (x, a, ε)] + (1 − θ) E ϕ [f (x0 , a0 , ε)]]
Therefore, (5) will be as follows
Proof.
T ϕ [θ x + (1 − θ ) x0 ] ≥ θ [u (x, a) + β E ϕ f (x, a, ε)]] +
+ (1 − θ) [u (x0 , a0 ) + β E ϕ f (x0 , a0 , ε)]]
=⇒ T ϕ [θ x + (1 − θ ) x0 ] ≥ θ T ϕ ( x) + (1 − θ ) T ϕ ( x0 )
5
Proving T maps STRICTLY CONCAVE functions into strictly concave functions
Proof. We have to use a two-step proof. First of all, let ϕ be a strictly concave,
continuous, and bounded real-valued function. Secondly, consider the following
assumptions:
Standard assumptions:
(i) x ∈ X ⊆ Rl , X is a convex set.
4
(ii) u (·) ∈ C
(iii) β ∈ (0, 1)
(iv) Γ (x) is a non-empty, continuous, and compact-valued correspondence.
(v) f (x, a, ε) is bounded and continuous in all its arguments
Additional assumptions:
* for concavity:
(xi) Γ (x) is convex in the following sense
∀θ
θ a + (1 − θ ) a0
∈ [0, 1] , ∀a ∈ Γ (x) , ∀a0 ∈ Γ (x0 )
∈ Γ (θ x + (1 − θ ) x0 )
(xii) f (a, x, ε) is concave in (x, a)
(xiii) ϕ (·) is increasing (in addition to strictly concave, continuous, and
bounded)
(xiv) u (·) is strictly concave in (x, a)
FIRST STEP: Given the above assumptions we can show that T ϕ is a
continuous, concave, and bounded real-valued functions. Now, since the space
of continuous, concave, and bounded real-valued functions (D) endowed with
the sup-norm is a complete metric space, and T is a contraction by assumption,
we can use the Contraction Mapping Theorem to ensure that there exists a
unique fixed point ϕ ∈ D (i.e. ∃! ϕ ∈ D s.t. ϕ = T ϕ) .
SECOND STEP: Since the space of continuous, strictly concave, and bounded
real-valued functions is not complete, we cannot invoke the Contraction Mapping Theorem. However, we will use the following reasoning. By assumption
(xiv) we have that u (·) is strictly concave. Since the sum of a concave function (β E ϕ [f (x, a, ε)]) and a strictly concave function (u (x, a)) , is a strictly
concave function. Then, u (x, a) + β E ϕ [f (x, a, ε)] is strictly concave. The
max operator preserves such a curvature property. So, T ϕ is strictly concave.
Therefore, since we already have ϕ = T ϕ, and T ϕ is stricly concave, it follows
that ϕ is strictly concave.
5