Differential games of inf-sup type and Isaacs equations
Hidehiro Kaise and Shuenn-Jyi Sheu
Institute of Mathematics, Academia Sinica,
Nankang, Taipei 11529, Taiwan, R.O.C.
E-mail: [email protected], [email protected]
Abstract. Motivated by the work of Fleming [5], we shall provide the general framework to
associate inf-sup type values with the Isaacs equations. We shall show that upper and lower
bounds for the generators of inf-sup type are upper and lower Hamiltonian in differential games
respectively. In particular, lower (resp. upper) bound corresponds to progressive (resp. strictly
progressive) strategy. Under Dynamic Programming Principle and identification of the generator, we can prove that the inf-sup type game is characterized as the unique viscosity solution
of the Isaacs equation. We also discuss about the Isaacs equation with Hamiltonian of convex
combination between lower and upper Hamiltonians.
1
Introduction
Let us consider dynamics of state x(s) ∈ RN influenced by two factors a, b.

 dx (s) = f (s, x(s), a(s), b(s)), t ≤ s ≤ T
ds

x(t) = x ∈ RN
(1.1)
where a(·), b(·) are measurable functions on [t, T ] taking its value in metric space A, B
respectively. For given a and b, we introduce game payoff function:
Z T
J(t, x; a, b) ≡
l(s, x(s), a(s), b(s))ds + Φ(x(T )),
(1.2)
t
where l : [0, T ] × RN × A × B → R, Φ : RN → R. In this problem, a(·) is considered
as control of minimizing player and b(·) as control of maximizing player. Under certain
assumption on available information for each player, we can define two types of game
value.
V (t, x) ≡ inf sup J(t, x; α[b], b)
α
W (t, x) ≡ sup inf J(t, x; a, β[a]).
β
(1.3)
b
a
(1.4)
In (1.3), supremum is taken over any b(·) and infimum is taken on a set of mapping α
from b(·) to a(·), which is considered as a class of strategies for minimizing player. (1.4)
is interpreted in the same way.
1
In conventional differential game, it is considered as a basic problem to find appropriate class of strategies which enable us to characterize V , W and to identify V with W
under min-max (Isaacs) condition. Evans and Souganidis [4] answered to these problems
by using theory of viscosity solutions. In [4], by taking Elliott-Kalton strategy (see Definition 2.2 in the present paper) for strategy class, it is showed that Dynamic Programming
Principles (DPPs) hold: For 0 ≤ t < t + δ ≤ T, x ∈ RN ,
Z t+δ
V (t, x) = inf sup[
l(s, x(s), α[b](s), b(s))ds + V (t + δ, x(t + δ))]
(1.5)
α
b
t
Z t+δ
W (t, x) = sup inf [
l(s, x(s), a(s), β[a](s))ds + W (t + δ, x(t + δ))].
(1.6)
β
a
t
Then, by using DPPs, it is proved that V and W are characterized as the unique viscosity
solution in the following Isaacs equations:

 ∂V (t, x) + H(t, x, ∇V (t, x)) = 0 in (0, T ) × RN
∂t
(1.7)

N
V (T, x) = Φ(x), x ∈ R ,

 ∂W (t, x) + H(t, x, ∇W (t, x)) = 0 in (0, T ) × RN
∂t
(1.8)

N
W (T, x) = Φ(x), x ∈ R ,
where H, H are defined as follows:
H(s, x, p) = inf sup[f (s, x, a, b) · p + l(s, x, a, b)]
a∈A b∈B
H(s, x, p) = sup inf [f (s, x, a, b) · p + l(s, x, a, b)]
b∈B a∈A
Here, we point out that the order of inf and sup in DPPs is flipped in the Isaacs equation.
Since comparison theorems for Isaacs equations are proved under some condition and
H ≤ H holds, one can have
V (t, x) ≤ W (t, x)
Note that under min-max condition H = H, we can see that V (t, x) = W (t, x) because
of uniqueness of viscosity solutions (see details in [4]).
In the above problem, both of minimizing and maximizing players are treated equally.
On the other hand, we can give a special role to maximizing player. For instance, maximizing player is regarded as disturbance in H ∞ -control (cf. [2]). In this interpretation,
inf-sup type value (1.3) is preferable to sup-inf type value (1.4). Recently, Fleming [5]
considers inf-sup type value in terms of max-plus stochastic control which gives a generalization of H ∞ -theory. In [5], sup-inf type value W in Elliott-Kalton sense is identified
with inf-sup type value defined by smaller class of strategies than Elliott-Kalton strategies
by using discretization method (see Theorem 4.1, [5]).
The aim of the present paper is to provide general framework how to relate infsup type games with the corresponding Isaacs equations. To utilize viscosity solution
methods, we have to work with two steps: one is DPP and the other is identification of
2
infinitesimal generator (cf. Chapter II, [7]). When we give a class of strategy, we can
define a inf-sup type value by (1.3). Then, it would be natural to expect that DPP holds
in the form of (1.5). If DPP (1.5) holds, we formally have the following equation:
Z t+δ
1
lim inf sup[
l(s, x(s), α[b](s), b(s))ds + V (t + δ, x(t + δ)) − V (t, x)] = 0.
δ→0+ δ α
b
t
Indeed, this can be rewritten as follows:
1
(Ft,t+δ V (t + δ, ·)(x) − V (t, x)) = 0
δ→0+ δ
lim
(1.9)
where Ft,s is defined for given function φ : RN → R as follows
Z s
Ft,s φ(x) = inf sup[
l(r, x(r), α[b](r), b(r))dr + φ(x(s))], x ∈ RN .
α
b
t
In general, it is known that V is not smooth. So, we consider (1.9) for smooth function
ϕ(t, x) instead of V :
1
(1.10)
lim (Ft,t+δ ϕ(t + δ, ·)(x) − ϕ(t, x)).
δ→0 δ
In Section 2, we shall study the infinitesimal generators (1.10). We give upper and
lower bounds of (1.10) for quite general classes of strategies. In fact, lower bound is
given by H and upper bound by H. In general, it is not easy to find a class which
has infinitesimal generator. However, we can show that the lower bound of generators
corresponds to Elliott-Kalton strategy (which we call progressive in this paper) and the
upper bound corresponds to strictly progressive strategy.
Relationships between DPP and infinitesimal generator are discussed in Section 3.
Once the infinitesimal generator is obtained, we can consider the corresponding Isaacs
equation. Following the general ideas of viscosity theory, we shall prove that if we have
infinitesimal generator and DPP, the inf-sup value is a viscosity solution of the corresponding Isaacs equation. In this procedure, we also have to see if DPP holds. DPP for
Elliott-Kalton case is proved in [4]. We shall show that DPP holds for strictly progressive
strategy.
One of the interesting problems is to find a strategy class whose inf-sup value is characterized as viscosity solution of Isaacs equation with Hamiltonian different from H, H.
In Section 3, we consider Isaacs equation with Hamiltonian given by convex combination
of H, H. Instead of giving a strategy class, following the techniques of Souganidis in
[9], [10], we consider approximated differential game. We prove that approximated value
converges to viscosity solution of the Isaacs equation with the convex combination of the
Hamiltonians.
Acknowledgement: The authors wish to thank Professor W.H. Fleming for valuable
comments and suggestions.
3
2
Infinitesimal generators for game values
As we mentioned in Introduction, it is an important step to study infinitesimal generators associated to game values. In this section, we shall give bounds for infinitesimal
generators of inf-sup type value for general class of strategies. In particular, we shall
identify infinitesimal generators for progressive and strictly progressive strategies.
In the rest of all the arguments, we always suppose the followings:
A, B are compact subsets of Euclidean space.
(A.1)
f, l and Φ are bounded and continuous.
(A.2)
There exists L > 0 such that for each s ∈ [0, T ], x, y ∈ RN , a ∈ A, b ∈ B
|f (s, x, a, b) − f (s, y, a, b)| ≤ L|x − y|
|l(s, x, a, b) − l(s, y, a, b)| ≤ L|x − y|
|Φ(x) − Φ(y)| ≤ L|x − y|.
(A.3)
Note that f and l are uniformly continuous on [0, T ] × K × A × B for any compact set
K.
At,s (resp. Bt,s ) is the set of A-valued (resp. B-valued) measurable functions on [t, s],
which is considered as all of the controls for minimizing player (resp. maximizing player).
We denote the set of mappings from Bt,s into At,s (resp. At,s into Bt,s ) as Γ0t,s (resp. ∆0t,s ).
Γ0t,s (resp. ∆0t,s ) is considered as all the possible strategies of minimizing player (resp.
maximizing player).
For given Γt,T ⊂ Γ0t,T , we define inf-sup type value:
V (t, x) ≡ inf
sup J(t, x; α[b], b)
α∈Γt,T b∈Bt,T
≡ inf
Z
sup [
α∈Γt,T b∈Bt,T
(2.1)
T
l(s, x(s), α[b](s), b(s))ds + Φ(x(T ))],
t
where x(·) is a solution of (1.1) with initial condition x(t) = x and controls a = α[b], b.
We also introduce the operators on C(RN ) associated to (2.1):
Z s
a,b
l(r, x(r), a(r), b(r))dr + φ(x(s)),
Ft,s φ(x) =
t
a[b],b
Ft,s φ(x) ≡ inf sup Ft,s φ(x), φ ∈ C(RN ),
α∈Γt,s b∈Bt,s
(2.2)
where x(·) is a solution of (1.1) with x(t) = x
Firstly, we obtain bounds for the generators in general case:
Proposition 2.1. Suppose Γt,s includes open loop strategies, i.e., α : Bt,s → At,s defined
in the following belongs to Γt,s :
α[b](r) ≡ a(r), t ≤ r ≤ s, b ∈ Bt,s ; a ∈ At,s .
4
(2.3)
Then, for ϕ ∈ C 1 ((0, T ) × RN ), we have
1
∂ϕ
(Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )) ≤
(t, x) + H(t, x, ∇x ϕ(t, x)),
δ→0+ δ
∂t
∂ϕ
1
(t, x) + H(t, x, ∇x ϕ(t, x)) ≤ lim (Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )),
∂t
δ→0+ δ
lim
(2.4)
(2.5)
where (tδ , xδ ) is any sequence such that (tδ , xδ ) → (t, x) ∈ (0, T ) × RN as δ → 0+.
Proof. If we identify a ∈ At,s with the strategy defined in (2.3), we may consider
At,s ⊂ Γ0t,s . By the assumption that Γt,s includes open loop strategies, At,s ⊂ Γt,s . Then,
we have
Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xtδ ) =
≤
α[b],b
inf
sup
Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xtδ )
inf
sup
Fta,b
ϕ(tδ + δ, ·)(xtδ )
δ ,tδ +δ
α∈Γtδ ,tδ +δ b∈Bt ,t +δ
δ δ
a∈Atδ ,tδ +δ b∈Bt ,t +δ
δ δ
≤ inf
sup
a∈A b∈Bt
Fta,b
ϕ(tδ + δ, ·)(xtδ ).
δ ,tδ +δ
(2.6)
δ ,tδ +δ
For a ∈ A, we consider a ∈ At,s by a(u) = a. Let a(u) = a ∈ At,s . By the definition of
Fta,b
,
δ ,tδ +δ
Z
Fta,b
ϕ(tδ
δ ,tδ +δ
+ δ, ·)(xtδ ) =
tδ +δ
l(s, xδ (s), a, b(s))ds + ϕ(tδ + δ, xδ (tδ + δ)),
tδ
where xδ (·) is a solution of (1.1) with initial condition xδ (tδ ) = xδ . Note that for solution
xδ (·) of (1.1) with initial condition xδ (tδ ) = xδ driven by a ∈ Atδ ,T and b ∈ Btδ ,T , there
exists K > 0 independent of δ, a, b such that
|xδ (s) − xδ | ≤ Ks, tδ ≤ ∀s ≤ T.
By (A.1)–(A.3), (2.7), we can see that
Z
tδ +δ
l(s, xδ (s), a, b(s))ds + ϕ(tδ + δ, xδ (tδ + δ)) − ϕ(tδ , xδ )
tδ
Z
tδ +δ
=
[
tδ
tδ +δ
Z
∂ϕ
(s, xδ (s)) + f (s, xδ (s), a, b(s)) · ∇ϕ(s, xδ (s)) + l(s, xδ (s), a, b(s))]ds
∂s
∂ϕ
(s, xδ ) + f (s, xδ , a, b(s)) · ∇ϕ(s, xδ ) + l(s, xδ , a, b(s))]ds + o(δ)
∂s
tδ
Z tδ +δ
∂ϕ
[f (t, x, a, b(s)) · ∇ϕ(t, x) + l(t, x, a, b(s))]ds + o(δ)
(t, x)δ +
=
∂t
tδ
∂ϕ
≤ [ (t, x) + sup{f (t, x, a, b) · ∇ϕ(t, x) + l(t, x, a, b)}]δ + o(δ) (δ → 0+)
∂t
b∈B
=
[
5
(2.7)
where o(δ) is uniform on a and b(·). Thus, we have
inf
sup
a∈A b∈Bt
Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )
δ ,tδ +δ
≤[
∂ϕ
(t, x) + inf sup{f (t, x, a, b) · ∇ϕ(t, x) + l(t, x, a, b)}]δ + o(δ) (δ → 0+).
a∈A b∈B
∂t
Therefore, (2.4) implies from (2.6).
Fix arbitrary α ∈ Γtδ ,tδ +δ and b ∈ Btδ ,tδ +δ . Since Atδ ,tδ +δ ⊂ Γtδ ,tδ +δ we have
α[b],b
Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) ≥
inf
a∈Atδ ,tδ +δ
Fta,b
ϕ(tδ + δ, ·)(xδ ).
δ ,tδ +δ
By taking supremum over b ∈ Btδ ,tδ +δ and then taking infimum over α ∈ Γtδ ,tδ +δ ,
inf
sup
α∈Γtδ ,tδ +δ b∈Bt ,t +δ
δ δ
α[b],b
Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) ≥
sup
inf
b∈Btδ ,tδ +δ
≥ sup
a∈Atδ ,tδ +δ
inf
b∈B a∈Atδ ,tδ +δ
Fta,b
ϕ(tδ + δ, ·)(xδ )
δ ,tδ +δ
Fta,b
ϕ(tδ + δ, ·)(xδ ).
δ ,tδ +δ
(2.8)
In the same way as the proof of (2.4), we can prove that
sup
inf
b∈B a∈Atδ ,tδ +δ
≥[
Fta,b
ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )
δ ,tδ +δ
∂ϕ
(t, x) + sup inf {f (t, x, a, b) · ∇ϕ(t, x) + l(t, x, a, b)}]δ + o(δ) (δ → 0+). (2.9)
∂t
b∈B a∈A
Then, by (2.8) and (2.9), we obtain (2.5).
In general, it is not obvious to find an explicit form of infinitesimal generators. However, we can identify the limit for progressive and strictly progressive strategies. We
introduce these two notions of strategies.
Definition 2.2. (cf. [3], [8], [11]) α ∈ Γ0t,s is called progressive strategy for minimizing
player if the following condition is satisfied:
For each r ∈ [t, s], if b ≡ b̃ a.e. on [t, r], then α[b] ≡ α[b̃] a.e. on [t, r].
We denote by ΓPt,s the set of progressive strategies for minimizing player. Progressive
strategy for maximizing player β ∈ ∆0t,s is defined in a similar way and the set of progressive strategies for maximizing player is denoted by ∆Pt,s .
Definition 2.3. (cf. [5]) α ∈ ΓPt,s is strictly progressive strategy if for any β ∈ ∆Pt,s , there
exist a ∈ At,s and b ∈ Bt,s such that
α[b] = a, β[b] = b a.e. on [t, s].
We denote the set of strictly progressive strategies as ΓSP
t,s .
Remark 2.4. ΓPt,s and ΓSP
t,s include open loop strategies.
6
We define the operators associated with these two classes of strategies:
α[b],b
P
Ft,s
φ(x) ≡ inf sup Ft,s
α∈ΓP
t,s
φ(x),
b∈Bt,s
α[b],b
SP
Ft,s
φ(x) ≡ inf
sup Ft,s
α∈ΓSP
t,s b∈Bt,s
φ(x)
We also introduce the corresponding inf-sup type game values:
V P (t, x) ≡ inf
sup J(t, x; α[b], b)
V SP (t, x) ≡ inf
sup J(t, x; α[b], b).
α∈ΓP
t,T b∈Bt,T
α∈ΓSP
t,T b∈Bt,T
Note that under (A.1)–(A.3), V P and V SP are bounded and Lipschitz continuos on
[0, T ] × RN
We first give the form of the generator for progressive case. The proof is implicitly
done in [4]. However, we shall give another proof for convenience.
Proposition 2.5. For ϕ ∈ C 1 ((0, T ) × RN ),
1 P
∂ϕ
(Ft,t+δ ϕ(t + δ, ·)(x) − ϕ(t, x)) →
(t, x) + H(t, x, ∇ϕ(t, x)), δ → 0+
δ
∂t
(2.10)
uniformly on each compact set in (0, T ) × RN .
Proof. Note that uniform convergence of (2.10) on each compact set is equivalent to the
followings:
∂ϕ
1 P
(Ftδ +δ,δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )) →
(t, x) + H(t, x, ∇ϕ(t, x)) (δ → 0+)
δ
∂t
where (tδ , xδ ) is any sequence converging to (t, x) ∈ (0, T ) × RN . In Proposition 2.1, we
already showed that
∂ϕ
1
(t, x) + H(t, x, ∇ϕ(t, x)) ≤ lim (FtPδ +δ,δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )).
∂t
δ→0+ δ
So, we shall prove the other side of inequality
∂ϕ
1 P
(Ftδ +δ,δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )) ≤
(t, x) + H(t, x, ∇ϕ(t, x)).
δ→0+ δ
∂t
lim
(2.11)
We take arbitrary θ > 0. Following the argument of Lemma 4.3 (b) in [4], we can
construct a measurable mapping ā : B → A satisfying
∂ϕ
(t, x) + H(t, x, ∇ϕ(t, x))
∂t
∂ϕ
(t, x) + inf [f (t, x, a, b) · ∇ϕ(t, x) + l(t, x, a, b)]
≥
a∈A
∂t
∂ϕ
≥
(t, x) + f (t, x, ā(b), b) · ∇ϕ(t, x) + l(t, x, ā(b), b) − θ, ∀b ∈ B.
∂t
7
(2.12)
Define a strategy ᾱ ∈ Γ0tδ ,tδ +δ as follows:
ᾱ[b](s) = ā(b(s)), tδ ≤ s ≤ tδ + δ; b ∈ Btδ ,tδ +δ .
In (2.12), if we take ā = ᾱ[b](s), b = b(s) and integrate both of sides on [tδ , tδ + δ], we
have
[
∂ϕ
(t, x) + H(t, x, ∇ϕ(t, x))]δ
∂t
Z tδ +δ
∂ϕ
≥
( (t, x) + f (t, x, ᾱ[b](s), b(s)) · ∇ϕ(t, x)
∂t
tδ
+ l(t, x, ᾱ[b](s), b(s)))ds − θδ, ∀b ∈ Btδ ,tδ +δ (2.13)
By (A.1)–(A.3) and (2.7), we can see that
Z
tδ +δ
(
tδ
∂ϕ
(t, x) + f (t, x, ᾱ[b](s), b(s)) · ∇ϕ(t, x) + l(t, x, ᾱ[b](s), b(s)))ds
∂t
Z tδ +δ
∂ϕ
=
( (s, xδ (s)) + f (s, xδ (s), ᾱ[b](s), b(s)) · ∇ϕ(s, xδ (s))
∂t
tδ
+ l(s, xδ (s), ᾱ[b](s), b(s)))ds + o(δ) (2.14)
where xδ (·) is a solution with xδ (tδ ) = xδ driven by ᾱ[b], b and o(δ) is uniform on ᾱ, b
and θ. Then, from (2.13) and (2.14), we have
[
∂ϕ
(t, x) + H(t, x, ∇ϕ(t, x))]δ
∂t
Z tδ +δ
∂ϕ
( (s, xδ (s)) + f (s, xδ (s), ᾱ[b](s), b(s)) · ∇ϕ(s, xδ (s))
≥
∂t
tδ
+ l(s, xδ (s), ᾱ[b](s), b(s)))ds + o(δ) − θδ
=
ᾱ[b],b
Ftδ ,tδ +δ ϕ(tδ
+ δ, ·)(xδ ) − ϕ(tδ , xtδ ) + o(δ) − θδ, ∀b ∈ Btδ ,tδ +δ .
Since b ∈ Btδ ,tδ +δ is arbitrary and ᾱ ∈ ΓPtδ ,tδ +δ ,
[
∂ϕ
(t, x) + H(t, x, ∇ϕ(t, x))]δ ≥ FtPδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xtδ ) + o(δ) − θδ.
∂t
By sending θ to 0 and dividing both of sides by δ, we finally have
∂ϕ
1
(t, x) + H(t, x, ∇ϕ(t, x)) ≥ (FtPδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xtδ )) + o(1).
∂t
δ
In the next result, we shall prove the infinitesimal generator associated to strictly
progressive strategy corresponds to upper Hamiltonian H(t, x, p).
Proposition 2.6. For ϕ ∈ C 1 ((0, T ) × RN ),
∂ϕ
1 SP
(Ft,t+δ ϕ(t + δ, ·)(x) − ϕ(t, x)) →
(t, x) + H(t, x, ∇ϕ(t, x)), δ → 0+
δ
∂t
uniformly on each compact set in (0, T ) × RN .
8
(2.15)
Proof. As the proof of Proposition 2.5, it is enough to show that
∂ϕ
1
(t, x) + H(t, x, ∇ϕ(t, x)) ≤ lim (FtSP
ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ ))
δ ,tδ +δ
∂t
δ→0+ δ
for any sequence (tδ , xδ ) converging to (t, x) ∈ (0, T ) × RN .
Take arbitrary θ > 0. In the same way as the argument in the proof of Proposition
2.5 (see the proof of Lemma 4.3 (b), [4]), there exists a measurable mapping b̄ : A → B
such that
∂ϕ
(t, x) + H(t, x, ∇ϕ(t, x))
∂t
∂ϕ
≤
(t, x) + sup[f (t, x, a, b) · ∇ϕ(t, x) + l(t, x, a, b)]
∂t
b∈B
∂ϕ
≤
(t, x) + f (t, x, a, b̄(a)) · ∇ϕ(t, x) + l(t, x, a, b̄(a)) + θ, ∀a ∈ A.
∂t
(2.16)
We define a new strategy β̄ ∈ ∆0tδ ,tδ +δ :
β̄[a](s) = b̄(a(s)), tδ ≤ s ≤ tδ + δ; a ∈ Atδ ,tδ +δ .
In fact, we can see that β̄ ∈ ∆Ptδ ,tδ +δ . By taking a = a(s), b = β̄[a](s) in (2.16) and
integrating on [tδ , tδ + δ], we have
[
∂ϕ
(t, x) + H(t, x, ∇ϕ(t, x))]δ
∂t
Z tδ +δ
∂ϕ
( (t, x) + f (t, x, a(s), β̄[a](s)) · ∇ϕ(t, x)
≤
∂t
tδ
+ l(t, x, a(s), β̄[a](s)))ds + θδ, ∀a ∈ Atδ ,tδ +δ .
From (A.1)–(A.3) and (2.7),
Z
tδ +δ
(
tδ
∂ϕ
(t, x) + f (t, x, a(s), β̄[a](s)) · ∇ϕ(t, x) + l(t, x, a(s), β̄[a](s)))ds
∂t
Z tδ +δ
∂ϕ
( (s, xδ (s)) + f (s, xδ (s), a(s), β̄[a](s)) · ∇ϕ(s, xδ (s))
=
∂t
tδ
+ l(s, xδ (s), a(s), β̄[a](s)))ds + o(δ), δ → 0+
where o(δ) is uniform on a, β̄, θ. Thus, we have
[
∂ϕ
(t, x) + H(t, x, ∇ϕ(t, x))]δ
∂t
Z tδ +δ
∂ϕ
≤
( (s, xδ (s)) + f (s, xδ (s), a(s), β̄[a](s)) · ∇ϕ(s, xδ (s))
∂t
tδ
+ l(s, xδ (s), a(s), β̄[a](s)))ds + o(δ) + θδ
a,β̄[a]
= Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ ) + o(δ) + θδ, ∀a ∈ Atδ ,tδ +δ .
9
(2.17)
Take arbitrary α ∈ ΓSP
tδ ,tδ +δ . By the definition of strictly progressive strategy, there
exist â ∈ Atδ ,tδ +δ and b̂ ∈ Btδ ,tδ +δ such that
α[b̂] = â, β̄[â] = b̂ a.e. on [tδ , tδ + δ].
(2.18)
By (2.18) and (2.17), we have
[
∂ϕ
â,β̄[â]
(t, x) + H(t, x, ∇ϕ(t, x))]δ ≤ Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ ) + o(δ) + θδ
∂t
α[b̂],b̂
= Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ ) + o(δ) + θδ
≤
sup
b∈Btδ ,tδ +δ
α[b],b
Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ ) + o(δ) + θδ.
Since α ∈ Γtδ ,tδ +δ is arbitrarily taken,
[
∂ϕ
ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ ) + o(δ) + θδ.
(t, x) + H(t, x, ∇ϕ(t, x))]δ ≤ FtSP
δ ,tδ +δ
∂t
By taking θ → 0 and dividing by δ, we obtain
∂ϕ
1
(t, x) + H(t, x, ∇ϕ(t, x)) ≤ (FtSP
ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )) + o(1).
∂t
δ δ ,tδ +δ
3
Dynamic programming principle and Isaacs equations
In the present section, we shall study relationships between Dynamic Programming Principle (DPP) and its infinitesimal generators. More precisely, if DPP holds and the infinitesimal generator is identified, the inf-sup type game value is a viscosity solution of the
corresponding Isaacs equation. Furthermore, if the Hamiltonian of the Isaacs equation
satisfies certain structural conditions, the value is characterized as the unique viscosity
solution.
In progressive and strictly progressive case, we can show DPPs hold. By combining
results in Section 2, we shall prove that value defined by progressive (resp. strictly progressive) strategy is characterized as a unique viscosity solution for lower Isaacs equation
(resp. upper Isaacs equation).
Firstly, we give a general result on relationship between DPP and the Isaacs equation.
For given class of strategies Γt,s ⊂ Γ0t,s , 0 ≤ t < s ≤ T , we define inf-sup type game value
V (t, x) as (2.1). For given class Γt,s , DPP is described as follows: For 0 ≤ t < t + δ ≤ T ,
x ∈ RN ,
Z t+δ
l(s, x(s), α[b](s), b(s))ds + V (t + δ, x(t + δ))].
sup [
V (t, x) = inf
α∈Γt,t+δ b∈Bt,t+δ
t
This is reformulated in terms of (2.2):
Ft,t+δ V (t + δ, ·)(x) = V (t, x)
10
(3.1)
Proposition 3.1. Suppose that (3.1) holds and the infinitesimal generator is identified,
i.e., there exists H(t, x, p) such that for ϕ ∈ C 1 ((0, T ) × RN ),
1
∂ϕ
(Ft,t+δ ϕ(t + δ, ·)(x) − ϕ(t, x)) →
ϕ(t, x) + H(t, x, ∇ϕ(t, x)), δ → 0 + .
δ
∂t
Then, V (t, x) is a viscosity solution of the equation:

 ∂V (t, x) + H(t, x, ∇V (t, x)) = 0, (t, x) ∈ (0, T ) × RN ,
∂t

V (T, x) = Φ(x), x ∈ RN .
(3.2)
(3.3)
Proof. Suppose that ϕ is smooth function on (0, T ) × RN and (t, x) ∈ (0, T ) × RN is a
local maximum point of V − ϕ. We note that for sufficiently small δ > 0,
Ft,t+δ ϕ(t, x) − ϕ(t, x) ≥ 0..
(3.4)
In fact, since (t, x) is a local maximum point of V − ϕ, there exists r > 0 such that
V (s, y) − ϕ(s, y) ≤ V (t, x) − ϕ(t, x), ∀(t, x) ∈ [t − r, t + r] × B̄r (x)
i.e.
V (s, y) − V (t, x) ≤ ϕ(s, y) − ϕ(t, x), ∀(t, x) ∈ [t − r, t + r] × B̄r (x).
(3.5)
By the definition of Ft,t+δ and (3.1),
Z t+δ
l(s, x(s), α[b](s), b(s))ds + V (t + δ, x(t + δ)) − V (t, x)] = 0.
inf
sup [
α∈Γt,t+δ b∈Bt,t+δ
t
By continuity of solution on initial point (2.7), we can take sufficiently small δ0 > 0
uniformly on α and b such that
x(s) ∈ Br (x), t ≤ s ≤ t + δ0 .
(3.6)
Then, from (3.5) and (3.6), we obtain (3.4).
By (3.2) and (3.4), we have
∂ϕ
(t, x) + H(t, x, ∇ϕ(t, x)) ≥ 0.
∂t
Thus, V is a viscosity subsolution of (3.3).
The proof of viscosity supersolution is the same as the above argument.
We shall next apply Proposition 3.1 to progressive and strictly progressive cases. In
Section 2, we proved that infinitesimal generators for progressive strategies and strictly
progressive strategies have explicit forms. To associate the values defined by these classes
with the corresponding Isaacs equation, we need to prove DPP.
In progressive case, DPP and characterization of V P (t, x) as viscosity solution is
proved in [4] (See Theorems 3.1, 4.1, [4]). By using DPP in [4], we can apply Propositions
2.5 and 3.1 to obtain characterization result for V P .
11
Proposition 3.2 (Theorem 3.1, [4]). V P (t, x) satisfies DPP:
P
V P (t, x) = Ft,t+δ
V P (t + δ, ·)(x)
Z t+δ
= inf
sup [
l(s, x(s), α[b](s), b(s))ds + V P (t + δ, x(s))].
α∈ΓP
t,t+δ b∈Bt,t+δ
t
We have another proof for characterization of V P .
Theorem 3.3 (cf. Theorem 4.1, [4]). V P (t, x) is unique viscosity solution in BU C([0, T ]×
RN ) for the lower Isaacs equation:

P
 ∂V
(t, x) + H(t, x, ∇V P (t, x)) = 0, (t, x) ∈ (0, T ) × RN ,
(3.7)
∂t

V P (T, x) = Φ(x), x ∈ RN ,
where BU C([0, T ] × RN ) is the set of bounded, Lipschitz continuous functions on [0, T ] ×
RN .
Proof. By Propositions 2.5 and 3.2, we can apply Proposition 3.1 to show that V P (t, x)
is a viscosity solution of (3.7). Under (A.1)–(A.3), Hamiltonian H satisfies structural
condition to ensure uniqueness results in BU C([0, T ] × RN ) (cf. Theorem 9.1, Chapter
.
II, [7], Exercise 3.9, Chapter II, [1]).
We consider value for strictly progressive strategies V SP (t, x). In [5], relationship
between V SP and upper Isaacs equation is studied and V SP is identified with upper
game value which is a viscosity solution of upper Isaacs equation. On the other hand,
we directly prove DPP and apply Propositions 2.6 and 3.1.
Proposition 3.4. V SP (t, x) satisfies DPP: For 0 ≤ t < t + δ ≤ T , x ∈ RN ,
SP
V SP (t, x) = Ft,t+δ
V SP (t + δ, ·)(x)
Z t+δ
l(s, x(s), α[b](s), b(s))ds + V SP (t + δ, x(t + δ))]. (3.8)
= inf
sup [
α∈ΓSP
t,t+δ b∈Bt,t+δ
t
SP
Proof. Set W (t, x) = Ft,t+δ
V SP (t + δ, ·)(x). We shall first show that W (t, x) ≥ V SP (t, x).
For any ² > 0, there exists α² ∈ ΓSP
t,t+δ such that
α² [b ],b1
sup Ft,t+δ1
W (t, x) ≥
V SP (t + δ, ·)(x) − ²
b1 ∈Bt,t+δ
Z
t+δ
≥
t
l(s, x²1 (s), α² [b1 ](s), b1 (s))ds + V SP (t + δ, x²1 (t + δ)) − ², ∀b1 ∈ Bt,t+δ ,
(3.9)
where x²1 (·) is a solution of (1.1) controlled by α² [b1 ] and b1 . We consider V SP (t + δ, z),
z ∈ RN . By definition of V SP (t + δ, z), there exists αz² ∈ ΓSP
t+δ,T such that
V (t + δ, z) ≥
sup
b2 ∈Bt+δ,T
Z
T
≥
t+δ
α² [b ],bs
z 2
Ft+δ,T
Φ(z) − ²
l(s, x²2 (s), αz² [b2 ](s), b2 (s))ds + Φ(x²2 (T )) − ², ∀b2 ∈ Bt+δ,T ,
12
(3.10)
where x²2 (·) is a solution of (1.1) with initial condition x²2 (t + δ) = z, controls αz² [b2 ], b2 .
Define a new strategy ᾱ² : Bt,T → At,T : For b ∈ Bt,T ,
(
α² [b|[t,t+δ] ](s), t ≤ s < t + δ
²
ᾱ [b](s) =
αx² ²1 (t+δ) [b|[t+δ,T ] ](s), t + δ ≤ s ≤ T
(3.11)
where x²1 (·) is a solution of (1.1) with controls α² [b|[t,t+δ] ], b|[t,t+δ] . We used the notation
b|[s,r] for restriction of b on [s, r].
P
We shall show that ᾱ² ∈ ΓSP
t,T . Take arbitrary β ∈ ∆t,T . For given â2 ∈ At+δ,T , define
β1 : At,t+δ → Bt,t+δ as follows:
β1 [a1 ](s) = β[a1 ⊕ â2 ](s), t ≤ s ≤ t + δ, a1 ∈ At,t+δ .
(3.12)
Here a1 ⊕ â2 is defined by
(
a1 ⊕ â2 =
a1 (s), t ≤ s < t + δ
â2 (s), t + δ ≤ s ≤ T.
Note that β1 does not depend on â2 ∈ At+δ,T because β1 [a1 ] = β[a1 ⊕ â2 ]|[t,t+δ] is determined by a1 on [t, t + δ] (See Definition 2.2). Since α² ∈ ΓSP
t,t+δ , there exist ā1 ∈ At,t+δ ,
b̄1 ∈ Bt,t+δ such that
α² [b̄1 ] = ā1 , β1 [ā1 ] = b̄1 on [t, t + δ].
(3.13)
Let x̄²1 (·) be solution of (1.1) by α² [b̄1 ], b̄1 . Define β2 : At+δ,T → Bt+δ,T in the following:
β2 [a2 ](s) = β[ā1 ⊕ a2 ](s), t + δ ≤ s ≤ T, a2 ∈ At+δ,T .
Note that β2 ∈ ∆Pt+δ,T . It implies from αx̄² ² (t+δ) ∈ ΓSP
t+δ,T that there exist ā2 ∈ At+δ,T ,
1
b̄2 ∈ Bt+δ,T such that
αx̄² ²1 (t+δ) [b̄2 ] = ā2 , β2 [ā2 ] = b̄2 on [t + δ, T ].
If we set ā = ā1 ⊕ ā2 , b̄ = b̄1 ⊕ b̄2 , then we have from (3.11), (3.13), (3.14),
ᾱ² [b̄] = ā, β[ā] = b̄ on [t, T ].
Therefore we proved that ᾱ² ∈ ΓSP
t,T .
By definition of α² , (3.9), (3.10),
Z
W (t, x) ≥
t
t+δ
l(s, x̄² (s), ᾱ² [b](s), b(s))ds
Z T
l(s, x̄² (s), ᾱ² [b](s), b(s))ds + Φ(x² (T )) − 2²
+
t+δ
²
≥ J(t, x; ᾱ [b], b) − 2², ∀b ∈ Bt,T ,
13
(3.14)
where x̄² (·) is a solution of (1.1) with ᾱ² [b], b. Thus, we have
W (t, x) ≥ inf
α∈ΓSP
t,T
sup J(t, x; α[b], b) − 2² = V SP (t, x) − 2².
b∈Bt,T
Since ² > 0 is arbitrary, we finally obtain
W (t, x) ≥ V SP (t, x).
We next show that W (t, x) ≤ V SP (t, x). Fix arbitrary α ∈ ΓSP
t,T . For given b̃2 ∈ Bt+δ,T ,
we introduce a strategy α1 : Bt,t+δ → At,t+δ :
α1 [b1 ](s) ≡ α[b1 ⊕ b̃2 ](s), t ≤ s ≤ t + δ, b1 ∈ Bt,t+δ .
(3.15)
Note that α1 does not depend on the choice of b̃2 because α ∈ ΓPt,T .
P
P
We shall see that α1 ∈ ΓSP
t,t+δ . For given β1 ∈ ∆t,t+δ , we can find β ∈ ∆t,T such that
β[a] = β1 [a|[t,t+δ] ] on [t, t + δ], a ∈ At,T .
For instance, we can construct β as follows:
(
β1 [a|[t,t+δ] ](s), t ≤ s < t + δ
β[a](s) =
b0 , t ≤ s ≤ T
where b0 ∈ B is a fixed constant. Since α ∈ ΓSP
t,T , there exist â ∈ At,T , b̂ ∈ Bt,T such that
α[b̂] = â, β[â] = b̂ on [t, T ].
(3.16)
In (3.15), if we take b̃2 = b̂|[t+δ,T ] , we have from (3.16)
α1 [b̂1 ] = â1 , β1 [â1 ] = b̂1 on [t, t + δ]
where â1 = â|[t,t+δ] , b̂1 = b̂|[t,t+δ] . Hence, α1 ∈ ΓSP
t,t+δ .
By definition of W (t, x) and α1 ∈ ΓSP
t,t+δ ,
Z
sup [
W (t, x) ≤
b1 ∈Bt,t+δ
t+δ
l(s, x1 (s), α1 [b1 ](s), b1 (s))ds + V SP (t + δ, x1 (t + δ))].
t
For any ² > 0, there exists b²1 ∈ Bt,t+δ such that
Z
t+δ
W (t, x) ≤
t
l(s, x²1 (s), α1 [b²1 ](s), b²1 (s))ds + V SP (t + δ, x²1 (t + δ)) + ²
where x²1 (·) is solution of (1.1) controlled by α1 [b²1 ], b²1 .
Define α2 : Bt+δ,T → At+δ,T as follows:
α2 [b2 ](s) ≡ α[b²1 ⊕ b2 ](s), t + δ ≤ s ≤ T, b2 ∈ Bt+δ,T .
14
(3.17)
P
In order to prove that α2 ∈ ΓSP
t+δ,T , we take any β̂2 ∈ ∆t+δ,T . Define β̂ : At,T → Bt,T :
( ²
b1 (s), t ≤ s < t + δ
β̂[a](s) ≡
β̂2 [a|[t+δ,T ] ], t + δ ≤ s ≤ T.
It is easy to see that β̂ ∈ ∆Pt,T . Since α ∈ ΓSP
t,T , there exist ā ∈ At,T and b̄ ∈ Bt,T such
that
α[b̄] = ā, β̂[ā] = b̄ on [t, T ].
From definition of β̂, we see that b²1 = β̂[ā]|[t,t+δ] = b̄ on [t, t + δ]. Thus, we have
b̄ = b²1 ⊕ b̄|[t+δ,T ] .
Thus, if we set ā2 = ā|[t+δ,T ] , b̄2 = b̄|[t+δ,T ] , we have
α2 [b̄2 ] = ā2 , β̂[ā2 ] = b̄2 on [t + δ, T ].
Hence α2 ∈ ΓSP
t+δ,T .
From definition of V SP (t + δ, x²1 (t + δ)),
Z T
SP
²
V (t + δ, x1 (t + δ)) ≤ sup [
l(s, x2 (s), α2 [b2 ](s), b2 (s))ds + Φ(x2 (T ))] + ².
b2 ∈Bt+δ,T
t+δ
For ² > 0, there exists b²2 ∈ Bt+δ,T such that
Z T
SP
²
V (t + δ, x1 (t + δ)) ≤
l(s, x²2 (s), α2 [b²2 ](s), b²2 (s))ds + Φ(x²2 (T ))]
(3.18)
t+δ
where x²2 (·) is solution with initial condition x²2 (t + δ) = x²1 (t + δ) controlled by α2 [b²2 ], b²2 .
We set b² = b²1 ⊕ b²2 . Then, by (3.17), (3.18),
Z t+δ
W (t, x) ≤
l(s, x²1 (s), α[b²1 ⊕ b²2 ](s), b²1 (s))ds
t
Z T
l(s, x²2 (s), α[b1 ⊕ b²2 ](s), b²2 (s))ds + Φ(x²2 (T )) + 2²
+
t+δ
Z T
l(s, x² (s), α[b² ](s), b² (s))ds + Φ(x² (T )) + 2²
=
t
= J(t, x; α[b² ], b² ) + 2²
where x² (·) is solution of (1.1) with α[b² ], b² . Since α ∈ ΓSP
t,T is arbitrarily taken, we have
W (t, x) ≤ inf
sup J(t, x; α[b], b) + 2² = V SP (t, x) + 2².
α∈ΓSP
t,T b∈Bt,T
Taking ² → 0,
W (t, x) ≤ V SP (t, x)
Then, we have characterization for V SP .
15
Theorem 3.5. V SP (t, x) is the unique viscosity solution in BU C([0, T ] × RN ) for the
upper Isaacs equation:

SP
 ∂V
(t, x) + H(t, x, ∇V SP (t, x)) = 0, (t, x) ∈ (0, T ) × RN ,
(3.19)
∂t

SP
N
V (T, x) = Φ(x), x ∈ R .
Proof. By Propositions 2.6, 3.1, 3.4, V SP is a viscosity solution of (3.19). By uniqueness
or comparison theorem in the class BU C([0, T ]×RN ), V SP is the unique viscosity solution
(See [1], [7] ).
4
Approximations of games for general Hamiltonians
P
For class of strategies Γt,s such that ΓSP
t,s ⊂ Γt,s ⊂ Γt,s , we can introduce an inf-sup type
value:
V (t, x) ≡ inf sup J(t, x; α[b], b).
α∈Γt,T b∈Bt,T
Note that V P (t, x) ≤ V (t, x) ≤ V SP (t, x). In the previous sections, we gave a general
framework to relate DPP and the corresponding Isaacs equations. Particularly, V P (resp.
V SP ) is characterized as the unique viscosity solution of lower Isaacs equation (resp.
upper Isaacs equation). As Proposition 2.1 indicates, it is an interesting problem to
study some class Γt,s for which the infinitesimal generator is different from H and H and
V (t, x) satisfies DPP. However, it seems not easy to find such class immediately.
In this section, we consider an approximation problem for Hamiltonian H (γ) of convex
combination between H and H:
H (γ) (t, x, p) = (1 − γ)H(t, x, p) + γH(t, x, p), 0 ≤ γ ≤ 1
We shall introduce an approximation of game problem by following techniques of SouganiSP
P
with some weight related to γ to define
dis in [9], [10]. We use the product of Ft,s
, Ft,s
discrete game problem. Then, by taking the limit with respect to the size of sub-interval,
we shall show that the discrete game value converges to the unique viscosity solution of
(3.3) with Hamiltonian H = H (γ) .
(γ)
For 0 ≤ γ ≤ 1, we define operator Ft,s as follows:
(γ)
SP
P
Ft,s φ(x) ≡ Ft,t+γ(s−t)
Ft+γ(s−t),s
φ(x), x ∈ RN , 0 ≤ t ≤ s; φ ∈ C(RN ).
At first, we obtain the result on infinitesimal generator for (4.1).
Proposition 4.1. For ϕ ∈ C 1 ((0, T ) × RN ),
1 (γ)
∂ϕ
(Ft,t+δ ϕ(t + δ, ·)(x) − ϕ(t, x)) →
(t, x) + H (γ) (t, x, ∇ϕ(t, x)), δ → 0+
δ
∂t
uniformly on each compact set in (0, T ) × Rn .
16
(4.1)
Proof. It is enough to show that for each sequence (tδ , xδ ) converging to (t, x) ∈ (0, T ) ×
RN ,
1 (γ)
∂ϕ
(Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )) →
(t, x) + H (γ) (t, x, ∇ϕ(t, x)), δ → 0 + .
δ
∂t
From (4.1), we have
(γ)
Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )
Z tδ +γδ
=
inf
sup [
l(s, xδ (s), α[b](s), b(s))ds
α∈ΓSP
t ,t
δ δ +γδ
+
=
FtPδ +γδ,tδ +δ ϕ(tδ
inf
sup
α∈ΓSP
t ,t
δ δ +γδ
+
b∈Btδ ,tδ +γδ
b∈Btδ ,tδ +γδ
tδ
+ δ, ·)(xδ (tδ + γδ))] − ϕ(tδ , xδ )
Z tδ +γδ
[
l(s, xδ (s), α[b](s), b(s))ds + ϕ(tδ + γδ, xδ (tδ + γδ))
FtPδ +γδ,tδ +δ ϕ(tδ
tδ
+ δ, ·)(xδ (tδ + γδ)) − ϕ(tδ + γδ, xδ (tδ + γδ))] − ϕ(tδ , xδ )
(4.2)
where xδ (·) is solution of (1.1) with initial condition xδ (tδ ) = xδ and controls α[b], b. By
continuity on initial condition (2.7) and Proposition 2.6,
FtPδ +γδ,tδ +δ ϕ(tδ + δ, ·)(xδ (tδ + γδ)) − ϕ(tδ + γδ, xδ (tδ + γδ))
∂ϕ
= (1 − γ)[ (t, x) + H(t, x, ∇ϕ(t, x))]δ + o(δ), δ → 0+
∂t
where o(δ) is uniform on α, b. Thus, from (4.2) and Proposition 2.5, we have
(γ)
Ftδ ,tδ +δ ϕ(tδ + δ, ·)(xδ ) − ϕ(tδ , xδ )
Z tδ +γδ
=
inf
sup [
l(s, xδ (s), α[b](s), b(s))ds + ϕ(tδ + γδ, xδ (tδ + γδ))]
α∈ΓSP
t ,t
δ δ +γδ
b∈Btδ ,tδ +γδ
tδ
∂ϕ
(t, x) + H(t, x, ∇ϕ(t, x))]δ − ϕ(tδ , xδ ) + o(δ)
∂t
∂ϕ
∂ϕ
= γ[ (t, x) + H(t, x, ∇ϕ(t, x))]δ + (1 − γ)[ (t, x) + H(t, x, ∇ϕ(t, x))]δ + o(δ)
∂t
∂t
∂ϕ
=
(t, x) + H (γ) (t, x, ∇ϕ(t, x))]δ + o(δ)
∂t
+ (1 − γ)[
This completes the proof of the present proposition.
(γ)
For given partition π : 0 = t0 < t1 < · · · < tk = T , define Vπ
recursively backward in time:
: [0, T ] × RN → R
Vπ(γ) (T, x) = Φ(x), x ∈ RN
Vπ(γ) (t, x)
=
(4.3)
Ft,ti+1 Vπ(γ) (ti+1 , ·)(x),
(γ)
x∈R
N
if ti ≤ t < ti+1 .
(4.4)
We consider the asymptotics of Vπ as kπk ≡ max0≤i≤k |ti+1 − ti | → 0. As we noted
in Section 2, under (A.1)–(A.3), it is proved that V P , V SP are bounded and Lipschitz
17
(γ)
continuous in [0, T ] × RN . Similarly, we can see that Vπ is also bounded and Lipschitz
continuous. Moreover, it is not difficult to show that L∞ -bound and Lipschitz constant
(γ)
of Vπ on [0, T ] × RN are independent of π. Therefore, by Ascoli-Arzelà’s theorem, there
(γ)
(γ)
exist a subsequence {Vπ(n) }∞
such
n=1 and a bounded Lipschitz continuous function V
that
(γ)
Vπ(n) (t, x) → V (γ) (t, x) as kπ (n) k → 0 uniformly on each compact set in [0, T ] × RN .
(4.5)
(γ)
(γ)
Indeed, we can prove V
is a viscosity solution of (3.3) with H = H , i.e. we have the
following result:
(γ)
Theorem 4.2. Vπ converges uniformly on compact set in [0, T ] × RN to the unique
viscosity solution V (γ) ∈ BU C([0, T ] × RN ) of Isaacs equation (3.3) with H = H (γ) :

(γ)
 ∂V
(t, x) + H (γ) (t, x, ∇V (γ) (t, x)) = 0, (t, x) ∈ (0, T ) × RN
(4.6)
∂t

(γ)
N
V (T, x) = Φ(x), x ∈ R .
Proof. Note that under (A.1)–(A.3), we can apply uniqueness or comparison results of
(γ)
viscosity solution in BU C([0, T ] × RN ) to (4.6) (cf. [7], [1]). So, we prove that if Vπ
converges to V (γ) uniformly on compact sets, then V (γ) is a viscosity solution of (4.6).
Let (t, x) ∈ (0, T ) × RN be a local maximum point of V (γ) − ϕ for sufficiently smooth
function ϕ on (0, T ) × RN . Specifically, we take [t − r, t + r] × B̄r (x) ⊂ (0, T ) × RN such
that
V (γ) (s, y) − ϕ(s, y) ≤ V (γ) (t, x) − ϕ(t, x), (s, y) ∈ [t − r, t + r] × B̄r (x).
(4.7)
We may suppose that (t, x) is a strictly local maximum point without loss of generality.
For each partition π : 0 = tπ0 < tπ1 < · · · ≤ tπk = T , take maximum point (tπ , xπ ) of
(γ)
(γ)
Vπ − ϕ in [t − r, t + r] × B̄r (x). Since (t, x) is a strict maximum point and Vπ converges
to V (γ) uniformly on compact sets, we see that
(tπ , xπ ) → (t, x) as kπk → 0.
Therefore, we have for sufficiently small kπk,
(tπ , xπ ) ∈ [t − r/2, t + r/2] × B̄r/2 (x).
(4.8)
(γ)
By definition of Vπ ,
(γ)
Vπ(γ) (tπ , xπ ) = Ftπ ,tπi+1 Vπ(γ) (tπi+1 , ·)(xπ ) if tπi ≤ tπ < tπi+1 .
(4.9)
Indeed, (4.9) reads
inf
α∈ΓSP
t ,t
π π +γδπ
sup
b∈Btπ ,tπ +γδπ
Z
[
tπ +γδπ
l(s, x(s)α[b](s), b(s))ds
tπ
+ FtPπ +γδπ ,tπi+1 Vπ(γ) (tπi+1 , ·)(x(tπ + γδπ ))] − Vπ(γ) (tπ , xπ ) = 0
18
(γ)
where we set δπ ≡ tπi+1 − tπ Then, by (2.7), (4.8) and the assumption Vπ
V (γ) uniformly on compact sets, we have for sufficiently small kπk,
converges to
FtPπ +γδπ ,tπi+1 Vπ(γ) (tπi+1 , ·)(x(tπ + γδπ )) − Vπ(γ) (tπ , xπ )
≤ FtPπ +γδπ ,tπi+1 ϕ(tπi+1 , ·)(x(tπ + γδπ )) − ϕ(tπ , xπ ),
which is uniform on α and b. Thus, we obtain
0 ≤ Ftπ ,tπi+1 ϕ(tπi+1 , ·)(xπ ) − ϕ(tπ , xπ ).
Hence, by using Proposition 4.1, we can see that
Ftπ ,tπi+1 ϕ(tπi+1 , ·)(xπ ) − ϕ(tπ , xπ )
∂ϕ
(γ)
(t, x) + H (t, x, ∇ϕ(t, x)) = lim
≥ 0.
kπk→0
∂t
tπi+1 − tπ
Therefore, V (γ) is a viscosity subsolution of (4.6).
The proof of supersolution is proved in a similar way.
References
[1] M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of
Hamilton-Jacobi-Bellman Equations, Birkhäuser, Boston, 1997.
[2] T. Basar and P. Bernhard, H ∞ -optimal control and related minimax design problems,
2nd ed., Birkhäuser, Boston, 1995.
[3] R.J. Elliott and N.J. Kalton, The existence of value in differential games, Mem.
Amer. Math. Soc. 126, 1972.
[4] L.C. Evans and P.E. Souganidis, Differential games and representation formulas of
Hamilton-Jacobi-Isaacs equations, Indiana Univ. Math. J. 33(5) (1984), 773-797.
[5] W.H. Fleming, Max-plus stochastic control, Lecture Notes on Control and Info. Sci.
280 (ed. B.Pasik Duncan) (2002), 111-119.
[6] W.H. Fleming, Max-plus stochastic Processes, preprint.
[7] W.H. Fleming and H.M. Soner, Controlled Markov Processes and Viscosity Solutions,
Springer-Verlag, 1991.
[8] E. Roxin, The axiomatic approach in differential games, J. Optim. Theory Appl. 3
(1969), 153-163.
[9] P.E. Souganidis, Approximation schemes for viscosity solutions of Hamilton-Jacobi
equations, J. Differential Eq. 59 (1985), 1-43.
19
[10] P.E. Souganidis, Max-min representations and product formulas for the viscosity
solutions of Hamilton-Jacobi equations with applications to differential games, Nonlinear Analysis, Theory, Methods & Appl. 9(3) (1985), 217-257.
[11] P.P. Varaiya, The existence of solutions to a differential game, SIAM J. Control
Optim., 5 (1967), 153-162.
20