Pursuit and Evasion Game under Uncertainty
Abiola, Bankole PhD
Department of Actuarial Science and Insurance
Faculty of Business Administration
University of Lagos
Akoka, Lagos
Email: [email protected]
Professor Ojikutu, R.K.
Department of Actuarial Science and Insurance
Faculty of Business Administration
University of Lagos
Akoka, Lagos
Email: [email protected]
Abstract: In this paper we examined a class of multidimensional differential games.
In particular we considered a situation in which the pursuer and evader are affected by
uncertain disturbances. A necessary and sufficient condition for the existence of
saddle point for this class of game was developed.
Key Words: Uncertain disturbances, Pursuer, Evader, Differential Games.
Introduction and System Description: Let IR n , IR m and IR l be Euclidean
spaces and . denote the Euclidean norm. We consider the uncertain dynamical
system modelled by:
x p (t )  A p x p (t )  B p u p (t )  v p (t )
...
x p (t 0 )  x p (0)  IR n ( given)
(1)
xe (t )  Ae xe (t )  Be u e (t )  ve (t )
xe (t )  xe (0)  IR m (given)
...
...
...
(2)
(3)
(4)
The subscripts p and e in equations (1)-(4) stand for pursuer and evader
respectively. Ap , Ae  IR nn , while B p , Be  IR nm .
The vectors v p (t ) and v e (t ) are uncertain vectors. The pursuer uses control
u p (t ) , to attempt to capture the evader, while the evader uses control u e (t ) to avoid
being captured. The problem of interest is the minimization and the maximization of
the final miss by pursuer and evader respectively, in the presence of uncertainties
v p (t ) and v e (t ) . The final miss is defined as a weighted quadratic form:
x p (T )  xe (T ) .
...
(5)
To make the game meaningful, we shall impose the following limitations:
T

0
T

0
2
u p (t )
Rp
u e (t )
dt   p
...
dt   e
...
2
Re
(6)
(7)
Where R p , Re are positive definite matrices such that
R p : IR m  IR n ,
Re : IR m  IR n
 p ,  e  0, T is the final time. Adjoining (6) and (7) to (5) , we have the following
pay-off functional defined as:
J (u p , u e ) 


T
2
1
x p (T )  xe (T )   [u p (t ) T R p (t )u p (t )  (u e (t ) T Ru e (t )] dt .. (8)
0
2
The controller u p (t ) is the minimizer of J (u p , u e ) and the controller u e (t ) is the
maximizer. In equation (1) and (2) the uncertainty v p (t ) is acting against the wish of
the controller u p (t ) and in equation (3) and (4) is also acting against the wish of the
maximizer u e (t ).
For subsequent development of this paper we make the following assumptions:
Assumption A1: Th uncertainties
v p (t ) : V p  IR
lp
ve (t ) : Ve  IR le
where V p  Ve   and IR p , IR le  IR l are assumed Lebesgue measurable and
ranged within a compact set. Then there exist constants such that
l
v p (t )   p , p  (0, ),  p  0
ve (t )   e , e  (0, ),  e  0


A2: There exist non-singular matrices P and E of appropriate dimensions such that:

u p (t )   Px p (t )

u e (t )   Exe (t )
. . . (9)
. . . (10)
 
Also given any Q1 and Q2  PD n there exist unique solution P, E  PD n such that
the following matrix Lyapunov equations are satisfied.


PA p  ATp P  Q1  0



EAe  AeT E  Q2  0
. . . (11)
Problem Formulation: Define a new state variable as follows:
z (t )  x p (t )  xe (t )
. . . (12)
Then from equations (1) – (4) we have
z (t )  Ap x p (t )  Ae xe (t )  B p u p (t )  Be u e (t )  v p (t )  ve (t )
. . . (13)
where

P  P 1 
 
E  E 1 
. . . (14)
We shall put (13) in compact form as:
z (t )  Fu p (t )  Gue (t )  v(t )
. . . (15)
where



T
G  Ae E  Be


v(t )  v p (t )  v e (t )
F  B p  ATp P
We also impose the condition that v(t )   ,   0.
On the basis of (15) we have the following problem:
. . . (16)
Problem:1
Min max J (u p , u e ) 


T
1
z (T ) T z (T )   [(u p (t ) T R p u p (t )  u e (t ) T Re u e (t ))]dt ,
t0
2
Subject to
z (t )  Fu p (t )  Gue (t )  v(t ), t  [t 0 , T ]
Problem (1) will be solved under the following assumption.
A3: The pay-off functional defined by J (u p , u e ) in Problem (1) can be separated to
be of the form:
min max{ J (u p , u e )}   ( z (T ))   {h1 (t , z, u p )  h2 (t , z, u e )}dt
… (17)
Based on the assumption A3, and noting that
v(t )  v p (t )  ve (t )
. . . (18)
We arrived at the formulation of the following two optimal control problems define
by Problems (2) and (3).
Problem (2):
MinJ (u p ) 
T
1
z (T ) T z (T )   {u p (t ) T R p u p (t )}dt 

t0
2 
Subject to
z (t )  Fu p (t )  u p (t )
z (t 0 )  z 0
… (19)
Problem (3):
MaxJ (u e ) 
T
1
z (T ) T z (T )   {u e (t ) T Re u e (t )}dt 

t0
2 
Subject to
z (t )  Gue (t )  ve (t )
. . . (20)
z (t 0 )  z 0
Solution and Necessary Condition for a Saddle Point
For problems (2) and (3) we introduce the following assumptions:
A4: The matrix functions R p (.), Re (.), F (.) and , G (.) are constant matrices of
appropriate dimensions.
A5: Control functions u p (.) , u e (.) and disturbances v p (.) , ve (.) in problems (2) and
(3) are generated by strategies.
Now consider problem (2) and define the Hamiltonian for the problem as
H ( z, u p ,  ) 
1
(u p (t ) T R p u (t ))  T v p (t )
2
. . . (21)
The adjoint equation satisfies
  0


 (T )  z (T )
. . . (22)
1
Hu p  0  u p (t )  R p (t ) F T (t ) (t )
Now,
max H  max {T v p }.
v p  p
v p  p
Therefore
v p (t ) 
 p  (t )
 (t )
. . . (23)
From (23) we deduce the following three cases, namely:
(i)  (t )  0 : v p (t )   p ,
(ii)  (t )  0 : v p (t )   p
(iii)  p (t )  0 : then any admissible v p (t ) is an optimal solution.
We knock off case (ii) since  (t ) cannot be negative as an adjoint vector.
Case (iii) is a trivial solution. Assume that the solution is not trivial, we consider case
(i).
Let  (.) : [t 0 , T ]  IR n 1 be continuous and let  (t ) be define as
 (t )  M (t ) z (t )  N (t ) p ,
. . . (24)
where M (t ) and N (t ) are square matrices and z (t ) : [t 0 , T ]  IR n is a solution of
(19) . We differentiate (14) to get:
 (t )  M (t ) z (t )  M (t ) z (t )  N (t ) p
Hence,
M (t ) z (t )  M (t ) z (t )  N (t ) p  0
. . . (25)
… (26)
Therefore on the basis of (12) we have
M (t ) z (t )  M (t ) F (t ){R p1 F (t ) (t )}  M (t ) p  N (t ) p  0
… (27)
Substituting (14) into (17) we get
M (t ) z (t )  M (t ) T F (t ) R p1 (t ) F (t ) M (t ) z (t )  M (t ) T F (t ) R p1 (t ) F (t ) N (t ) p
 ( M (t )  N (t )) p  0
. . . (28)
For (18) to hold, it is necessary and sufficient that
M (t )  M (t ) F (t ) R p1 (t ) F (t ) M (t )  0


M (T )  I


1
N (t )  M (t )  M (t ) F (t ) R p (t ) F (t ) N (t )  0

N (T )  I

. . . (29)
Let  ( z (t ), t ) and  ( z (t ), t ) be strategies for u p (t ) and v p (t ) respectively, we
then summarize the saddle point solution as follows:
 ( z (t ), t )  u p (t )   R p1 (t ) F (t ) T {M (t ) z (t )  N (t ) p }
 ( z (t ), t )  v p (t )   p
. . . (30)
For problem (3) the Hamiltonian is defined as
1
H (u e , ve , )   (u e (t ) T Re (t )u e (t ))   (t ) T {Gue (t )  ve (t )}
2
. . . (31)
Following the same procedure as we did for problem (2) we found that the adjoint
vector satisfies
 (t )  0 

 (T )  z (T )
. . . (32)
Now,
Hu e  0  u e (t )  Re1 (t )G (t ) (t )
. . .(33)
Since u e is a maximizer, we choose v e (t ) such that
min H  min { (t ) T ve }
ve  e
ve  e
. . . (34)
Therefore
ve (t )  
 e (t )
 (t )
. . . (35)
The following three cases can be deduced from (25)
(i)
 (t )  0, ve (t )   e
(ii)  (t )  0, ve (t )   e
(iii)  (t )  0, then any admissible v e (t ) is an optimal solution. … (36)
Since v e (t ) is a minimizer and from (26) we shall knock off case (ii) and we shall
also assume a non-trivial solution, hence we shall let
 (.) : [t 0 , T ]  IR n 1 be continuous and be defined as
 (t )  D(t ) z (t )  E (t ) e
. . . (37)
Where D(t ) and E (t ) are square matrices and z (t ) : [t 0 , T ]  IR n is a solution of
(10). We observed that
 (t )  D(t ) z (t )  D(t ) z (t )  E (t ) e
. . . (38)
From (22)
D(t ) z (t )  D(t ) z (t )  E (t ) e  0
. . . (39)
Therefore
D(t ) z (t )  D(t )Q(t )u e (t )  D(t )ve (t )  E (t ) e  0
. . . (40)
which becomes
D (t ) z (t )  D(t ) Re1G (t ) T  (t )  D(t ) e  E (t ) e  0
. . . (41)
On substituting (37) into (41) we have the following equation
D (t ) z (t )  D(t )G (t ) Re1 (t )G (t ) T D(t ) z (t )  D(t )G (t ) Re1 (t )G (t ) E (t ) e
+ D(t ) e  E (t ) e  0
. . . (42)
For (42) to hold
D (t )  D(t )G (t ) Re1 (t )G (t ) T D(t )  0


E (t )  D(t )  D(t )G (t ) Re1 (t )G (t ) E (t )  0
. . . (43)
Let  ( z (t ), t ) and  ( z (t ), t ) be strategies for control u e (t ) and disturbance
respectively, then the saddle point solution is summarized as follows:
 ( z (t ), t )  u e (t )  Re1G (t )[ D(t ) z (t )  E (t ) e

 ( z (t ), t )  ve (t )   e
. . . (44)
Value of the Game
We now employ the results in (20) and (34) to compute the value of the objective
functional defined in problems (2) and (3)
We recall that
z (t )  Fu p (t )  v p (t )
… (45)
 (t )  0
. . . (46)
 (T )  Z (T )
. . . (47)
Multiply (45) by  (t) T and equation (46) by z (t ) to get
 (t ) z (t )   (t )T Fu p (t )   (t )T v p (t )
. . . (48)
 (t ) z (t )  0
. . . (49)
add equations (48) and (49) together to get
 (t )T z (t )   (t )T z(t )   (t )T Fu p (t )   (t )v p (t )
d
[ (t ) z (t )]   (t ) T Fu p (t )   (t )v p (t )
dt
=
. . . (50)
Integrating (50) we get the following:
T

t0
T
d
[ (t ) T z (t )]   { (t ) T Fu p (t )   (t ) T v p (t )}dt
t0
dt
T
=  (t ) z (t ) |Tt0   { (t ) T Fu p (t )   (t ) T v p (t )}dt
t0
T
T
t0
t0
=  (T ) T z (T )   (t 0 ) z (t 0 )    (u p (t ) R p u p (t )dt   { (t ) T  p ]dt
We know from (47) that  (T )  z (T ) , therefore,
T
T
t0
t0
z (T ) T z (T )   {u p (t ) T R p (t )u p (t )}dt   (t 0 ) z (t 0 )   { (t ) p }dt
Hence,
T
2 J (u p (t ))   (t 0 ) z (t 0 )   { (t ) T  p }dt , and from
t0
 (t )  M (t ) z (t )  N (t ) p , therefore
T
2 J (u p (t ))  M (t 0 ) z (t 0 ), z (t 0 )   N (t 0 ) p z (t 0 )   {( M (t ) z (t )  N (t ) p ) T  p }dt
t0
. . . (51)
Similarly, we consider
z (t )  Gue (t )  ve (t )
 (t )  0
 (T )  z (T )
. . . (52)
. . . (53)
. . . (54)
From (52) and (53) we have
 (t ) T z (t )   (t ) T Gue (t )   (t ) T ve (t )
. . . (55)
 (t ) T z (t )  0
. . . (56)
Combining (55) and (56) we have
 (t ) z (t )   (t ) z (t )   (t ) T Gue (t )   (t ) T ve (t )
. . . (57)
Therefore
T
T
d
{ (t ) z (t )}dt    Re u e (t ), u e (t )  dt   ( (t ) T  e )dt
t 0 dt
t0
t0
. . . (58)

T
Now, given that  (T )  z (T ), then
T
T
t0
t0
z (T ) T z (T )   ( Re u e (t ), u e (t )dt   (t 0 ) T z (t 0 )   ( (t ) T  e )dt
. . . (59)
From (37) substitute for  (t ) to get
T
z (T ) T z (T )   ( Re u e (t ), u e (t )) dt  z (t 0 ) T D(t 0 ) z (t 0 )  E (t 0 ) e z (t 0 )
T
to
T
  {( D(t ) z (t )  E (t ) T  e }dt
t0
. . . (60)
Therefore
2 J (u e )  z (t 0 ) T Dz (t 0 )  E (t 0 ) eT z (t 0 )
T
  {( D(t ) z (t )  E (t ) e ) T  e }dt
. . . (61)
t0
Combining (51) and (61) , the value of the game is given as
1
J (u p , u e )  { ( M (t 0 )  D(t 0 ), z (t 0 ) 
2
T
 ( N (t 0 ) p  E (t 0 ) e ) z (t 0 )   [( M (t )  D(t ) z (t )
t0
 ( N (t ) p )T  p  ( E(t ) e }dt}
. . . (62)
Sufficient Condition
We shall employ the sufficiency theorem given by [Gutman, S. (1975)] to show that
the solution got for each of the cases is indeed a saddle point. To achieve this let
V ( z, t ) : IR n  IR1  IR1
be defined as
V ( z, t )  z(t )T Mz (t )  N (t ) p z(t )
. . . (63)
We assume that V (.,.) is continuous on [t 0 , T ] and it is a C 1  function.
Define
L( z, t , u p , v p )  u p (t )T R p (t )u p (t )  gradV ( z, t ) F ( z, t , u p , v p )
 u p (t ) R p (t )u p (t )  2 z(t )T M (t ) z (t )  N (t ) p z (t )
 z (t ) M (t ) z (t )  N (t ) p z (t )
= u p (t ) R p (t )u p (t )  2 z(t )T [M (t )( F (t )u p (t )  v p (t ))]
 N (t ) p [ F (t )u p (t )  v p (t )]  z(t )T M (t ) z(t )  N (t ) p z(t )
. . . (64)
By virtue of (23) and (24) we have
1
L( z, t , u p , v p )  u p (t ) R p (t )u p (t )  z (t ) M (t ) z (t )  2 z (t )M (t ) T F (t ) R p (t ) F (t ) T M (t ) z (t )
1
 2 z (t ) M (t ) T F (t ) R p (t ) F (t ) N (t ) p  2 z (t ) M (t ) p
1
1
 M (t ) T F (t ) R p (t ) F (t ) T N (t ) p z (t )  N (t ) T F (t ) R p (t ) F (t ) N (t ) p2
 N (t ) p  N (t ) p z (t )
. . . (65)
1
= u p (t ) R p (t )u p (t )  z (t )( M (t )  M (t ) F (t ) R p (t ) F (t )M (t ) z (t )
T
- z(t )M (t )T F (t ) R p1 (t ) F (t )T M (t )  N (t )  M (t )
1
- M (t ) T F (t ) R p1 (t ) N (t ) z (t ) p  2M (t ) T F (t ) R p (t ) F (t ) N (t ) p z (t )
+ M (t ) z(t ) p  N (t ) F (t ) R p1 (t ) F (t ) N (t ) p2  N (t ) p
. . . (66)
By using (29), (66) reduces to
L( z, t , u p , v p )  u p (t ) R p (t )u p (t )  M (t ) z(t ) p  2M (t )T F (t ) R p1 (t ) F (t ) N (t ) p z(t )
- N (t )T F (t ) R p1 F (t ) N (t ) p2  N (t ) p  z(t )M (t )T F (t ) R p1 (t ) F (t )T M (t ) z(t )
. . . (67)
In (66) we substitute for u p (t ) R p (t )u p (t ), using (23) and (24) to get
L( z , t , u p , v p )  M (t ) z (t ) p  N (t ) p
...
(68)
Now
min L( z, t , u p , v p )  minm [ M (t ) z (t ) p  N (t ) p ]  0
u p IR m
u p IR
...
(69)
and
maxl L( z, t , u p , v p )  maxl [ M (t ) z (t ) p  N (t ) p ]  0, v p   p
v p V
p
v p V
p
On the basis of (69) and (70), (30) is indeed a saddle point.
. . . (70)
Conclusion: The idea of saddle point (min, max) controllers arises in
engineering problems where extreme conditions are to be overcome
(Gutman,1975) A natural example is the “boosted period” of missiles when
high thrust acts on the body so that every small deviation from the designed
specifications cause unpredictable (input) disturbances in three nodes.
In this work we have considered situations where the disturbances affect the
motions of the pursuer and the evader respectively. In our subsequent paper
we hope to apply the results in this work to problems arising from pricing of
general insurance policies, particularly in a competitive and non-cooperative
market.
References:
[1] Abiola, B.(2012) “On Generalized Saddle Point Solution for a Class of
Differential Games” International Journal of Science and Advanced Technology,
2(8):27-31
[2] Abiola, B.(2009) “Control of Dynamical Systems in The Presence of Bounded
Uncertainties” Unpublished PhD Thesis , Department of Mathematics University of
Agriculture Abeokuta.
[3] Arika, I. (1976). “Linear Quadratic Differential Games in Hilbert Space” SIAM
Journal of Control and Optimization, 1(1)
[4] Gutman,S, (1975). “Differential Games and Asymptotic Behaviour of Linear
Dynamical Systems in the Presence of Bounded Uncertainty” PhD Dissertation,
University of California, Berkley
[5] Leitmann, G, (2004) “ A Direct Optimization Method and its Application to a
Class of Differential Games” Journal of Dynamics of Continuous and Intensive
Systems, 11, 191-204