Bellman-Isaaks equations
for differential games with random duration
Ekaterina Shevkoplyas
[email protected]
Supervisor Leon A. Petrosjan
REVIEW
E.J. Dockner, S. Jorgensen, N. van Long, G. Sorger. 2000 [1]
• Hamilton-Jacobi-Bellman equation for differential games with prescribed duration or infinite
time horizon
• A game-theoretic model of nonrenewable resource extraction with infinite time horizon
Petrosjan L.A., Murzov N.V., 1966 (in Russian) [3]
• 2- person game of pursuits with random duration (terminal payoffs)
Petrosjan L.A., Zaccour G., 2003 [5]
• Non-standard algorithm of characteristic function values calculating was proposed («Nash
equilibrium» approach: if k players form coalition K, then the remaining players stick to their
feedback Nash strategies)
• Time-Consistency of the Shapley Value was proved.
Petrosjan L.A. , 1977 [2]
•The notion of the time-consistency for differential games solution (prescribed duration of the
game)
Definition of the game
Differential n-person game (x0). The final time instant T is a random variable
with distribution function F(t). Let hi(x(τ)) be an instantaneous payoff of the
player i. Then the expected integral payoff of the player i, i=1,...,n is as follows:
 t
K i ( x0 , u1 ,..u n )    hi ( x( )) ddF (t ).
t0 t0
Cooperative game
n
( x0 )
n
n  t
max  Ki ( x0 , u1 ,..un )   Ki ( x0 , u1*,..un *)     hi ( x * ( )) d dF (t ) V ( N , x0 ).
u i 1
i 1
i 1 t t
0 0
Non-standard problem of dynamic programming (see form of functional)!
An Example (A Game-Theoretic Model of Nonrenewable Resource Extraction)
Let x(t) and ci(t) denote respectively the stock of the nonrenewable resource and
player i's rate of extraction at time instant t [1].The final time instant of the game is
random variable T with the exponential frequency distribution. The utility function (or
instantaneous payoff) for player i at time instant τ
hi (ci ( ))  A ln( ci )  B.
Results
1.
2.
We introduced a new class of differential games such that differential games with
random duration.
We derived the Isaak-Bellman equation (or the Hamilton-Jacobi-Bellman equation)
for the problem with random duration.
W ( x, t )
W ( x, t )


f (t )
W ( x, t ) 
 max 
H ( x, u ) 
g( x, u ) 

,
u
1  F (t )
t
x


3. Using non-standard approach of Petrosjan and Zaccour [5] we proposed an
algorithm of the characteristic function construction with the help of the new HamiltonJacobi-Bellman equation .
4. We applied our algorithm to game-theoretical model of nonrenewable resource
extraction with random duration.
5. We introduced a notion of time-consistency for differential games with random duration.
IDP (imputation distribution procedure) was derived in analytic form. Moreover we
proposed a method of optimality principle regularization if instantaneous payoffs are
positive.
6. We proved the time-inconsistency of the Shapley Value in the game of nonrenewable
resource extraction with random duration.
The questions and the way forward
What about regularization under condition of arbitrary sign of instantaneous payoffs
of the players (not only positive)?
2. Can we use incentive strategies in our model of nonrenewable resource extraction?
1.
We are going

to consider another form of utility function ( instantaneous payoff) in the model
of resource extraction such as follows:
hi  A
ci
1
1
 B,  1.
 to calculate PMS-value for all examples.
 to consider asymmetric payoffs of the players in the resource extraction game.
 to investigate the agreeability of the optimality principle in the game with random
duration.
References
1. E.J. Dockner, S. Jorgensen, N. van Long, G. Sorger. Differential Games in
Economics and Management Science. Cambridge University Press, 2000.
2. Petrosjan L.A.. Differential Games of Pursuit. World Sci. Pbl.2003. p. 320.
3. Petrosjan L.A., Murzov N.V. A Game Theoretic Model in Mechanics // Litovskyi
matematicheskyi sbornik, vol.VI, Vilnjus, Litva, 1966, pp. 423- 432. (in Russian)
4. L.A. Petrosjan, E.V. Shevkoplyas. Cooperative Solutions for Games with Random
Duration. Game Theory and Applications, Volume IX. Nova Science Publishers,
2003, pp.125-139.
5. Petrosjan L.A., Zaccour G. Time-consistent Shapley Value Allocation of Pollution
Cost Reduction. // Journal of Economic Dynamics and Control, Vol. 27, 2003, pp.
381-398.
6. E.V. Shevkoplyas. On the Construction of the Characteristic Function in
Cooperative Differential Games with Random Duration. International Seminar
"Control Theory and Theory of Generalized Solutions of Hamilton-Jacobi Equations"
(CGS'2005), Ekaterinburg, Russia. Ext.abstracts, Vol.1,pp. 262-270. (in Russian)