The Expected Value of Perfect
Information in the Optimal
Evolution of Stochastic Systems
Dempster, M.A.H.
IIASA Working Paper
WP-81-055
April 1981
Dempster, M.A.H. (1981) The Expected Value of Perfect Information in the Optimal Evolution of Stochastic Systems. IIASA
Working Paper. IIASA, Laxenburg, Austria, WP-81-055 Copyright © 1981 by the author(s). http://pure.iiasa.ac.at/1706/
Working Papers on work of the International Institute for Applied Systems Analysis receive only limited review. Views or
opinions expressed herein do not necessarily represent those of the Institute, its National Member Organizations, or other
organizations supporting the work. All rights reserved. Permission to make digital or hard copies of all or part of this work
for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial
advantage. All copies must bear this notice and the full citation on the first page. For other purposes, to republish, to post on
servers or to redistribute to lists, permission must be sought by contacting [email protected]
NOT FOR QUOTATION
WITHOUT PERMISSION
OF THE AUTHOR
THE EXPECTED VALUE OF PERFECT
INFORMATION I N THE OPTIMAL
EVOLUTION OF STOCHASTIC SYSTEMS
M.A.H.
Dempster
A p r i l , 1981
WP-81-55
W o r k i n g P a p e r s a r e i n t e r i m r e p o r t s o n work o f t h e
I n t e r n a t i o n a l I n s t i t u t e f o r Applied Systems Analysis
a n d h a v e r e c e i v e d o n l y l i m i t e d review.
Views or
o p i n i o n s e x p r e s s e d h e r e i n do n o t n e c e s s a r i l y r e p r e s e n t t h o s e o f t h e I n s t i t u t e o r o f i t s N a t i o n a l Member
Organizations.
INTERNATIONAL INSTITUTE FOR APPLIED SYSTEMS ANALYSIS
A-2361 L a x e n b u r g , A u s t r i a
To appear in: S t o c h a s t i c D i f f e r e n t i a l S y s t e m s ; P r o c e e d i n g s o f
t h e IFIP Working C o n f e r e n c e , V i s i g r a d , Bungary, S e p t e m b e r 1 9 8 0 .
M . Arato & A.V. Balakrishnan, eds. Springer-Verlag Lecture Notes
in Control and Information Science.
ABSTRACT
This paper uses abstract optimization theory to characterize
and analyze the stochastic process describing the current m a r g i n a l
e x p e c t e d v a l u e o f p e r f e c t i n f o r m a t i o n in a class of discrete time
dynamic stochastic optimization problems which include the familiar optimal control problem with an infinite planning horizon.
Using abstract Lagrange multiplier techniques on the usual nonanticipativity constraints treated e x p l i c i t l y in terms of adaptation of the decision sequence, it is shown that the marginal
expected value of perfect information is a nonanticipative supermartingale. For a given problem, the statistics of this process
are of fundamental practical importance in deciding the necessity
for continuing to take account of the stochastic variation in the
evolution of the sequence of optimal decisions.
CONTENTS
1.
Introduction
2.
The dynamic recourse problem as an abstract optimization problem
3.
Characterization of optimal solutions
4.
The marginal expected value of perfect information supermartingale
5.
Possible extensions
6.
Acknowledgements
References
This paper uses abstract optimization theory to characterize and
analyze the stochastic process describing the current marginal expected
value o f perfect information in a class of discrete time dynamic stochastic optimization problems.which include the familiar optimal control
problem with an infinite planning horizon. Using abstract Lagrange
multiplier techniques on the usual nonanticipativity constraints treated
zxplicitly in terms of adaptation of the decision sequence, it is shown
that the marginal expected value of perfect information in a nonanticipative supermartingale. For a given problem, the statistics of this
process are of fundamental practical importance in deciding the necessity
for continuing to take account of the stochastic variation in the evolution
of the sequence of optimal decisions.
'
Let x = Cxtjt=l be a sequence of d e c i s i o n s in Rn and let
be a discrete time stochastic process in ( Z , C , p ) of sub_F = i;t)t=l
sequent obscr2ations. A ?olicy ( d e c < s i o n r u l e or r e c o u r s e f u n c t i o n ) is
a meesurable nap x : S - c x ( S )
Consider the problem
-
.
(where nt 2 n T = n , mt ;mT = m) and the n o n a n t i c i p a 5 i v e condition
that the current decision xt depends only on the sequence of observations 51,52,..., St-l , and realised decisions x1,x2,..., x ~ -(and
~
thus 5 ) to date. Here ft: = x Xtll nnt+IEl is assumed measurable
in its first argumsnt and Bore1 mehsurable in its second and
(A full set of technical
f (XI: = ft ( ,x ( ) ) , and similarly for qt (5)
-t
assumptions will be introduced in $$2 and 3.)
The problem (RPI--termed the d y n a m i c r e c o u r s e p r o b i e m - - h a s a number
of important applications in the mathematical sciences ( c j . Dempster,
1980). Special cases include stochastic dynamic linear or quadratic
programming formulations of e n e r g y - e c o n o n i c p l a n n t n g models, 3irge (1980),
Louveaux and Smeers I1980,1981); t n v e n t o r g z g n t r o l models, see e.g.
Veinott (1966); Mcrkov d e c i s i o n p r o c e s s e s with random transition matrices
for m s n p o w e r p l a n n i n g , Grincld (1976,1980) and the classical d i s c r e e e
t i m e o p t i m a l c o n t r o l model. To see the last assertion in more detail,
make the following substitutions in (,W) :
-
-
-
.
Then (RP) reduces to the familiar c o n t r o l p r o b l e m
Control and state space constraints are easily added to (C) by suitable
definition of Pt, t=l,...,T
Characterization of the (optimal) solutions to the general problem
(RP) for finite
has been treated by Rockafellar and Wets(1976aIb,
1978) for the convex case under a Slater r e g u i a r i t y c o n d i t i o n ( c o n s t r a i n t
a u ~ t i , + ' i ~ a t t o n ) using the duality theory of convex conjugate functions.
:lore recently (19811, they have given a similar treatment of the convex
Bolza problem--a special case of (C)--for finite r
Hiriart-Urruty
(1978,1981) has considered a more. general class of nonlinear special
cases of (RP) for finite T
He applied a theory characterizing in
terms of generalized gradients the minimum of an integral fmctional
involving a measurable locally Lipschitz integrand subject to a measurable closed valued multifunction constraint. The version of (Re) he
treated has some nondifferentiable and 'some differentiable constraints
and a correspondingly mixed Slater/llangasarian-Fromowitz constraint
qualification is enforced.
This paper discusses the use of recently developed abstract mathematical programing theory (Dempster,l976;Zowe and Kurcyusz,l979;
Brokate,l980) to extend these results to the i n f t n i t e h o p i t o n ( r
infinite) problem for the general nonlinear case under appropriate regularity conditions involving problem functions in LThe s a p p o r t
r e p r e s e n $ a ? i o n problem (Dempster,l976) for (RP) is addressed to obtain
the agropriate stochastic maximum principle and nonanticipative supermartingale representation of the L1 multiplier process corresponding
to the nonanticipative constraints. The general results were announced
in Dempster (1980); details and proofs will appear elsewhere. In this
paper, emphasis is placed on precise problem formulation, results and
interpretation ( $ 5 2 and 3 ) and attention is focused on the supermartingale multiplier process corresponding to the nonanticipative constraints
($4). Technical limitations to extending the analysis of this marginal
expected value of perfect information process to continuous time systems
--involving say diffusion or jump dynamics--are discussed briefly in $ 5
with a view to their possible relaxation jn future work using recent
theories of ~athwiseintegration of stochastic differential equations
(Sussman,l978;Marcus,l981).
.
.
.
.
2.
THE DYNMIIC RECOURSE PROBLEM AS AN ABSTRACT OPTIMIZATIOX PROBLEM
The purpose of this section is to set up the problem (RP) as an
abstract mathematical programme of the form
(PI
supxeP f
(2;)
s.t.
g
(x) E: '2
where P and Q are sets in appropriate linear topological spaces U
and V , f:U + R and g:U + V
A natural assunption to make in the applications cited in $1 is
that feasible policies should be e s s s n t t c l l y bounded--i.e. in Lm -on (S,Z,p) , c j . Rockafellar and Wets (1976,1978,1981), although a
similar treatment of policies in L (1 ;p cm) is also possible, c f .
P
Eisner and Olsen (1975,1980), Hiriart-Urruty (1978,1981). Hence assume
Z completed with respect to u and define
.
and
equipping U with the usual equivalent of the product topology defined
(we shall be party to the usual
in t e m s of the norm (ul = suptlutlm
abuse of terminology by referring to the equivalence class elements of
Banach function spaces as functions and subject ?o the standard analyst/
probabilist's schizophrenia by denoting the elements of. for example,
.
-
IJa, as both u and u depending on whether the analytic or probabilistic interpretation is to the fore.) The abstract objective function of
(PI is obviously
but a little more analysis will be necessary in order to define the
abstract constraint function g and its range (image) space V
The problem lies with the e z ~ l i c i tcharacterization of nonanticipation. Its solution was first proposed in the context of stochastic
i i n e c r programming by Eisner and Olsen (1975,1'980) Let {Zt) be the
usual increasing tower of a-fields,
.
.
generated b y the process
-5 .
Cvzry feasible policy
x-
is assumed to
n
After canonically embedding Lm
define the closed projections
for t=l,...,r
.
in
-
Then x is adapted to
n
L_
-5
in the obvious wag. we may
if, and only if,
Here (when we make the usual assumption that x l is deterministic)
, where K 1 denotes the linear sgan of the constants in
": ="1
n
embedded in Lm
.
La,
We may now define the constraint function of (PI as
such that
with V equipged with the normed-defined equivalent of the product
topology (as with U ) . All these considerations apply equally well to
In the sequel we shall consider only the case
finite or infinite r
of infinite r ; necessary changes for the simpler case of finite T
are easily supplied by the reader. Further. define ttIxt to be the
,~c,...,x-t , of the observation,
respechisrory, i.e. 51,;2,...,5t;
tively policy, process to tine t
We shall specialize in what follows
to the case--relevant to all the practical examples of $1--of t r i ~ n g u t a ~
(RP), for which current constraints depend only on observation and
decision histories to date, i.e.
.
.
(Thus the measurability properties for the
strictec? to It rather than : . )
gt
cited in $1 may be re-
This completes the basic set of assumptions needed to specify and
analyze the dynamic recourse problem (RP) over an infinite horizon.
Further(i1lustrative)technical assumptions will be introduced as required to complete the analysis in $3.
The characterization of optimal solutions for the abstract mathematical programming problem (P) necessitates consideration (perhaps only
implicitly) of its L a n g r a n ~ i c n f u n c t i o n
for X E U and m u l t i p z i e r vectors ~ ' E V ,' the duaZ space of V (consisting of all linear functionals on V continuous in the given topology
for V).
*
We shall thus here need the following characterization of L- , due
essentially to Yosida and Hewitt (1952) (see also Dubovitskii and
Ililyutin, 1965, Valadier, 1974, and Denpster, 1976). A finitely additive
R "'
on (1.1) is p u r e l y f i n i t e l y
row n-vector valued measure T ' : L
a Z d i t t 3 e ( p . j . a . ) if, and only if, for all countably additive real valued
measures v on (:,XI and for all E>O there exists AEcE S U C ~that
n'
i. e. a p. f a. measure is carried by
i v (AL)I < L and 1' ( hE) = O I L R
sets assigned arbitrarily small measure by any countably additive measure.
(Prime is used to denote a dual e1ement;in the finite dimensional case
n* the
this is consistent with vector transposition.)
Denote by Ln (as defined in (2.1)1 , by ':L
(Banach) dual spaca of Lthe space of
(coordinatewise) absolutely integrable row n-vector valued functions on
(E,1) and by pn' the space of purely finitely additive row n-vector
valued measures on ( Z X >
+
.
8
.
F'ropositiox:
3.1. Given the neasure space (f,Z,u)
with respect to the u-finite measure p , then
,
if
Here = denotes isometric isomorphism and the action of
n-vector valued function xr~: is given by
(3.2)
y'x: =
:-,y ' (;)x
(2:)
i
-
/-;?.'(dS)x(~)
X
is complete
n*
ylcLm on an
The f i r s t i n t e g r a l i n ( 3 . 2 ) i s s i m p l y an a b s t r a c t Lebesque i n t e g r a l ; t h e
second r e q u i r e s t h e a n a l o g o u s i n t e g r a t i o n t h e o r y d e v e l o p e d f o r f i n f t e Z y
a d d i t i v e measures by Dunford and Schwartz ( 1 9 5 6 ) .
In f a c t , Valadier
( 1 9 7 4 ) e x t e n d e d t h e r e s u l t o f P r o p o s i t i o n 3.1. t o t h e a - f i n i t e c a s e from
t h e E i n i t e c a s e e s t a b l i s h e d by Yosida and H e w i t t
(1952), whileDubovitskii
and C l i l y u t i n ( 1 9 6 7 ) i n d e p e n d e n t l y gave a c o m p l e t e t r e a t m e n t o f
*
in
Lm
(y;- of ( 3 . 2 ) w i t h o u t r e f e r e n c e t o t h e i r
i n t e g r a l r e p r e s e n t a t i o n s . A f i n e r c h a r a c t e r i z a t i o n o f L: i n t e r m s of
n a t u r a l s u b s p a c e of p n r a p p e a r s i n Dempster (1976)
terms of s i n g u L a r functior.aZs
.
tJe a r e now i n a p o s i t i o n t o ma!ce p r e c i s e s e n s e of t h e L a n s r a n g i a n
.
According t o ( 2 . 4 ) w e a r e i n t e r e s t e d i n r e p r e mt*
n N
n N
s e n t a t i o n of t h e d u a l s p a c e , XtP1 L,
x (L,)
)
o f V (=xt_Y L? x (L,)
where IN denota-s t h e n a t u r a l numbers.
A s t r a i g h t f o r w a r d a p p i i c a t i o n of
function (3.1) f o r (P)
P r o p o s i t i o n 3.1
Q
yields
a s g i v e n by
u s i n g t h e f a c t (Yosida and H e w i t t ,
-.
1952) t h a , t a l l p . f . a .
m e a s u r e s on IN
( w i t h c o u n t i n g measure % t a k e n a s ground measure) a r e c a r r i e d by
neighbourhoods of
XI
m
I n (3.3)
E P [ ( I x ~E X
, Y ( N ) Iu
X # )
;
=m+n ' 1
t h e s p a c e o f row (m+n)-vector
v a l u e d measures on t h e p r o d u c t a - f i e l d shown, a n d i n t h e c o r r e s p o n d i n g
integral
gt
h a s been c a n o r . i c a l l y embedded i n lRm
Next we c h a r a c t e r i z e an optimum x of
0
c o n c e p t of d e r i v a t i v e s of t h e L a g r a n g i a n
@
.
(PI i n t e r m s of a s u i t a b l e
g i v e n by ( 3 . 3 ) .
Rather than
u s e minimal c o n c e z t s and i n t r o d u c e h i g h l y t e c h n i c a l , c o n d i t i o n s on ( P ) ,
we s h a l l by way o f i l l u s t r a t i o n u s e ~ r g c h e td e r i v a t i v e s and g i v e regul a r i t y c o n d i t i c n s o:iliv f o r ; 2 P ) sufficient t o e n c c r e c h e t r u t h 02 t h e
f o l l o w i n g : : . i ; : , 6 - J s c k z r, ' ; ~ ~ c i l ~
f omr (P),
P.=.
Zowe and :turcyusz
(1979)
.
Suffice it to say here that versions of Proposition 3.2 below are available involving both generalized derivatives (cj. Niriart-Urruty, 1981)
and (one-sided) Gateaux directionpl derivatives (Dempster, 1976) under
m i n i m a l regularity conditions (cf. Dempster, 1976; Brokate, 1980) for
(P) posed in locally convex Hausdorff topological vector spaces. We
shall need the concept of the Jz.4~1 coze Q ' C V ' of a set Q C V as
and similarly for sets in U.
P r o p o s i t i o n : 3.2.
Let U and V be Banach spaces and the problem functions f and g of (P) be Frdchet differentiable with derivatives V f
and Vg respectively. Then under suitable regularity conditions on (P),
xO an optimum for ( - 0 ) implies that there exists y i Q'
~ such that
Vue P
Conditions (3.5) are termed Kuhn-Tucker (necessary 1 c o n d i t i o n s for an
optimum of (PI.
1f' 0 e P C U and
n e a s condit'ions
0E QCV
,
the last two imply the c o m p Zementary s l o c k -
Applying Proposition 3.2 under suitable regularity assumptions on
(RP) yields a (necessary) characterization of its optimal policies in
terms of the a b s t r a c t Lagrangian of (PI given by (3.3). However, inspection of (3.3) raises the question of conditions under which this characterization remains valid if the awkward terms involving integrals with
respect to purely finitely additive measures are dropped. This is a
special case of the ger~ersls ~ ; F J ~ * _ r; z ~ ~ l ~ ~ ~p r on j ltz ran t(Denpster,l975)
~ ~ n
I
which has appeared in the control (Dubovitskii and tlilyutin,1965), economics (e.g. Frescott and Lucss ,1372) 2nd o~;irr,iz~tic;n(Rockalcllzr ~ n d
Wets,1976a,b,1378) literature.
To solve the support representation problem for (RP) we must give
conditions on the problem sufficient to make both the stochastic p.f.a.
measures T t' and $I;, t=1,2,..., and the inter'enporal
p.f.a. measure :X
of (3.3) vanish. The followinc conditions are a distillation of the
literature cited above. Some terms and definitions will be needed. A
policy history xt is termed f e a s i b l e if it satisfies the constraint
structure of (RP) to time t, i.e. if we have for its components
Define
"t
xt: = lxt E I :xt = xt (6) , 6 E I
, xt
-.
feasible
1,
and
Clearly Ct essentially lies in Xt, but the analytical (computational)
intractability of (RP) arises from the fact that this inclusion is in
general essentially strict. The c o n t r o l l a b i i i t y (or relatively cornpletz
r e c o u r s e ) condition
ensures almost sure decision recourse at all times from any realization
of the observation (and decision) process to date and forces the optimal
stochastic p.f.a. measure T t' of Proposition 3.2 to vanish. Rockafellar
and Wets (1976a) obtain more technicai sufficient conditions. They show
by example that, without the ezplicit introduction into the problem of
the constraints which bind at an optimum induced on Ct by lhter stages,
the support representation problem in terms of L1 multipliers for (RP)
is insolubZe. Since a nonanticipative constraint (2.3) cannot lead to
infeasibilities of subsequent nonanticipative constraints, we can conclude immediately that the stochastic p.f.a. measures $,;
t=l,Z,.
always vanish at the optimum. To ensure that the optimal intertemporal
p.f.a. measure X:
vanishes,'it suffices .to assume that 0 EPt, t=1,2,
and the finite horizon a p p r o = i . ~ a t t o n conditioa:
..
...
(3.8)
for some r (211, x feasible implies (xt,O,O,. . . )
feasible for all t 1 r
.
Examples of nontrivial p.5.a. measnres (in the absence of this condition)
a r e known ( s e e P r e s c o t t a n d L u c a s , 1 9 7 2 ) .
N e x t we m u s t s t a t e s u i t a b l e r e g u l a r i t y c o n d i t i o n s o n (RP) s u f f i c i e n t
t o ensure t h e c o n c l u s i o n s of P r o p o s i t i o n 3.2.
i n terms o f
L, m u l t i p l i e r s .
a n d Q t c IRmt a r e
Hence w e a s s u m e ( b y way o f i l l u s t r a t i o n ) t h a t P t C IF?t
c l o s e d convex c o n e s , 0 c Q t and t h e p r o b l e m . f u n c t i o n s f t , g t are d i f f e r e n t i a b l e w i t h r e s p e c t t o t h e i r p o l i c y c o m p o n e n t s w i t h g r a d i e n t s V f t , Vgt,
t=1,2,
Given a set C ( i n a l i n e a r t o p o l o g i c a l s p a c e ) and a p o i n t
X E C d e f i n e t h e inner approximation coze
... .
w h e r e t h e i n t e r s e c t i o n is t a k e n o v e r n e i g h b o u r h o o d s x o f x .
assume t h a t f o r a n o p t i m a l p o l i c y
We shail
xy
( l i n e a r ) p r o j e c t i o n s I -!I o f ( 2 . 3 ' ) a r e c l o s e d , ( 3 . 9 ) i s
t
s u f f i c i e n t t o e n s u r e t h a t t h e a b s t r a c t problem f u n c t i o n g o f (2.4)
Since the
satisfies
i n terms o f i t s F r e c h e t d e r i v a t i v e a t a n optimum
xo o f
(PI f o r t h e
c l o s e d c o n v e x c o n e Q C V , cf. D e m p s t e r ( 1 9 7 6 ) , Zowe a n d K u r c y u s z ( 1 9 7 9 )
F i n a l l y w e a r e i n a p o s i t i o n t o a p p l y P r o p o s i t i o n 3.2
t o obtain a
Zocal v e r s i o n o f t h e Kuhn-Tucker n e c e s s a r y c o n d i t i o n s ( 3 . 5 ) f o r a n o p t i mal p o l i c y 2O o f ( R P ) , i n terms o f a s t o c h a s t i c mazimum p r i n c i p l e i n v o l v i n g t h e L m u l t i p l i e r p r o c e s s e s y' a n d p' , o f t h e f o r m
1
"
-
.
I n t h e c a s e t h a t a l l problem f u n c t i o n s a r e concave i n t h e p o l i c y v a r i ables, conditions (3.11) a r e a l s o sufficient--in general, they a r e not.
I n t h e n e x t s e c t i o n , w e t u r n t o an a n a l y s i s and i n t e r p r e t a t i o n of
t h e (optimal) m u l t i p l i e r process p ' corresponding t o t h e nonanticipative
c o n s t r a i n t s (2.3) of (RP)
.
4.
-
THE MARGINAL EXPSCTED VALUE O F PERFECT 1YT'ORI.IATION SUPERMARTINGALE
-
I n t h i s s e c t i o n we s h a l l assume t h a t a f i x e d o p t i m a l p o l i c y x f o r
0
(RP) i s s p e c i f i e d . ( V a r i o u s f u r t h e r a s s u m p t i o n s may be adduced t o t h e
problem t o g u a r a n t e e e x i s t e n c e and even u n i q u e n e s s o f t h e o p t i m a l p o l i c y ,
b u t t h e s e w i l l n o t c o n c e r n u s h e r e . ) W e s h a l l a p p l y modern p e r t u r b a t i o n
t h e o r y f o r t h e a b s t r a c t programme (PI o f $ 2 , see e.g. Lempio and Maurer
(1980), t o study t h e nonanticipative c o n s t r a i n t m u l t i p l i e r process p '
c o r r e s p o n d i n g t o t h e chosen o p t i m a l p o l i c y xo f o r (RP). Of i n t e r e s t
a r e p e r t u r b a t i o n s t o t h e n o n a n t i c i p a t i v e c o n s t r a i n t s ( 2 . 3 ) o f t h e form
-
-
where t h e zt a r e a r b i t r a r y n - v e c t o r v a l u e d f u n c t i o n s m e a s u r a b l e w i t h
r e s p e c t t o Z s ( t ) ( s ( t > t ) and hence r e p r e s e n t i n g information on t h e
fueuz-e of t h e o b s e r v a t i o n p r o c e s s 5
More s p e c i f i c a l l y , f i x t and
an a r b i t r a r y n - v e c t o r v a l u e d f u n c t i o n z,L m e a s u r a b l e w i t h r e s p e c t t o
f o r s o m e f i x e d s > t and c o n s i d e r p e r t u r b a t i o n s o f t h e tth nonIS
a n t i c i p a t i v e c o n s t r a i n t of t h e form
-.
.
f o r a E [ O , 6 1 ( 6 > 0)
Denote t h e o p t i m i z a t i o n problem r e s u l t i n g from
t h e p e r t u r b a t i o n ( 4 . 2 ) a s ( R P [ a z t l ) and i t s a b s t r a c t e q u i v a l e n t as
D e f i n e t h e a b s t r a c t p e r t u r b a t i o n f u n c t i o n n : V + I R U I-}
as
(P[ a z t ] )
.
Then, u n d e r c o n d i t i o n s on t h e problem d a t a o f (RP) s u c h t h a t t h e o r i g i n a l
problem h a s an o p t i m a l s o l u t i o n , t h e p e r t u r b e d problems P [ a z t l w i l l
have f e a s i b l e s o l u t i o n s . W e s h a l l assume t h a t w e nay f i n d a c u r v e x ( a )
Then,
s u c h t h a t x (a) i s f e a s i b l e f o r P [ a z t ] and l i m o s o x ( a ) = x O E U
s i n c e t h e c l o s e d p r o j e c t i o n ( I - !I t ) d e f i n s a s u b s p a c e of La , t h e
n'
Lagrange m u l t i p l i e r P; E: L, f o r t h e c o n s t r a i n t ( 2 . 3 ) i s an a n i h i l a t o r
( s u p p o r t i n g h y y e r ? l a ? e ) o f t h i s subs?aco. Under Dur a s s u ~ p t i o n s ( a p p l y i n 9 Theorem 4 . 3 , L e c p i o and biaurer,l9SC) we zay t h u s c o n c l u d e t h a t we
.
'
may choose
where
Vta denotes theFr6chet derivative of the perturbation function
(4.3) of the abstract problem (PI evaluated at 0 under perturbations of
the form (0.2) at time t
That is, the current state E; of the nonanticipative constraint multiplier process
in Ln1 ' represents the
marginal expected value of perfect i n f o r m a t i o n (EVPI) at time t with
respect to future states of the observation process E
We first establish that this marginal EVPI process p ' --like the
optimal policy process
itself-is adapted to the observation process
.
_pi
-.
zO
Lemma: 4.1.
The process
..'
in 1:'
P
-
is nonantiripatiue, i.e.
This fact follows from the observation that expression (4.4) for
P:
does not depend on any particular perturbation (4.2) representing
some future knowledge of the observation process E
m.
Next we show that the process o ' has the s u p e r m a r t i n g a t e property.
This reflects the fact that the earlier information on the future observation process F, is available, the more its marginal expected worth
to optimal decision making.
C
-.
-
-
Theorem: 4.2.
(4.6)
!?t'>
The process
E(.P;I
Ltl
By virtue of (4.5
-
P'
n'
in L ,
a,s.
is a s u p e r m a r t i n g a l e , i,e.
for (12) t < s
.
we must show for fixed t and s> t that
But a further consequence of (4.5) is that for all
and hence (4.6) is equivalent to showing that
s$t
-
But information on the future of 5 after time s-1, as represented
by an n-vector valued perturbation function z measurable with respect
to ZU for u ~ ,s cannot be -worth less in expectation the earlier it
is known, i.e.
Indeed, an optimal policy for the problem perturbed
where zS.. - zt: = z
at time t can iake this information into account earlier than a corresponding policy for the problem perturbed at s Hence, subtracting ~ ( 0 )
from each side of (4.7), dividing by a > 0 and passing to the limit as
a + 0 , yields
.
.
Since integration is nonnegativity preserving
5.
POSSIBLE EXTENSIONS
As noted in the introduction, th2 marginal expected value of perfect
information (EVPI) process 9' is of considerable jotential importance
for-stochastic systems of the dynamic recourse type arising in practice
n' : =
(see $1). If this nonanticipative supermartingale process in L1
Li [ (I,L ,u ) ; IRn' ] remains in a ball of (problem dependent) radius i > 0
for all t after some time s 2 l , then the stochastic elements of the
problem are practically inessential from time s onward and a deterministic model--and simpler computational procedure--should suffice. Of
course, this statenent raises the knotty problems of prior numerical
computation of the marginal EVPI process, or--more realistically--of
bounds on this process, etc. (in this context, see Birge,lSBO).
Nevertheless, it'would be interesting to have theoretical results
similar to those derived in $4 for familiar optimization of stochastic
system problems in contiquous time involving dynamics driven by semimartingales (see e.g. Shiryaev,l980). The difficulty in attempting an
analogue of the analysis presented in this paper for such systems is
that the corresponding pertxrbed abstrzct problem (as utilized in :4)
must make sense. Put 5ifferrntly, the ori~inelstochastic gptimization problem must remain well defined when nonanticipativity is
r a e Jsing the Ito calculus a F r ~ % c ; :I3r.2 1-2-. z1csnt t~;t~r,s.'c.-.s
- -.
to semimzrtingslzs 3ener~:ir.; mixec C i 2 i ~ r i c z3 2 2 j..-- dynamics) this
-
-I.,
is not possible, since the rigorous analytic i n t e g r a l form of the dynar n i c s r e q u i r e s n o n a n t i c i p a t i v i t y of the integra~ldin the stochastic integrals involved. This t e c h z S c s Z requirement of the stochastic integration
theories utilized has been relaxed for integration of G a u s s i o n processes
but
with respect to similar processes by Enchev and Stoyanov (19'80)~
this setting is of insufficient generality for many systems of interest.
More promising is the application to the problem at hand of the recent
p a t h w i s e theory of stochastic integration introduced for the study of
stochastic differential equations whose integrals are driven by processes
with continuous sample paths by Sussman (1978) and developed for semimartingales with jumps, for example, by Marcus (1981).
In the case of successful application of the approach of this
paper to optinization of stochastic systems in continuous t h e , with
differential dynamics in lRn of the form ;= f
- .(x)
" , it may be conjectured
that the f u l l expected value of perfect information process u" in L1
may be recovered from the m a r g i n a l EVPI process "
p a in 1;' by (Lebesque)
integration as
-
zS .
for an appropriate definition of
This is again a statement of'some
potential practical importance for stochastic system modelling.
6.
ACKNOWLEDGEMENTS
I would like to express my gratitude to J-M. Bismut, who first
pointed out to me the technical difficulties discussed in $4 regarding
extension of the present analysis to stochastic systems in continuous
t h e , and to M.H.A. Davis, whose conversation lead to the suggestions
made therein for surmounting them.
REFERENCES
.
1. J. Birge (1981 )
Stochastic dynamic linear programs. Ph .D. Thesis
Operations Research Department, Stanford University.
2. M. Brokate (1980). A regularity condition for optimization in
Banach spaces: Counter examples. Appl.Math.Optim. 6, 189-192.
3. M.A.H. Dempster (1976). Lectures on Abstract Optimization and Its
Applications. Department of Matheaatics, Melbourne University.
4 . M.A.H. Derr,pster (1980).
Introduction to.stochastic programming. In:
Dempster, ed. Academic, London.
S t o c h a s t i c P r o g r a m r i n g . M.A.H.
3-59.
5. A . Y . Dubovitskii & A.A. Milyutin (1965). Extremum problems in the
presence of restrictions. USSi? Comput. Math. & N c t h . P h y s . 5 . 3 ,
1-80.
6. N. Dunford & J.J. Schwartz (1956). L i n e a r O p e r a t o r s . Vol.1. Interscience, Mew York.
7. M.J. Sisner & P. Olsen (1975). Duality for stochastic programming
interpreted as L.P. in L -space. SIAM J.Appl.Math. 28, 779-792.
P
8. M.J. Eisner 6 P. Olsen (1980). Duality in probabilistic programming.
In: S t o c h a s t i c Programming, o p . c i t . , 147-158.
9. O.B. Enchev & J.M. Stoyanov (1980). Stochasti-c integrals for
Gaussian rzndom functions. S t o c h a s t i , n s 3 , 277-289.
10. R.C. Grinold (1976).
Operations Res.
Manpower planning with uncertain demands.
387-399.
24,
11. R.C. Grinold (1980). A class of constrained linear control problems
with stochastic coefficients. In: S t o c h a s t i c Programming, op.cit.,
97-1 08.
12. J-B. Hiriart-Urruty (1978). Conditions necessaires d'optimalitg pour
un programme stochastique avec recours. SIAM J . ControZ O p t i n .
1 6 . 2 , 317-329.
13. J-B. Hiriart-Urruty (1981). Extension of Lipschitz integrands and
minimization of nonconvex integral functionals: Applications to
the optimal recourse problem in discrete time. P r o b a b i l i t y c n d
5 t c t i s ; t c s I , to appear.
14. F. Lempio & A. Maurer (1980). Differential stability in infinitedimensional nonlinear programming. A?p I . Mcth. O p t i m . 6 , 139-152.
15. F. ~ouvea& & Y. Smeers (1981a). A stochastic node1 for electricity
generation. In: ModeZZ<ng o f L a r g e - S c a l e Energy S y s t e m s .
W. Haefele & L.K. Kirchmayer, eds. IIASA Proc.Series No.12.
Pergamon, Oxford, 313-320.
16. F. Louveaw & Y. Smeers (1981b). Optimization of a stochastic model
for slectricity generation. S t o c h a s t i c a . Special issue containing the Proceedings of the IIASA Task Force Meeting on
Stochzstic Optimization, 9-13 December, 1980. To appear.
17. S.I. Marcus (1981). 1.1odelling and approxination of stochastic
differential equations driven by semimartingales. S t o c h a s t i c s 4.3,
223-246.
18. E.C. Prescott & R.E. Lucas, Jr. (1972). A note on price systems in
infinite dimensional space. 1 n t Z . E c o n . B e v . 1 3 , 416-422.
19. R.T. Rockafellar & R.J-B. Wets (1976a). Stochastic convex programming:
Relatively complete recourse and induced feasibility. SIAM
J.ControZ Op5im. 1 4 , 574-589.
20. R.T. Rockafellar & R . J-3. ICets ! 1976b) . ?:onanticipativity and L ' martingales in stochastic optimization problems. ~ , ' z t i . 2 r o g r c x ~ t x ;
S t x d i e s 6 , 170-187.
21. R . T .
Rockafellar & R.J-B.
Wets (1978). The o p t i m a l r e c o u r s e
problem i n d i s c r e t e time: L 1 - m u l t i p l i e r s f o r i n e q u a l i t y cons t r a i n t s . SIAM J . C o n r r o Z O p t i m . 1 6 . 1 , 16-36.
22. R . T .
Rockafellar & R.J-B.
Wets (1981). S t o c h a s t i c o p t i m i z a t i o n
problems o f Bolza t y p e i n d i s c r e t e t i m e . S t o c h a s t i c s . IIASA
S p e c i a l I s s u e , o p . c i 5 . To a p p e a r .
23. A.N.
S h i r y a y e v (1980). M a r t i n g a l e s : Recent Developments, R e s u l t s ,
Applications.
L e c t u r e n o t e s d i s t r i b u t e d a t t h e 1980 European
Meeting of S t a t i s t i c i a n s , B r i g h t o n , England.
24. H . J .
Sussmann (1978). On t h e gap between d e t e r m i n i s t i c and
s t o c h a s t i c o r d i n a r y d i f f e r e n t i a l e q u a t i o n s . Ann.Prob. 6 . 1 ,
19-41.
,
25. M. V a l a d i e r (1974). A n a t u r a l supplement of L1 i n t h e d u a l of L
Seminaire d l A n a l y s e Convexe, M o n t p e l i e r .
26. A.F. V e i n o t t (1966). The s t a t u s of m a t h e m a t i c a l i n v e n t o r y t h e o r y .
Management S c i . 1 2 , 745-777.
27. X. Yosida & E . H e w i t t (1952).
A . M . S . 7 2 , 46-66.
F i n i t e l y a d d i t i v e measures. T r a n s .
28. J . Zowe & S. Kurcyusz (1979). R e g u l a r i t y and s t a b i l i t y f o r t h e
m a t h e m a t i c a l programming problem i n Banach s p a c e s . AppZ.Math.
O p t s m . 5, 49-62.
.