BAYES SEQUENTIAL TESTING
A DIRECT AND ANALYTIC APPROACH
by
Xizhi Wu
A dissertation submitted to the faculty of the
University of North Carolina at Chapel Hill in
partial fulfillment of the requirements for the
degree of Doctor of Philosophy in the Department
of Statistics
Chapel Hill
1987
Approved by
Advisor
Reader
Reader
XIZHI WU.
Bayes Sequential Testing, A Direct and Analytic Approach.
(Under the direction of GORDON SIMONS.)
The problem of finding the Bayes sequential test of a population
mean 8
~
8 0 versus 8
analysis.
> 80
is a classical problem in sequential
As one of the optimal stopping problems, it has been
attacked by many researchers.
One traditional approach is to approximate this discrete time
problem by a related continuous time problem, and to use the power of
analytic methods which this makes possible.
This will usually lead to
a free boundary problem involving a heat equation with boundary
conditions.
Typically, the free boundary problem can not be solved
analytically, and most results can only be asymptotic in the sense
that the observation cost tends to zero or infinity.
Following thi".
the results obtained for the continuous time version of the problem
have to be converted back to the original discrete problem.
When the original problem is truncated, one might be able to
attack it numerically by a direct backward induction.
But pure
numerical calculating does not give analytic characteristics of the
problem, which are very important for the purpose of our research.
To avoid the difficulties caused by the above approaches, our
work attacks this discrete time sequential testing problem directly
and analytically.
We do not attempt to solve any corresponding
continuous time problems, and we do not apply any asymptotic
procedures.
before.
Few particular cases have been approached from this angle
Under a very general assumption. which proved suitable to a wide
range of cases. we proved the existence of the optimal stopping rule.
For some general and specific cases. we gave various representations
for the solutions of the difference equation involved with the dynamic
equation.
Based on selected solutions of the difference equation. we
presented a series of inner and outer approximations of the optimal
continuation region.
We particularly discribed the shape of that
region in the Bernoulli case.
In addition. we introduced a useful transformation to the
backward heat equation for Bayes models.
With the aid of the
transformation. a precise solution for a free boundary problem linked
with SPRT is obtained.
ACKNOWLEDGEMENTS
I want to express my gratitude to my advisor. Gordon Simons, for
introducing me to the topics considered in this work and for the
amount of time and effort he put forth.
I appreciate his unlimited
patience and support, and thank him for his guidance and encouragement
throughout the preparation of this thesis.
I acknowledge the members of my committee, Charles R. Baker,
Stamatis Cambanis, Edward Carlstein and Douglas G. Kelly, for their
interest and their careful reading of the manuscript.
I also thank the members of the staff who have been most helpful
throughout the past five years, especially June Maxwell for somehow
always managing to keep things in perspective.
I also want to thank my fellow graduate students for their
friendship and support.
Finally I want to thank my family and friends for their encouragement throughout my education.
It is truly appreciated.
My graduate study has been supported in part by the Nankai
University, and by the National Science Foundation under grant
DMS-84-00602.
TABLE OF OONTENTS
CHAPTER I
INTRODUCTION
1.1
Introduction
1
1.2
Model. basic assumption and testing problem
2
1.2.1
2
Model
1.2.2 Basic assumption
3
1.2.3 Testing problem and posterior risk function
4
1.3
Optimal stopping problem
6
1.4
Background
8
1.5
The motivation for this study
CHAPTER I I
11
MATHEMATICAL MACHINERY AND BASIC TECHNIQUES
2.1
Introduction
13
2.2
Optimal stopping problem. dynamic equation
14
2.3
The existence of the optimal stopping rule
for our model
16
2.4
To the r-k plane
18
2.5
Cases satisfying the basic assumption
20
2.5.1
One-parameter exponential family case
20
2.5.2
A nonparametric setting based on
Dirichlet processes
23
Some other cases
25
2.5.3
2.6
Solutions of difference equation (2.14)
26
2.6.1
General case
26
2.6.2
One-parameter exponential family case
28
2.7
Basic techniques for inner and outer
approximations
29
2.7.1
Inner approximations
30
2.7.2
Outer approximations
32
CHAPTER III OlITER APPROXIMATION FOR BOUNDED OBSERVATIONS
3.1
Introduction
35
3.2
Outer approximation for bounded observations
35
CHAPTER IV
TIIE BAYES TESTS FOR p
~
2'1
VERSUS P
> 2'1
4.1
Introduction
41
4.2
The shape of the continuation region
44
4.3
Inner approximations
52
4.4
Outer approximations
63
4.5
Conunents
65
CHAPTER V
OlITER AND INNER APPROXIMATIONS FOR
NORMAL RANDOM VARIABLES
5.1
Introduction
66
5.2
Solutions of relevent difference equations
67
5.3
Outer approximations
70
5.4
Inner approximations
79
5.5
Conunents
82
CHAPTER VI
INNER APPROXIMATIONS ON AN EXPONENTIAL
DISTRIBUTION CASE
6.1
Introduction
84
6.2
A model and a testing problem
84
6.3
Inner approximations
86
CHAPTER VII TRANSFORMING INTO THE BACKWARD HEAT EQUATION
FOR BAYES MODELS
7.1
Introduction
89
7.2
The Wiener process with drift 8
90
7.3
Bayes models and partial differential equations
91
7.4
Transformations
93
7.5
A free boundary problem linked with the SPRT
100
7.6
Normal prior distribution case
106
7.7
A improper prior case
107
7.8
Transformations from one Wiener process into
another
108
REFERENCES
109
aIAPTER I
INTRODUCfION
1.1
Introduction.
The objective of this study is to conduct a Bayes sequential test
of a population mean: 8
~
8 0 versus 8
> 80 •
where 8 is the mean of
potential observations X .X2 ••••. and 8 0 is a constant.
1
For fixed 8.
the observations are assumed to be independent and identically
distributed.
However. 8 will be viewed as a random variable.
As is
common with contiguous hypotheses. the loss for choosing the wrong
hypothesis is assumed to be proportional to 18 - 8 0
1.
And as is
common for sequential tests. a fixed cost per observation is included
in the loss structure.
There are two components to a Bayes sequential test: (i) a
"stopping rule" which determines how many of the potential
observations are actually observed. and (ii) a "terminal decision
rule" which decides between the the hypotheses.
The Bayes terminal
decision rule is obtained simply by choosing the hypothesis which
minimizes the posterior Bayes risk.
The Bayes stopping rule is the
stopping rule which optimally stops the sequence of (minimized)
posterior Bayes risks.
We shall impose a basic assumption on the posterior mean.
This
2
assumption. in fact. is very general. and it is suitable to a wide
range of cases.
We shall show that under the basic assumption and with a reward
function. our testing problem is related to a Markov discrete time
optimal stopping problem.
We shall prove that the optimal stopping
rule of this problem does exist.
In this study. an attempt is made to approximate optimal stopping
rules of selected discrete time problems directly and analytically.
Unlike most former studies which approximated discrete time problems
by the aid of corresponding continuous time procedures and asymptotic
methods. we do not attempt to solve any corresponding continuous time
problems. and we do not apply any asymptotic procedures.
Instead, we
employ the techniques exploited and developed by Bather (1962, 1983)
and Simons (1986). based on dYnamic programming. to obtain outer and
inner approximations of optimal continuation boundaries. and seek to
describe their shapes.
1.2
Model. basic assumption and testing problem.
1.2.1
Model
For a probability space (0.
~. P),
let {X ,
n
n~1}
be a sequence of
real random observations taking values in the infinite-dimensional
00
00
Borel space (R • B ). and let 8 be a random variable taking values in
the Borel line (R. B).
Let
~n
=a(X 1 .X2 ,
a-fields generated by X .X .···.X and
1 2
n
~1C ~2C ••• C~'
e
• •. Xn ) and
~
respectively.
=a(8)
be the
We then have
Suppose X .X •••• have a common distribution which
1 2
depends on the parameter
e.
which will be given a known prior
3
distribution. and we further suppose the sequence {Xn.n
conditionally independent given
~l}
is
~.
From the Bayesian standpoint. with the prior distribution of e
and the common conditional distribution of the X.·s given e. we can
1
obtain the joint distribution of X .X .···.X and e for any
1 2
n
n~l.
and
acquire the so-called posterior conditional distribution of e given
X1 .X .···.Xn (or given
2
~n).
The concept of the posterior distribution
is the most important feature of a Bayes model.
The prior
distribution represents our belief or information about e before
sampling.
The information about e improves as observations are taken.
and the original prior is replaced by a sequence of posterior
distributions.
We use
E(·
I~
n
)
to represent the corresponding
posterior expectation.
1.2.2 Basic assumption.
As we mentioned before. a represents the common conditional mean
of Xi's given e. i.e .. e
= E(Xil~)
for any i~l.
We now present a basic assumption on the posterior expectation of
X + given
n 1
Xl,~,···,Xn'
and this assumption will be carried
throughout our entire study. and adapted to many cases.
the conditional independence of the X. 's. we have
1
= E(X
~
where
~XJn
11~XJ).
n
refers to the a-field a(e.X .X .···.Xn ).
1 2
E(al~n )
Thus
= E(E(Xn +ll~XJ n )I~n ) = E(Xn+ ll~).
n
Because of
4
We now assume that for n
~l
k
n
(1.1)
=~.
n
some constants r
We suppose that r
O and k O satisfies E(8)
which is the prior mean.
E(X
O
and k .
O
=E(E(Xll~» = E(X l ) = ~
r '
O
We further assume E(X
2
I
2
2
2
1 )= E(X i ) for all i~l, and therefore E(X i )<
)<
00
00
Notice that
for any
i~l. This
condition guarantees the existence of the optimal stopping rule, which
we shall discuss later.
Basic assumption (1.1) is satisfied by many common and important
cases.
We shall demonstrate in Chapter 2 that the one-parameter
exponential family, the nonparametric case which takes a Dirichlet
process as its random probability, and some other cases are adequate
to. this assumption.
Due to the nature of the assumption, when the
k
sample size n is large, the posterior mean ~ is almost identical to
r
n
the sample mean! ~ IX., which is independent of the choice of the
n
given prior.
1=
1
In a sense, therefore, this assumption could be
acceptable for non-Bayesian and even nonparametric statistics.
1.2.3
Testing problem and posterior hayes risk function.
Under the above model and assumption (1.1). our testing problem
is
versus
•
5
where 80 is given.
A cost c is assigned for each observation.
And an
additional cost 18 - 8 0 1 is assessed if the wrong hypothesis is
chosen.
Then the posterior expected loss at stage n is
E«8
eo)+I~ )+ cn
n
if Ho is accepted;
E«8
80)-1~ )+ cn
if Hi is accepted.
n
The Bayes policy on the terminal decision when we stop sampling at
choose Hi otherwise.
By choosing the Bayes policy which takes the
best hypothesis in order to minimize the expected loss. one obtains a
posterior Bayes risk function (which includes both the posterior Bayes
risk due to a wrong decision and the cost of observing):
min
=
{E«8 - eo)+I~n ).E«e - 80)-1~n )} + cn
~
E( 18 - 80 II~n)-
~IE(81~n)-80 1+
cn.
(1. 2)
We have obtained the posterior Bayes risk function. which is a
random variable depending on the current data.
Our interest is to
decide at each stage whether to continue or to stop sampling in order
to minimize the expected posterior Bayes risk.
Our testing problem
therefore is an optimal stopping problem. which will be discussed in
the next section.
6
1.3
Optimal stopping problem.
As mentioned in the previous section. our attention will be the
corresponding optimal stopping problem. and our task is to find the
optimal stopping rule which minimizes the expected risk (the formal
definition of the optimal stopping rule will be given later).
First.
for convenience. instead of using the posterior risk function (1.2).
we shall consider a posterior reward function. which can be obtained
simply by changing the sign of the corresponding posterior Bayes risk
function (1.2).
In fact. for a reward function. multiplying by a
positive constant factor. or adding (or removing) a constant term does
not change the results of optimal stopping problem.
For our specific
problem (truly. for any case where the optimal stopping rule exists).
removing or adding a uniformly integrable martingale term (here
removing
E(IS - Sol I~n »
will not change the optimal stopping rule
(its existence for our problem will be proved in Chapter 2).
With
changes on constant factors and terms in (1.2). and after replacing
)
n
E(el~
k
by
n
r'
n
a reward function for stopping at the state n is
obtained as
Rn == R(kn .rn )
(1.3)
2cr .
n
which is a random variable depending on the data X .X .···.Xn (or
1 2
~n)'
Now we have a stochastic sequence {R .~ }. where the R is a reward
n n
n
function for stopping sampling at time n.
Now we need some definitions.
A random variable T taking values
in 1.2 .•••. 00 is called a stopping time if [T=nJE
~
n
for n=I.2.···.
An
7
alternative and equivalent definition for a stopping time is
(1.4)
n
n
where AnEB , a Borel set in R , and inf{~} will be understood as 00
We define
C
={stopping
times T: P(T< 00)=1 and ER- < oo}
T
and
C - {TEC: TEA}.
A
A stopping time a n EC[ n,OO ) is called the optimal stopping time or the
optimal stopping rule from n onward if it maximizes the expected
reward, i. e. ,
ER
a
Let a
=
aI'
= sup
n
ERT .
TEC[n,oo)
so that
ER
a
= sup
TEC
ER .
T
2
Under the condition E(X. )< 00 for any
1
i~I, we shall, in the next
chapter, verify the existence of the optimal stopping rule with
respect to our stochastic reward sequence {R
n
,~}.
n
We shall further
demonstrate that the sequence {(r ,k )} in the r-k plane is a Markov
n
representation of {R
n
,~
n
n
}, and the optimal stopping rule therefore
divides the plane into the "optimal stopping region" consisting of
"optimal stopping points" and the "optimal continuation region"
consisting of "optimal continuation points", the meanings of which
8
will be discussed in Chapter 2.
In our study, we are going to approximate the optimal
continuation region from both inside and outside.
to as inner and outer approximations, respectively.
These are referred
With the help of
special techniques which will be describe in the next chapter, we
shall discover some sufficient conditions for a point in the r-k plane
to be an optimal continuation point (for the inner approximation) or
an optimal stopping point (for the outer approximation) in various
selected problems.
1.4 Background.
The development of sequential testing problems was stimulated by
Abraham Wald's celebrated book SequentiaL AnaLysis (1947).
Based on a
method for constructing the most powerful test provided by NeYman and
Pearson (1933), Wald established what is called the sequential
probability ratio test. abbreviated SPRT.
To explain the SPRT, let
X .X .···.X be a random sample, with each Xi having a common density
1 2
n
f(x,a), and consider the testing problem Ho : a = a o versus Ht : a = at
n
n
f(x 1.. a t )
n
f(x.,a
o)
1
i=1
An = n
i=l
The SPRT depends upon two constants A and B such that 0
and, at stage n, it accepts Ho if
continues if B
< An < A.
An~
B, rejects Ho if
< B < A < 00;
An~
A, and
The corresponding stopping time then is
9
N=inf
{n~l:
Anf{B,A)}.
The SPRT has been studied and developed by
many researchers because of its remarkable optimum properties.
Subsequently, Arrow et at. (1949) considered the Bayes problem and
proved the existence of Bayes solutions.
Due to the difficulties involved in explicitly computing Bayes
solutions, Wald (1950) introduced asymptotic sequential analysis in
the context of estimation as the cost per observation c
~
O.
Thereafter, there were several approaches to optimal stopping
problems.
Chernoff (1959) investigated the asymptotic properties of
sequential testing, and the testing theory was developed further by
Schwarz (1962).
Kiefer and Sacks (1963) generalized these results and
those from Albert (1961) and Bessler (1960).
For a problem of testing
two composite hypotheses without an indiffernce zone, Kiefer and Sacks
(1963) employed the asymptotic stopping rule (combined with a Bayes
acceptance decision function) s * (c):
n~O,
for which R{T,X ,X ,···,X )
n
1 2
~
Stop at the smallest integer n,
c, where R is the posterior Bayes
risk, a function of observations X 'X .···.X and a prior distribution
n
1 2
T, and they demonstrated under some conditions that s * (c) is
asymptotically optimal. In Schwarz's case X ,X ,···,X are summarized
n
1 2
by T =~l X., a minimal sufficient statistic (the posterior Bayes Risk
n
1
takes the form of R{n,T
n
».
In a problem of testing two composite
hypotheses, with an indifference zone, for a one-parameter exponential
family, Schwarz presented an upper bound and a lower bound of the
optimal continuation region with the aid of continuation regions
determined by s * (.), and thereby described the asymptotic shape of the
continuation region.
From a slightly different point of view. Bickel
10
and Yahav (1967) provided some stronger results concerning asymptotic
pointwise optimal (A.P.O) procedures.
sequence of random variables {Y .
n
n~1}.
They considered a general
where P(Y >O)=1 and Y
n
n
~
0
a.s .. and defined Z (c)=Y + nco where Y represents the
n
n
n
(non-observational part of the) risk at stage n. and c is the cost per
observation.
The specific form of the term Y can be given for
n
testing and estimating problems.
A procedure s(c). which is a
.
Zs(c)(c)
stopping time. is called A.P.O. H 11m sup Z
(c)
c-()
t(c)
other stopping time t(o).
~
1 a.s. for any
Bickel and Yahav (1969) described some
examples. both for estimation and testing.
Another approach is represented by Chernoff (1961. 1972). and
Bather (1962. 1983).
In the research on the problem of sequentially
testing for the sign of the mean of a normal distribution. they
indicated that this problem and many other problems which are discrete
time optimal stopping problems can be approximated by continuous time
problems involved with a Wiener process given a normal prior.
These
continuous time optimal stopping problems prove to be eqUivalent to
free boundary problems involved with a heat equation (or a backward
heat equation) and some boundary conditions. which vary with problems.
Van Moerbeke (1974a. 1974b. 1975) gave a detailed exposition of the
relation between optimal stopping problems and free boundary problems.
Asymptotic solutions for free boundary problems were obtained by
Breakwell and Chernoff (1964) and Chernoff (1965a). and a theorem for
revisit to the original discrete time version was provided by Chernoff
(1965b).
Moriguti and Robbins (1962). Chernoff and Ray (1965). Petkau
(1978) and Simons (1986) specifically discuss problems concerning
11
Bernoulli observations.
Our consideration, in a sense, is a parallel in the discrete
setting to that of free boundary problems.
1.5 The Motivation for this study.
Some discrete time sequential analysis problems are truncated;
i.e., stopping must take place by the n-th observation for some n, and
the resulting problems might be directly attacked numerically by the
backward induction.
It is nevertheless very important to be able to
approximate the optimal rule analytically.
for this:
There are several reasons
1. There are usually one or more continuous parameters
(e.g. the cost per observation).
It is impossible to analyze the
properties for all values of continuous parameters by backward
induction.
2. A good analytic approximation can be used to describe
asymptotic behavior.
3. Sometimes, an explicit analytic approximate
solution for an optimal stopping rule can be very close to the optimal
but implicit solution (cf. Simons(1986».
As we mentioned before, many researchers approximate discrete
time problems by continuous ones with the aid of the potential power
of analytic methods.
There are, however. disadvantages in so dealing
with continuous time problems.
Partial differential equations in
continuous time cases are the counterparts to the difference dYnamic
equations in discrete time cases.
These partial differential
equations are presumably easier to treat by analytic techniques.
optimal stopping problems lead, however, to free boundary problems
involving heat equations (or backward heat equations) which are
Such
12
usually insolvable and can be reached only by numerical
approximations.
And. after these approximate solutions have been
obtained a suitable correction is needed when a continuous time
problem is converted back to the original discrete time version.
For these reasons. we are encouraged to attempt to attack
discrete time problems directly and analytically.
This was
established as feasible by Simons (1986) in a clinical trials model.
where a very precise almost optimal inner approximation was obtained.
It is impossible to reach this level of precision by using continuous
time approximations.
Aside from Simons. however. few researchers have
approached discrete time problems from this angle and. as a result.
little is known about such an approach.
aIAPfER II
JlA11IEIIATICAL JlAaIINERY AND BASIC TEaINIQUES
2.1
Introduction.
We have introduced the concept of the optimal stopping rule in
the previous chapter.
In this chapter we shall first introduce the
optimal stopping reward function at stage n and the dynamic equation
involved with a difference equation.
We shall then prove the
existence of the optimal stopping rule for our reward sequence {R
n
,~
n
}
introduced in the previous chapter.
We shall verify that cases such as the exponential family, a
nonparametric setting and some other cases satisfy the basic
assumption (1.1).
For these and more general cases, the solutions of
the difference equation involved in the dynamic equation will be
discussed.
We shall conclude this chapter with the basic techniques which we
shall employ throughout our entire inner and outer approximations.
These techniques are essential for our research.
14
2.2
OptillBl stopping problem. dynamic equation.
We first consider a finite horizon case.
The "horizon"
associated with the optimal stopping problem is simply the number of
available observations (or observation stages).
A finite horizon
problem can be handled by means of a backward induction. such as the
following (2.1).
Suppose there are only a finite number of
Let ~ denote the optimal stopping reward
n
at stage n (for n=1. 2. ···.N) which is defined by the backward
induction
(2.1)
~n = max(Rn
E(~n+ 11~n »
•
which are called dYnamic equations.
for n = N-1. N-2.···.1.
Thus ~ is ~ -measurable.
n
n
The
optimal stopping time from time n onward aN defined in Section 1.3
n
(tne only difference is here T€[n.N] instead of [n. ro
that aN
= min{n<J·<N:
n
- -
~=R.}; in other words. stop at
J
J
».
j
It turns out
N
J
if S.=R. and
J
continue if the equation
~J = E(~J+ 11~.)
J
holds.
(2.2)
It can be proved that the definition of SN is equivalent to
n
the following expression. which can also be considered as a
defini tion.
~=ess
sup E(R I~ )
n
T n
T€C[n.N]
(2.3)
15
(see Section 1.3 for the notation CAl.
The notation ess sup f will
a EA
a
be understood to be an extended random variable T} ("extended" means it
~
may be + 00) such that peT}
peT}
~ ~)=1
for any extended random variable
f a )=1 for each aEA.
P(~ ~
f a )=1 for each aEA. and such that
~
that satisfies
Here "ess sup" is short for "essential
supremum". which is unique up to an almost sure equivalence.
In infinite horizon cases. let S
n
reward at stage n.
We however cannot define S
n
We instead define S
n
n
n
in the way of (2.1).
as
E(R I~
S =ess sup
The S
represent the optimal stopping
TEC
[n. oo )
T n
).
(2.4)
defined by (2.4) is uniquely specified almost surely. and it
satisfies the dYnamic equation (cf. Chow et at. (1971»
Sn
= max(Rn
. E(Sn+ 11~»
n
for n=1.2.···.
(2.5)
Generally. the dYnamic equation (2.5) does not necessarily determine
S uniquely.
n
We now see under what conditions S is in principle
n
computable by backward induction.
Let
S' == lim ~
n
N~ 00 n
(2.6)
The limit must exist. and it can be shown that S' satisfies (2.5).
n
Therefore. if S'=S for n=1.2.···. then S is uniquely determined by
n n
n
(2.5). i.e .. it is in principle computable by backward induction.
16
Chow et at. (1971) describe conditions under which S =S (their
n
n
Theorems 4.3. 4.4 and their corollary) and conditions (Theorem 4.5 and
4.5') under which the stopping time
a
n
=
is an optimal stopping time.
inf{j~n.
S.=R.}
J
J
In other words. under these conditions,
our optimal stopping rule is: stop at the n-th stage if
S =R
n
n
(2.7)
and continue i f
Sn
= E(Sn+ 11~).
n
(2.8)
We shall prove in the next section that our optimal stopping
problem satisfies these conditions and therefore can be handled
through the dYnamic equation (2.5).
2.3
The existence of the optimal stopping rule for our model.
2
From our reward sequence {R .~ } and the assumption E(X )(
I
n n
00
given in Chapter 1. we are able to verify the existence of the optimal
stopping rule for the corresponding optimal stopping problem by
checking the conditions of theorems and corollaries in Chow et at.
(1971) .
The sequence {~~ lX.,~ } is not necessarily a martingale. but
1=
1
n
{~~ I(X.- 9).~ } is. where 9 represents the common conditional mean of
1=
1
n
X.·s given 8. i.e .. 8 = E(X. I~) for any i~1 as we assumed in Section
1
1
17
1.2.
For employing the following Hajek-Renyi inequality, let
=X B for i~1. We have EY.=O for i>1 and E(Yn+ 11~n )=0. Let
Tn =~ 1Y.' the sequence {Tn ,~n } therefore is an L martingale
2
Y1.
.1
1=
1
-
1
because
E(Tn +11~n )
E(Tn I~n ) + E(Yn+ 11~)
n
=
and the assumption E(X 2 )<
1
00,
Tn + 0
=
which implies E(y. 2 )<
=
00.
1
Tn ,
It is easy to
check that
Rn
=
Ikr n
-
Bol -
2cn ~ a
I~1= 1(X.-B)1
1
n
+ ~IBI + ~ - 2cn,
n
=al~=~Yil
+
for some constants a,
pial
~
+
and
~ - 2cn =al:nl
n
00,
which implies E(sup Rn +)<
n~1}
+
~ - 2cn. (2.9)
From (2.9), it is sufficient to
00
inequality (cf., e.g., Chow (1978»: If {Tn
martingale and {b ,
n
pial
~.
First we verify E(sup R +)<
prove E(sup I:nl)<
+
00.
=2~
1=
By Hajek-Renyi
1Y.'~
,n>1} is an L 2
1
n -
is a positive, non-decreasing real sequence,
then for any X > 0 and any n,
Thus
00
P{ supl:nl > X}
Hence
<_I\,
- 1\2
L
i=1
E(Y 2)
.
1
i
2
~2E(y21)
="
6X2
18
Therefore we have verified E(sup R
n
Rn~
- 00.
+
)
< 00.
Notice our reward function
By Theorem 4.5' of Chow et at. (1971), for the stopping
time defined by
for the n onward,
and a = aI' we conclude that pea
< 00)=1,
(2.10)
a € C and a is optimal in C.
By the corollary of Theorem 4.4 of Chow et at (1971), we have S'=S
n n
for n=1,2,···.
In other words, for our problem, there exists the
optimal stopping rule (a or a ) and it is in principle computable by
n
the dynamic equation (2.5), and the optimal stopping rule can be
described by (2.7) and (2.8).
2.4 To the r-k plane.
Now we are going to provide a Markov representation of the
sequence {R ,~}.
n n
2
2
Let (D ,B ) be the canonical measurable space for
n n
Then R is a B2 -measurable function of (r ,k ), since r
n
n
nn
n
{(r ,k )}.
nn
actually has one-to-one correspondence with n.
From the definition of
(r ,k ), it is clear that
n
n
for any B€B
2
n
We therefore say {Z ,B } provides a Markov
n n
19
representation of the sequence {R ,~ }, and we have, by Theorem 5.1 of
n n
Chow et at.,
Sn
sup E(RT I~n )
TEC[n, 00)
= ess
= ess
TEC
sup
[n,oo)
E(R I(r ,k »,
T
n n
where the notation C[n,OO) was introduced in Section 1.3.
Therefore
our optimal stopping rule can be described in terms of (r ,k ) in the
n
n
r-k plane (we only consider r>O) which is taken as the state space of
{(rn,kn )} for all possible rO>O and k .
O
We then can employ the
notation E(·I(r .k » to represent E(· I~ ) for our study.
n n
n
Generally
given a point (r,k) in the state space (r-k plane), for a new
observation X, the posterior conditional expectation takes the form of
(2.11)
E[XI(r,k)]
by our assumption (1.1).
A new observation X sends a state (r,k) into
a new state (r+1, k+X) , and a new state contributes new information to
the knowledge we have obtained so far.
For a generic state (r,k), the
reward function, which is the function of (r,k), takes the form
R(r,k)
=I~
r
-
80
1-
2cr.
(2.12)
We therefore may employ the notation S(r,k) to denote the optimal
stopping reward function S defined by (2.4), which represents the
n
best expected payoff which can be obtained starting from state (r.k).
Then the dYnamic equation (2.5), defined in Section 2.2, can be
20
expressed in the form
S(r.k) = max {R(r.k). E[S(r+l.k+X)I(r.k)]}.
(2.13)
According to our optimal stopping rule. a state (r.k) will be
called an optimal stopping point if S(r.k)=R(r.k). and an optimal
continuation point if
S(r.k) = E[S(r+l.k+X)I(r.k)].
(2.14)
Some states may have both appellations when the equation
R(r.k) = E[S(r+l.k+X)I(r.k)]
holds.
The set of optimal continuation (stopping) points is called
the optimal continuation (stopping) region.
The purpose of our
research is to approximate these regions.
Equation (2.14) plays a vital role in our study. and it is a
discrete time analog of the heat equation (or backward heat equation)
which has been encountered by many researchers when working with
continuous time problems.
2.5
cases satisfying the basic assumption.
.
2.5.1
One-parameter exponential family case.
Suppose X .X •••• are iid random variables from the exponential
1 2
family (See Raiffa
& Schlaifer (1961) and Diaconis & Ylvisaker (1979)
21
for details on this and conjugate priors.) with density
exp(wx-~(w»dA(x).
dFw(x) =
< X < 00
where
-ro
which
J~e w x
check that
•
Wen . an open interval on the real line over
dA(x)
~'(w)=
(2.15)
< 00;
and
EWX and
~(W)=lOg[J~e W x
~"(w)=var
X.
W
dA(x)].
It is easy to
Let 8 denote E X.
W
The natural conjugate prior density for w then has the form
(2.16)
where
°O(w) = C(
1
k) exp(kOw-rO~(w».
rOo 0
(2.17)
and the function C(r.k) is determined by
o < C(r.k)
-
JneXP(k w-r
~(w»dw <
00
(2.18)
We then can obtain the posterior conditional density of w given the
(2.19)
Notice that
depends on (r .k ) only.
n
n
d~
n
(w)
From Diaconis and Ylvisaker (1979). we have
22
E[X +l l (r .k )]
n
n n
=
k +X +X +···+X
k
0 1 2
n - rn.
r + n
O
n
More generally. given a generic state (r.k) the expectation of a new
observation X which has the corresponding posterior distribution with
(r.k) as parameter takes form of
k
E[XI(r,k)]
(2.20)
r
That means the general assumption (1.1) is satisfied for the
one-parameter exponential family.
It therefore is a special case of
our general setting.
One advantage of choosing conjugate prior distributions is that
both the prior and posterior distributions belong to the same
exponential family with convenient additive parameters k
n
and r .
n
Now we examine the difference equation corresponding to (2.14).
The right hand side of equation (2.14) becomes
E[S(r+1.k+X)I(r.k)]
II
• n
RS (r+1,k+x)e
= C(;.k)
x w- ~(w)
J S(r+1.k+X)
R
= C(;.k)
= C(;.k)·
dX(x)·e
k w-r ~(w)
Jne(k+X)W-(r+1)~(W)d w
d w
dX(x)
J C(r+1.k+X) S(r+1.k+x) dX(x).
R
where C(r.k) is defined in (2.18).
Accordingly (for an exponential
family setting). (2.14) can be written as
S(r.k) = C(;.k)
23
J
(2.21)
C(r+l.k+X) S(r+l.k+x) dA(x).
R
In the following section. we shall discover some solutions of
difference equation (2.21).
2.5.2
A nonparametric setting based on Dirichlet processes.
This nonparametric setting was introduced by Ferguson (1973).
Suppose that
ZI.Z2.···.~ are
distribution Gamma(a j .l) with
j=I.2.···.k.
independent. and Zj has the ganuna
aj~
0 for all j and a j > 0 for some j.
The distribution Gamma(a.b). where parameters
a~O
and
b>O. is defined by the following density function. when a>O.
1
-a -zlb a-I
fez Ia.~) = rea) ~
e
z
and is degenerate at zero for a=O.
The Dirichlet distribution
for z>O.
We now introduce two definitions.
D(al.···.~)
with parameter
(al.···.~)
is
defined as the distribution of (Y .Y .···.Y ). where
1 2
k
_
Yj -
For a measurable space
Zj
k
2.1= lZ.1
(~.~).
for j=I.2.···.k.
we call P a Dirichlet process with
D
parameter a. a finite non-null measure on
k=I.2.···. and measurable partition
(~.~).
(Bl.···.~)
if for each
of
~.
the distribution
of (PD(Bl).···PD(~» is Dirichlet D(a(Bl).···.a(~». Generally
(~.~)
is taken as the Borel line (R.B); for the case of bounded
observations described in the next chapter.
(~.~)
is a bounded
24
measurable space
(~'Bb)
on the real line, where
of R, such as a interval, and
Bb
~
is a bounded set
is a suitable a-field on
~
Similar to the model given in Section 1.2, in the probability
space (0,
~,
n~1)}
P), {(X '
n
and
{~n}
have the same meaning.
the parameter a used here is slightly different.
However.
Let the parameter a
be a random measure and take values on a space (0 , B ) instead of the
a
a
Borel line, and let
independent.
~
= a(a).
Let X ,X ,··· have a common random probability P which
D
1 2
is a Dirichlet process on
parameter a.
We still let X ,X ,··· be conditionally
1 2
(~,~)
depending on the random measure
From results of Ferguson (1973), the posterior
distribution of P given X ,X ,···,X is also a Dirichlet process with
1 2
n
D
the parameter
a
where
n
= a+o X +OX +·.·+OX '
1
2
n
°X. 's are measures and satisfy
1
if X.€ A
1
for any A C
~
otherwise
We can, consequently, obtain the posterior expectation of a
generic observation X 1 given X ,X ,···,X (or
~
n
1 2
~
n
):
a(~)~+X1+X2+···+Xn
a(~)+n
(2.22)
where ~ = E(X 1 ) =E(E(X11~».
Let r O=
a(~),
(2.22) becomes
k O=
a(~)~,
rn=rO+n and kn=kO+X1+X2+···+Xn' then
25
k
n
=r-'
n
i.e .. the assumption (1.1).
This nonparametric setting therefore is a
specific case that satisfies our general assumption.
Clayton (1985)
proved that. in this nonparametric case. the stopping problem is
truncated when a has a bounded support.
This result can also be
obtained directly. as a specific case. from our more general results
of Chapter 3.
2.5.3
Some other cases.
Ericson (1969. 1970) gives some general conditions under which
(1.1) is satisfied.
Let X .X .···.X be conditionally independent
n
1 2
with a common distribution depending on parameter S. which is given a
prior.
Without loss of generality. let S
-_1~
X
= -n
2..
1=
IX 1..
=E(XI~)
(~
= a(S»
and
Assuming the existence of all the following
expectations and the existence of the prior variance of S. and
var(S»
O. we obtain:
If E(Xn+ 11~)=
a X + ~. where a and ~ are independent of
n
Case 1.
the Xi's then
k
E(X 11~)= _n_.
n+
n
r
(2.23)
n
E(S)E(v~
holds. where k =
- S)
. k =kO+X 1+X2 +·· ·+X ; and
O
va'l
n
n
1 ~
E(var(X
I
r O=
var(S)
. r n = rO+n. where E(S) = E(E(X 1 ~» = E()
Xt and
E(var(XI~»
given
is the prior expectation of the conditional variance of X1
~.
Case 2.
Suppose G is a class of distributions of w having
f
26
density g(wlx' .n· .y') for (x' ,n' .y') € Y. say.
If one more
observation X=x changes (x' .n· .y') to (x+x' ,l+n' .y') and i f
g(wlx·.n· .y') and the new posterior distribution g(wlx+x·.l+n· .y')
x'+a
both belong to G and E(9)= n'+b for any constants a and b.
f
(2.23) holds.
Here (2.23) is (1.1).
Then
It is clear that Case 1 above subsumes
Case 2. and Case 2 subsumes our exponential family case.
Assumption
(1.1) therefore adapts to very general cases.
2.6
Solutions of difference equation (2.14).
2.6.1
General case.
Working with equation (2.14). an important and difficult task is
to discover appropriate solutions.
There are many solutions or
solution families for this difference equation.
In the rest of our
work. we use notation Z(r.k) to represent a generic solution of
equation (2.14).
The simplest and most convenient solution family
obviously is Z(r.k)
=a.
solution is Z(r.k) =
a constant for any (r.k).
Another useful
~.
for then. the right hand side of (2.14) will
r
be equal to
E(Z(r+1.k+X) I(r.k»
k+XI (r.k»
= E(r+1
k I(r.k»
= E(r+1
1 E (X I(r.k» = r+1
k +
+ r+1
rk+k
k
=( r+ 1) r =-=Z(r.k).
r
the left hand side of (2.14).
Let {M(r .k ).G }. n=1.2.···. be a martingale which satisfies
n n
n
e·
27
E(M(r mm
.k )IGn ) = E[M(r mm
.k )I(rn.k
n )] = M(r n.k
n)
a.s.
then M(r.k) is a solution of (2.14).
for
m~n.
In fact
E[M(r+1.k+X)I(r.k)] = M(r.k).
Le .• (2.14) holds for S(r.k)=M(r.k).
k
The solution Z(r.k)
r
actually is a specific case of the above martingale solutions because
k
the solution
~
(= E[XI(r.k)]) is from the trivial martingale ~
r
r
n
(= {E[XI(r n .kn )].Gn }).
Suppose Z(r.k) is a solution of (2.14).
combination of it. Zc (r.k)
=
aZ(r.k)+~.
Then any linear
for constants
a and~.
is
still a solution since
E(Zc (r+1.k+X)I(r.k»
= E(aZ(r+l.k+X)+~I(r.k»
= aE(Z(r+1.k+X)I(r.k»+~ = aZ(r.k)+~ = Zc (r.k).
i.e .. Z (r.k) satisfies (2.14).
c
For a number of cases. e.g. the exponential family case. some
derivatives or integrals of solutions are also solutions.
Choosing
suitable solutions largely depends on the character of each specific
difference equation and the risk function or. in other words. on the
character of the discrete time version of the free boundary problem.
28
2.6.2
One-parameter exponential family case.
Now we examine solutions of the difference equation (2.21). which
is a special case of equation (2.14) under the one-parameter
exponential family setting.
Of course. all those solutions of (2.14)
mentioned in 2.6.1 are likewise solutions of (2.21).
Equation (2.21) has some other solutions.
One family of
solutions takes the form
Z(r.k)
1
= C(r.k)
e
k w-r l#I(w)
for any w € O.
(2.24)
and its general form is
Z(r.k)
1
= C(r.k)
(k+J.L)w-(r+v)I#J(w)
e
for any w
€
O. J.L.v)O
(2.25)
For verifying that (2.25) is a solution. replace S in (2.21) by Z.
then the right hand side of (2.21) becomes
C(;.k)
1
= C(r.k)
I(
JRC(r+1.k+X) Z(r+1.k+x) d'A(x)
RC r+1.k+x
)
1
e(k+J.L+x) w-(r+v+1) l#I(w) d~(x)
C(r+1.k+x)
"
I
_
1
(k+J.L) w-(r+v) l#I(w) _
x w- l#I(w)d~( )
" x
- C(r.k) e R e
(k+J.L) w-(r+v) l#I(w) -1 - Z( k)
_
1
- C(r.k) e
r. .
Since the parameters w. /-L. and v in (2.25) are arbitrary in their
corresponding domains. derivatives and integrals of solution (2.25)
with respect to w. J.L and v are solutions also.
With this fact. one is
29
able to create more solutions as we will see later.
2.7 Basic techniques for inner and outer approximations.
We have transferred our original testing problem to an optimal
stopping problem, and for our model, the optimal stopping rule exists.
At the end of Section 2.3, we demonstrate that the optimal rule is in
principle computable by (2.5).
We therefore can use (2.5) to
approximate the optimal stopping rule.
In this section, all results are subject to our model and the
assumptions we made before.
And all conclusions, such as those we
obtained in the previous sections of this chapter, are available here.
As before, R(r,k) denotes the posterior Bayes reward function for
stopping at point (r,k), and S(r,k) represents the optimal stopping
reward at stage (r,k), which was defined by (2.4) and computable by
the dYnamic equation (2.5), or equivalently (2.13), which we rewrite
here as (2.26):
S(r,k) = max {R(r,k), E[S(r+l,k+X)I(r,k)]}.
(2.26)
We have known that the optimal stopping rule at stage (r,k) is: stop
if S(r,k)=R(r,k) , continue to take one more observation if the
difference equation
S(r,k) = E[S(r+l,k+X)I(r,k)]
(2.27)
holds, and the point (r,k) is called an optimal stopping point or
30
optimal continuation point. respectively.
If (r.k) is both an optimal
stopping and continuation point. then stopping or continuing makes no
difference. i.e .. one may stop or continue.
Suppose Z(r.k) is a generic solution of (2.27).
We are
interested in the behavior of the relation among S. R and Z near the
border between the optimal continuation region and the optimal
stopping region in order to approximate the boundary.
kinds of approximations.
We have two
The inner (outer) approximations are to
approximate the border by finding a sufficient condition for a point
to be an optimal continuation (stopping) point.
The techniques and terminology described here for approximations
are based on Simons (1986).
In the following text. "inunediate (or
all) successors" of (r.k) represent all possible values of (r+l.k+X)
(or (r+n.k+X +X +---+X ) for all
1 2
n
n~1).
where X is confined to its
support under the posterior distribution given (r.k).
2.7.1
Inner approximations.
In this part. a point (r.k) will be called "good" if
Z(r.knR(r.k). and called "warm" if Z(r.kHS(r.k).
We have the
following results:
Lemma 2.1
(i)
(Inner approximations)
If a point (r.k) is good and all its immediate successors
are warm. then it is an optimal continuation point.
(ii)
If all the immediate successors of a point (r.k) are warm.
then (r.k) is warm.
31
Proof.
(i) The given conditions suggest that Z(r,k)lR(r,k) and
S(r+1,k+X)lZ(r+1,k+X), which imply
E[S(r+1,k+X)I(r,k)] l E[Z(r+1,k+X)I(r,k)]
= Z(r,k) l R(r,k).
Therefore, by equation (2.24),
S(r,k) = E[S(r+1,k+X)I(r,k)],
which indicates that (r,k) is an optimal continuation point.
(ii) By the condition,
Z(r+1,k+X)~S(r+1,k+X), we
obtain
Z(r,k) = E[Z(r+1,k+X)I(r,k)]
~ E[S(r+1,k+X)I(r,k)] ~ S(r,k).
o
It is clear that the curve defined by equation Z=R divides the
r-k plane into two regions.
because ZlR.
z~R(~S).
One of them consists of good points
The other region consists of warm points because
But among the good points obtained in this way, there might
also be warm points.
We will demonstrate that it is important, for
our purpose, to identify as many of these warm points as possible by
using (ii).
With abundant good and warm points identified, we are
able to use (i) to determine which points are optimal continuation
points.
32
2.1.2 Outer approximations.
In this part. a point (r.k) will be called "good" i f
Z(r.k)~R(r.k).
and called "warm" if Z(r.knS(r.k).
Although employing
the same terms. the definitions of "good" and "warm" points are here.
of course. different from those in the case of inner approximations;
the direction of the inequalities used in the definitions are
reversed.
We note the following results:
Lemma 2.2
(i)
(Outer approximations).
If (r.k) is good and all its immediate successors are warm.
then it is an optimal stopping point.
(ii)
Z(r.k)~R(r.k)
If
and all the immediate successors of (r.k)
are warm. then (r,k) is warm.
(iii)
If
Z~R
for all successors of (r.k), then (r,k) is a warm
point.
Proof.
(i) The given conditions suggest that
S(r+l,k+X)~Z(r+l,k+X), which
Z(r,k)~R(r,k)
imply
E[S(r+l,k+X)I(r,k)] ~ E[Z(r+l,k+X)I(r,k)]
= Z(r,k)
~
R(r,k).
Therefore, by (2.24),
S(r ,k)=R(r, k),
which indicates that (r,k) is an optimal stopping point.
(ii) By the condition,
Z(r+l.k+X)~S(r+l,k+X). we
obtain
and
33
Z(r.k) = E[Z(r+1.k+X)I(r.k)] ~ E[S(r+1.k+X)I(r.k)]
and
Z(r.k)
which imply
Z(r.k)~S(r.k)
~
R(r.k).
from (3.1).
(iii) Under the given conditions and S's definition (2.4). we
conclude
Z(r.k)~S(r.k)
by backward induction.
0
Unlike Lemma 2.1. while applying Lemma 2.2. we cannot obtain warm
points simply by dividing the r-k plane by the equation Z=R.
But we
can instead use (iii) to discover warm points and then use (i) to
identify optimal stopping points.
Remarks
As mentioned before. we do not attempt to obtain the exact form
of S(r.k): instead we employ the above lemmas to discover the position
of each point in the optimal stopping rule. i.e .. whether a point is
an optimal continuation (stopping) point.
While these lemmas provide
very useful and general techniques to identify points. they do not
automatically give answers.
It is possible that with inadequate
solution families of (2.25). these lemmas cannot identify a single
point.
It is also possible that a solution family which is suitable
for inner approximations is totally useless for outer approximations.
and vice versa.
In practice. we must first discover solution families of (2.25).
Then a particular family conducted from them must be selected in order
to suit specific problems. which vary with different distributions and
different (outer or inner) approximations.
Finally. we have to
34
determine the most adequate solution for each individual point within
the chosen family.
A general method for finding adequate solutions has not so far
been discovered.
Hence. one should make a concrete analysis of
concrete conditions.
In the next few chapters. specific problems will
be discussed with the aid of the techniques developed in this section.
aJAPI'ER I I I
OUTER APPROXIJlATIOO FOR BOUNDED OBSERVATIOOS
3.1
Introduction.
The results obtained in this chapter are the generalizations of
those in Simons and Wu (1987) with similar techniques of proof.
On the base of the general setting provided in Chapter 1. here we
are confined by a further condition: the observations X.·s are bounded
1
random variables.
2
This implies E(X )(
1
00.
All results here therefore
are suitable for those bounded observation cases which belong to the
cases described in Chapter 2 such as one-parameter exponential
families. nonparametric cases and some others.
3.2
Outer approximations for bounded observations.
In this section. we assume
a~X~b
and
a~8o~b.
k
and let s - r'
(r' .k·) represent successors of state (r.k). where r'=r+m for
m=1.2.··· and k·=k+X.
As before. R(r.k) denotes the posterior Bayes
reward function for stopping at point (r.k). i.e.
where c is the cost per observation.
R(r.k)=I~
- 8o l-2cr.
r
We have the follOWing results.
36
Theorem 3.1
A point (r.k) is an optimaL stopping point iF For
(B~-a)(b-s)~(r+1)(b-a)+(b-s).min{i-1+ r(s- Bo ). i=1.2 •••• }.
ci
(3.1)
(b-cBo)(s-a)~(r+1)(b-a)+(s-a).min{i-1- r(s-
ci
Bo )
i=1.2 .••• }.
(3.2)
The minimum occurs when
i=[~lr(s-Bo)/cl~] where brackets [ ] denote
the integer part of the number inside.
Proof.
For convenience. let T : ~ - Bo . T': k: - Bo . B : b - Bo
r
r
and A : a - Bo .
It is easy to check that s-B o is a solution of the
difference equation (2.21).
and~.
So is Z :
a+~(s-Bo)
for any constants a
If Z(r.k)~R(r.k). Le .. (r.k) is "good". and Zero .k·HR(r· .k·)
for all successors of (r.k). which implies (r+1,k+X) is "warm". then
the point (r.k) is an optimal stopping point. by Lemma 2.2.
In other
words. for point (r.k) to be an optimal stopping point. we need
a + ~ ~ ITI-2cr
(3.3)
+ ~. ~ IT·I-2cr·.
(3.4)
and
a
1). For T>O: (3.3) becomes
a
+
~ ~
T-2cr.
37
Let 'Y --~
2c ' we have
'Y(T' -T)
~
(T')
-
c
-
(3.5)
- m.
It is enough to find a proper 'Y (depending on (r,k»
such that (3.5)
holds for all pairs (r' ,k').
For
When T'=T, (3.5) holds for then (3.5) leads to O)-m.
T'~O:
When T')T, (3.5) requires
-m
'Y ~ T'-T"
(3.6)
,
For fixed r , it is most stringent when T' is as large as possible:
T'k+mB
r'
~
to 'Y
e0
-r'
B-T.
r+l
B-T.
and, therefore, T'-T = m(B~T).
r
So (3.5) holds for all (r',k') with T') T, providing
When T'( T, by (3.5) we have upper bounds for 'Y:
'Y
~
-
'Y
~
-m
T'-T' so the best choice for 'Y is
'Y -
For T=O and
As a result, (3.6) leads
T'~O,
r+l
B-T
(3.7)
the 'Y in (3.7) makes (3.5) hold automatically.
(3.5) requires
For T'(O:
r+l (T'-T) ) - -T'
- - m
B-T
c
or, identically,
[B-T
c
If
B-T
c
-
(r+l)~O,
holds automatically.
_ (r+l)JT'+ m(B-T) + T(r+l) ~ O.
(3.8)
then the most stringent case is T'=O, then (3.8)
If
B-T
c
- (r+l)~O, then it is most stringent
38
when T' is chosen as small as possible .. i.e .. T'
k+mA
It
r
follows from (3.8) that
(B-T)m2-(-(r+l)(B-A)+(B-T)-
A(:-T) )m + ~ (B-T)rT ~ O. (3.9)
We need (3.9) to hold for all m=I.2.···. and that requires. by
algebraic arguments.
A(B-T)
c
~ (r+l)(B-A)+(B-T) min(i-l+~;
i=I.2 .... ). (3.10)
Cl
2). For T<O. we also achieve
B(T-A)
c
~ (r+l)(B-A)+(T-A) min(i-l-~;
i=I.2 .... )
Cl
in a similar way except replacing A. B. and T by -A. -B. and -T
respectively.
After changing T. A and B to s-So. a-So and b-S o
respectively. we obtain the inequalities (3.1) and (3.2).
Corollary 3.1
State (r.k) is an optimaL stopping point if for
(So-a)
(b-s)
c
and for s
~
0
~
(r+1)(b-a)
(3.11)
(r+ 1)(b-a).
(3.12)
So.
(b-S o )
(s-a)
c
Proof.
~
This is a direct consequence of Theorem 3.1.
0
39
Corollary 3.2
s
~
80
State (r.k) is an optimal stopping point if. for
•
(8 0 -a) (b-s) ~ (r+l)(b-a)+(b-s)·min {2.
c
and, for s
~
(3.13)
r(So-s)}.
c
(3.14)
So.
(b-So)(s_a)
c
Proof.
r(s-So)}
c
.
~
(r+l)(b-a)+(s-a)·min {2,
In the proof of Theorem 3.1, m=1 is the most stringent
case for the inequality (3.9) if and only if
(So-a)(b_s) ~ (r+l)(b-a)+2(b-s).
c
(3.15)
When m=l, (3.9) itself becomes
(So-a)(b_s) ~ (r+l)(b-a)+(b-s)r(s- So)/c.
c
(3.16)
Combining (3.15) and (3.16), (3.13) is achieved. and similarly we can
obtain (3.14).
0
Corollary 3.3
If
r ~ (b-So)(So-a) - 1,
c(b-a)
(3.17)
then for any k. (r.k) is an optimal stopping point; in other words.
the r
O
which satisfies
gives an upPer bound of the truncation point.
40
Proof.
This is a direct consequence from Corollary 3.1.
0
Let k=b(r) represent the boundary function of the continuation
region. which is a function of r in the r-k plane. in the sense that
(r.k) is an optimal stopping point. for fixed r. if and only if
k~b(r).
We then have the following corollary. which describes the
asymptotic characteristic of the function b as r
Corollary 3.4
~
O.
For s = ~ fixed. as r ~ O. the boundary function
r
of the continuation region b(r) satisfies. for s
~
Bo .
(3.18)
and. for s
~
Bo •
(3.19)
Proof.
By Corollary 3.2 and Taylor eXPansions (very tedious).
Corollary 3.5
0
In a symmetric setting. where b- Bo= Bo-a. for
any fixed k. when r is smaLL enough the point (r.k) must be an optimaL
s topping point.
Proof.
By Corollary 3.2 directly.
0
Corollary 3.5 describes the shape of the boundary function b(r)
near the point (O.B o ).
It suggests that the optimal continuation
region just "touches" the line r=O at most at one point (0. Bo ).
aIAPTER IV
1
mE BAYES TESTS FOR p 5; 2 VERSUS P
4.1
> 21
Introduction.
This problem has been discussed by many researchers in many
different special cases. especially by Moriguti and Robbins (1962).
Let X'X 1 'X2 .••• be independent Bernoulli random variables with common
mean p. 0
~
p
~
1 i.e.
with probability p
with probability I-p
The hypothesis testing problem is
Ho : p
~ ~
versus
It is easy to see that this is a specific case of the one-parameter
exponential families described in 2.5.
1
is the testing problem with Bo =2.
By the notation in 1.1. this
A suitable conjugate prior density
for p is a beta distribution. Beta(ko.ro-k o }. which is an extension of
42
the natural prior distribution.
This extension is used, instead of
the natural prior distribution Beta(ko+l,ro-ko+l), in order to keep
the domain of the parameters non-negative.
The posterior conditional
distribution of p given (r,k) is also beta distribution Beta(r,r-k).
For comparison, notation similar to that used by Moriguti and Robbins
(1962) will also be taken here:
Let x
=k, n =r, y =n-x and A =;c'
the state plane therefore is x-y plane.
The reward function then
takes the form
V Y - x-y
R(x,y) = A x x+y
or, identically,
R(x,y) = A x V Y - n,
n
where x V y
=max(x,y)
(similarly x A y
(4.1 )
=min(x,y)).
The dynamic
function (2.13) becomes
S(x,y) = max (R(x,y), ~ S(x+l,y)+ ~ S(x,y+l)).
n
n
(4.2)
A state (x,y) is an optimal stopping point if S(x,y)=R(x,y), and a
continuation point if
S(x,y) = ~(x+l.y) + ~(x,y+l).
n
n
(4.3)
which is a specific case of (2.14).
As before, Z(x,y) represents a generic solution of the difference
equation (4.3).
The terminologies such as "good" and "warm" points
for inner and outer approximations mentioned in Chapter 3 are still
43
used here.
Lemmas 2.1 and 2.2 will also be employed.
The only change
in notation is that we will here use (x,y), rather than (r,k), to
represent states.
Therefore, the immediate successors of (x,y) are
(x+1,y) and (x,y+1).
For convenience, let d=x-y, we can therefore consider the
equivalent d-n plane.
The immediate successors of (d,n) are (d+1,n+1)
and (d-1,n+1), corresponding to (x+1,y) and (x,y+1) respectively.
Obviously, x =
~d
2
and y =
n-d
2
Moriguti and Robbins (1962) present the following results in
their theorem: A point (x,y) is an optimal stopping point if
2(n+1)~A,
and a point (x,x), which is on the line x=y, is an optimal
continuation point if and only if
2(n+1)~A,
they obtain a few asymptotic results.
where n=2x.
In addition,
We shall prOVide alternative
proofs for their theorem with our techniques in order to demonstrate
the nature of our methods.
We shall furthermore introduce the
boundary function of the optimal continuation region and prove its
legitimacy, and describe the shape of the continuation region,
especially near two ends of the continuation region on the line x=y.
In 4.3, we shall present an attempt at inner approximations and
compare our results with the optimal rule.
In the final section,
results for outer approximations in Chapter 3 are employed and similar
comparisons are made.
This chapter is based on Simons and Wu (1987).
44
4.2 The shape of the continuation region.
Let Q
=S-R.
It can be checked by (4.3) and by the definition of
R that Q satisfies the recursive equation
Q(x,y) = {A xAy
- (1- Ix-y I) +- 1+ x- Q(x+1,y)+ v~ Q(x,y+1)} + .
n(n+1)
n
n
(4.4)
Obviously, Q is non-negative and symmetric to the line x=y.
If
Q(x,y»O, then (x,y) is an optimal continuation point by the
definition of Q.
1.eDIIB 4.1
If
2(n+1)
then the point (x,x) (or (~,~»
~
A,
(4.5)
on the diagonaL x=y is an optimaL
continuation point.
Proof. Then (4.4) becomes
A
+
Q(x,y) = {2(n+1)
+ Q(x,y+1)} ,
o
which is greater than zero.
An alternative proof of Leoma 4.1 by inner approximations.
Choosing a solution of (4.3) Z(x,y)
simplest solution.
Let F
=Z-R.
=a,
a constant, which is the
We determine the constant a by
F(x,x)=O, i.e., Z(x,x)=R(x.x), which means (x,x) is a good point.
That requires
45
F(x.x)
or a
= 2A -
n.
In order that the point (x.x) be an optimal
continuation point. we need. by Lemma 2.1. that the immediate
successors of point (x.x) are warm. and it requires the following
inequality:
F(x+1.x)
= F(x.x+1) = 2A -
n -A
x+1
n+1
n+1- ~
2
= --n-+-""1;;;'"
~ O.
It is clear that the above inequality holds if
~
2(n+1)
That ends the alternative proof.
Lemma 4.2
2(n+1)
~
A.
0
A point (x.y) is an optimaL stopping point if
A.
Proof.
We use outer approximations described by Lemma 2.2.
Due
to the symmetry of the problem with respect to x and y. we only
consider
x~y
without loss of generality.
A constant solution Z(x.y)=a
is also selected. and the constant a will be determined so that
F(x+1.y)=O. this leads to
a
=
A(x+1)
n+1
-n-1.
46
As a result of our choice of a,
= a+n+1
F(x,y+1)
holds.
-
A x
A
= -n-+;";;""'1- > 0
n+1
This inequality and the inequality y ~ ~ imply the following
inequali ty
F()
x,y
Therefore, if
= a+n
2(n+1)~A,
Ax
n+1
-
then
= (n+l)n
Ay
F(x,y)~O,
_ 1
(4.6)
which implies that the point
(x,y) is a good point (cf. Section 2.7).
F=O, together with the other half for
< A
1
- 2(n+1) - .
x~y,
inside which points are good points due to
The curve represented by
forms a closed region
F~O.
It can be verified,
with the help of techniques of geometry and calculus, that all
successors of (x+1,y) and (x,y+1), as well as themselves, satisfy
prOVided
2(n+1)~A.
are warm points if
F~O
By Lemma 2.2, all successors of (x,y), therefore,
2(n+1)~A.
optimal stopping point.
Consequently the point (x,y) is an
0
These two results were proved by Moriguti
very different and complicated way.
& Robbins (1962) in a
From the above applications of
our inner and outer approximations, one might see that even the
simplest solution of the difference equation, (4.3) could help us to
describe some aspects of the problem, and that our techniques for
inner and outer approximations are not difficult to apply under
certain circumstances.
A special case of Corollary 3.5 for this problem is that:
When x
(or y) is fixed if y (or x) is small enough, then (x,y) must be an
•
47
optimal stopping point, in other words, anyone step successor of any
optimal continuation point which is close enough to the origin is an
optimal stopping point.
Also, for any optimal continuation point
which is close to the line n=
-2
~ n ~ ~
~
~
-I, at least for the interval
-I, its next immediate step must be in the optimal stopping
region according to Lemma 4.2.
Therefore, one could employ the "one
step looking ahead policy" for points in these locations.
introduce the boundary function first.
We now
Let d=b(n), which is a
function of n in the n-d plane, denote the boundary of the optimal
continuation region in the sense that (x,y) will be an optimal
continuation point for 0
<n ~ ~
-1 if and only if Idl=lx-yl
~ b(n).
The legitimacy problem due to this definition will be clarified by the
following Lemma 4.3.
Recall that (d,n) and (x,y) refer to the same point (as well as
(d+l,n+l) for (x+l,y) etc.).
The function Q(x,y) (and other functions
of states), of course, can be viewed as a function of d and n.
We
therefore understand that the notation Q(d,n), used below, means the
same as Q(x,y) (as well as for other functions of states).
We know
that Q is non-negative, continuous and sYmmetric (to the straight line
x=y or d=O).
fixed n.
We shall prove that Q(d,n) is non-increasing in Idl for
That will imply that the definition of b(n) is reasonable.
Lemma 4.3
Proof.
The function Q(d,n) is non-increasing in d and convex
We only need consider
(d,n), Q in (4.4) becomes
x~y
or
d~O.
As a function of
48
A(n-d)
+
x
= {2n(n+1)(1-d)
-1+ n Q(k+1,n+1)+
Q(d,n)
~
+
n Q(k-1,n+1)} .
For convenience, the expression inside the out parentheses {.} will be
denoted by the function
A(n-d) (1-d)+-1+ ~ G(k+1,n+1)+ ~ G(k-1,n+1). (4.7)
2n(n+1)
n
n
=
G(d,n)
Clearly, it is sufficient to prove that G is non-increasing in d.
From the way that Q is defined and from the fact that Q(d,n)=O
for large n (e.g. n>
2A
-1), it is easy to check that the first partial
derivative of Q(d,n) with respect to d exists except at a finite
number of points.
We further know that the first right and left
partial derivatives of Q(d,n) with respect to d exist at every point.
Let Gd(d,n) denote the first partial derivative (or the left
derivative if the derivative does not exist at the point (d,n), but
right derivative for points (d,n) where d=O).
Induction.
Since, for large n, we have Q(d,n)=O for all d, we
are able to assume that G (d,n+1) is non-positive and non-decreasing
d
for all n
~
d
O. Under this assumption, we are going to prove that
~
Gd(d,n) is also non-positive and non-decreasing
Case 1. d
~
1: We have
Gd(d,n)
1
=n
{x
1
Gd (d+1,n+1)+ y Gd (d-1,n+1)}
+ 2n {G(d+1,n+1)-G(d-1,n+1)}
and
(4.8)
49
OGd(d,n)
1
= 2n
{[(2x+ad) OGd (d+1,n+1) + (2y-ad) OGd (d-1,n+1)]
+ ad [Gd (d+1,n+1) - G (d-1,n+1)]
d
+ [OG(d+1,n+1) - OG(d-1,n+1)]},
(4.9)
where af(d):f(d+ad)-f(d) for any function f and any positive increment
ad.
By assumption, all terms on the right side of (4.9) are
non-negative, hence
d
~
OGd(d,n)~O,
i.e., Gd(d,n) is non-decreasing for
1.
On
the right side of (4.8), by the assumption, terms with G are
d
non-positive and
Case 2. 0
~
G(d+1,n+1)-G(d-1,n+1)~O,
d
< 1:
therefore,
Gd(d,n)~O.
We have
= A[2d-(n+1}]
2n(n+1)
+
n1 [x
Gd (d+1,n+1} + y Gd (d-1,n+1}]
1
+ 2n [G(d+1,n+1)-G(d-1,n+1)]
(4.10)
and
OGd(d,n)
A ad
= n(n+1)
1
+ ~[(2x+ ad) OGd (d+1,n+1) + (2y-ad) OGd (d-1,n+1)]
+ ad [G (d+1,n+1) - G (d-1,n+1)]
d
d
+ [OG(d+1,n+1)-OG(d-1,n+1)]}.
(4.11)
Examining terms on the right hand side of (4.11), by assumption, it
follows that
Gd (d+1,n+1) ~ Gd (O,n+1)
-A
= 2(n+1)'
ad
= -A
2(n+1)'
-A ad
Gd (O,n+1) = 2(n+1)'
OG(d+1,n+1) ~ ad Gd (O,n+1)
-OG(d-1,n+1) ~ ad
and
-Gd (d-1,n+1)
= Gd (1-d,n+1)
~
-A
2(n+1)'
50
and terms with OG
are non-negative.
d
Therefore. from the above
inequalities. we obtain
A 6d
OGd(d.n) ~ n(n+l)
6d
- 2n [Gd (d+l.n+l)-Gd (d-l.n+l)+G(d+l.n+l)-G(d-l.n+l)]
> A 6d
4A 6d _ 0
- n(n+l)
4n(n+l)-
or Gd(d.n) is non-decreasing.
On the right side of (4.10). Gd(d+l.n+l)~O and
1
2n
[G(d+l.n+l)-G(d-l.n+l)]~O.
therefore. we obtain the following
results (in two cases):
i). For n
~
1,
A( I-n)
(n-l)
2n Gd (-O.n+l) =
G (0 +1)
d .n
A(I-n) < 0
4n(n+l) - .
~ Gd (l.n) ~ 2n(n+l) +
- A(I-n) _ (n-l)
- 2n(n+l)
2n
_ A(I-n) + (n-l)A _
- 2n(n+l)
4n(n+l) i i). For n
< 1,
Then we have proved that Gd(d.n) is non-positive and
non-decreasing. i.e. G(d.n) is non-increasing in d and convex in Idl.
and so is Q(d.n).
0
By a direct backward induction (with the one step looking ahead
policy as we mentioned before) we can obtain:
51
Lemma 4.4
1 A 1
(4A - 2'
4 - 2)
For the points cLose to the origin and near the point
(at Least n is in the intervaL
[2A -2
,
2A -1])
the
boundary function satisfies the equation (hyperboLa)
2
Ab -A(n+1)b+n(A-2(n+1»
In fact, when A
~
= O.
(4.12)
6 this hyperbola describes the exact whole
boundary because any point after one step will be beyond the line
n= ~ -1 under this circumstance.
As an example of inner
approximations for A~6, we employ Z= a+ ~ n(n;l) , where a and ~ are
constants to be determined, as a solution of the difference equation
(4.3).
Here we only consider x)y without loss of generality.
defining the constants a
and~,
let F(x,y)=O «x,y) then is a good
point) and F(x+1,y)=O (then (x+1,y) is a warm point).
determined parameters, if
F(x,y+1)~O
an optimal continuation point.
For
With the
«x,y+1) is warm), then (x,y) is
The condition
F(x,y+1)~O
inner boundary of the optimal continuation region.
provides an
It is interesting
that this inner boundary coincides with the exact boundary described
in (4.12).
Moreover, as we shall demonstrate in the next section, for
small A (e.g. <100), this inner approximation gives precise solution
for integer pairs (x,y).
We must say that the way to choose
a
and
~
highly depends on the related nature of the geometry reflecting the
problem.
By above argument, we conclude that there exists a non-negative
boundary function d=b(n) defined on
(x,y) (or(d,n»
Idl~b(n).
(O,~ -1] with the property that
is an optimal continuation point iff
2(n+1)~A
and
The optimal stopping rule for each state is completely
52
specified through the function ben).
In the area near the two ends of
the line x=y, i.e., near the origin (0,0) and the point
A
(2
-1,0), ben)
has the shape of the hyperbola, as we mentioned before.
In the same
way that we attained Corollary 4.4, we obtain:
(4.13)
which appears in Simons and Wu (1987).
4.3
Inner approximations.
In the previous section, a constant solution family of the
difference equation (4.3) is selected in both inner approximations and
outer approximations for proving Lemma 4.1 and 4.2.
Clearly, with a
constant solution, one cannot achieve a very precise approximation to
the exact boundary of the optimal continuation region.
In this
section, we attempt to obtain more precise inner approximations with
selected solutions, and a method involved with the approximation will
be described for this specific problem.
There are a number of solution families for equation (4.3).
One
accessible family takes the form
Z(x,y)
= B(x+~,y+v)
B(x,y)
for all
~ ~
0, v ~ 0,
where B(x,y) represents the beta function defined by
(4.14)
53
1
x-I
-1
s
(l-s)y ds.
1o
B(x.y)=
~
If we choose. say.
for x>O and y>O.
= v = 0.1.2.···. one can obtain solutions like:
Zo(x.y) - 1.
which represents all constant solutions.
which is the solution we used in the previous section to obtain an
inner approximation when A
~
6.
_
x(x+l)y(y+l)
Z2(X.y) - n(n+l)(n+2)(n+3)'
As a limit of the above solutions when
~
etc.
= v
~
00.
we obtain the
solution
Obviously. the above Z. 's are symmetric in x and y.
1
Other than
solutions from this family. one can acquire an expression for
E(lp -
~I I(x.y». which is easily seen to be a solution for positive
integer x and y.
It takes the form Z (x.y)=
*
1
n
W(x.y). where W(x.y) is
defined recursively by
W(x.O) = W(O.x) = x;
W(x.y) =
1
~W(x-l.y)+W(x.y-l)]
x.y = 1.2.···.
54
The form for nonintegers is unknown.
In practice, we will naturally
use linear combinations of these Z's as solutions of (4.3):
Z= a+
~1'
Z= a+
~oo
and Z= a+
~*
for appropriate constants a and
which will be determined to suit each individual pair of (x,y).
~,
For
convenience and for comparing our results with the optimal stopping
rule prOVided by backward induction, we consider only integer lattice
points.
sections.
We shall use the notation introduced in the last two
Without loss of generality, only the case x>y will be
considered because of the symmetry of the solutions and the stopping
reward function R (and results for x=y are completely known by Lemma
4.1 and 4.2).
In the following two cases, where Z= a+
respectively, and the case where Z= a+
~*'
~1
and Z= a+
~oo
we shall undertake the
following procedure:
We shall determine parameters a and
~
for each individual point.
Suppose we are interested in the point (x,y) and want to know whether
it is an optimal continuation point. (in the section on inner
approximations we discuss a sufficient condition for an optimal
continuation point.)
Let F(x,y)=l(x,y)-R(x,y), for convenience.
and
~
First, we choose a
such that
and
F(x,y)=O
which makes parameters a and
~
F(x+l,y)=O,
functions of x,y and A.
( 4.15)
This makes the
(x,y) a good point (as well as "warm" point), and (x+1,y) a warm point
(as well as "good" point).
The reason for this way of determining the
55
coefficients for "parameters" has to do with the nature of the
geometry involved in the problem.
With the parameter a and
the function F=O divides the x-y plane into two parts:
area where all points satisfy condition
F~O
(or
Z~R)
~
chosen,
One is the
and hence are
good points. However, in this area there are also many unknown warm
points which satisfies
points satisfy
F~O
or
S~Z~R.
(S~)R~Z
The other part is the area where all
and hence are warm points.
The entire
picture is sYmmetric to the line x=y.
So far, we know that (x,y) is good and (x+l,y) is warm.
If
(x,y+l) is also a warm point, then (x,y) is an optimal continuation
point by Lemma 2.1 (i).
In fact, usually F(x,y+l)
> 0,
so that
(x,y+l) is not among the warm points which can be automatically
determined from division of the x-y plane by F=O.
One possible case
that (x,y+l) is a warm point is that all its immediate successors are
warm.
Similarly, the immediate successors of (x,y+l) can be warm
points if either
warm.
F~O
there or all of their immediate successors are
We could repeat the same argument again and again.
Therefore,
if each of all possible sample paths from (x,y+l), which is formed by
a chain of successors of point (x,y+l), sooner or later meets the warm
points determined by
F~O,
then (x,y+l) will be a warm point.
Intuitively we are searching the case that the successors of (x,y+l)
are "blocked" or "trapped" by those warm points determined by
F~O.
This idea will run through the rest of this section.
Among the future points of (x,y+l), the warm points determined by
F~O
x=y.
form two geometrical parts: one on each side of the sYmmetric axis
We consider the following two conditions: Condition I, there
exists an no ~ n+l where the two parts have a common point (~,~),
56
F{xo.xo)~O.
i.e .•
Xo ~
x
~
where 2xo =no. or there exists a point (xo.xo). where
2. on the symmetric line such that
(4.16)
i.e .. the successors of (xo.xo) are warm and hence (xo.xo) is warm.
Condition 2. the points (xo+l.xo) and (x.y) both belong to one
connected area where
F~O
entire part determined by
(on any side of the symmetric line x=y. the
F~O
does not have to be connected).
Here
for our integer lattice points we call a point set A is connected in
the sense that if any two subsets form a partition of A. then there
must exists a point such that its two immediate successors belong to
two subsets respectively.
In other words. these two conditions
guarantee that the point (x.y+l) is trapped by a connected warm point
set.
Therefore. (x.y+l) is also a warm point by Lemma 2.1 (ii). and
we can further conclude that the point (x.y) is an optimal
continuation point by Lemma 2.1 (i).
The above condition 1 can be easily represented by (4.16).
Xo
While
which satisfies condition 1 is discovered. the next task is to
verify the condition 2.
It is not easy.
Even the connectedness for
real point set does not necessarily guarantee the connectedness for
our lattice case.
point.
Therefore we have to check the condition 2 point by
However. the results based on computer studies for
that if (4.15) and (4.16) are satisfied for some
Xo ~
x
~
A~5000
show
2. then
condition 2 will be satisfied automatically for Z= a+ ffZt and
Z= a+ ffZ* and conditionally (if
F{x+l.y+l)~O)
for Z= a+ ffZoo •
Now let us examine these conditions for cases involved with Zt
57
and Zoo.
A~5000
We assume
Case (1).
for the rest of this chapter.
We try
Z
=a
+ (3Z1
=a
+ f3
(n )
x+y
x+y+l '
therefore,
F(x,y)
= Z(x,y)-R(x,y) = a + f3 ~(x-+-y~jY~(-x+-y-+~I~)
Ax
= a + R n( xy
n+ 1) - -n- + n.
-
-x~-y-
+x+y
tJ
By (4.15), parameters a and f3 are determined by
a
=A
_ (x+y)(2x-y)+2x
x(x+l)
x-y
(x+y+1)(x-y)
(4.17)
and
f3
= (x+y+2)
x-y
{(x+y )(x+y +1) - A}.
y
(4.18)
The condition (4.16),
for some
Xo ~
x
~
2,
takes the form
for some
Xo ~
x
~
2
(4.19)
It is clear that only if
A
=(2a+ ~ -A+4)2- 16(a-A+l»O,
(4.20)
58
Fig. 1
Point (x,y+l) is not "trapped" by lmown warm points.
Fig. 2
Point (x,x+l) is "trapped" by known warm points.
e.
59
there exist two roots for the equation
2
4x +(2a+ ~ -A+4)x-(a-A+1)
Let
Xi
it
can
and
denote the two roots.
X2 (X 2 )Xi)
be concluded that
= O.
(4.21)
From the argument above,
point (x,y) is an optimaL continuation
a
point if (4.19) hoLds, or (which is a LittLe stronger)
A ) 0,
(4.22)
where [ ] means the integer part.
We shall discuss the results from Case (1) and those from below
Case (2) later.
Case (2).
We try
Z
=a
+ f3Zoo
= a+
1
(3 - - - -
2~(x,y)
therefore
F(x,y)
= a+
(3
1
2~(x.y)
As with Case (1), we determine
a
Ax
x+y
- ---- +x+y.
and {3 by (4.15): F(x,y)=O and
F(x+1,y)=O in order that to make (x,y) a good point and (x+1,y) a warm
point.
As a result, the two parameters are given by
a
and
=A
x(x+y+1}
(x+Y+ 1)( x.....y )
---,.""';;';;'Jo.,;_~_;.....,..;;.."t_~_
-
X
-
Y
2x
- -x-y
60
= (1-
ff
A
y
(x+y) (x+y+1)
) 2x+y + 1
xB(x.y}
x-y
As before. our task is to discover the condition which makes point
(x.y+1) a warm point.
As we mentioned before. for achieving Condition
2. the additional condition
F(x+1.y+1}~O
is required. which can be
expressed as
2
2
(x-y) -(x-y)-2(y+1) + (x+y+1)(x+y)-4y
A
(x+y+1) (x+y+2)
y
~
For the selected
a
o.
(4.23)
and ff. the condition described by (4.16) takes the
form
A {
Ay
2x+y+1-2xo xB(x.y)
x(x-y+1)
_ xo+1 } + 12xo+1
(x+y+1) (x-y)
{(x+y}(x+y+1}}
(x-y}B(xo.xo)
+ 2xo + 1 -x-y- x~ ~ 0
for some Xo ~ x.
(4.24)
To conclude. a point (x.y) is an optimaL continuation point if (4.23)
and (4.24) are satisfied.
We can still try some other solutions of (4.3). such as Z2.
One may notice that it is not very easy to neatly
describe the complex geometrical nature caused by the involved beta
function. and above required conditions in Case (1) and (2) are a
little tedious. although it is easy to program on a personal computer.
Based on computer studies. we have found that our method described for
Case (2) gives a very precise approximation for this optimal stopping
problem.
Generally, with the linear solution family a+ ffZ1 in Case
(1). when A is small. for example A <100. our method works very well.
and we can find all of the optimal continuation points (x.y) with x
61
and y integer value.
When A is large, however, it cannot find optimal
continuation points (x,y) for which Ix-yl>l.
for the families of the form a+
~.
1
But performance improves
as i increases.
performance is provided by the family a+
~oo
The best
described in Case (2).
It (apparently) is capable of finding optimal continuation points for
arbitrarily large values of Ix-yl, for arbitrarily large values of A.
Unfortunately, the family of form a+
~*
yields only slightly
better results than does the family of the form a+
~1'
For integer-valued pairs x and y, the next two tables show two
types of comparisons of the
results for different solution families
of equation (4.3) at various values of A (A=200, 500 and 1000).
The
comparisons are between the minimal Bayes risk at (x,y) and the Bayes
risk at (x,y) that results from continuing according to the optimal
continuation points that can be discovered using the linear family in
question (stopping at any point not discovered to be an optimal
continuation point).
Table 1 shows the difference between the larger
risk and the smaller, and Table 2 shows the ratio of the larger risk
and the smaller.
The units in the tables are the numbers and the
percentage of the points among the whole considered points.
62
Table 1
Number (and the percentage) of points
with a Bayes risk difference in [a,b)
A
200
Family
[.0001, .(01)
[.001, .01)
7 (.14%)
15 ( .30%)
0 ( .00%)
7 (.14%)
15 ( .30%)
0 ( .00%)
2 ( .04%)
5 (.10%)
0 (.00%)
100 (.32%)
8 ( .03%)
0 ( .00%)
98 (.31%)
6 ( .02%)
0 ( .00%)
00
47 (.15%)
6 ( .02%)
0 ( .00%)
a+{3Z1
137 ( .11%)
102 ( .08%)
0 ( .00%)
a+{3Z*
a+{3Z00
102 (.08%)
102 (.08%)
0 ( .00%)
84 (.07%)
14 (.01%)
0 ( .00%)
a+{3Z1
a+{3Z
*
a+{3Z
00
500
a+{3Z1
a+{3Z
*
a+{3Z
1000
[.01,
00)
e
Table 2
Number (and the percentage) of points
with a Bayes risk ratio in [a,b)
A
200
500
1000
Family
[1.1, 1.2)
[1. 2,
00)
a+{3Z1
16 ( .32%)
4 (.08%)
0 ( .00%)
a+{3Z*
a+{3Z00
16 ( .32%)
4 (.08%)
0 ( .00%)
3 ( .06%)
2 ( .04%)
0 ( .00%)
a+{3Z1
89 ( .29%)
4 ( .01%)
0 ( .00%)
a+{3Z*
a+{3Z00
76 (.24%)
4 (.01%)
0 (.00%)
44 (.14%)
4 (.01%)
0 (.00%)
a+{3Z1
a+{3Z
139 (.11%)
37 ( .03%)
38 ( .03%)
111 ( .09%)
37 ( .03%)
38 ( .03%)
72 ( .06%)
6 ( .00%)
8 ( .01%)
*
a+{3Z
00
Note:
[1.01, 1.1)
The total number of integer-pairs (x,y) with x+y
~ ~ 1 (the
remaining points are optimal stopping points by Lemma 1 and Lemma 2)
is 4,950 for A=200, 31,125 for A=500, and 124,750 for A=1000.
e
.
63
4.4 Outer approximations.
All theorem and corollaries in Chapter IV can be used in this
specific testing problem
(p~~ vs p>~). By Theorem 3.1, for instance,
here we correspondingly obtain that a point (x,y) is an optimal
stopping point whenever
which is the theorem in Simons and Wu (1987).
In the folloWing Table
3, we compare the results between those from this theorem and the
optimal stopping rule prOVided by backward induction based on computer
studies for only integer-valued x and y.
various values of the parameter B.
when
n~B
The comparison is for
B is the truncation point (i.e.
all points are optimal stopping points.
related to A and C by A=2(B+1) and C=(4B+4)-1).
Of course, B is
The comparison is
between the minimal Bayes risk at (x,y) and the Bayes risk at (x,y)
that results from stopping according to the optimal stopping points
that can be discovered using our theorem (continuing at any point not
discovered to be an optimal stopping point).
Table 3 shows both the
difference and the ratio between the larger Bayes risk and the
smaller.
Note: Table 3 only gives the percentage of points among
points before the truncation line instead of the number.
64
Table 3
Comparison between the optimal stopping rule and our theorem
(The unit here is percentage (%) of points
among all points before truncation)
3.1
% of points with ratio of risks in intervals:
B
OK pts*
[1, 1.01]
< 10
100
96.30
93.30
92.24
90.27
89.75
89.19
100
94.44
89.95
88.79
87.83
87.86
87.66
10
20
30
50
75
100
3.2.
OK pts*
[0 •. 001]
< 10
100
96.30
93.30
92.24
90.27
89.75
89.19
100
94.44
89.95
88.79
87.83
87.86
87.60
50
75
100
0
0
2.87
2.37
2.51
2.42
2.30
(1.1,1.2]
0
5.56
3.35
3.66
2.59
2.11
2.14
(1.2. 00 )
0
0
3.83
5.17
7.06
7.62
7.90
% of points with difference of risks in intervals:
B
10
20
30
(1.01, 1.1]
(.001, .01]
0
0
1.44
1.29
1.33
2.00
2.04
(.01, .02]
( .02,(0)
0
5.56
7.18
4.96
3.06
2.25
2.08
0
0
1.44
4.96
7.77
7.96
8.28
*Note: "OK points" represents the percentage of the number of optimal
stopping points which are correctly determined by our theorem and has
no relation with the ratio or difference of risks. so figures are the
same for both table 3.1 and 3.2 only for reference.
e
65
4.5
COmments.
In the last two sections, for the problem of testing p
p
1
> 2'
~
21
versus
we found that the inner approximations we made are much more
precise but rather tedious analytically.
However, our approach
demonstrates that one can obtain precise inner approximations on this
problem by our method.
Apparently, if we understood more about the
analytic characteristics of the involved beta function, we would be
able to improve our inner approximations significantly.
there are many ways to improve our results.
Of course,
For example, there might
be more adequate solutions for difference equations with which one can
work easily, or a better way to decide the parameters such that the
results could be simpler or more precise.
The results from outer
approximations which we have achieved so far are much neater than
inner approximations but less precise in this particular problem.
One
obvious reason is that the theorem in Chapter 3 for outer
approximations is under very general conditions, where the selected
solution of our difference equation is almost universal, unlike those
for inner approximations.
Therefore, we have to sacrifice some degree
of precision when we apply these general results to individual
problems, each of which has its own characteristics.
If we can find
specific theorems for specific problems, we might get more precise
results.
<lIAPI'ER V
0UfER AND INNER APPROXIJIATI<XiS FOR NORMAL RANIDI VARIABLES
5.1
Introduction.
Very few studies have dealt with approximating discrete optimal
stopping problems directly and analytically. and even fewer have ever
discussed these problems for unbounded random observations directly.
Various cases of inner or outer approximations have been introduced in
previous sections. but only the cases related to bounded random
variables are involved so far.
Optimal stopping problems with
unbounded observations. however. are very important and inevitable in
practice. especially the normal observation cases.
dealing with those unbounded cases are obvious.
Difficulties in
In bounded
observation cases. whatever the assigned distributions are. there are
always limits for observations.
Therefore it is easy to employ a
universal solution of the involved difference equations to obtain a
general rule to approximate optimal stopping rules. without worrying
about any troubles caused by those "uncontrollable outliers".
The
situation changes while discussing unbounded observation cases. where
limiting properties of observations. solutions of relevant difference
equations and stopping reward functions highly depend on their
eo
67
individual distributions.
Hence. as a beginning. it might be feasible
to work on these unbounded observation cases individually instead of
by package.
In this chapter. the case where observations have a
normal distribution will be studied.
Moreover for the normal distribution case, some link between the
discrete time case and its corresponding continuous time case for
outer approximation will be discovered.
One may obtain suitable
solutions for involved difference equations with the aid of solutions
for the heat equation.
In fact. all solutions for involved difference
equations. which appear in this chapter (some solutions will be
selected for approximations) are solutions of the heat equation.
The
main reason for this phenomenon apparently is the close link between
the normal distribution and the heat equation. If this were generally
true. then we could solve the discrete time problems directly with the
knowledge of free boundary problems.
Throughout the entire chapter. suppose observations X1 .X2 .•••
have a normal density function
FX(x)
N(~.l)
12
= -1
- exp[
- ~x-~) ] •
(5.1)
..&
which is a particular case of the exponential families mentioned in
Chapter 2.
5.2
Results obtained in Chapter 2 will be used here.
Solutions of relevant difference equations.
We shall employ the results for one-parameter exponential family
obtained in Chapter 2 to this specific normal case.
The natural
68
conjugate prior distribution described by (2.16) for parameter w of
our normal distribution (5.1) is also a normal distribution.
The
posterior distribution of w given (r.k) is normal too. with a density
function of the form
1
1
C(r.k) exp(k w - 2 r w).
k
2
where C(r.k)=exp(2r)Y2i7r by (2.17) and (2.18).
Furthermore. the
difference equation (2.21) takes the form of
S(r.k) =
../i
v211"( r+ 1)
2
k
[
1 2 (k+x)2
exp(- 2r)
_exp{+ 2(r+1)}S(r+1.k+x) dx.
ZK
~
(5.2)
A solution family of (5.2) given in (2.24) now has the form (notice
that w is also a variable)
2
../i
k
r 2
Z(r.k.w) = ----- exp(- --2 + k w - -2 w )
.../Xi
../i
r
1
k
,.-2
= ----- exp{- 2 (-- - w vr)
../i
.g;:;
../i
21
,2
(y - w vr) ]
= ----- exp[-
.g;:;
k
h
were
y -= --.
v
=l.r
for any w
€
n.
For convenience. we also use the notation U
../i
Taking w = O. we obtain a solution
../i
1
2
Zo(r.k) = ----- exp(- 2 y )
.g;:;
or. equivalently.
(5.3)
=kr and
69
1
Zo(r,k) == Zo(UV)
where
use
~
~
=r
2
~(y)
1
2"
=V
~(UV
- 2"
),
(5.4)
represents the standard normal density function, and we also
as the standard normal distribution function.
In order to obtain suitable solutions of (5.2) we need the
following lemma:
LenIDa. 5.1
For any integer n
~
I, the n-th derivative and the
n-th multiple integral (all limits are from -
00
to U) of the solution
Zo with respect to U, i.e.,
and
are still solutions of (5.2).
Proof.
Because of the arbitrariness of w in the solution
den)
Z(r,k,w), its n-th derivative =---l(r,k,w), a function of w, is still
n
dw
a solution of (5.2) for any w.
=
w::O
_ n+l
V
2
In addition, it can be verified that
(n)
~(x)
n
dx
den)
. Jr,-1 = =---lo(U, V),
x=Uvv
dUn
which leads to the first conclusion of the lemma.
Moreover, for any n, the n-th multiple integration of Z(r,k,w)
70
~~
•••
w w
~Z(r,k,W) dw
···dwdw is still a solution of (5.2) for any w.
w
It can be checked that
which concludes our proof.
0
It is worth noticing that the solution (5.4) and solutions from
Lemma 5.1 are all solutions of the heat equation
5.3
Outer approximations.
e-
Under the model provided in previous sections, we consider the
Bayes testing problem, which is a special case described in 1.3.
the notations of 1.3, suppose 9 0 = 0 and c
are normal random variables).
= 21
By
(here the observations
The reward function then is
R(r ,k)= I~I-r.
r
The counterpart of this case, which is a free boundary problem
with the same reward function, has been discussed by Bather (1983).
Actually, one can demonstrate that it is feasible to employ the exact
technique Bather used for his free boundary problems to our discrete
case.
But, here, we prefer to use the techniques which are described
in Chapter 3.
However we select the exact solution Bather chose for
71
the heat equation.
By Lemma 5.1, double integration of Zo(r,k)
~-oo{~-ooZo(r,k)dU}dU = ~-oo{~-oor2~(Y)dU}dU
=
~ ~(y)dU
1
= r-
2
-00
{~(y)+~(y)}
1
is a solution. and it is known that
k
r=y
r
-
2
is a solution too. As a
result, their linear combination
1
r
- 2
{~(y)+y[~(y)-
1
2]}
is also a solution, which is the same solution of the heat equation
used by Bather (1983).
For our purpose, we select the linear
combination
1
Z(r,k) = a+
where a and
~
~
r
- 2
{~(y)+y[~(y)-
1
2]}'
(5.5)
are the parameters which will be determined properly.
Considering the outer approximation, a sufficient condition for
point (r,k) to be an optimal stopping point is, by Lemma 2.2,
Z(r,k)=R(r,k)
(5.6)
and
Z(r' ,k')
~
R(r' ,k')
for all (r',k'),
(5.7)
72
where (r'.k·) represent all successors of (r.k): r'= r+m
m=1.2.··· and
k'=k+x
for
-00
for
< X < 00.
With selected solution (5.5). (5.6) becomes
1
r
a+ ~
and after assuming k
~
- 2
1
{.(Y)+Y[~(Y):2]} =
k
Irl-r.
0 without loss of generality because of
symmetry. it leads to
3
~
2
y-r -
1
2
ar
= _----a.._.....;;.._=-..."...._
.(y)+y[~(y)-
(5.8)
1
2]
Let F(r.k)=2(r.k)-R(r.k). then (5.7) is equivalent to
F(r'.k·) = F(r+x.k+m)
a function of m and x for m=1.2.··· and
F(x.m}=F(r·
-00
~
eO.
< x < 00.
(5.9)
The notation
.k·} will be used when we emphasize that x and mare
variables for given (r.k) without ambiguity. and we use y' to
1
represent
k 'r'-
2
Then
1
F(x.m) = a+
~«r+m)- 2 [.(y')+y'(~(y')-
1
k x
-2)]-l + l+ r + m.
r+m
The following steps are for discovering the minimum of F(x.m) for
a fixed m.
Let G(m) =
m~n
F(x.m).
Because of the symmetry of above function F in k·. we assume
k'~O
73
without loss of generality.
We therefore obtain
1
F~
= ~(r+m)
- 2
1
1
[-y';(y')+y';(y')~(y')-
=
{~ [~(y')
2](r+m)
1
- 2 ]-1} (r+m)
-
2
-(r+m)
-1
-1
(5.10)
and
3
F"
= (r+m)-
x
where F' and F"
x
2~
;(y'),
(5.11)
represent the first and the second partial
x
derivatives of F with respect to x respectively.
For fixed m, we consider F as a function of x only.
If F is
convex, then we are able to find the condition under which the x's
function F acquires its minimum value at some point, which either
satisfies F'=O or is a boundary point (including infinite points).
x
,
*_ -1 1
First, we choose y =y
inverse function
of~.
=~
(~+
1
~
2)
0 , where
~
-1
represents the
Obviously, with this y* , F'=O at the point
x
1
(x* ,m) where x* =y* (r+m) -2-k and
F~'~O
..
(I.e.
F IS convex) for any x E
R (one may notice that the definition of y* itself implies
~ ~
2), and
we obtain
1
G(m) =
Second, if
m~n
~
F(x,m) = a+
~ ;[~
-1 1
(~+
1
2)](r+m)
< 2, then F'< 0 and
x
G(m) = lim F(x,m) = -
00
x-lOO
Hence we are only interested in the case
~ ~
2.
-
2
+r+m.
(5.12)
74
If G(m) is non-decreasing, and if
F(r',k')=F(x,m)~G(m)~O for
requires Gm "
G(l)~O,
then
all m, and hence (5.7) holds.
That
the derivative of G with respect to m (which is
temporarily considered as a continuous variable here) to be
non-negative.
In other words, it requires
Gm'=
~(m)
= -
3
~r+m)-
2
~ .[~-1(~ + ~)]+1 ~ o.
(5.13)
This implies that
and the most stringent case takes place when m=l:
3
~
~
2(r+l}2
(5.14)
_ ~ _~J-_~_
.[~-1(~ + ~)]
or, equivalently,
~ .[~-1(~ + ~)] ~
3
2(r+1)2.
(5.15)
By (5.8) and (5.12),
3
G(l) =
-{~ [.(y)+y(~(y)- ~)]_y+r2}
1
= ~ {(r+1)
- 2
r- 2+
~
1
(r+1)
- 2
1
(~+ 2)]-(.(Y)+Y[~(Y)- 2]) r
-1 1
(.[~
1
1
-1 1
[~(~
1
2
} +y r
1
+ 2)]+r+1
1
- 2
+1.
(5.16)
Then, we have proved the following result:
75
Lemma
5.2
If a point (r.k) satisfies
max G(l)
/3€B
~
O.
(5.17)
where G(l) is given by (5.16) with Iyl in y's position. and
B
={ /3:
(5.15) hoLds}. then (r.k) is an optimaL stopping point.
The next task is to discover the value of /3 which makes G(l) as
large as possible.
Consider G(l) as a function of /3.
After taking the first and the
second derivatives with respect to /3. we obtain:
1
d
d~(l)
1
= -[~(y)+y(~(y)- 2)]r
-
2
1
+
{~[~
-1 1
1
1 -1 1
(73 + 2)]~ ~ (73
1
- 2
+ 2)} (r+1)
(5.18)
and
2
d~G(l)
< o.
(5.19)
As a function of /3. G(l) therefore is concave and G(l) has a maximum
at the /3 which makes
d~(l)
= o.
Let /31 denote the solution of the equation
/3
~[~-l(k
3
+
~)] = 2(r+1)2,
(5.20)
76
which originates from the inequality (5.15).
with~.
side of (5.20) increases
B={~: ~ ~ ~1}.
Let
~2
Because the right hand
~ ~ ~1'
(5.15) holds for
i.e ..
denote the solution of
d
i{f(l}
(5.21)
= O.
We then have that
max G(l}=G(l}I~_~ when
~€B
- 1
~2
max G(l}=G(l}I~_~ when
~€B
- 2
~2 ~ ~1·
It is easy to see that the range of
~2
> ~1'
is [2. 00]
as y tends to zero and 2 as y tends to infinity).
(~2
tends to infinity
From Lemma 5.2. we
therefore obtain:
Lemma 5.3
A point (r.k) is an optimal stopping point if
G(l}1
~=min(~1 '~2}
~
(5.22)
O.
where 2 ~ ~2 ~ 00. and G(l} is defined by (5.16) with Iyl in y's
position.
In fact. we could rewrite Lemma 5.2 by transformation
~
=
~
-1 1
(~+
1
.
2)'
I.e .•
increasing function of
~
=
~.
[~(~)-
1 -1
2]
it becomes
Then. noticing that
~(~)
is an
77
Lemma 5.2'
max {[~(~)-
A point (r.k) is an optimal stopping point if
1
1
1
1 -1
- 2
1 - 2 I I - 2
2] {(r+l) ~(~)-(~(Y)+Y[~(Y):2])r }+ y r +1}
~
0.
~~~
(5.23)
~1
where
is the solution of the equation
(5.24)
with range
~
€[o. G]. where G ~ 0.677.
Similarly. Lemma 5.3 can be rewritten as:
Lemma 5.3'
[~(~)
A point (r.k) is an optimal stopping point if
1 1 1
1 -1
- 2
1 - 2 I I - 2
- 2] {(r+l) ~(~)-(~(y)+y [~(y)- 2])r }+ y r +1
~
0.
(5.25)
where
~=max (~1'~2)'
where
~1
satisfies (5.24) and
~2
1
1
[~(~)+[~(~)- 2]~](r+l)
where the range of
If we let
~
~2
-
2
-[~(y)+y(~(y)-
is [0,00].
tend to 00 in (5.25). we obtain
satisfies
1
1
2)]r
-
2
= 0,
(5.26)
78
Corollary 5.1
A point (r,k) is an optimaL stopping point if
1
{lyl-2[.(y)+y(~(y)- ~)]}r- 2 +
1
~
O.
(5.27)
Obviously, Corollary 5.1 is less precise than Lemma 5.3' but easy
to employ.
Furthermore, Corollary 5.1 is more precise when y is
large, i.e., Corollary 5.1 is closer to Lemma 5.3' when y is large.
From equation (5.24), which defines
function of
~1'
and denote it by
~1'
we consider r as a
rl(~l):
2
_-:.......(_~.....
d'-::l- } 3"_l.
From function (5.26), which defines
when y=O,
~2
(5.28)
2»
2(~(~1-
~2'
we know that, for any fixed r,
can obtain its smallest value.
~2'
(5.26), we consider r as a function of
After taking y=O in
and let
r2(~2)
denote it:
(5.29)
It can be checked that both
decreasing, and that
the inequality
rl(~)~ r2(~)
~1~ ~2.
rl(~l)
for
and
any~.
r2(~2)
are monotonically
Consequently, we obtain
Therefore, after modifications and
simplifications on Lemma 5.3', we acquire:
79
Theorem 5.1
A point (r.k) is an optimal stopping point if
ill r
where
~
1
-2" +
~(r+l)
1
~
O.
(5.30)
is the solution of the equation
(5.31)
where y
=
5.4
k
r;
Inner approxiDBtions.
So far. we have discussed inner approximations for bounded
observation cases only.
This section will discuss whether it is
possible to apply our Lemma 3.1 to obtain inner approximations to
normal random variable cases.
The setting in this section is the same as in the previous
section but with a slight change on R(r.k).
we use a general cost c)O.
1
Here. instead of c = '2'
The stopping reward function therefore
takes the form
R(r.k)=I~I-2cr.
r
(5.32)
The notation employed in this section is the same as in previous
sections of this chapter.
First. solutions of the difference equation (5.2) are selected.
Unfortunately. the solution (5.5). which works well in outer
approximations. is totally unusable for inner approximations. although
the optimal stopping problem remains unchanged.
3
With the aid of Lemma 5.1. we choose solution
basic one.
~(y)(1_y2) r 2" as
a
A linear combination of this solution and a constant takes
80
the form
3
Z(r,k)=a+~
+(y)(1-y
2
2
)r ,
1
where y=k r-
(5.33)
1
2 as
before (similarly, y'=k'r'-
2).
A point is an optimal continuation point if
Z(r,k)=R(r,k)
(5.34)
and if there exists at least one r'=r+m (i.e. one m) such that
Z(r' ,k')
~
where k'=k+x as before.
consideration:
good point.
for all
R(r' .k')
k'€(- 00.00),
(5.35)
The above statement is from the following
The equation (5.34) implies that the point (r.k) is a
The inequality (5.35) indicates that there must be at
least a line r=r' made up of warm points defined by
Z~R.
All
immediate successors of (r,k) therefore are "trapped" by this "warm
wall" and hence is warm.
The point (r.k) then is an optimal
continuation point by Lemma 3.1.
From (5.34).
3
2
a = -~ +(y)(1_y2)r -1~1+2cr.
r
We only consider
k~O
(i.e.
y~O)
without loss of generality. and for
any given (r,k), we only consider
sYmmetry.
We then have
(5.36)
k'~O
(i.e. Y lO) also because of the
81
F(r' ,k') :: Z(r' ,k')-R(r' ,k')
=
~(-
3
311
+(y)(1_y 2)r 2 + +(Y')(1_y,2)r,2)+ Y r- 2_Y'r'- 2+2cm .
(5.37)
For fixed (r,k), F(r' ,k') actually is a function of y' and m, hence we
write f(y',m)::F(r',k') for convenience.
We have
3
8f =
By'
L et
R
~
R
~
1
~( y ' ) ( y ,3- 3y ') r ,2-r ,- 2
(5.38)
~
• f
8f = 0 , i.e. ,
satls
y By'
(5.39)
Let Yo satisfies
(5.40)
and that leads to
exp(-
1 2
2
4
2Y
o )(l+2yo -Yo )-1
which is independent of m, and
yo~l.l386.
(5.41 )
= 0,
It can be checked that when
y'=Yo or 0, the function f acquires a maximum, and then
~
is
determined as a function of r' (or a function of m):
(5.42)
from (5.39).
With the chosen a
the condition (5.35) holds.
and~,
if
f(Yo,m)~O
for some m, then
We therefore obtain the result (under the
82
setting of this section):
Theorem 5.2
A point (r,k) is an optimaL continuation point if
there exists an integer m (or r') such that
~
hoLds, where
331
1
(_.(y)(1_y2) r 2 + r,2(2v)- 2)+lyl r- 2+2cm
~
Remark.
~ 0
(5.43)
is defined in (5.42).
In this example for inner approximations, from a
solution family, one of which may be chosen as our basic solution,
1
z
3
222
= C .(y)r +.(y)(l-y )r ,
(5.44)
the one from C=-r is better than C=-r' (Note: here, C is regarded as a
constant, and only in the final step of the criterion, it is replaced
by -r or -r'), and the one from
C=O
leads to an unusable solution.
With the solution
is better that both.
However C=r
Z(r,k)=exp(-A2~)cosh(AU), where A is any constant, we can only
discover the optimal continuation points on the line k=O.
5.5
COmments.
There are many solutions of (5.2), but not every solution can be
used for both or even one of inner and outer approximations.
Experience demonstrates that a solution which is feasible for outer
e-
83
(or inner) approximations may not be usable for inner (or outer)
approximations. As we mentioned before. for instance. with the
solution (5.5) which works well in outer approximations. we are not
able to discover any single optimal continuation point in inner
approximations.
The reason for this apparently is due to opposite
requirements for these two different kinds of approximations. and that
is implied by Lemma 2.1 and 2.2.
On the other hand. it is clear that solutions for free boundary
problems are helpful to us in finding our solutions. especially when
the way to approximate the optimal stopping rule for continuous time
cases is similar to our way in discrete time cases like the case in
5.3.
In section 6.3. our method for outer approximations is different
from that of Bather (1983). however. the principles are similar.
aIAPTER VI
INNER APPROXDlATIONS ON AN EXlUiENTIAL DISfRIBUfION CASE
6.1
Introduction.
A normal observation case has been discussed in the previous
chapter as an example of inner approximations for unbounded
observation optimal stopping problems.
With a similar procedure. in
this chapter another unbounded case. the exponentially distributed
e-
observation case will be considered.
6.2
A model and a testing problem.
In this chapter. suppose the random observations X .X .••• (X as
1 2
the generic one) have exponential distribution.
density function Ae
8 corresponds to
1
~
-;>..x
.
here.
Assume X has the
By notations employed in Chapter 1. the mean
Taking 8 0 =1. the testing problem in 1.2.3
becomes
Ho :!.>1
A -
versus
H1: !.
A
< 1.
We select a conjugate prior distribution of A with the density
function
85
r
k
r-1-Ak
f(;\) = f(r);\
e
for
r )
O.
(6.1)
where f(r) is the gamma function defined by
f(r)= [
o
u
r-1 e-u duo
Obviously. density (6.1) is the density function of the gamma
distribution Gamma(r.k). which is a extension of the natural conjugate
prior distribution of;\.
This extension guarantees the domain of r is
greater than zero for the prior distribution.
As a special case of (1.3). the stopping reward function takes
the form
= I~ -
R(r.k)
where c is the observation cost.
11 - 2cr.
(6.2)
Corresponding to (2.21). we have the
difference equation
S(r.k)
=[
S(r+1.k+x)
o
kr
r
1 dx.
(k+x)r+
(6.3)
A natural solution of (6.3) takes the form of
Z(r.k)
for any;\)O.
=
k
r
f(r)
;\r-1 e -
Ak
Let Zl(r.k) represent the above solution with ;\=1.
86
6.3
Inner Approximations
For our inner appr'ximations, we select a linear combination of
Zl(r,k) and a constant (similar to what we have done before):
r
k
-k
Z(r,k) = a+ ~ f(r) e
where a and
(r,k).
~
(6.4)
are parameters which will be determined for each point
Let F=Z-R, as before.
With Lemma 2.1 and the similar argument
we employed in 5.4 for describing condition (5.34) and (5.35), we
have:
A point (r,k) is an optimal continuation point if there exists
at least one integer m>O such that
F(r,k)=O
(6.5)
and
F(r' ,k')
where r'=r+m and k'=k+x.
~
0
for all
k'> k,
(6.6)
Condition (6.5) implies (r,k) is a good
point, and condition (6.6) then guarantees all immediate successors of
(r,k) are warm points similar to what we discussed in 5.4.
a
By (6.5),
is determined by
a
We use notation
r
k
-k k
= -(~ fer) e -I
r - 11+2cr).
u= k-r
k'
and U': --,.
therefore, can be written as
r
Without ambiguity, F(r' ,k'),
e-
87
Urr r -Ur (U'r,)r
F(r' ,U') = ~(- fer) e
+
f(r')
e
- U' r ,
)
+IU-II-IU'-Il+ 2cm.
(6.7)
For discovering properties of maxima of the function F, which is
considered as a function of U' (as a continuous variable temporarily),
we take the first and second derivatives of F with respect to U', and
we obtain
aF
au'
,
,r, 1
= ~ f ( r ' ) U,r - e -
U'
,
(6.8)
r r' (l-U ') 1= I
and
I
1
,r '+2
'2 U' ,
- -2
- -2
a?- = ~ rf(r')
au~~
U,r - e- r (U'-(I+r'
»(U'-(1-r'
».
(6.9)
With the aid of (6.8) and (6.9), it can be checked that, as a function
of U', F has three local maximum points: U'=I, U'=U 1 and U2
I
and U2 satisfy
make
a~ a
au,2 .
aF
au'-O,
and U1
We determine
~
~
I-r'
-
2 and
,
where U1
1
U2
~
- 2,where
1+r'
1
I±r'-
by the condition
F(r',!) = max (F(r',U 1 ),F(r',U 2 ».
After a lengthy argument, it can be verified that the
(6.10)
~
we are going
to choose satisfies
(6.11)
where
(6.12)
where U1 and U2 are two solutions of equation
2
(6.13)
1
and Ut
~
1 - r'
-2
1
and U2
~
1 + r
-2
Hence, the corresponding
(absolute) maximum of F is
F
= - A er'-Ur Ur rr-r
max
If
Fmax~
f(r'} + IU-11 + A + 2cm.
fer)
0, then condition (6.6) will be satisfied.
That proves our
result (under the setting in this section):
Theorem 6.1
A point (r,k) is an optimaL continuation point if
there exists an integer m (or r') such that
_ A r'-Ur Ur
e
r
r-r' f(r'} + IU-11 + A + 2cm ~ O.
fer)
where A is defined by (6.12).
(6.14)
<lIAPTER VII
TRANSFORMING nm> mE BA<XWARD HEAT
7.1
~ATI(l'(
FOR BAYES MODELS
Introduction.
In section 5.2 and 5.3, A normal random variable case is
discussed, where observations have an normal distribution N(B,I).
Given a normal prior distribution for the parameter B and a stopping
reward function which links with a Bayes testing problem, a difference
equation is obtained for our outer approximations.
We select
solutions of this difference equation with the aid of a free boundary
problem solution introduced by Bather (1983).
Despite our having the
same stopping reward, Bather's free boundary problem does not
necessarily originate from the same Bayes testing problem as ours
does.
A free boundary problem might originate from very different
problems, such as testing problems or estimating problems, Bayes
problems or non-Bayes problems.
Even the same original problem may
lead to different free boundary problems (the same heat or backward
heat equation with different boundary conditions) depending on what
transformation is taken.
In Bather's model, a standard Wiener process
is considered, and it proved to be true that the optimal stopping
problem with a standard Wiener process is a free boundary problem.
In
90
this chapter. we shall discuss transformations from partial
differential equations involved in Bayes stopping problems of a Wiener
process with drift 0 to the heat equation involved in free boundary
problems.
The Wiener process with drift 0 is the continuous time
counterpart of the discrete normal observation case with which we were
dealing in Chapter 5.
In Chernoff (1961.1972). Anscombe (1963). Chernoff and Ray
(1965). and Petkau (1978). various transformations are given for
different problems.
We shall demonstrate that the above
transformations are special cases of our general transformations.
As
an example. we shall completely solve a free boundary problem related
with a SPRT problem with the help of our transformations.
7.2 The Wiener process with drift 9.
In Chapter 6. XI .X .···.Xn •••• are i.i.d random observations with
2
normal distribution N(O.I). which is the normalization of the general
2
normal distribution N(O.a ) with a known.
S
n
=i?x
.
1 1
The sum of observations
therefore has N(On.n) distribution.
continuous time version of this setting.
We now consider the
Let Z(t) denote a random
variable depending on the continuous time t. and let Z(t) replace the
sum S.
n
Z(t) then has a normal distribution N(Ot.t).
process Z={Z(t).
t~O}
a Wiener process with drift O.
We call the
Formally. the
Wiener process Z can be defined through the standard Wiener process as
follows.
Let process W={W(t).
which satisfies
(a)
W(O)=O;
t~O}
be the standard Wiener process
91
(b)
W(t)-W(s) '" N(O. It-sl):
(c)
(independent increments) For all -
00
<tl~t2~ ••• ~tn<
00.
random variables [W(t 2 )-W(t 1 )]. [W(t )-W(t )].···.
3
2
[Wetn )-W(tn- 1)] are independent.
Let Z(t)=W(t)+ St. then the process Z={Z(t).
process with drift S.
t~O}
is called the Wiener
Obviously Z has the same properties as W.
except (b) must be replaced by Z(t)-Z(s) '" N(S(t-s). It-sl).
7.3 Bayes models and partial differential equations.
We consider the optimal stopping problem for the process
Z={Z(t).
t~O}.
Elsl<
the posterior expectation of S given Z(t)=z. which represents
00.
Assume that S has a prior distribution G(S).
Assuming
the present state (z.t). takes the form
J
S exp{zS -
~S
2
} G(dS)
<p(z. t) == E[S IZ(t)=z] = - - - - - - - : : : : - 2 - - -
J exp{zS -
(7.1 )
tS
2' } G(dS)
Let R(z.t) be a given stopping risk function at the state (z.t). which
is obtained from some loss structure related to an optimal stopping
problem in which we are interested.
The loss structure may represent
a testing problem or an estimating problem or any other possible
problem.
Denote b(z.t) as the optimal stopping risk at the state (z.t).
For the process Z. each state belongs to either the optimal
continuation region or the optimal stopping region.
By dYnamic
92
programming, if a state (z,t) is in the optimal continuation region,
then it must satisfy the equation
b(z,t) = E{b[Z(t+o),t+oJIZ(t)=z}+ 0(0)
as o!O,
(7.2)
and if it is on the boundary or in the optimal stopping region, then
b(z,t)=R(z,t).
By properties of the process Z, equation (7.2) can
take the form
b(z,t) = E{E(b[W(t+o)+9(t+o),t+oJIW(t)+9t=z,9)IZ(t)=z}+ 0(0)
= E{E(b[JoY+W(t)+9t+90,t+oJ IW(t)+9t=z,9) IZ(t)=z}+ 0(0)
= E{E[b(JoY+z+90,t+o) IZ(t)=z,9J IZ(t)=z}+ 0(0),
where Y, independent of W, has N(O,l) distribution.
Then we obtain
the partial differential equation
1
2-bzz +~(z,t)bz +b t = 0,
by the Taylor expansion of b(JoY+z+90,t+o) and letting 0 ~
(7.3)
o.
In the
next section, we shall introduce transformations which convert (7.3)
to the backward heat equation
1
~2
= 0,
1
= O.
uu+h r
or the heat equation
~2
uu -hr
The backward heat equation or the heat equation, together with proper
e·
93
boundary conditions (for unknown boundary), constitute free boundary
problems.
Our transformations will build a bridge between the optimal
stopping problems involving (7.3) and those involving the backward
heat or heat equation.
7.4 Transformations.
~(z,t),
Consider any function
derivatives
~z' ~zz
and
~t·
and suppose there exist partial
First we prove the following lemma on
transformations for this general
function~.
e
that the posterior expectation of
by (7.1), which is also denoted by
Secondly we will show
given the state (z,t) determined
satisfies the condition in the
~,
lemma and hence could be transformed by our methods.
As the first
step, we have the following lemma.
Lemma 7.1
IF a Function of (z,t),
2~ +2~ +~
t
z
zz
~(z,t)
say, satisFies
= 0,
(7.4)
then the partiaL diFFerentiaL equation
1
-b +~(z,t)b +b t
z
2 zz
=0
(7.5)
can be transFormed into the backward heat equation
1
-h +h
2 zz t
=0
(7.6)
by the transFormation
h(z,t)
= b(z,t)
1
2
exp{- ~~ +~z)}.
(7.7)
94
The proof will be presented later.
function
-
~(z.t)
It can be verified that the
defined in (7.1) satisfies the condition (7.4). and
~~2+~z)=ln[~(z.t)] where ~ represents the denominator of ~:
J
~(z.t)=
exp{za-
ta2
(7.8)
~}G(da).
Therefore. by lemma 7.1 the following theorem is proved.
Theorem 1.1
The transformation
h(z.t) =
b(z.t)~(z.t)
(7.9)
transforms (7.3) into the backward heat equation ~21 +h =0. where
zz t
~(z.t)
is defined by (7.8).
Lemma 1.2
For any constants
a.~.~.o
and K where
transformation
(u.r)
and
or
2
Kat
p(u.r)=h(z.t)exp{~±
transforms h
zz
+h =0 into p +p =0.
t
uu r
~
zva/o}
(for
~=O)
A=ao-~~>O.
the
95
The proof of Lemma 7.2 as well as the proof of Lemma 7.1 is
postponp.d.
In a way that is similar to Lemma 7.2, we have the
following:
Lemma 7.3
For any constants
a,~,~,o
and K where
A=ao-~~<O,
the
transformation
(u,r)
and
or
2
p(u,r)=h(z,t)exp{- K2~t ± zJ-a/o}
transforms h
zz
(for
~=O)
+ht=O into p -p =0.
uu r
The transformations in Lemma 7.1 and 7.2 transform the backward
heat equation into itself or the heat equation.
Theorem 7.1 together
with Lemma 7.2 or 7.3 therefore provide various choices of
transformations, which can transform the p.d.e. (7.3) into either
backward heat equation or heat equation.
Recall that any linear
combinations of above transformations and solutions of heat (or
backward heat) equation are still the same type of transformations.
After transformations described above, the new risk function R*
takes the form
R* (u,r) = R(z,t)k(z,t),
where k(z,t) is the factor in associated transformation
(7.10)
96
h{u,r)=b{z.t)k{z.t) and u=u{z.t),r=r{t).
It is obvious that the only difference between the transformations in
Lemma 7.2 and 7.3 is the sign of the parameters a and
Proof of Lemma 7.1. 7.2.
~.
To convert (7.5) into (7.6). we first
try the transformation indicated by the product
h{u.r)=b{z.t)k{z.t)
or with K{z.t)=k{z,t)
-1
b{z.t)=K{z.t)h{u.r).
(7.11 )
where u=u{z.t). r=r{z.t) and K{z.t) are unknown functions to be
determined.
Equation (7.5) therefore takes the form
For our purpose. we need
Ku z
2
=
1
-:3(2
r zz+K z r z +Kr z cp+Kr t .
Kr zu z
= O.
(7.14)
2
= O.
(7.15)
Kr z
1
-:3(2
u
(7.13)
zz +Kz u z +Ku t +Kuz <p
=0
(7.16)
and
(7.17)
97
In fact, equation (7.15) implies (7.14), and r is a function of t
alone by (7.15).
By simple verification, (7.13) leads to
U
2
z
=rt~O
and
uz=~ or
u = ±z~+g(t),
where get) is a function of t, and u
zz
=0.
(7.18)
Equation (7.16) therefore
indicates
k
-1
2
= K = exp{Jcpdz+A( t )+z
r tt
gt
± z"
}.
t
r
""4r
(7.19)
t
where terms Jcpdz+A(t) represent the indefinite integral of ~ with
respect to z.
From (7.19), we can obtain expressions of K , K and
z
zz
K , and (7.17) then becomes
t
r(3)r -r 2
r
z2{!J-!£)2_ t
t tt
8'- r t
4 2
} +
rt
2
1 2
gt
1
+ - ~ + --- - 2
2
2r
t
~
r
1
Z
g
2g r r g
tt t
tt t- tt t
r-3) f a }
v rt
2~
{~
r tt
- --- z 4r
t
~
2
- A -(fcpdz) =
t
t
o.
(7.20)
Equation (7.20) requires that the following partial differential
equations hold:
3r
tt
2-2r (3)r = 0
t
t
'
(7.21)
(7.22)
gtrtt-gttrt = O.
and
2
1 2
gt
1
-2 ~ + --2r - -2
t
~
r tt
z - --4r t
~
2
- At -(Jcpdz) t =
o.
(7.23)
98
By solving equation (7.21), we obtain the solution
_ at+{3
(7.24)
r ( t ) - 'Yt+o'
where a,
~,
'Y and 0 are constants such that
ao-~'Y)O.
From equation
(7.22) function get) is a linear combination of function ret).
Corresponding to (7.24), we therefore obtain
at+{3
(7.25)
get) =k 'Yt+o'
for any constant k.
With these determined ret) and get), equation
(7.23) takes the form
(7.26)
where
2
k (ao-{3'Y)
J2('Yt+o)
B(t)=A(t)- (
2
'Y
+ 2( t+o»dt.
'Y
(7.27)
Equation (7.26) is equivalent to the condition (7.4) of the lemma.
By
replacing functions ret) and get) in (7.18) and (7.19) together with
(7.24) and (7.25), we obtain the expression for k(z,t)
1
z2'Y ± z,~
L a6 -{3r}.
k(z,t) = exp{ f ~z +A(t)- 2('Y+o)
'Yt+o
By (7.26) and (7.27),
where
(7.28)
...
99
and
for 'Y#O
for 'Y=O.
The transformation h(z.t)=b(z.t)k (z.t) is the one in Lemma 7.1. and
1
the transformation p(u.r)=h(z.t)k (z.t) together with
2
is the one in Lemma 7.2.
of these two.
A general transformation is the combination
0
Corollary 7. 1
A transformation from the combination of Theorem
7.1 and Lemma 7.2 (or 7.3) can transform (7.3) into the backward heat
equation (or heat equation).
By our transformations the original optimal stopping problem then
is equivalent to the following free boundary problem (A) for the heat
equation (or backward heat equation). where ~ and ~~c denote the
optimal stopping and continuation regions respectively. and
boundary of
o~
the
~:
1
-h (u.r)fh (u.r)=O
2 uu
r
h(u.r)=R* (u.r)
(A)
h (u.r)=R* (u.r)
u
u
~.
for (u.r)€
for (u.r)€
for (u.r)€
~
O~.
Of course. we could also consider a stopping reward function instead
100
of a stopping risk function without any difficulties.
Free boundary
problems like (A) have been studied by many researchers such as
Chernoff (1961. 1972. 1986). Bather (1962. 1983) and Van Moerbeke
(1974a. 1974b. 1975).
For any suitable problem with
certain~.
there is a variety of
transformations available. and therefore there is a group of free
boundary problems with the same backward heat equation but different
stopping risk (or reward) functions.
Nevertheless. these problems
must lead to a same optimal stopping boundary since they originate
from the same optimal stopping problem.
A choice of transformations
apparently depends on the nature of functions
~(z.t)
and R(z.t).
It
is simply a matter of convenience.
7.5
A free botmdary probleaa 1 inked wi th the SPRT.
Under the same model introduced in this chapter. suppose that the
drift
a has
a two point distribution
1
p(a = ± Jl) = 2".
It can be verified that the posterior distribution given z(t)=z takes
the form
1
= 2exp(± Jlz) sech(Jlz).
(7.30)
Furthermore. the posterior expectation given state (z.t) is
101
~(z.t) = E[alz(t)=z] = ~ tanh{~z).
(7.31)
which is a function of z alone.
We consider the following testing problem:
(7.32)
with a loss function which equals
~-a+ct
if H is accepted
O
~+a+ct
if HI is accepted.
where the constant c is the cost per union time.
The posterior Bayes
risk function given (z.t). which is our optimal stopping risk function
at state (z.t). therefore. is
R(z.t)
=min
{E[~-a+ctlz(t)=z]. E[~+a+ctlz(t)=z]}
(7.33)
In this case. (7.3) takes the form of
(7.34)
We take a transformation from Corollary 7.1.
h(u.t) = b(z.t)
cosh(~z)
e
z-(~2-1)t/2
.
(7.35)
102
where (u.t)=(t+z. t).
Results in SPRT suggest that. in this special case. the optimal
continuation region must be the area between a pair of parallel lines.
z=±a. where a)O. unknown.
Our task in this section is to determine
the value of a. which is the solution of our free boundary problem.
The optimal stopping risk b(z.t) at (z.t) therefore must be equal to
the posterior expectation of the loss which is counted from state
(z.t) to the parallel lines. i.e .. the average expected loss from
state (z.t) to parallels z=±a.
These two parallels construct the
boundary of the optimal stopping region.
Hence by this b(z.t) which
is a solution of the free boundary problem one is able to obtain the
value of the unknown constant a. and solve the problem.
Let T be the stopping time when Z(t) first hit the boundary of
the continuation region (-a<z<a) from inside the region.
that Z(T)= ±a.
P
Zl
It is clear
Let
=P[Z(T)=als.(z.t)]
and
P
Z2
=P[Z(T)=-aIS.(z.t)].
the probabilities that Z(t). starting from (z.t). hits the parallel
lines.
It is clear that
Let m(s) be the moment generating function of X
m(s)
= exp(Ss +
s
2
~).
(~N(S.l)).
then
103
Let h satisfy m(h)=1 and
h~,
then h=-2S.
By Wald's Fundamental
Identity
EehZ(T)(m(h»-T = 1
or
EehZ(T) = 1,
we obtain
We therefore have expressions for P
Zl
and P
Z2
(7.36)
and
(7.37)
We then can attain
ESZ(T-t) = (a-z)P +(-a-z)P
Z2
Z1
-2Sz
= a coth(2Sa)- ae
cosech(2Sa)- z.
(7.38)
The loss at the boundary is
l(S) = Jl- S+cT
= Jl+ S+cT
on z
on z
=a
= -a.
We therefore obtain
ES[l(S)] = Jl+ Se
-2Sz
csch(2Sa)- S coth(2Sa)+ cEST,
(7.39)
where
EST =
ES[Z(T-t)]
a
a -2Sz
z
E X
= e coth(2Sa) csch(2Sa)- e
S
ae
(7.40)
104
by Wald's equation.
Combining (7.31). (7.39) and (7.40). we finally
obtain the average loss from (z.t) to the boundary:
E[1(9)I{z.t)] = E{E 9 [1(9)]I{z.t)}
=
c
~- ~ tanh{~)+ ~
[a
tanh{~)-z tanh{~z)]+
ct.
(7.41)
As we mentioned before. the optimal stopping risk b{z.t) must be
equal to the above expectation. i.e .•
b{z.t) = E[1(9)I{z.t)] =
= ~-~ tanh{~)+ ~[a tanh{~)- z tanh{~z)]+ ct.
~
(7.42)
~ ,
In fact. by direct verification or by the way that b{z.t) is
determined. this b{z.t) is a solution of partial differential equation
(7.34). and it satisfies the boundary condition
b{±a.t) = R{±a.t).
(7.43)
From u=z+t. we rewrite function b as
b{u.t) =
~-~ tanh{~)
+ ~ {a tanh{~) - (u-t) tanh[~{u-t)]}+ct.
(7.44)
After the transformation provided by (7.35). the new function takes
the form
-'
105
h(u.t) = {J.l-J.l
tanh(~)
+ ~a tanh(~)-(u-t) tanh(J.l(u-t»]+ct}ecosh[J.l(u-t)]e
t
2
z~J.l -1)
J.l
(7.45)
with corresponding risk function
t
*
R (u.t) = R(u.t) cosh[J.l(u-t)] e
2
z~J.l -1)
t
= {J.l[l-tanh(J.llu-tl)]+ct} cosh[J.l(u-t)] e
2
z~J.l -1)
(7.46)
By (7.43).
h(±a+t.t) = R* (±a+t.t).
We also want h =R* at u=±a+t. and this is true if b =R.
u u
u u
(7.47)
It can be
verified that the condition
3
sinh(2J.La)=2(~
- ~)
c
(7.48)
will lead to b =R and hence h =R* (it is easy to check that ht=R*
u u
u u
t
under the condition (7.48) also).
The boundary of the optimal
continuation region consists of the parallel lines z=±a with the
constant a determined by (7.48).
So we have proved the following
result:
Theorem 7.2
Under the modeL introduced in this section, the
optimaL continuation region is between the pair of paraLLeLs z=±a,
where the constant a is determined by equation
3
sinh(2J.La)=2(~
- J.La).
c
106
We have solved our specific free boundary problem and the related
SPRT problem.
7.6
Normal prior distribution cases.
In problems, such as the problem of testing for the sign of a
normal mean, Chernoff (1961,1972); the one-armed bandit problem,
Chernoff & Ray (1965); the sequential medical trial involving paired
data, Anscombe (1963); and the sequential medical trials for comparing
an experimental with a standard treatment, Petkau (1978), S is
assigned a normal prior distribution
N(~O'oO).
The posterior mean
therefore takes the form of
~(z,t) = E[SIZ(t)=z]
(7.49)
By Corollary 7.1, the optimal stopping problem can be converted to
free boundary problems (A) by various transformations.
In order to
obtain the same transformation as that in above papers, we take the
transformation
(7.50)
h(u,r) = b(z,t),
where
(u,r)
=[
-2
,(oO+t)
-1
].
(7.51)
This is the same as their transformation.
Mathematically, the differences among all problems mentioned in
107
this section reflect the differences of those different risk (or
reward) functions.
These functions originate from different optimal
stopping problems. such as testing problems. estimating problems etc ..
and determine different boundary conditions in free boundary problems
after transformations.
7.7 An improper prior case.
In Chernoff and Ray (1965) and Petkau (1978). the original
problem concerned Bernoulli trials with a success probability p which
is given a beta prior distribution Beta(aO.b ).
O
Let N be the lot
size. s and f be the numbers of successes and failures after n
distribution of p after n observations is also a beta distribution
Beta(a.b).
A dynamic equation then is obtained.
normalization and transformation. for M ~
00.
After a kind of
a partial differential
equation with the form
!b + ~b +b = 0
2 zz t z t
follows.
(7.52)
It could be considered as a special case of our normal prior
case provided in the previous section. except the prior is an improper
prior with 00=
00.
which leads to a proper posterior distribution
z 1
N(t' t)' also a normal distribution.
We employ a transformation from
Corollary 7.1:
h(u.r) = B(z.t)-u
with
(u.r)=(~. ~-1).
108
which transforms (7.52) into the heat equation. and it is equivalent
with theirs.
7.8. Transformations from one Wiener process into another
From Lemma 7.3 one can easily find a group of transformations
which transform the backward heat equation into the heat equation.
we simply select a=O=K=O and
~~=1
If
we have the transformation
(7.53)
with
(u.r)=(~.t).
which inverse is
(z.t)=(~.~) and
1
1
z 1
b(z.t) = ~(t't) [(2wt)
- 2
2
z-l
exp(- 2t)]
The transformation (7.53) changes a Wiener process
.
{W(t).t~O}
(7.54)
into
{rW(~).r~O}. It is well known that {rw(~).r~O} is also a Wiener
process but in -r scale.
processes
{W(t).t~O}
and
Optimal stopping problems for the Wiener
1
{rW(r).r~O}
are respectively corresponding to
those free boundary problems for the backward heat equation and heat
equation with related reward functions.
Transformation (7.54). as a
special case of the reverse of the transformations in Lemma 7.3. is
still called Appell (1892) transformation. which is mentioned by Van
Moerbeke (1974) to transform a reward function with infinite horizon
into that with finite horizon.
Albert. A. (1961). The Sequential Design of Experiments for
Infinitely Many States of Nature. Ann. Math. Statist. 32.
774-799.
Anscombe. F.J. (1963). Sequential Medical Trials.
Assoc. 58. 365-83.
J. Am.
Statist.
Appell. P. (1892). Sur lequation o2/ox2-oz/oy=O et la theorie de la
chaleur. J. Math. Pures AppL. 8. 187-216.
Arrow. K. J .• Blackwell. D.. and Girshick. M. A. (1949). Bayes and
Minimax Solutions of Sequential Decision Problems. Econometrica.
17. 213-244.
Bather. John (1962). Bayes Procedure for Deciding the Sign of a
Normal Mean. Proc. Cambridge PhHos Soc .. 58. 599-620.
Bather. John (1983). Optimal Stopping of Brownian Motion: a
Comparison Technique. Recent Advances in Statistics. Papers in
Honor of Herman Chernoff. Academic Press. New York.
Bessler. S. A. (1960). Theory and AppLication of SequentiaL Design of
Experiments. k-actions and InfiniteLy Many Experiments. Tech.
Rep. No. 55. Dept. of Statist .• Stanford Univ. Stanford.
California.
Bickel. P. J. and Yahav, J. A. (1967), Asymptotically Pointwise
Optimal Procedures in Sequential Analysis. Proc. Symp. Math.
Statist. ProbabiLity, 5th. BerkerLey, 1965, 1, 401-413. Univ. of
California.
Bickel, P. J. and Yahav, J. A. (1969). Some Contributions to the
Asymptotic Theory of Bayes solutions. Z.
WahrsheinLichkeitstheorie Yerm. Gebiete 11, 257-276.
Breakwell, J. A. and Chernoff. Herman (1964), Sequential Tests for
the Mean of a Normal Distribution II (large t). Ann. Math.
Statist. 35, 162-173.
Chernoff, Herman (1959). Sequential Design of Experiments.
Math. Statist. 30, 755-770.
Ann.
Chernoff. Herman (1961), Sequential Tests for the Mean of a Normal
Distribution. Proc. 4th BerkeLey Symp. 1. 79-91.
110
Chernoff. Herman (1965a). Sequential Tests for the Mean of a Normal
Distribution III (small t). Ann. Math. Statist. 36. 28-54
Chernoff. Herman (l965b). Sequential Tests for the Mean of a Normal
Distribution IV (discrete case). Ann. Math. Statist. 36. 55-68
Chernoff. Herman (1972). SequentiaL AnaLysis and OptimaL Design.
Regional Conference Series 8. SIAM. Philadelphia.
Chernoff. Herman (1986). Numerical Solutions for Bayes Sequential
Decision Problems. SIAlf. J. SCI. STAT. COMPUT. Vol. 7. No.1,
47-59.
Chernoff. Herman and Ray. S.N. (1965). A Bayes Sequential Sampling
Inspection Plan. Ann. Math. Statist. 36. 1387-407.
Chow. Y. S. (1978).
Inc ..
ProbabiLity Theory.
Spring-Verlag. New York
Chow. Y. S .. Robbins. Herbert and Siegmund. D. (1971). Great
Expectations: The Theory of OptimaL Stopping. Houghton Mifflin.
New York.
Clayton. Murray k. (1985). A Bayesian nonparametric sequential test
for the mean of a population. The AnnaLs of Statistics. Vol.
13. No.3 1129-1139.
Diaconis. Persi and Ylvisaker. Donald (1979). Conjugate priors for
exponential families. The AnnaLs of Statistics. Vol. 7. No.2.
269-281.
~ricson.
W. A. (1969). Subjective Bayesian Models in Sampling Finite
Population. J. Roy. Statist. Soc .. Series B. 31. 195-233.
Ericson. W. A. (1970). On the Posterior Mean and Variance of a
Population Mean. J. Am. Statist. Assoc. 65. 330. 649-652.
Ferguson. Thomas S. (1973). A Bayesian Analysis of Some nonparametric
problems. The AnnaLs of Statistics. Vol. 1. No.2. 209-230.
Kiefer. j. and Sacks. j. (1963). Asymptotically Optimum Sequential
Inference and Design. Ann. Math. Statist. 34. 705-750.
Lerche. Hans Rudolf (1984) On the optimality of Open Ended Sequential
tests with parabolic boundaries.
Moriguti. Sigiti and Robbins. Herbert (1962). A Bayes test of p~~
versus p>~.Rep. Statist. AppL. Res Un. Japan Sci. Enqr. 9. 39-60.
Neyman. j. and Pearson. E. S. (1933). On the Problems of the Most
Efficient Tests of Statistical Hypothesis. PhiLos. Trans. Roy.
Soc. London Ser. A. 231. 289-337.
111
Petkau. A.J. (1978). Sequential Medical Trials for Comparing an
Experimental with a Standard Treatment. J. Am. Statist. Assoc.
13. 328-38.
Raiffa. H. and Schlaifer. R. (1961). AppLied StatisticaL Decision
Theory. Graduate School of Business Administration. Harvard
Univ .. Boston.
Schwarz. Gideon (1962). Asymptotic Shapes of Bayes Sequential Testing
Regions. Ann. Math. Statist. 33. 224-236.
Shiryayev. A. N. (1978).
York Inc ..
OptimaL Stopping RuLes. Spring-Verlag. New
Simons. Gordon (1986). Bayes Rules for a Clinical-trials Model with
Dichotomous Responses. Ann. Statist. Vol. 14. 3. 954-970.
Simons. Gordon and Xizhi Wu (1987). On Bayes Tests for p~~ versus
p>~: Analytic approximations.
Adaptive StatisticaL Procedures
and ReLated Topics. IMS Lecture Notes.
Van Moerbeke. P.L.J. (1974a). Optimal Stopping and Free Boundary
Problems. the Proceedings of the Conference on Stochastic
Differential Equations (July 1972). Edmonton (Alberta). Rocky
Mountain Math. J. 4. 3. 539-578.
Van Moerbeke. P.L.J. (1974b). An Optimal Stopping Problem with Linear
Reward. Acta Mathematica. 1-2. 132. 111-151.
Van Moerbeke. P.L.J. (1975). On Optimal Stopping and Free Boundary
Problems. Archive for Rat. Mech. Ana. 60. 101-48.
Waldo Abraham. (1947).
New York.
SequentiaL AnaLysis.
John Wiley & Sons. Inc ..
Waldo Abraham. (1950). Asymptotic Minimax Solutions of Sequential
Point Estimation Problems. Proc. Symp. Math. Statist.
ProbabiLity. BerkLey and Los AngLes. 1-11. Univ. of California
Press.
Whittle. Peter (1982). Optimization Over Time: Dynamic Programing and
Stochastic ControL. John Wiley & Sons.