EASILY DETERMINING WHICH
URNS ARE "FAVORABLE"
Gordon Simons
University of N.C.
Chapel Hill, N.C. 27514
,
Mimeo Series #1596
February 1986
DEPARTMENT OF STATISTICS
Chapel Hill, North Carolina
•
',.
Eas-:l.1y
Determ-:l.n-:l.ng Wh-:l.ch
Urns
a.re
nFa.vorab1e n
by Gordon Simons
University of North Carolina at Chapel Hill
•
e
ABSTRACT
The optimal sampling strategy for an urn, containing known numbers nf
plus and minus ones, can be simply described with the use of a empirically
ju.tified rule, based upon what appears tn be a legitimate third-order
asymptotic expansion of "the optimal stopping boundary" as the urn size
goes tn infinity.
The rule performs exceedingly well.
There is a known
first-order asyaptoticexpansion due to Shepp.
The reader is invited to
try to justify a second-order asymptotic expansion of a type described by
Chernoff and Petkau.
The evidence presented in its support is very
persuasive.
AMS 1980 Subject Classifications: Primary 62L15
Keywords: optimal stopping, asymptotic expansions,
stopping boundary
•
The author's work is supported by the National Science Foundation, Grant
Number DMS-8400602.
Q.
Motiyation.
The author's interest in the urn problem described below was
spawned by a desire to apply the methods of approximation developed in [7] and
[8] to a rather different type of optimal stopping problem.
was largely unfulfilled, the author found that:
•
While this desire
(i) a stopping rule based upon
an asymptotic approximation described by Shepp (1969) performs well for most
urns, (ii)a simple modification, motivated by Chernoff and Petkau (1976),
performs
substantially
better,
and
(iii)
a
second,
empirically-fashioned
modification performs stupendously; it hardly ever makes a mistake: Two urns of
about 1.5 billion considered are misclassified.
It
is
certainly hoped that
someone with a
theoretical
bent will
be
inspired, by the evidence,to find justifications for these modifications; the
numerical evidence leaves no doubt that the first modification is the right
--
thing to do.
The practical value of working with the urn problem needs to be stressed.
The backward recursion, based on the dynamic equation, is so simple that the
optimal rule can be worked out exactly for very large urns.
And without such
calculations, the empirical aspects of this paper would be impossible.
FAVORABLE URNS
2
An illustration is provided by the asymptotic approximation (7) below.
which only applies to some
optimal
boundaries.
There
are
other
optimal
boundaries with equally valid asymptotic approximations which do not satisfy
•
(7) .
Since any reliable asymptotic description of any optimal boundary will do,
it does not seem appropriate to single out a particular optimal boundary. a
priori, and .callit "the optimal boundary."
The optimal boundaries arising in [3], [7] and [8] are not unique.
the issue is ignored in the first paper; the object is to pursue
approximations.
But
~articular
(See Hogan (1985) for sOlie helpful insight.) . And the issue
does not arise directly in the latter papers.
For the main focus of attention
is on discovering optimal continuation and optimal stopping points, not optimal
boundaries.
Here,
the non-uniqueness issue is addressed,
but, as in [3],
particular approximations are pursued.
1. The urn problem.
One may draw at random without replacement froa an urn
containingm minus ones and p plus ones, and stop whenever one wishes.
object is to obtain a large sum.
The values a and p are known at the outset,
and one is free not to draw at all.
is "favorable";
I.e.,
for which
R(m,p)
The task is to determine whether the urn
whether its expected
stopping, is (strictly) positive.
.The
Letting
~
return
R(a,p),
under
denote the class of
optimal
(m,p)
urns
is positive, Shepp (1969) observed that an optimal policy is
to stop as soon as .one reaches a depleted urn not in
~.
..
FAVORABLE URNS
For small values of
R(m,p)
a,p
=
by induction:
m
R(m,O) =
and
3
p, it is easy and practical to determine
° (m=O,l,"'),
R(O,p) = p (p=O,l,···). and for
1,2,"',
•
R(m,p)
Naturally,
R(m,p)
urn when
R(m, p) >0
m-p
m
p
max(O,--a+p R(m-1,p) + --m+p R(m,p-1) - m+p )..
when
p>m.
can be positive when m>p.
m>p
(1 )
What is somewhat surprising is the fact that
Clearly it is better not to draw from the
than it is to proceed with a policy which calls for a fixed
number of draws.
But, because the sampling is performed without replacement,
it may be desirable to proceed with a more sophisticated policy which depends
upon the outcome of the draws.
e_
Let
paper).
n = m+ p
When
and
k = m - p (notation that will be used throughout the
n = 4,5,6,7,8,9 and 10, it is best to draw at least once from
the urn if k is no larger than 0,1,0,1,0,1 and 2, respectively.
When n=3 and
k=1, i t does not matter whether one refrains from drawing or one draws until
the first plus one is obtained; the expected return in either case is· zero.
(This phenomenon occurs for no other urn with
conjectures there are no other occurrences.)
is a "boundary sequence"
n
~
54,000.
Boyce (1973)
Shepp (1969) has shown that there
b(1),b(2), ... such that
~
= ((m,p): k<b(n»).
And he
has shown that
e
b(n) = an
where the coefficient
a
1/2
+ o(n
1/2
)
as
n
~ ClO
,
(2)
0.83992 ... is the unique solution of the equation
FAVORABLE URNS
00
(l-(X2)J
The sequence
same par i ty as
b(n)
n,
semi-open interval
o
4
2
exp(aX-X /2) dx =a.
is not unique.
odd or even,
(be(n),bu(n)]
Since
each
k
b(n)
(3)
is always an integer of the
must be specified within a
•
of length two.
Equation (2) suggests a "first-order asymptotic stopping rule":
stop as soon as the current values }
of
k
and
n
satisfy
k
~
an
1/2
(4)
.
We shall compare this rule with a "second-order asymptotic stopping rule",
stop as soon as the current values of }
k
and
n
(5)
k ~ an 1/ 2 - .5
satisfy
and with various "third-order asymptotic rules" of the forll:
stop as soon as the current values of
and
where the sequence
c(n)
n
satisfy
k }
e·
(6)
is an empirically
fJ
~
c(n).
"-.5", appearing in (5) and (6), is suggested by the work of
Chernoff and Petkau (1976, page 888);
interval
n, and
goes to zero with
determined constant, depending on
The
k ~ an 1/ 2 -.5 +pc(n)
When
n
is large
and
k
is near the
(be(n),bu(n)], the proportion of minus ones in the urn must be close
to one-half.
Consequently, as
n
begins to decrease, the changing of
closely approximated by a random walk with steps sizes
for which the correction
"-.5"
k
is
± 1, the circumstance
is appropriate.
Note that stopping rules (5) and (6) are modest refinements of (4); the
adjustments " -.5"
and
"- . 5 + pc (n) ", when
when compared to the length of the interval
boundary sequence of the form
n
is large, are both small
(bt(n),bu(n)].
·So if there is a
.
FAVORABLE URNS
= an 1/2 -.5
ben)
as
+ 0(1)
5
n..
there must be others which are not of this form.
00
(7)
,
These small refinements can
make a significant difference, as we will see .
.
.
,
Based on a careful examination of the locations of the allowable intervals
,
(b (n), bu(n)], i t seems likely that there exist optimal boundaries which are
i
increasing and concave.
It also seems likely that all increasing and concave
So unique
optimal boundaries will agree up to a suitable third-order term.
asymptotic results might hold for "smooth optimal boundaries.
number of terms in the asymptotic result,
i . e.,
The desired
JI
the desired accuracy, would
determine the amount of smoothness required.
2. The favorable urns.
•
urn sizes
n
~
54,000
The author has determined all of the favorable urns for
using about 100 hours of computing time on an
A concise.summary is given in
k ~ ko
n ~ no
and
Table I.
for some pair
favorable for sampling.
IBM - AT .
The interpretation is as follows:
(ko,no )' then
R(m,p) > 0
If
and the urn is
Otherwise, it is optimal not to sample.
Formally, no
is define by:
Do
==
mine integer l
Boyce
•
(1973)
of parity
argues,
~
(8)
incorrectly,
that
these calculations
conducted with adequate precision in floating-point
how they can be performed in fixed-point
sizes.
and
Table I
the
arithmetic.
can not
be
arithmetic, and he shows
(integer) arithmetic for small urn
was produced using double precision floating-point arithmetic,
results
were
rechecked
using
single
precision
floating-point
Both gave identical results in all but one instance, a genuinely
close case which simply could not be resolved with the significant digits
available in single-precision ari.thmetic.
Accumulated round-off error was not
FAVORABLE URNS
a problem.
R(m,p)
-
6
A simple variant of (1) was used, based on the function
(m-p) ,
because,
unlike
(1),
it
can
be
programmed
to
S(IIl,p)
avoid
=
all
subtractions of positive floating-point numbers.
Boyce's paper contains many interesting results.
that
(m+1,p+1)
For instance, he shows
is an opti.al continuation point whenever
(lI,p) is, a fact
which helps make possible the concise summary shown in Table I:
1,458,081,000 urns of size
ko
-1
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
n0
1
2
5
10
19
30
43
60
81
102
129
156
187
222
259
298
341
386
435
486
539
596
655
718
783
850
921
996
1071
1152
1233
1318
1407
ko
no
32
33
34
. 35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
1498
1591
1686
1787
1888
1993
2100
2211
2324
2441
2560
2681
2806
2935
3064
3197
3334
3473
3614
3759
3906
4057
4210
4365
4524
4687
4850
5017
5188
5361
5536
5715
5896
n
~
ko
There are
54,000.
no
65 6081
66 6268
67 6457
68 6650
69 6847
70 7044
71 7245
72 7450
73 7657
74 7866
75 8079
76 8294
77 8513
78 8734
79 8959
80 9184
81 9415
82 9648
83 9883
84 10120
85 10361
86 10606
87 10851
88 11102
89 11353
90 11608
91 11867
92 12128
93 12391
94 12658
95-12927
96 13200
97 13475
ko
no
ko
no
ko
no
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
13752
14033
14316
14603
14892
15183
15478
15777
16076
16379
16686
16995
17306
17621
17940
18259
18582
18909
19238
19569
19904
20241
20582
20925
21270
21619
21970
22325
22682
23041
23404
23771
24140
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
24511
24884
25261
25642
26025
26410
26799
27190
27583
27980
28381
28782
29189
29596
30007
30422
30837
31258
31679
32106
32533
32964
33397
33834
34273
34716
35161
35610
36059
36514
36969
37430
37891
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
38356
38823
39294
39769
40244
40723
41206
41691
42178
42669
43162
43657
44156
44659
45162
45671
46180
46693
47210
47729
48250
48775
49302
49833
50366
50901
51440
51981
52526
53073
53622
54001+
TABLE I
e·
•
•
FAVORABLE URNS
3. Empirical studies .
7
An empirical assessment of the asymptotic rules, based
on the information in Table I, will now be described.
when
n. is large, the more accurate value
a
For greater precision
= 0.8399236757
will be used with
all calculations; all of the digits shown are significant.
A sequence
., ( 1 ), ., ( 2) ,
'1(n)~(bl(n),bu(n)],
will be said to be
"optimal at
if
n"
Le., if the favorable urns of size n are those for which
k < '1(n).
The "first-order asymptotic rule" (4) performs well; the sequence
is optimal at
single urn,
"-
f
n
about
75%
of the time.
The
of a given size, as "favorable."
"second-order asymptotic
1 2
of
the
time when
converge to
100%
optimal at
n
as
n
n
~~.
~
For the number of times
n
is optimal at
n
about
One can expect this percentage to
54,000.
appears to grow with
1/2
When it errors, it misclassifies a
rule" (5) performs much better; the sequence an / _.5
99.7%
an
1 2
an / _.5
n 1/2
. log 2n.
like
fails to be
When it errors, it
misclassifies a single urn, of a given size, as "not favorable."
There
is
a
simple
performance rate for
conjecture
which,
if
true,
explains
the
75%
and the reason that (5) performs so much better.
(4),
Consider the "empirical distribution functions"
FN(X)
=
N
-1 '\
N
•
L
1(an
1/2
-.5
~
,x~R,
bt(n) + x)
N~1,
(9)
n=1
where
1(')
denotes the indL:;<ltor function.
proportion of times
F (2)
N
- FN(O)
~ N that the sequence
F (1.5) - F (-.5)
N
N
1 2
an /
is optimal at
is the same proportion for the sequence
conjecture is that
[0,2], so that
n
Then
F
N
1
an / 2 -
is the
n.
.5.
And
The
has a limiting uniform distribution on the interval
FAVORABLE URNS
8
(10)
The evidence for the conjecture is strong.
the situation whenN = 54,000.
See Table II, which describes
The entry
I
f
= N[F (i/50) - F «i-l)/50)]
i
N
N
is the number of times
n ~ 54,000
that an 1 / 2 - .5 - b (n)
t
interval «1-1)/50,i/50].
Under a uniform distribution on
cell frequencies would be. 540
1
= o.
In fact,
is 1n the
[0,2], the expected
1 = 1,2,···,100, and would be
for
i f the cells for
1
=
0
and
10d
0
for·
are comb1ned, then the
chi-square goodness of f1t statistic beco.es
99 (f.-540)2
l
o
+
1
i=l
(f +f
f
o
x < 2, one still has
= 181
-540)
2
- 4.66,
Even though
F (-.0078) = 0
fails to be optimal at
n.
FN(O) > 0
n
and
FN(X) = 1
F (1.995) < 1.
and
N
in Table II is the number of times
1 2
an / -.5
(11 )
540
540
which describes an excellent fit.
some
100
54,000
that the sequence
As noted earlier, this count appears to
as
If so, then
i
f.
0
1
2
3
4
5
6
7
8
9
10
11
12
181
540
535
536
534
538
547
540
544
528
549
539
538
1
i f.
1
13
14
15
16
17
18
19
20
21
22
23
24
25
547
534
542
541
541
544
541
539
541
546
529
552
541
1 f
26
27
28
29
30
31
32
33
34
35
36
37
38
1
536
550
534
547
536
549
544
539
540
537
542
539
534
iff
39
40
41
42
43
44
45
46
47
48
49
50
51
540
540
540
538
534
541
540
536
542
539
532
538
541
The entry
N
~
for
i f
52
53
54
55
56
57
58
59
60
61
62
63
64
TABLE II
i
538
540
547
536
535
535
547
541
538
542
535
545
541
1 f.
1
65 534'
66 544
67 544
68 538
69 547
70 538
71 543
72539
73 544
74 539
75 535
76 544
77 548 .
i fi
78 546
79 537
80 548
81 536
82 537
83 554
84 536
85 545
86 531
87 537
88534
89 541
90 542
N ..
i f
91
92
93
94
95
96
97
98
99
100
i
531
535
542
545
534
537
535
538
543
364
00.
FAVORABLE URNS
"uniformly distributed modulo 1".
as
N ...
an
-
1/2
[0,2]
"fJc (n) "
wi th
c (n)
an
that the resulting sequence
is
F
N
N
In
F has a
N
(2) - FN(O)
-+
1
2
=
~
54,000, are caused because
1/2
c(n)
-
-1/2
.5 + fJc(n)
fJ
~
n.
There is no
and then try to fit
is optimal for
1
~
so
fJ
n
~
N,
Some results are shown in Table III,
The corresponding fJ
0.008890
By using any
n
refers to an iterated logarithm: "log log".
n -1/2
. log 2n.
is a positive
But an effective empirical approach is to
chosen as large as possible.
"log"
c(n)
n
converging to zero wi th
c(n).
make a reasonable choice, such as
where
if, and only if,
So
is slightly too small; what is needed in (6)
-.5
theory to guide the choice of
with
is
b (n)
t
See Hlawka (1984), pages 17,18, and 23.
of the 181 mistakes of (5), when
correction
,
.5
00 •
All
,
1/2
fact, it can be shown that it is uniformly distributed modulo 2.
limiting uniform distribution on
•
an
should be mentioned that the sequence
It
,
9
The apparent winner
satisfies the inequality
fJ S 0.008976.
(12)
in this interval together with
-1/2
2
c(n) = n
log n, the
third-order rule described in (6) will perform flawlessly for every urn of size
n
,
•
~
30,836, for a total of 475,475,702 urnsl
1/2
Choice of n
c(n)
1
log n
log2n
log2 n.log (n+2)
2
2
log n/log (n+2)
2
Maximum possible N
100
849
30,836
1495
17,305
TABLE III
FAVORABLE URNS
Even if
cautious
c(n) = n
-1/2
2
log n
with inequality (12).
10
goes to zero at the right rate, one must be
For there is likely to be a fourth-order term,
leading to a "fourth-order asymptotic stopping rule", which slightly perturbs
the calculations that give rise to (12).
There is evidence for this:
near the lower end of (12), the sequence an
optimal at an
n
~
54,000
1/2
a total of five times.
the total drops to three.
And for
-.5 + pn
-1/2
For
(
p
2
.
log n faJ.1s to be
•
Near the upper end of (12),
p = 0.009165, outside the range shown in
(12), the total is only two.
There is an analog to (9) which takes into account the additional term
"pc(n}".
As with (9), the empirical distributions apparently have a limiting
uniform distribution on [0,2].
The corresponding analog of Table II suggests a
much better fit near the endpoints of [0,2], and a comparable (excellent) fit
elsewhere.
REFERENCES
[1] Boyce, W. (1973). On a simple optimal stopping problem. Discrete Mathematics. 5
297-312.
[2] Chernoff, H. (1965). Sequential tests for the mean of a normal distribution
(discrete case). Ann. Math. Statist. 36 55-68.
iv
[3] Chernoff, H. and Petkau, A.J. (1976). An optillal stopping probleM for
dichotomous random variables. Ann. Prob. 4 875-888.
of
SUIBS
[4] Hlawka, E. (1984). The Theory of Uniform Distribution (A B Academic Publishers,
Berkhallsted) .
[5] Hogan, M. (1985). Comments on a problem of Chernoff and Petkau.
Unpublished.
[6] Shepp, L.A. (1969). Explicit solutions to sOlie problells of optillal stopping.
Ann. Math. Statist. 40 993-1010.
[7] Simons, G. (1986). The Bayes rule for a clinical-trials model for dichotomous
data. Ann. Statist. To appear.
[8] Simons, G. and Wu, X. (1986) On Bayes tests for p ~ 1/2 versus
p > 1/2:
Analytic approximations.
To appear in a volulle honoring Professor Herbert
Robbins.
•