A BAYESIAN ESTIMATOR OF
THE LINEAR REGRESSION MODEL
WITH AND~ICERTAIN INEQUALITY CONSTRAINT
W.E. Griffiths and A.T.K. Wan
No. 74 - May, 1994
ISSN
ISBN
0 157 0188
1 86389 191 9
A BAYESIAN ESTIMATOR OF THE LINEAR REGRESSION MODEL
WITH AN UNCERTAIN INEQUALITY CONSTRAINT
William E. Griffiths
Department of Econometrics
University of New England
and
Alan T.K. Wan
Department of Econometrics
University of New South Wales
March 1994
ABSTRACT
In this paper, we consider Bayesian estimation of the normal linear
regression model with an uncertain inequality constraint. We adopt a
non-informative prior and uncertainty concerning the inequality restriction
is represented by a prior odds ratio. Furthermore, we derive the sampling
theoretic risk of the resulting Bayesian inequality pre-test estimator and
numerically determine the optimal odds ratio according to a mini-max regret
criterion. An example taken from Geweke (1986) is used to illustrate our
methodology.
ADDRESS FOR CORRESPONDENCE : Professor William E. Griffiths, Department
of Econometrics, University of New England, Armidale, N.S.W. 2351,
Australia.
FAX : +61-67-733607;
PHONE : +61-67-732319
EMAIL : [email protected]
1.
INTRODUCTION
The problem of estimating the coefficients of a linear regression model
subject to inequality constraints has been a subject of interest for both
Studies from a sampling theory
sampling theorists and Bayesians.
standpoint (see, for example, Lovell and Prescott (1970), Judge and Yancey
(1981, 1986), Yancey and Bohrer (1992) and Wan (1993, 1994a)) examine the
properties of various inequality constrained and pre-test estimators and
the dependence of the pre-test estimators’ properties on test size.
Studies from a Bayesian standpoint include O’Hagan (19"/3), Davis (1978),
Geweke (1986, 198Ba, 1988b), Griffiths (1988), Barnett, Geweke and Yue
(1991) and Chalfant and Wallace (1992).
O’Hagan and Davis investigate
posterior quantities that result from a natural conjugate prior and single
The later
and multiple linear inequality restrictions, respectively.
papers are more computationally based and show how importance sampling can
be used to estimate posterior quantities for a number of more general
inequality restrictions, and noninformative priors.
One difficulty with inequality restricted estimation, whether it be
from a sampling theory or a Bayesian point of view, is that it assumes an
unfaltering belief in the prior information implied by the inequality
constraints. Few researchers are likely to be so stubborn that they would
continue to believe in the prior constraints, when faced with strong sample
information to the contrary.
Within the sampling theory framework,
allowing for the possibility that inequality restrictions might be
incorrect,
even if this possibility is remote, means using a statistical
test and
the resulting so called inequality pre-test estimator.
This
estimator has received considerable attention in the literature (see Wan
(1993, 1994a) for a list of references). However, to our knowledge, the
analogous Bayesian estimator has not been considered. Although Hasegawa
(1989) has considered a "Bayesian inequality pre-test’° estimator, this
estimator is defined by a rule which chooses between the Bayesian
inequality restricted and the unrestricted maximum likelihood estimators
based on the outcome of a statistical test. A more appropriate Bayesian
procedure is to allow for the (remote) possibility that the inequality
restrictions are incorrect by placing a nonzero prior probability on the
region of the parameter space not covered by the restrictions.
The
resulting Bayesian "inequality pre-test" estimator that recognizes this
nonzero prior probability will be a weighted average of the Bayesian
inequality restricted estimators from different regions of the parameter
space, with weights given by the posterior probabilities attached to each
region.
In this paper, we examine the nature and properties of this Bayesian
estimator. We adopt a non-informative prior with uncertainties concerning
the inequality hypotheses measured by a prior odds ratio defined over the
null and alternative hypotheses. In Section 2, for the case of a single
inequality constraint, expressions are derived for the posterior means
under each hypothesis and for the Bayesian inequality pre-test estimator
(the posterior mean that recognizes the hypothesis uncertainty).
In
Section 3 we compare the performance of this estimator with its sampling
theory counterpart; we derive and numerically evaluate its sampling
theoretic risk, and determine the optimal prior odds ratio according to a
mini-max regret criterion. Under this criterion, the Bayesian estimator is
1.7 times more efficient than its sampling theory counterpart. An example
taken from Geweke (1986) based on the Pindyck and Rubinfeld (1981) rental
data is used to illustrate the methodology for the more general case with
more than one inequality restriction.
2.
MODEL FRAMEWORK AND THE ESTIMATORS
Consider the standard linear regression model
y = X~ + c ;
c ~ N(o,mzl)
(1)
where y and ~ are n x 1 vectors; X is a n x k non-stochastic matrix of full
If we adopt a
rank; ~ is a k × I unknown coefficient vector.
noninformative prior p(~,~) ~ ~-i, and no prior inequalities, the marginal
posterior pdf for ~ is the multivariate t pdf
p(~ly) = c[v~-z + (~-~)’
S{~-~)]-(v*k}12
(2)
where S -- X’X, the location vector is the unrestricted maximum likelihood
estimator ~ = S-IX’y, v = n-k is the degrees of freedom parameter, ~r2 =
(y-X~)’ (y-X~)/v, the
precision matrix is [~rZS-1]-I, and c is the
normalizing constant.
2
Now let us introduce information about the coefficient vector /~, in
the form of a single inequality hypothesis:
(3)
where C’ is a 1 x k known vector and r is a known scalar. For a researcher
with the noninformative prior p(l~,~r) ~ ~r-1, and no special prior weight on
H: C’~ ~ r, the posterior probability of the inequality hypothesis being
o
true is
(4)
~(HolY) = f I(Ho) P(~IY)CI~
where I(Ho) is the indicator function which is equal to 1 when ~ satisfies
H and 0 otherwise. Given the noninformative nature of the prior pdf from
o
which ~(HolY) was derived, we can also view ~(HolY) as a measure of the
sample support for Hoo As popularized by Geweke (1986), we can estimate
~(HolY) by sampling (via computer) from the pdf in (2) and computing the
proportion
of observations
~ thatprior
satisfy
Ho.
Consider
a researcheron
whose
information
is such that H° holds
with certainty, but, within the region C’~ >- r, the prior for ~ is
noninformative. We write this prior pdf as
(5)
p(/S,o-~Ho) ¢x I(Ho) o"-~
The
corresponding marginal
posterior pdf for ~ is the truncated
multivariate t pdf
p(~ZI
I(Ho)P(~ly)
(6)
Y,Ho) =
~(HolY)
Given this posterior pdf, and a quadratic loss function, the posterior mean
E[~ I Y,Ho] is the Bayesian (inequality restricted) estimator for 8.
As
described by Geweke (1986) we can estimate this mean by sampling from
p(~ly) and computing the sample average of those observations on ~ that
satisfy H .
o
A useful analytical expression for E[~Iy, Ho] can be derived for the
case where we have an inequality on a single coefficient. Suppose that it
is the
element
of Cthe
that
is subject
to the inequality
that
C’first
= (I,0
..... O) and
restriction
becomes
E1 -> r. Inrestriction,
Appendix so
A it
is shown that E[~IIy,Ho] -- ~I’~o is given by
3
2 2 -]-(v-1)/2
all~mll + {r-~1)2/(v¢~ all)j
_
,
(7)
~I,Ho---- ~1 +
where m =
r(Z~-)v
a is the 1st diagonal element of S-1, the
F(_~) (~v) 1/2(v_i)’ 11
(~(r-~1)
probability P{HoIY) = 1 - ¢,~j measures the sample support for Ho, and
11
~ is the Student’s $ distribution function with v degrees of freedom. A
similar expression for estimating a normal mean is given by Hasegawa
O’Hagan gives an alternative expression in terms of beta
Note that, in (7), the unrestricted estimator ~1 is adjusted
functions.
(1989).
upwards to accommodate the restriction 61 -~ r. If ~i is much larger than r
(the sample strongly supports the restriction), the numerator in the
adjustment term will be approximately zero and the denominator
approximately one, leading to a negligible adjustment. For ~I much less
than r (the sample contradicts the restriction), both the numerator and the
denominator will approach zero, and the adjustment will depend on the
relative rates of convergence.
The posterior mean for elements in B that are not subject to inequality
restrictions is also derived in Appendix A. Using Sz as an example, its
posterior mean (inequality restricted estimator) is given by
~2,H0 = ~2 + h:1Ih12(~1,H0 ~I)
(8)
where h and h depend on the correlation structure of X’X.
II
See
12
Appendix A for their exact definitions. Note that the adjustment to the
unrestricted estimator depends on the correlation structure in X’X and the
difference between the restricted and unrestricted estimators for 6f
As mentioned in the introduction, one difficulty with the inequality
restricted estimator is that it assumes an unquestioning belief in Ho. If
~(HolY) is very small, we may want to question our prior information that
H is true. The analogous sampling theory situation is to (pre-)test the
o
-> r vs. Hi: ~i < r, set the level of significance ~
hypothesis
that
Ho: Bithan
to
something
greater
0 and to question our prior information if H° is
rejected.
This pre-testing procedure gives rise to the sampling theory
4
inequality pre-test estimator, which chooses between the estimators implied
by the null and the alternative hypotheses according to the outcome of the
statistical test.
A great
deal has been learnt about the sampling
properties of this estimator in recent years (see Wan (1993, 1994a) for
references).
The Bayesian alternative to this pre-testing procedure is to assign a
non-zero probability to H1.
Let this probability be denoted by P(H1) and
let P(Ho) = 1 - P(HI). Presumably, the prior odds ratio T = P(HI)/P(Ho) -~
1, as we think that H is at least as likely to be true as H. This
0
1
follows from the sampling theory procedure that sets the critical value, c,
such that c - r -~ 0. As soon as we recognize a pr~or~ that H may be true,
1
the appropriate "Bayesian inequality pre-test estimator", our counterpart
to the sampling theory inequality pre-test estimator, is
(9)
~ = P(Ho[Y)~H + P(HI[y)~H ,
0
1
where the elements in ~ are given in (7) and (8) and where
0
all~rn[l + (r -~l)2/(v~’2a~l)l-(v-1}/2
(10)
As an example of the other elements, the second one is given by
(11)
In equations (9), (I0) and (II) ~ is the Bayesian inequality restricted
1
estimator under HI, ~(H11y) = I - ~(HolY) measures the sample support for
H~, and P(Hjly), j = O, 1 are the posterior probabilities when P(Ho) ~
P(HI).
Noteon
that
view
~(HolY)
and ~(H~ly) as the posterior
probabilities
H° we
and can
HI when
P(H
o) = P(HI) (we do not discriminate
a pr~or~ between H
o
and H ).
I
Under these circumstances it seems reasonable
to define
~(H.]
, j = 0, I
P(H.~Iy) =
~(HolY)P(Ho) + ~(HllY)P(H1)
from which we obtain
S
(12)
(13)
P(Hl{y) ~(Hl{y) P(H1)
P(Ho{Y) ~(Ho{Y) P(Ho)
A more rigorous justification of (12) and (13), as limiting posterior
probabilities from proper prior pdfs is given in Appendix B.
One major difference between ~ and the sampling theory inequality
pre-test estimator is that the latter is a discontinuous function of the
data, while the former is not. Also, it can be seen from equations (7) to
(13) that if we regard H° and HI as a prlori equally likely (Le., P(Ho) =
P(HI) -- 0.S such that ¯ = I), then ~ = ~. The sampling theory counterpart
is to set c = r in which case the inequality pre-test estimator reduces to
However, if P(Ho) ~ P(HI), the sampling theory pre-test estimator and
~ are different. Considering the first element first we have, by
substituting (7) and (I0) into (9),
(14)
"[~(HoIY) "
Using (13), we can write (14) as
~1 ~1+ ~’allm [1
2 2 2"] -(v-1)/2
P(Ho[Y) { P(H1)1
This expression confirms that when P(H1) -- P(Ho)’- ~l = ~i" Also, when
P(H1) -- 0, P(Ho[Y) = i and ~l reduces to ~I,HO, the Bayesian inequality
restricted estimator under Ho. Furthermore, when P(Ho) > P(H1), we adjust
the unrestricted estimator upwards. This prior adjustment will be greater,
(a) the smaller the prior odds ratio P(HI)/P(Ho),
(b) the greater the value of P(HolY)/15(HolY)
[note that P(H1)/P(Ho) -< i
implies that P(Hol y)AS(H° I Y) -> I],
(c) the greater the value of ~rz, and
2
2 2 "1 -(v-1)/2
(d) the greater the value of 1 + (r-~I) /(Vall~r )J
.
This last term can be viewed as a measure of the strength of the sample
6
information. When r - ~i = 0, the sample information does not discriminate
-(v-l)/2
between H0 and HI, the term [i + (r-~l)z/(va:l~Z)] is maximized,
and the adjustment implied by the prior information is relatively large.
+ ~J
2
2 2 "l-(v-1)/2
declines, the impact
As Ir - ~ll increases, I
of the sample information is greater, and the prior adjustment away from ~i
is smaller.
Similarly, for other elements in ~, for example ~z’ we have
-(v-I)/z
z zl
P(H ly)[
mF1 + (r-@? z/va11;
F(HoJ[y)¯ 0/1
~2 = ~2 ÷ h-lh ;aII lZ 11 L
P(H1)I
(16)
Thus, the same kind of remarks can be made about ~2’ except that the
direction and magnitude of the adjustment will also depend on the
correlation structure of X’X.
Next, we examine the finite sample risk performance of the Bayesian
inequality pre-test estimator that we have derived.
3. THE PREDICTIVE RISK FUNCTION OF ~
Sampling theory properties do not have to be used to justify Bayesian
estimators to Bayesians.
It is nevertheless interesting to derive the
sampling theoretic risk of the Bayesian inequality pre-test estimator and
to compare this risk with that of its sampling-theory counterpart.
The
results given in the previous section primarily focus on the special case
where the constraint is in the form of 61 ~ r. To examine the risk of the
Bayesian inequality pre-test estimator under the more general constraint
C’~ ~ r given in (3), it is convenient to follow the procedure given in
Judge and Yancey (1981, 1986) and reparameterize (I), (3) and (S) as
y = HO + ~: ;
(17)
H : 0 -> r
(18)
o1
and
1
(19)
p(O, ~’) = I(Ho)/O"
respectively, where H = XS-I/2Q’; O = QS~/2B; r~
= r/hl; hl is the first
’
element of h’ = C’S-I/2Q and is assumed to be positive without loss of
7
generality; 0 is the first element of ~; and O is an orthogonal matrix
1
such that Q$-I/2c(C.S-Ic)-Ic.s-I/2Q. = (I 0"I.
O0
Using these results, the procedure described in the previous section,
1-~((r1-~)/~1
and noting that ~ = S-I/2Q’~ and P(HolY) T + (l-T) 1 it is straightforward to show that the
~i)/
Bayesian inequality pre-test
estimator of /~ can be written as
+ ~rmll + (r-~)2/V~2]-(v-1)/2
(20)
(k-l)
where ~ is the first element of ~, the unrestricted estimator of e; and
~(k-l) is a (k-i) × I vector containing the remaining elements.
Now, we consider the predictive risk function of ~ under squared error
loss defined as R(X~)= E[(X~ - X~)’(X~ - XB)]/~2.
We examine this
quantity rather than the risk of ~ itself so that
our results are
independent of the data matrix.
In terms of the
~ space, this is
equivalent to assuming orthonormal regressors.
Making use of (20) and the usual definition of risk under squared error
loss, the predictive risk function of the Bayesian inequality pre-test
estimator ~ can be written as
R(X~) -- E [(~-~)’ S(~-~)]/o"2
+ m2(1-T2) E
[n-w)
2
2 ~-(v-1)/2 ,~
(~-w) /q~
+ 2 m(l-z) El ql/2w[1 +
~
dq dw
dq dw { 21)
where w = (~1-ei)/~, q = v~.Z/o.2 and ~ = {r1-el)/~. When the inequality
restriction is correct, ~ -< O. The predictive risk of ~H’ the Bayesian
o
inequality restricted estimator under Ho, results when T = O.
To further examine the risk behavior of X~, we performed numerical
evaluation of (21) for v = 5, 15, 25 ..... 80; T = O, 0.I, 0.3, O.S, O.V, I;
and ~ -- [-6, 6]. The subroutines BETAI and GAMMQ from Press et at. (1986)
were used
to calculate the cumulative distribution function for the
Student’s f distribution and the Gamma function respectively. Numerical
integrations were carried out using the NAG (1991) subroutines DOIAHF and
D01AMF. These were incorporated into a FORTRAN program written by the
authors and executed on the VAXV610 and VAX6000 machines. Figures I and 2
illustrate some typical results.
For purposes of comparisons, the
predictive risk functions of the sampling theory inequality restricted and
pre-test estimators given in Wan (1993, 1994a) are also depicted in Figure
2. The inequality pre-test estimator was first derived by Judge and Yancey
(19B1) for the case in which ~2 is known.
Their results were later
generalized by Hasegawa (19B9) and Wan (1993, 1994a) to the ~z unknown
case.
From the diagrams, we observe the following features. First, the
predictive risk function of ~H is bounded and approaches that of ~ as @ ÷
o
-~, but it is unbounded as @ ÷ =. The predictive risk of ~ is bounded as
9
101
-> ~. When the direction of the inequality constraint is correct (Le.,
~ -~ 0),than
the the
predictive
risk
of ~ is smaller than that of ~, but it is
greater
risk of X~
H.
The risk of X~ reaches a maximum in the
0
region ~ > 0, then declines monotonically and approaches the predictive
risk of the unrestricted estimator as ~ increases. Also, for T close to i,
the predictive risk function of ~ more nearly approximates the predictive
risk of ~, and vice versa. These results concur qualitatively to the risk
performance of the corresponding sampling theory estimators.
However, contrary to the risk behavior of the sampling theory
inequality pre-test predictor, the predictive risk of the Bayesian
inequality pre-test estimator ~ is enveloped by the risks of X~H and X~
0
over the entire parameter space, except for a very small region near ~ =
0.
With the sampling theory estimators, there is always a finite, but
non-negligible range in 0, where the sampling theory pre-test estimator has
risk greater than the risk of both the unrestricted and the inequality
restricted maximum likelihood estimators. Regardless of the choice of T,
the intersection of the risk functions of X~ and X~ tends to occur in the
close neighborhood of ~ = 0, whereas the corresponding intersection point
for the sampling theory estimators is in the region ~ > 0. ~ is always
risk superior to ~H when ~ > 0, while the sampling theory inequality
o
pre-test estimator has smaller risk than the corresponding maximum
likelihood inequality restricted estimator only when @ is sufficiently
large.
If minimization of sampling theoretic risk is a major criterion for
estimator choice, we can ask the question whether there exists a prior odds
ratio T which is "optimal" in this or a related sense. This problem is
analogous to specifying an optimal critical value for the sampling theory
inequality pre-test estimator; it is explored in the next section.
4.
THE OPTIMAL ODDS RATIO FOR THE BAYESIAN INEQUALITY
PRE-TEST ESTIMATOR
In the pre-test literature, the mini-max regret criterion has been used
by many authors for selecting an optimal critical value of a pre-test (see
for instance, Brook (19"/6) and Wan (1993, 1994b)). Here we adopt this
I0
criterion to find an optimal prior odds ratio such that the maximum regret
of not being on the minimum risk boundary is minimized. Now, for 0 -~ ¯ -~
I, it is observed empirically that except in the close neighborhood of ~ =
O, min R(X~), the minimum of R(X~), is given by either R(X~[~=0) or
R(X~[~=I). That is, for almost all 0, minimum risk is achieved from either
the Bayesian inequality restricted estimator under H or the unrestricted
o
maximum likelihood estimator. Near @ = O, the risk of X~ can lie above or
below the minimum of R(X~) and R(X~ ), but, since this region is typically
o
very small, it does not influence the results.
For any given ~, the regret function is defined as
P.EG(~I) = R(X~) - min R(X~) .
(22)
The mini-max regret odds ratio, ~M’ is the odds ratio such that the maximum
regret over all values of @ is minimized. It is found that decreasing
reduces the maximum regret for ~ -~ O, but increases the maximum regret for
@ > O, and vice versa. Because of this monotonicity property, ¯ is the
M
value of ¯ which equalizes the maximum of the regret function in the
regions @ > 0 and @ - O. Values of ¯
for different degrees of freedom are
computed using the Golden Section Search Routine given in Press et
(1986), and are reported in Table 1.
From Table I, we observe that ¯ fluctuates only slightly over the
M
entire range of degrees of freedom, and is around 0.43 for moderate to
large degrees of freedom. In practical terms, this result means that if H
is believed to be more likely than HI,
o
but no further information is
available regarding the relative likelihoodof the two events, then from
the standpoint of sampling theoretic risk, an odds ratio of 0.43 will lead
to an optimal predictor according to a mini-max regret criterion.
For purposes of comparison, cM, the optimal critical value for the
sampling theory inequality pre-test estimator derived by Wan (1993, 1994b)
and the corresponding values of the regret function, are also given in
Table I. As with the Bayesian case, the optimal critical value is not very
sensitive to the degrees of freedom. Although neither the Bayesian nor the
sampling theory pre-test estimator strictly dominates each other in terms
of risk, at least for the cases that we have considered, ~ dominates c in
M
M
the sense that the maximum regret of using the Bayesian inequality pre-test
11
estimator
with
T --cT is
always smaller than the corresponding maximum
regret value
when
Mis chosen as the critical value for the sampling
M
theory inequality pre-test estimator.
Also, the difference in regret is
considerable. The Bayesian estimator is twice as good for small degrees of
freedom and 1.68 times better for large v.
Figure 2 illustrates these
results for the case of v = 15.
5.
AN EMPIRICAL EXAMPLE
The purpose of this section is to illustrate our methodology described
in Section 2 with an example taken from Geweke (1986) based on the rental
data provided by Pindyck and Rubinfeld (1981, p.44).
The estimated
equation attempts to explain rentals paid by students at the University of
Michigan and is given as follows :
(23)
+ 63(l-st)rt + 64stdt + 65(l-st)dt + ct’
Ytis--the
61 +
62strt
where Yt
rent
paid per
person; rt is the number of rooms per person;
d is the distance from campus in blocks; st is a sex dummy, one for male
t
and zero for female.
In what follows we present several estimates of posterior probabilities
and posterior means. These values were computed using the BAYES command in
SHAZAM (White ef al. (1993)). The number of antithetic replications for
Monte Carlo integration was 20,000 for each case. Corresponding estimates
in
Geweke
(1986) differ slightly because of differing random number
sequences.
The first hypothesis of interest is that where the coefficients have
the expected signs.
That is,
Ho: 6z-> 0, 63 >- 0, 64- 0, 6s-~ 0
We estimate the sample support for this hypothesis as ~(HolY) = 0.0477. A
question
naturallytoarises
Given for
thisthe
low
data-based
for Ho, dothat
we proceed
obtainis:estimates
6’s
assuming probability
H° holds
with certainty, or do we entertain the possibility that alternative signs
could be possible? If we are going to let the data persuade us to consider
other possible hypotheses, then we should not begin with the dogmatic prior
P(H0) = 1.
12
The other hypotheses that we consider are
H~: ~32 ~- 0, ~3 -~ 0, ~4 > 0 and ~5 > O;
H2:~2 -- O, ~3 >- 0, ~4 > 0 and ~s -~ 0;
H3:~2 -~ O, ~3 -~ 0, ~4 - 0 and ~s > 0;
The hypothesis H might be relevant if more distant housing has attractive
features
housing close
to campus
possess.
Hypothesis
could be that
an appropriate
modification
ofdoes
Hi if not
females
are not
prepared H
to 2
pay
for transport.
more distant
housing because
of security
problems
associated
withmore
public
Hypothesis
H3 might
be relevant
if there
was a
security problem in the campus neighborhood, but not with public transport.
Note that, because there is more than one inequality involved, we no
longer have 3ust two hypotheses, as was the case in the earlier sections of
= 16
this paper. In fact, with four inequality restrictions, there are
hypotheses, one for each different combination of inequalities. For our
illustration we are assuming all hypotheses other than Ho, Hi, Hz and H3
When we have multiple inequalities, and
have zero prior probabilities.
consequent multiple hypotheses, analytical expressions for the posterior
means, conditional on each H (see equations (7) and (I0)), are no longer
available.
However, these conditional posterior means can be routinely
estimated using SHAZAM’s BAYES command.
The Bayesian inequality pretest
estimator can then be computed from
(24)
where
3
P(H~ly) = ~(H]Iy)P(H])/ ~- ~(H, Iy)P(H,)
(25)
I=0
Table 2.1 contains the inequality restricted estimates under each of
the hypotheses and the sample based posterior probabilities ~(Hily). Note
that the hypotheses and corresponding estimates differ through the signs
they attach to ~ and ~s" The sample information strongly supports Hz
where ~4 > 0 and ~s ~ 0. The sum of the P(H|Iy) is greater than 0.99,
indicating little sample support for the other 12 regions to which we
attach zero prior probabilities.
Bayesian inequality pre-test estimates
are presented in Table 2.2 for three different sets of prior probabilities.
13
In Case I, we attach a prior probability of 0.8 to what we consider to be
the most probable hypothesis Ho, but we place some credence on the argument
that more distant housing is more attractive and so allocate a 0.2 prior
probability to HI, where ~4 > 0 and 65 > 0. Cases 2 and 3 introduce
nonzero prior probabilities for H2 and I-I3 and so allow for the two security
arguments that suggest females are more concerned with security than males.
In Case 2 the prior probability on H remains high (0.7), but it is
o
weakened
to
0.4
in
Case
3.
In Case I, because the data suggest Hi is even less likely than H° and
because we have excluded Hz, we find the sample information strengthens our
prior belief in Hsmall
the estimates
for (0.1)
64 and
65 are to
consequently
probability
attached
H2 in Case 2
o, andprior
negative. The
allows the data to dominate, leading to a posterior probability of P(Hzly)
= 0.718 and to signs for 64 and 65 that satisfy H2. This effect is more
pronounced when we increase the prior probability to P(H = 0.2.
2)
6.
CONCLUDING REMARKS
We have introduced a Bayesian inequality pre-test estimator as a
counterpart to the sampling theory inequality pre-test estimator.
Furthermore, the predictive risk function of the Bayesian inequality
pre-test estimator was derived and evaluated numerically. Although in many
ways, the predictive risk behavior of the Bayesian inequality pre-test
estimator bears resemblance to that of the analogous sampling theory
pre-test estimator, the two estimators differ quali.tati.vely in terms of the
risk comparisons with their corresponding component estimators.
Our
numerical calculation shows that over almost the entire ~ space,
the
predictive risk of the Bayesian inequality estimator is enveloped by the
risk of Bayesian inequality restricted and unrestricted predictors. On the
other hand, there is always a non-negligible region in ~ in which the
sampling theory inequality pre-test estimator has predictive risk higher
than the predictive risk of both the unrestricted and inequality restricted
estimators. We have also tabulated optimal odds ratios for the Bayesian
inequality pre-test estimator according to a mini-max regret criterion. We
find that the pptimal odds ratio is approximately constant for moderate to
large degrees of freedom. Also, the Bayesian pre-test estimator dominates
14
the sampling theory pre-test estimator in the sense that the maximum regret
¯ corresponding to the optimal odds ratio uniformly dominates the
corresponding maximum regret value when the sampling theory pre-test
estimator is applied with the optimal critical value. A numerical example
is given to illustrate our methodology. In particular, we show how the
methodology is modified for multiple inequality restrictions and how the
specification of prior probabilities representing the uncertainty of
hypotheses can affect the final estimates. Finally, our risk results were
confined to a single inequality restriction; whether or not these results
also hold for non-orthonormal regressors and multiple inequality
restrictions still needs pursuing.
Appendix A: Derivation of Inequality Restricted Estimator
For the posterior mean of BI’ the parameter subject to the inequality
constraint, we have
where p(~11y) is the univariate t distribution with location ~I’ degrees of
freedom
v and precision
(a~l~rZ)-1 where
a211 is the first diagonal element
S-I. Making
the transformation
81 = ~1
+ ~r a t where t is a
of
11
standardized t random variable and where we define t’ = (r-~
obtain
E[~x [Y’H°] =~(H-I° [y) ~’ (~I+ ~allt)p(t)dt
15
a11,
we
-(v+l)/2
dt
~r allC’V I
-(v-1)/~
= ~I + ~(HolY)(V_l) i +
N2 2 l-(V-1)/2
+
= ~I + an~ m[l
where c’ is the normalizing constant for the standardized t distribution
and m = c’v/(v-1).
For the posterior mean of a parameter not subject to the inequality
constraint, say ~z’ we have
_ 1 ~ ~z I(Ho)P(~llY)P(~zI~I’Y)P(~ ..... ~kI~’~z’y)cl~
~(HolY)
-
~(Hol y) r
~
~{H0lY) r
The elements h
II
and h are elements from the precision matrix for the
12
bivariate t pdf p(~1,~zly). To obtain them, let V = S-i = (X’X)-I and let
V. be the {2x2) submatrix obtained from the first two rows and columns of
V. Let H = V-1
¯ " Then, h and h are the elements in the first row of H.
Ii
12
16
Appendix B: Derivation of a Limiting Posterior Odds Ratio
In the context of the model in the paper, suppose that the prior pdf
for ~ is multivariate t and that ~ is a priori independent of ~ whose prior
pdf is inverted gamma. That is,
where
p(~) = c1[vI + (~-b)’A(/~-b)]-(k+P
i )/2
p(~) = c ~-Iv2 +I) exp(-v s2/2~z)
2
22
In these expressions cI and cz are normalizing constants and the prior
b, A, v2 and sz. This prior was suggested by Dickey
parameters are vl,
(197S) as an alternative to the more common natural conjugate prior which
has
some
Judge etthat
al. (198S,
p.lll)
for undesirable
details. Asproperties.
A ÷ 0 andSee
v2 Dickey
÷ 0 this(197S)
prior or
becomes
used in
the
p(~, ~)posterior
~ ~-i probabilities
Thus, of interest
usvwill
be the limiting
formpaper,
of consequent
as A ~ 0to
and
2 ÷ 0.
The hypotheses introduced in Section 2 were
H:
o
C’~ -> r
I
and H:
C’~ < r
Suppose that the prior mean b is chosen such that
fpC~)d~ = f p(~)d~ = O.S
{Ho~
{HI}
That is, t.he prior pdf for ~ is centered at the boundary of the two
hypotheses. A procedure for selecting such a b is outlined below. For
prior pdf’s for 6, conditional on H° and H~, we take truncated multivariate
t distributions, obtained by truncating p(~) at the hypothesis boundary.
That is,
-(k+P1)/2
p(~~Hj) = 2c1[PI + (~-b)’A(~-b)]
I(Hj) j = 0, 1
17
where I(Hj) is an indicator function equal to 1 when ~ is such that Hj is
satisfied. For ~, we assume
Our objective is to find expressions for the posterior probabilities
for each hypothesis
P{H])p(y[Hj)
P(Hj[y)- P(Ho)p(y]Ho) + P{H1)p(y[H1)
j=0,1
given prior probabilities P(Ho) and P(H1), and where
(n+12)/2-1
= 2ClC2(2~)-n/zF[(n+vz)/2] 2 Z Qj
where
-(k+P )/2
Substituting for f(y[Hj) in the expression for P(H][y) yields
P(Hj[y) = P(Ho)Q
P(Hj)Qj
° + P(HI)Q1
For the noninformative prior implied by A ÷ 0 and v2 ÷ O,
where c~ is a collection of constants that are common to both Qo and QI"
Making this substitution for Q] yields the result in equation (12), namely
~(H] [ y)P(H~)
P(H][y) =
~(HoIY)P(Ho} + ~(HllY).P(H1)
18
The remaining task is to indicate how the prior p(/3) can be chosen so
that it is centered at the hypothesis boundary. For this purpose we can
use the transformations introduced in Section 3 of the paper. That is, we
begin with a prior pdf for e that has the kernel
p(19) ~ IvI + (O-t)’A*(19-t)]-(k+l~ )/2
I
being determined by the hypothesis
, tk)’, rl
where t -- (rI, tz, t3 ....
boundary (see Section 3), and t2, t3, ..., tk being arbitrary prior means.
Then, using notation introduced in Section 3, we define ~3 = S-I/2Q’~ so
that the quadratic form in the kernel of the prior pdf becomes
[e-t)’A*[e-t) = [Qsl/Z~-t)’ A~[QSI/2~-t)
-- [J~-S-112Q’ t)’ S1/2Q’ A~Qs1/2[~-s-II2Q’ t)
= (B-b)’ A(~-b)
where A = SI/ZQ’A*QS1/z and, to make the prior pdf symmetric around the
hypothesis boundary we choose the prior mean b = S-I/2Q’t.
19
~=0.1
~=0.3
X=0.5
X=0.7
Figure h Predictive risk functions of ~ for n = 20 and k = 5
~ (mini-max)
inequality restricted
pre-test (mini.max)
Figure 2: Risk comparisons between ~ and sampling theory inequality
pre-test predictors for n ffi 20 and k ffi 5
20
Table h Optimal odds ratio and critical values according to the
mini-max regret criterion and the corresponding maximum regret
values
v
5
I0
15
20
25
30
35
40
45
50
55
60
65
70
75
80
~M
0.516
0.460
O. 445
O. 438
O. 434
O. 432
0.430
0.428
0.427
0.427
0.426
0.426
0.426
0.425
0.425
0.425
regret(~M)
0.189
0.224
O. 234
O. 239
O. 242
O. 243
0.245
0.246
0.247
0.247
0.248
0.248
0.249
0.249
0.249
0.250
C
-i
-I
-1
-i
-1
-i
-1
-i
-i
-i
-i
-i
-i
-i
118
122
124
125
126
127
127
127
127
127
127
127
127
128
-I 128
-i 128
regret(c~)
0 410
0 414
0 416
0 418
0 418
0 419
0 419
0 419
0 419
0 419
0 419
o
0
0
0
0
420
420
420
420
420
regret(z~) and regret(c~) denote the maximum value
of the regret function corresponding to TM and c~
respectively.
21
Table 2.1: Bayesian inequality restricted estimates
Hypothesis
H
0.0477
37.843
136.77
H1
H2
H
0.0244
0.9179
0.0014
36.104
38.104
42.182
104.57
102.43
133.61
o
123.57
99.76
123.26
92.15
-0.926
3.562
3.530
-i.011
-1.2016
0.2475
-i. 1933
0.2764
Table 2.2: Prior and posterior probabilities and Bayesian inequality
pre-test estimates
Hypothesis
Case 1
Ho
H1
H2
H
0.8
0.8867
0.2 0. 1133
0.0
0.0
0.0
0.0
37. 646
133.12 120.87 -0.4171 -1.0375
0.7
0. I
0. i
0. I
0.2618
0.0191
0.7180
0.0011
38. 002
I11.48
122.86
2.3612 -1.1662
0.4
0.2
0.2
0.2
0.0919
38. 038
105.67
122.70
3.1151 -1.1582
Case 2
H
Case 3
H
0.0235
0.8833
0.0013
REFERENCES :
Barnett, W.A., J. Geweke and P. Yue (1991), "Seminonparametric Bayesian
estimation of the asymptotically ideal model: The AIM consumer demand
system" in W.A. Barnett, J. Powell and G.E. Tauchen, eds.,
Nonparamefric and Semiparamefric Methods in Econometrics and
Statistics:
Proceedings of" the Fifth International Symposium in
Economic Theory and Econometrics, Cambridge: Cambridge University
Press, 127-173.
Brook, R.B. (1976), "On the use of a regret function to set significance
points in prior tests of significance", Journal of the Arnertcan
Stafisfical Iissoc~ation 71, 126-131.
Chalfant, J.A. and N.E. Wallace (1992), "Bayesian analysis and regularity
conditions on flexible functional forms: Application to the U.S. motor
carrier industry" in W.E. Griffiths, H. Ltitkepohl and M.E. Bock, eds.,
ReadinF, s in Economefric Theory and Pracfice: A I[olume tn Honor of
GeorEe JudEe, Amsterdam: North-Holland, 159-174.
Davis, W.W. (1978), "Bayesian analysis of the linear model subject to
linear inequality constraints", Journal of fhe Amerfcan Sfaf£sfical
Associafion 73, 573-579.
Dickey, J.M. (1975), "Bayesian alternatives to the F-test and least-squares
estimates in the normal linear model", in S.E. Fienberg and A. Zellner,
eds., Sfudies ~n Bayesian Economefrics and Sfafis~cs, Amsterdam:
North-Holland, 515-554.
Geweke, J. (1986), "Exact inference in the inequality constrained normal
linear regression model", Journal of Applied Economefr~cs i, 127-141.
Geweke, J. (1988a), "Exact inference in models with autoregressive
conditional heteroskedasticity", in W. Barnett, E. Berndt and H. White,
eds., Dynamic Economefric Model ~nE: Proceedin~,s of fhe Third
Infernafional Symposium in Economic Theory and Econornefrics, Cambridge:
Cambridge University Press, 73-104.
Geweke, J. (1988b), "The secular and cyclical behavior of real GDP in 19
OECD countries", Journal of Business and Economic Sfaffsf£cs 6,
479-486.
Griffiths, W.E. (1988), "Bayesian Econometrics and how to get rid of those
wrong signs", Review of Marketing and Agricultural Economics 56, 36-56.
Hasegawa, H. (1989), "On some comparisons between Bayesian and sampling
theoretic estimators of a normal mean subject to an inequality
constraint", Journal of the Japan Statistical Society 19, 167-177.
Judge G.G., W.E. Griffiths, R.C. Hill, H. Ltitkepohl and T.C. Lee (1985),
The Theory and Practice of Econometrics, 2nd ed., New York, Wiley.
Judge, G.G., and T.A. Yancey (1981), "Sampling properties of an inequality
restricted estimator", Economics Letters 7, 327-333.
Judge, G.G., and T.A. Yancey (1986), Improved Methods of Inference ~n
Econometrics, North-Holland, Amsterdam.
Lovell, M.C. and E. Prescott (1970), "Multiple regression with inequality
constraints: Pre-testing bias, hypothesis testing and efficiency",
Journal of the American Stat~.sf~cal ~ssoc~at~on 65, 913-92S.
Numerical Algorithm Group Ltd. (1991), "NAG FORTRAN Library Manual, Mark
15, Vol. I", NAG Inc., U.S.A.
O’Hagan, A. (1973), "Bayes estimation of a convex quadratic", B£ometr£ca
60, 565-571.
Pindyck, R.S. and D.L. Rubinfeld (1981), Econometric Models and Economic
Forecasts (2nd edition), McGraw-Hill, New York.
Press, W.H., B.P. Flannery, S.A. Teukolsky and W.T. Vettering (1986),
Numerical Recipes: The ~ of Scientific Computing, New York, Cambridge
University Press.
Wan, A.T.K. (1993), "Inequality restricted and pre-test estimation in a
mis-specified Econometric model", Ph.D thesis, Department of Economics,
University of Canterbury.
Wan, A.T.K. (1994a), "The sampling performance of inequality restricted and
pre-test estimators in a mis-specified linear model", Australian
Journal of Statistics, forthcoming.
Wan, A.T.K. (1994b}, "The optimal size of a preliminary test for an
inequality restriction in a mis-specified regression model", mimeo.,
Department of Econometrics, University of N.S.W.
White K.J., S.D. Wong, D. Whistler and S.A. Haun {1993}, SHAZAM user’s
reference manual: version 7.0.
24
Yancey, T.A. and R. Bohrer (1997-), "Risk and power for inequality pre-test
estimators: general case in two dimensions",
in W.E. Griffiths,
H. Ltitkepohl and M.E. Bock, eds., Readings in Econometric Theory and
Pracf~ce: A Volume ~n Honor of George Judge, Amsterdam: North Holland,
33-55.
WORKING PAPERS IN ECONOMETRICS AND APPLIED STATISTICS
~o2.~~ P.D%eaa Red~. Lung-Fei Lee and William E. Griffiths,
No. I - March 1979.
~minQ Ro~. Howard E. Doran and Rozany R. Deen, No. 2 - March 1979.
~ A/ate on ~ Za~ ~a~imntan ~n an ~ @aaan
William Griffiths and Dan Dao, No. 3 - April 1979.
D.S. Prasada Rao, No. 5- April 1979.
Red2/: M ~9ixw.az~daa o~ ~ o~ @~/~. Howard E. Doran,
No. 6 - June 1979.
" Reo~ Roda~a. George E. Battese and
Bruce P. Bonyhady, No. 7 - September 1979.
Howard E. Doran and David F. Williams, No. 8 - September 1979.
D.S. Prasada Rao, No. 9 - October 1980.
~u~ ~o2~ - 1979. W.F. Shepherd and D.S. Prasada Rao,
No. I0 - October 1980.
~aZa~Yi~ne-Yeaieaand ~aoaa-YecZLaa ~oiat~ $atimaZea ~aodur_Zh~a Yuaciiaa
aWiA ~o~aadNeq~Romu~ ~/~. W.E. Griffiths and
J.R. Anderson, No. II - December 1980.
Howard E. Doran
and Jan Kmenta, No. 12 - April 1981.
~inai Oadea ~u~on2~~~~. H.E. Doran and W.E. Griffiths,
No. 13 - June 1981.
Pauline Beesley, No. 14 - July 1981.
Yo~ ~oia. George E. Battese and Wayne A. Fuller, No. 15 - February
1982.
26
~)~. H.I. Tort and P.A. Cassidy, No. 16 - February 1985.
H.E. Doran, No. 17- February 1985.
J.W.B. Guise and P.A.A. Beesley, No. 18 - February 1985.
W.E. Griffiths and K. Surekha, No. 19- August 1985.
~n~ ~a~. D.S. Prasada Rao, No. 20 - October 1985.
H.E. Doran, No. 21- November 1985.
~ae-Fex~ gatim~~tAie~ru~ ~ede/. William E. Griffiths,
R. Carter Hill and Peter J. Pope, No. 22 - November 1985.
William E. Griffiths, No. 23 - February 1986.
~~ ~anduntinn ~unct2nn~o~ ~ane/ ~ata: ~tA ~p~tathe
Ma~ ~a~ ~~. T.J. Coelli and G.E. Battese. No. 24 February 1986.
~aadu~u~ I!~Mggn2-q~~ata. George E. Battese and
Sohail J. Malik, No. 25 - April 1986.
George E. Battese and Sohail J. Malik, No. 26 - April 1986.
~a~. George E. Battese and Sohail J. Malik, No. 27- May 1986.
George E. Battese, No. 28- June 1986.
Nam/~.n~. D.S. Prasada Rao and J. Salazar-Carrillo, No. 29 - August
1986.
~un~ ~ex~en 9n~ 8at2~~na~M~(1) 8anna~edet. H.E. Doran,
W.E. Griffiths and P.A. Beesley, No. 30 - August 1987.
William E. Griffiths, No. 31 - November 1987.
27
Chris M. Alaouze, No. 32 - September, 1988.
G.E. Battese, T.J. Coelli and T.C. Colby, No. 33- January, 1989.
8~n~ Ze ~ gcon~-1~ide ~ade!2~. Colin P. Hargreaves,
No. 35 - February, 1989.
William Griffiths and ~orge Judge, No. 36 - February, 1989.
~%e~anag~o~ 8~ ~oiea Dan~ Druma~Z. Chris M. Alaouze,
No. 37 - April, 1989.
~ ~ 1~aZen M22~ ~r~. Chris M. Alaouze, No. 38 July, 1989.
Chris M. Alaouze and Campbell R. Fitzpatrick, No. 39 - August, 1989.
~a. Guang H. Wan, William E. Griffiths and Jock R. Anderson, No. 40 September 1989.
o~h~ Re~ Opt. Chris M. Alaouze, No. 41 - November,
1989.
~ ~he~ ozui ~~ ~. William Griffiths and
Helmut Lfitkepohl, No. 42 - March 1990.
Howard E. Doran, No. 43 - March 1990.
~!a2n~FheXa~ ~iAtea~ g~~uA-~apa~. Howard E. Doran,
No. 44 - March 1990.
Howard Doran, No. 45 - May, 1990.
Howard Doran and Jan Kmenta, No. 46 - May, 1990.
and $a/~ ~nizea. D.S. Prasada Rao and E.A. Selvanathan,
No. 47 - September, 1990.
28
~con~ YZudZe~uiZha ~~ o~ Nea~ ~ng/~. D.M. Dancer and
H.E. Doran, No. 48 - September, 1990.
D.S. Prasada Rao and E.A. Selvanathan, No. 49 - November, 1990.
~ ~n ~ gC~. George E. Battese,
No. 50 - May 1991.
Y~leann~ gannn~o~ ~~ ~oa~n. Howard E. Doran,
No. 51 - May 1991.
Fe~ Non-N~ ~ed2.2~. Howard E. Doran, No. 52 - May 1991.
~~ M~v~. C.J. O’Donnell and A.D. Woodland,
No. 53 - October 1991.
A(~ Yer_~. C. Hargreaves, J. Harrlngton and A.M.
Siriwardarna, No. 54- October, 1991.
Colin Hargreaves, No. 55 - October 1991.
~Jx~~a 90z&i9 ~vune~n $na~. G.E. Battese and T.J. Coelli,
No. 56 - November 1991.
2.0. T.J. Coelli, No. 57 - October 1991.
Barbara Cornellus and Colin Hargreaves, No. 58 - October 1991.
Barbara Cornellus and Colin Hargreaves, No. 59 - October 1991.
Duangkamon Chotlkapanich, No. 60 - October 1991.
Colin Hargreaves and Melissa Hope, No. 61 - October 1991.
" " ~ec~. Colln Hargreaves, No. 62 - November 1991.
O&a~ ~ EunA~e. Duangkamon Chotlkapanich, No. 63 - May 1992.
29
V~. G.E. Battese and G.A. Tessema, No. 64- May 1992.
~~ ~u~2.hO%~. C.J. O’Donnell, No. 65- June 1992.
~rmp~. Guang H. Wan and George E. Battese, No. 66 - June 1992.
~nz.on~n~fl~ 0% ~~ia, 1960-1985: ~ ~nAog~b~o~
Ma. Rebecca J. Valenzuela, No. 67 - April, 1993.
~o2~ D% tAe ~.F. Alicia N. Rambaldi, R. Carter Hill and
Stephen Father, No. 68 - August, 1993.
@a~izi~ ~recZa. G.E. Battese and T.J. Coelli, No. 69 - October,
1993.
Sex~ ~~. Tim Coelli, No. 70 - November, 1993.
~h~in W~ ~ ~. Tim J. Coelli, No. 71 December, 1993.
G.E. Battese and M. Bernabe, No. 72 - December, 1993.
Getachew Asgedom Tessema, No. 73 - April, 1994.
$aequz/iZ~ ~o~. W.E. Grlfflths and A.T.K. Wan, No. 74 May, 1994.
3O