CONSTRAINED LINEAR MODELS:
ASAS IMPJ..EMENTATION
by
Thomas M. Gerig and A. Ronald Gallant
Institute of Statistics
Mimeograph Series No. 899
Raleigh - November 1973
,:r.~~"'--'
;'-"'-
Cons trained Linear -Models:- A.
-S,AS
Implementati on
by
Thomas M•. Gerig and A. Ronald Gallant
ABSTRACT
This technical report contains the documentation of a
Statistical Analysis System
(SAS)
implementation of the
Constrained Linear Models estimation method reported in
Mimeo Series Number 814.
A monograph briefly describing
the computational method and relevant theory is included
to make this report self contained.
-'J'HillGLM PRQCEDlJRE
The CLM procedure is designed to analyze linear models whose parameters
are subject to linear equality constraints. No rank conditions are
imposed on the X matrix or on the restriction matrix. The methods
applied are discussed in the attached monograph.
For a given model, sets of linear parametric functions can be estimated
and linear hypotheses can be tested. The procedure automatically checks
to see if linear parametric functions are estimable or specified, and
checks all constraints and hypothesis equations for consistency.
OUTPUT
For each model entered, the eLM procedure gives model dimensions, error
variance information, restrictio~ ~pecification, and parameter estimates
including standar.derro~s and an indication of whether the estiinates are
unbiased or specified by the restrictions. The variance-covariance or
correlation matrix of the estimated parameters may be requestedEach request for the estimation of linear parametric functions results
in the printing of the combining matrix and the estimates with standard
errors and an indication of whether the estimates of the linear func~
tions are unbiased or specified by the restrictions. The variancecovariance matrix or correlation matrix of the estimated linear func·"'·
tions may be requested.
Each request to test a linear hypothesis results in the estimation of
the associated linear functions (with output as described in the previous
paragraph) in addition to the printing of the hypothesized values of the
linear functions and the appropriate F value, degrees of freedom, and
p-value. The variance-covariance or correlation matrix of the estimated
linear functions may be requested-
2
THE PROCEDUREu CIM STATEMENT·
The PROCEDURE CIM statement is of the form
PROC CIM
<DATA=data set name>
PROCEDURE INFORMATION STATEMENTS
VARIABLES Statement
The VARIABLES statement tells whichSAS variables will be used in the
analysis to follow and only these variables will be included in the
internal computation of the sums of squares and cross products matrix.
DROP Statement
The DROP statement serves the same purpose as the VARIABLES statement
except to specify which BAS variables not to include.
BY Statement
The BY statement works as explain~d in the SAS User's Guide.
CLASSES and ID Statements
The CLASSES and ID statements may not be used with this procedure.
PARMCARDS Statement
There are five· types of parameter cards: MODEL, CONSTRAINTS (or.
RESTRICTIONS), ESTIMATE, TEST, and OUTPUT•.
Each MODEL statement must be immediately followed by a CONSTRAINTS
statement (if there are any constraints).
The ESTIMATE statement causeS linear parametric functions to beestimated (under the current model) and the TEST statement causes linear
hypotheses to be tested (in the framework of the current model).
There is no limit to the number of ESTIMATE or TEST statements that
can be used. The OUTPUT statement causes predicted values and
residuals to be saved in an output file.
3
PARAMETER CARD SPECIFICATION
MODEL, CONSTRAINT, ESTIMATE, and TEST Statements
MODEL, CONSTRAINT, ESTIMATE, and TEST statements are of the form
MODEL numeric variable =: numeric variable <additional numeric variables>
NOINT
<!NOMEAN> <VA:H> <CORR> #
CONSTRAINTS
<linear combination of independent numeric variables
=:
constant>
<linear combination of independent numeric variables
=:
constant> #
ESTIMATE
<linear combination of independent numeric variables
<linear combination of independent numeric variables
<!VAR> <CORR> #
TEST
<linear combination of independent numeric variables
<linear combination of independent numeric variables
<!VAR> <CORR> #
=:
?
>
=:
?
>
,
=:
constant> #=
=:
constant>
Options and Parameters
NOMEAN
NOINT
When this option is absent, the procedure will
include an intercept term in the model. When
NOMEAN or NOINT is specified, no intercept will
be included.
VAR
This option causes the variance-covariance matrix
of the appropriate set of estimates to be printed.
CORR
This option causes the correlation matrix of the
appropriate set of estimates to be printed.
OUTPUT Statements
OUTPUT statements are of the form
4
OUTPUT OUT=data set 2
<PREDICTED predicted variable name>
<RESIDUALS residual_variable_uame> #
where predicted_variable_name and residual variable name are valid SAS
names.
An OUTPUT statement refers to the current MODEL statement. The OUTPUT
statement tells SAS to build a new data set to be named data set 2
containing all the variables in the data set to which CLM is-applied
in addition to other new variables named in the OUTPUT statement. If
PREDICTED and a SAS variable name are given, the predicted values for
the current model will be included in the new dataset under the given
SAS variable name. Similarly,' if RESIDUALS and a SAS variable name are
given, the residuals for the current model will be included in the new
data set under the given SAS variable name.
JOB CONTROL STATEMENT REQUIREMENTS AT TUCC
!!job name
JOB xxx.yyy.zzz,p~ogrammer name
IlsTEPI
EXEC SAS
IlsAs. STEPLIB
DD DSN=NCS. ES. B4139. GALLANT .SASLIB, DISP=SIffi
II
DD DSN=NCS.SAS,DISP=SIffi
!!SAS.SYSIN
DD *
SAS
···
statements and data cards
5
EXAMPLE .
The following example is discussed in the attached monograph.
//CLMEG
JOB xxx.yy.zzzzz,programmer name
IlsTEPl
EXEC SAS
_- .
IlsAs.sTEPLIB DD DSN=NCS.ES.B4139.GALLANT.~SLIB,DISP=SHR
DD DSN=NCS.SAS,DISP=SHR
!!SAS.SYSIN
DD *
INPUT Y 5-10 X 15-20;
BETA 1=0; BETA 2=0; BETA 3=0; BETA 4=0; BETA 5=0;
IF X<:L2 THEN GO TO A; IF-X>=12 THEN GO TO B;A: BETA 1=1; BETA 2=X; BETA 3=X**2; RETURN;
B: BETA:=4=l; BET~5=X; RETURN;
CARDS;
0.46
0·50
0.47
1·50
0·56
2·50
II
0·99
70·50
1.04
71.50
PROC CLM;
PARMCARDS;
MODEL Y = BETA 1 BETA 2 BETA 3 BETA 4 BETA 5 / NOINT :#:
CONSTRAINTS
BETA 1 + 12*BETA 2 + 144*BETA 3 - BETA 4 - 12*BETA_5 = 0
BETA-2 + 24*BETA-3 - BETA 5 =-0 #=
ESTIMATE BETA 1 + 12*BETA 2 + 144*BETA 3 = ?
BETA-2 + 24*BETA-3 = ? #=
TEST
BETA 5 = 0 #=
OUTPUT OUT=DATA 2 PREDICTED YHAT RESIDUAL EHAT #=
/~OC PRINT DATA;"DATl\_.2;
,
i
II
j
NUMBER OF PARAMETERS IN THE
NUMBER OF RESTRICTICNS
~U~3ER
MAXIMUM
NUM~~R
=
CF INDEPENDENf RESTRICTIONS
OF LINEARLY
INOEPENDE~l
ESTIMABLE lINEAR FUNCTlCNS =
MAXIMUM NUMDER OF UNSPECIFIED LINEARLV INDEPENDENT ESTIMABLE
..•
***.**.*•••**.*••••*** ••*.***.*.*•• *.~ *.*.*••••••••• *••••*••••••••••
•
•
ERROR
VARIANCE
•
•.*••**••••••*.*****.*••••*•••••*•••••••••••••••••*•••••••••••••••••••••
ERROR SUM OF SQUARES
=
ERROR VARIANCE ESTIMATE =
3.791?4350D-02
5.49455S8Co-04
DEGREES OF FREEDOM ASSOCIATED WITH THE
ESTI~ATE
OF ERROR VARIANCE
c
69
5 T Ii
r I S.
···lOI·~:';lUtJc~~, . ;'at8£1t.·6.•..
CA'T\
I
h
I
·.•
..
.
THE
...
*
*****.*"'***.*******••***••••*
i
,.,,
II
I
.
RESTRI
..
I
••
..•••••**"'*.*•••"'••*.*"'••••••••••••••••• *••••••••••••• *•••••••••••••••••
. ..
RR • B '" R
,
tl
I,
I
I
.!
I
I"
I
i
II
,
' .•
**•••••• *."'•••••"'•• "'*••••**••••"'•••*.*••••••"'••••••••••••••••••••"'."'••
.•
LEFT H~ND SIDE. RR, OF ThE RESTRICTIONS RR*B6R
I
I
•
•
•
*•••••••• "'*."'••••
*••• *•••**••••••••••••••"'••••••"''''.'''.*••••••••••••••••
I
8ETA_l
BETA_2
BETA_3
BETA...4
II
6ETA_5
I
ROW
1
1.0000
ROW
2
0.0
12.0000
144.0000
1.0000
24.0000
-1.0009
'0.0
*••••••• *.*••**••**.*•••*••**••••• ••••• *••••••••• *•••••*
•*'
~
.••***••••••••••••
RIGHT HANO :iIOE, R, OF THE RESTRICTIONS RR.B=R
-12.coeo
-1.0000
**••
••
~
*~*
••••••••• *•••••••••••••*••• ***•••••*•••• *.*•••••••
i
,'
,,
RCW
1
0.0
RCW
2
0.0
I
!
,
!
'!.
I
S TA T
s·~ "c ~ l
II' ".'
**u**••**.**.**.********.*.**********
..
..
•
. '
*****
.•
.
*••••••
PARAMETER EST IMATES
,.
.
.
•••*•••*••••*.*••••••••••••••••••••••••••••••••••••••• *•••*•• *•••• **.*
VARIABLE
--------
COEFFICIENT
-----------
STANDARD ERROR
...
------'------
-----
III".
I
i'" .
I"
I
I
I
BETA_l
0.4235225
0.01565432
lJNBIASED
BETA_2
0.05500032
0.001006519
UNBIASED
BETA_3
-0.002126766
0.0001284148
UNBIASED
BET A_4
0.1297768
0.006131528
UNBIASED
BETA_5
0.003957931
0.0001542194
UNBIASED
I
I '.
i
I
i
I
I
I
I
,
I
I.
i
I
!
I"
!
I
I
I
i
I
I'
I"
II
I
I
II,,, -,",
j" ;~
I'
I·
II
I
I
I
I
r
I
/·,1
II
I
I
I
I
L;
I"
I
i
i
:.
I
I
I
i
I\ , '
t.
I
!t
I
!,
i
I
I
I
I
I..
j'"
,I·
I
I
II
i
I
I
I
t"
e
'\
L
:>1 'tAL
I·"
!
I
I
!
I
I
I
.
***.******************.. *.*.. *... ***••• ****.......**••••**0'*.....*••••••••.
•
G*no.
•*
.
****************************************** •••****•••••*•••••••••••••••
I
I
I
I
i
I
I!
!
ROW
1
658.1434
I
I, .
1°·
f
t
II
I
!
Ir' .
t:
~
!
,,
. I
:-:~~,~~, ..;.:::;,; . ····,-'>'::·-:~i;·;: ':-';,-.-,.',
'e
:.",',
3
4
5
6
0.. 46
0 .. 't7
0.56
0 .. 61
0.. 61
0.67
0.68
l ',,,.:.,,.
0.18
C.69
10
8.5
0 .. 14
11
0 .. 77
9.5
10.5
12
0.78
0.75
12.5
0.111
13.5
H.5
ll.S
14
15
o.eo
III
17
,0.82
0.71
0.80
0.81
15. ')
16.5
17.5
0.711
19.5
20.5
21.5
18
19
20
21,
22
23
24
25.
0.87
o.eo
o.n
O.IH
18.5
22.5
23.5
21f.5
-26
o;al)
O.lll
21
0;.83
26
o.o?
26.5
2'J
1l.82
30
O.A6
27.5
2a.5
29.5 '
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
46
0.90
0.91
0.89
0.89
0.92
0.89
49
0.92
50
0.96
0.92
51
52
5:3
54
55
25,.5
0.82
,30.5
O.IlS
31.5
32.5
33.5
34.5
35.5
36.5
31.5
30.5
39.5
40.5
41.5
42.5
43.5
44.5
45.5
46.5
47.5
48.5
49.5
50.5
o.eo
0.86
0.91
O.lll
O.Il7
O.lll
0.85
0.90
0,.87
0.91
0.91
0.9')
0.<)"3
0.93
.
tt
'.,
/J,
,"
.
I
,
I
0.. 5
1.5
2.5'
3.. 5
4.5
').5
(;.5
7.5
7
S
9
13
h
I
,,"f' ,,:"::-
", ,..•,...."'...
1
2
~
. ,~..,,:,:,'y'
51.5
52.5
53.5
54 •.0;
1
1
1
1
1
1
1
1
1
1
1
1
0
'0.. 5
1.5
2.5
3.5
4.5
5.5
6.5
7.5
8.5
9.5
10.5
U.S
0.0
0 .. 25
2.~5
6.25
12.. 25
20 .. 25
30.25
42.25
56.25
.72.25
90.25
110.25
132.25
' 0.00
0.0
0 .. 0
. 0.0
0.. 0
0.0
0.0
Ci.o
0.0
0.0
0 ..0
0.0
0.0
12.5
0
0.0
0 .. 0
0.0
0.0
o.do
13.5 .
0
0.00
0.00
0.0
'0.0
0.00
0.0
14.5
15.. 5
16.5
17.5
18.5
19.5
0
0
0
0
0
o.do
o.do
0
8 ..0 '
o·
0-.0
0.00
0.«)0
Q.()O
0.00
0
0
00
0
0 ..0
0.0
0.00
Q.O
0.0
0.00
0.00
0.6
0
o.oa
0.0
0.00
0
0.0
0.0
0.0
0.0
0.00
0.00
0.00
0.60
0.9 0
0.0
0·90
o·
0
0
0
0
0
0
a
0
a
0
0
0
0
0
0
0
0
0
0
a
0
0
0
0
0
0.0
0.0
0.0
0.0
0.0
0.0
0.0 '
0.0
0.0
0.0
0.0
0.0
0.0
0 .. 0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.00,
0.00
0.00
0.00
0.00
o.bo
0.00
0.00
0.00
0.00
0.00
0.00
. 0.00
o.oc
0.00
0.00
0.00
0.00
0.00
0.00
0.00
20.5
21.5
17.5
21.5
24.5
t5.5
26.5
21.5
21!.5
29.5
30.5
31.5
32.5
:n.5
34.5
35.5 ,
36.5
37.5
30.5
39.5
40.5
41.5
42.5
43.5
~4.5
45.5
46.5
47.5
48:5
49.5
50.5
51.5
52.5
5'.5
54.5
.4504<11
.501238
.541731
.589971
.627957
.t6H:90
.6911{;9
.116394
.737366·
.754085
.7t655.0
.774761
.719251
.1enC9
.787167
• 791125
.195C83
.799041
.8Q2.999
.eC69'H
.81C914
.e14fH2
.0095090
-.0312378
.0122690
.0200293
' -.0179569
.C083104
-.0111687
.0636057
0473664
C14084,9
.C034501
.0052366
';".0292510
..CI61911
-.0071668
.0286752' .
-.0250821
.000959"
.0070014
-.0269565
.(590856
.ueno
-.·Clif9121t
.0111691
.822189
.82tH6
.f!3C1C-4
-.CI21892
.C532538
-.0207041
..834662,
.838620
-.00 /.6620
-.0186200
.8112578
-.C225119,
.846536
.850494
.854452
.858410
.862368
.866325
.810203
• I! 74241
.871l199
.882157
,.886115
.01)4642
-.C304937
-.C044517
.0215904 ;
-.0023615,
.0436745
-.C002631t
-.C042413
-.C081993
-.0321512
C138849 "
-.C200131
.0159690
.0020111
.0280531
-.0159048
-.01911627
.' .C061194
-.0217186
-.0017365
.0343056
-.0096524
-.0236103
.0124318
-.C1l5262
-.C154841
~eC;CC13
.894031
.891989
.901947
.905905
.909863
.913621
.911719·
.9217l1
.925694
.929652
.933610
.937568
.941526,
.9454.E4
.:,:}~,':
I.
.1
I
I
I
I·
. I
~,
I
.'1
I
I
r
I
I
l:
1
l
I
r
I
I
I
I'
I
I
I
I
,I
I
,
I
I
I""
I
i
J
I
I
I
I,
i
I!' "
'J
.
.....,'
\.':'
,-;."
AL
A N..A..LV S t
I
s
, 'I
i
I
0.99
55 •. 5
0.95
0,.97
0.91
0.96
0.97
0.9'.
0.96
1.03
0.99
1.01
0.99
0.99
56.5
51.5'
58.5
59.5
60.5
6h5
62.5
63.5
64.5
t5.5
66.5
0.91
67.5
68.5
1.el
69.5
0.99
70.5
11.5
1.0',
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
.
0
0
n
I
1
1
1
55.5
56.5
51.5
58.5
59.5
60.5
61.5
62.5
0.94944
0.9'5340
0.95136
0.96132
0.96527
0.9(,923
0.97319
0.91115
63.5
61t.5
0.98111
1
1
1
1 .,
1
1
65.5
66.5
61.5
(:8.5
69.5
10.5.
1
1
1
1
1
1
1
,
7l.S
0.985C6
O.ge9C2
0.Cj9298
O.S9694
1.CCC90
1.C04.eS
l.ce801
' 1.01211
.0305580
-.O()34000
.CI26421 .,
.00861142
-.0052738
.0001683 .
. -.0331896
-.0171476
i
II
I
I: '
I"·
.';":',
I
I
I
II
.0488945
I
.COlt9366
.0209787
-.0029793
-.C069312
-.0308951
.COl)1469
-.OleOllO
I
i
!,I
I
.~2123U
i,.
0,
j
!
I,
I.·
,"
I
,
..,
i
I
I
,'I
'1
case to reparametrize using the restrictions
to use the method of La.g;range muJ.tipliers;see
Pring~e
Our approach is more direct and, computationa.1..1Y, rapid
In addition, no rank conditions are imposed on X or R.
of our method arises when
of the form
X is not of
fUl~
Bt3--- = o and are non-estimable.
a g-inverse (XIX) -
such that
rank and
In this
73= (x/xfx';t.:
so~ves the
This can be used to find-the
usual constraints".
our notation and set
the matrices appearing
y
is an
constants,
in the model above are as
(n X 1) vector .0fobservations,X
~
isa
(p
X~)
is an
vector of unknown consts-nts,e
(n X~) vector of uncorrelated random variables each having mean
(J"2,
R
is a
(q xp)
matrix of known constants, and
constants.
(n X p)
The equations
matrix
is
Notation which is
speci~ic
to the model Y=
~
+ e
+
.
+ Rr ,
and
subject to
2·
..
cr- = SSE$)/ (n-rank(X~»
respectively,(p X 1),
(1 Xl), and
The fol.'J.owing matrix relations maybe verified using the
Much of this verification maybe
*
PA'
*
QA
are symmetric and idempotent,
(A/)+
A+(A+)I
RR+r = r
provided
+.
R(t3 - R r)
+
- Rr)
= (A+)1
=
,
(A/A)+ ,
$ = r·· are consistent ,
..
=0
= (t3 -
+.
R r ) provided
RP =R .•
R
remaining
TOA LINEAR CONSTRAINT
In this section,
w~dem.onstratethat the
vector
(Y- )$)' (y - Xf3)
linear equationsBf3 = r .
on a
FORTRAN
THEOREM 1.
. PROOF.
$
= r
Next, we discuss a method of
subroutine by Businger
~
minimizes
SSE(~)
We will first verify that
= r.
$
There is
since we assumed a consistent set of constraint
$= R (XQR) + (y
We nOTti verify that
+
~. XR
SSE@) ~ SSE(~)
r) + RR+r
o
computation
o~
the minimizing vector
matrix computations provided one has a means
f"oJ
~
requires
~orcomputing
the Moore-Penrose
This may be obtained using the singUlar value decomposition
Fora given matrix
A
A o~ order
(mx n)
as
A :: USV' }
where
U is
elements}
(m X n)}
V'is
S
is an
(nx n) and
(n X n)
withm~· n
o~
a.
the singular value decomposition of
.Sets:J.
si =0
=1/ s.J.
and form the diagonal matrix
A.
ifsJ.' >
0,
Let
s.
~
s.+
set
J.
=
from the elements . s.+ .
J.
S+
given by
. +
+
A= VB. U'
is the same as the number of non-zero. s..
J.
(If·· m < n
·)
of a
FORTRAN
subroutine to obtain the singular value de-"
A may be found in Businger and Golub (1969).
listed is fora COMPLEX
matrix
COMPLEX
setting the parameters
if
A
J
but we had no difficulty in converting
version.
ETA
The subroutine
We have had
= 1. D-
14 and
TaL
= 1. D
- 60 ; we
si"< sl X (1 X 10- 13 ) •
X . are too large for storage in fast core but the sums of
y/y,
used.
X/y,
and
XIX,
can be stored, then. the
If the formula
,..,
+
+
+
f> = (XQR) (y - XR r) + R r
its use should improve the accuracy of the computations by
avoiding unnecessary matrix multiplications.
'
f'oJ
~The
~
statistical properties of
linear model
the
~
pri ori
BI3 = r
following four theorems.
see for example Pringle atld'R'ayner (1971,p. 98).
proofs which depend heavily on the properties of the Moore-Penrose
in the Appendix; we believe that these proofs are new.
. THEOREM 2..
There is a
~
of the form
~=p..y+c
such that
e(A'~)
= A/~
~
for every
satisfying the consistent equations
BI3 = r . if an.d only if there are vectors
0
and
p
such that
A' = 0 'X + plR •
THEOREM 3.
of the form
Let
(3
be any estimator of the form
A' = O/X + p'R.
consistent equations
}$. =
r
If
e(A/~)
then'
= A/~
i3 = A:y
for all
~
+
C
and
A be
e(SSE@»
Let
e
,
.. 2
- [n-rank(X~)Jcr·
as
be distributed
2
(J
A/
I.
If
= t:.'X
provided $
= r
a multivariate normal with mean
A is a matrix of the form
+ F'R
distributed as· a (possibly singular) multivariate normal with
cr2 A' GA . where
2
88:8$')/cr . is independently distributed having a chi-squared.
distribution with n-rank (X~)
.GOROLLARY~
Let
degrees freedom.·
ebe distributed as a multivariate normal with mean
2
cr I .
If
0
A is a matrix
then under the hypothesis
H:
A'f3 = A/ f30 ,
statistic
where
f
l
= rank· (A'GA)
f'reedom and
f
2
has the
:::: n-rank(XQR)
F-distribution with
fl· numerator degrees
denominator degreesfreedomprovidedf
l
> 0 .
In a typical application of the preceeding theory; one might be interested
in estimating the linear function
,,'13
for the model
y = Xf3 +e
sUbject to
-
.,
. . .
.~
= r
assuming that the errors are normallY distributed.
not, the . (n+q X p)
matrix
necessity, be of the form
have rank
..
.:~
p. tne
.~)
A' =c5 / X + p/R.
. of variance is· 'O-'2. A'CA
Rf3
3,
A'~ •
is
degree~ freedom by Theorem 4.
is of the form
A'
A' =p'R
A'f3
restrictions
to be the constant
.
where
A'f3
with n-rank(XQR)
Note, however, that i f A'
,...,
+
A'f3 = A'R r . That is, i f
e
(If the matrix
condition must be checked directly.) By Theorem
best linear unbiased estimate of
....estimate.
will have rankp
then
= p/R
thenA/CA =0
is specified by the
+
.
,,'R
. r ; the data contribute
Onemay test a b;y:pothesis of the form
0
= r , and
fl.'
= !:::.'X
+ P/R
by applying the Corollary in a straightforward fashion.
As
can be
seen from the above, the application of Theorem 1
estimation of an estimable function
.' and
A/ j3
requires the computation of
applications for which the matrices
y
and
X
are too large for
core, one may compute the sum of squares and
X/y , and
X'X
and substitutethe formulas:
..
.~
75'2
=
(y/y "- $/X/y + ~/X/X;)/ (n-rank(C))
computations we willconsiderfitting a grafted poly-
nomial to data collected in a nutrition stUdy (Epprightetal., 1972).
~
The
data shown in Figure 1 are preschool boys' weight/height ratios (= y)
"(= x) ; the numeric values are given in Table I .•
a model which is li.near for ages above twelve months and quadratic for ages
Further, we will require that the response function be continuous
and have continuous firs t derivative in x.
More formally, the model is
subject to a continuity constraint at the point of join
~
'~_i~
::i
- ,- ~~
'.'~'~.:,,),
:
':'-;"~'
_:""'i.~,
'!: "- ;
INCH
9
o
co
t;....
o _ - -__--=-.....----r-----.,.---.,..-------r---~--,------,-----,.
N
~
I'\)
(1J
(;J
~.
1'\).
_;.ila
o
Z()J
-10)
::I:~
en o
~
~
~
ex>
01
f\)
01
en
en
o
;;
...-:-; -
~ ~
"-l~
-
~~.
..
_~
TABLE I
Boys,' Weight/Height v~.
W/H
AGE
0.46
0.47
0.56
0.61
0.61
0.67
0.68
0·78
0.69
0·74
0·77
0·T8
0·75
0.80
0·78
0.82
0·77
0.80
0.81
0·78
0.87
0.80
0.83
0.81
0·5
1.5
2·5
3·5
4·5
5·5
6·5
7·5
8·5
9·5
10·5
11. 5
12·5
13·5
14·5
15·5
16.5
17·5
18·5
19·5
2005
21. 5
22·5
23·5
AGE
0.88
0.81
0.83
·0.82
0.82
0.86
0.82
0.85
0.88
0.86
0·91
0.87
0.87
0.87
0.85
0·90
0.87
0·91
0·90
0·93
0.89
0.89
0·92
0.89
24.5
25·5
26·5
27·5
28·5
29·5
30·5
31·5
32·5
33·5
34.5
35'-5
36.5
37·5
38·5
39· 5
40.5
41.5
42.5
43·5
44.5
45.5
46·5
47·5
0·92
0·96
0·92
·0·91
0·95
0·93
0·93
0.98
0·95
0·97
0·97
0·96"
0·97
0·94
0·96
1.03
0·99
1.01
0·99
' 0·99
0·97
1.01
0·99
1.04
48.5 ,
49.5
50·5
51.5
. 52·5.
53·5
54·5
55·5
56·5
57·5
58.5
59·5
60·5
61.5
62.'5
63·5
64.5 .
""
65·5
66.5
67· 5
68·5
69·5
70·5
71·5
""
constraint. on the first derivative in
C ,and
,...2
(J'
-.0812
.00342
.0165
-.000702
-.000702
.00342
.0000300
.0199
.
.
(J'
degrees freedom.
.000930
.0199·
-.000396
.0000183
.0825
.0000183
....,2
-.0469
.-.000900
-.000900
-.000396
1.
.
.
= (.424, .055, -.00213, .730,.00396) ,
. -.0812
.446
at the join :point,..
..
computed from these data are:
./
ft.
x
-.00171
.0000433
-.00171
=. 000549
Thefitted equation is plotted against
If, say, one wished to estimate the slope of the
equation for ages :past twelve months· one would consider the estimation of
A. / (3
wi th
AI
=
(O, 0, 0, 0, 1 ) •
Since
x).
eR has rank 5 ,
The estimate of slope is
A. /tr
= 73 =
5
A.
.00396
is of the form
wi thstandard
4.
SOLVING NORMAL EQUATIONS SUBJECT TO
NON-ESTIMABLE CONSTRAINTS
the problem of solving the normal
XI]$
= X'y
the non-estimable parametric constraints
$=0.
By non-esti.mable, we mean that there do not
such that
.e
O/X
= plR
•
Our solution consists of showing that
is a generalized inverse of
X'X provided that
.. non-estimable.' It then follovlS (Searle, 1971,p. 8) that
1! = CX/y
a solution of the normal equations.
He have seen
satisfying. the non-estimable parametric constraints.
THEORElyl
6.
then
If there do not exist non-zero vectors
0
and
p
non-singular, the ranks of
Z
and
Since the
linear spaces generated by the rows of X and R are disjoint by hypothesis
we have that
X
"rank(R) .= rank(X)
generated by the rows of
+
rank(R).
Thus
X~
rank(XQ.R )
+
rank(R)"
Since the linear spaces
and R are orthogonal
=
Since the columns
rank (X) ,"
linear combinations of the columns of
X and the
are equal, there is a non-singular matrix
= X'X"
0
S
r~~s
such that
of the two matrices
X~
=
XS.
Thus,
As anexanrple of the use of this theorem in applications consider a
randomized block design with two treatments and two blocks :
1
1
010
1
1
0
1
0
110
0
1
+ e
=
lOt
0
1
to the usual non-estimable parametric constraints.
[:
1
1
0
0
0
1
:]
iJ.
t
2
b
IL
g-inverse of
R
R+
=
t1
l
b2
is
=
r:
It
01
0
0
,
~l
'0 '2'
I
I
/0
L
so that
I
il
-l
[:]
4
0
0
0
0
o
1
-1
0
0
o
-1
1
00
o
0
0
l-l
o
0
0
-1
fi
0
0
0
0
0
~
J.
4
0
0
0
-~
4
~
4
0
0
0
0
0
~
4
-~
0
0
0
-~
4
""'
1
4
,1
,
'4j
It is easy to verify that
r
I
Y11 + Y12 + Y21 + Y22
Y11 + Y12 - Y21 - Y22
=
r::x'y
=
i
-Yll - Y12 + Y21+ Y22
'
l
J
Y11- Y12 + Y21 - Y22
-Y11 + Y12 - Y21 + Y22
and that these are the usual parameter estimates.
Businger, P. A.and Golub, G. H.
complex matrix".
(1969). "Singular value decomposition of a
CO~11lunications of the ACM, 12,
,.."....
564-65.
18
E. S, Fox, H. M., Fryer, B. A. ,Lamkin, G. H., Vivian, V.M., and
Fulle:r, E. S.
(1972). "Nutrition of infants and preschool children in
the north central region of the United states of America".
of Nutrition and Dietetics, ,..,..,
14, 269~332.
Pringle, R. M. and Raynor, A. A.
(1971).
Generalized Inverse Matrices
Applications to Statistics. Hafner Publishing Company, New York.
Searle, S. R.
(l97l). Linear Models. John Wiley and Sons, Inc., New' York•..
APPENDIX .
PROOF OF THEOREM 2.
.,-.
(If)
Let
= a'x
A'
+ p/R
'.,
-~~,:;'~
= a I XQR (f3
= a'x(f3
=
Ca'x
= Aft?>
,
+)
R r
+ .
+
--R r) +6'XR r + pl$
+ p'R)f3
•
+
+ a'XR r + pi $
(Only if)
If
~
is of the form
.
then Rf3
=r
~
= R+r
+~i'
for all choi'ces of.. "i' • . We will take '13
examine the consequences' of various choices of
i'
to be of this form and
under the assUIllJ?tion that
there is a
~=Ay+c
e(A'~)
= A'~
+
A'AXR r +
First set
r·
for all
~'AX~r
Under this assumpt;i.on,fq:r
-,,-,....:--....:'
..
+ A'c
r = 0 , hence
A'ft~
+
+
r +A'c = A'R r ,
so that
for all choices of
we obtain
'Yihence
I'
By successive choice of the elementary vectors for
= A'AX
. ..
+ A'P\XPR. + A'PR
= [A'A]X
= O/X
+
[A'AXE+ + A'R+]R
+ p'R.
0
From the proof of the previous
The variance of A'~
W=X~,the
variance of
= A'A[PTtl*
A/~·
*
is
is
+ QwJA'A~2
~- Var (A'p).
0
PROOF OF THEOREM 4.
"Let
= lIy
W = X~.
Then
+
+
+ 2
- X(X~) (y - XRr) - XR rll
~.-
.=
* + Q,JC(13'"
* . R+r)1\ 2 •
lI~l
Now
since
*
~w
;: :;
0
and
P
R
+
(13 - R r)
=...0
.
proVJ.ded
*
SSE @') = e' QW e
where
*
~
*
e(e''4vre ) ;: :;
.
2
[n-ra.nk(vr)]rr.
5.
=r
J
is symmetric and idernpotentwi th rank
PROOF OF THEORD1
•
$
*
~.
= n-rank(W).
IT
We may rewrite
13 as follows •
.'
······e
..
.•.~
Thus}
mean
A'f5
A/ f3
Let
= A' (XQR) +e
+
+
+ L\'X(f3 - R r) + L\'XR r + P'R/3
·+
= A/(XQ
) e
R
"
+ A/ f3 •
has the (possibly singular) multivariate nbrmal distribution with
and variance-covarial'lce
2
0-
A/CA .
W = XQR • "From the proof of Theorem 4 we have that
* 2
SSE It5j2
\1-') 0- = e '~Te/ey
.n-rank:(X~).
Thus,
*
Q'\tl
i'There
is symrnetricand idempotent with ralL'ft
SSE(Jr)/ey2
has the chi-squared distribution with n-rank(XQR)
degrees freedom.
Consider the randcra variables
variance matrix
2(
)+~r
* =0
o-.XQR
.
ra.."1dom variables
(XQR) +e + f3
they. are independent.
..L
= AI (XQp ) . e + /\ I f3
...,
i
and
arid .
S~~ (p)
Q* e.Since their coW
As a consequence, the
are