•
...
•
A METHOD OF ESTIMATING
THE COEFFICIENTS IN DIFFERENTIAL EQUATIONS
FROM TIME-DISCRETE OBSERVATIONS
by
Walter E. Bell and H. R. van der Vaart
Institute of Statistics
Mimeograph Series No. 845
September 1972
•
•
•
iv
TABLE OF CONTENTS
Page
........................
1.
INTRODUCTION
2.
REVIEW OF LITERATURE
1
4
2.1 Estimation of Coefficients of Differential
3.
10
THE SOBOLEV NORM-TYPE EXPRESSION, SMOOTHING AND
DERIVATIVE ESTIMATION, AND ESTIMATOR COMPARISON
13
3.1
Considerations Regarding Use of a
Sobolev Norm-type Expression • • • • • • •
5-point Moving-Arc Polynomial Smoothing and
Derivative Estimation • • • • • • • • •
3.2.1 Smoothing and Derivative Estimation
Independent of Estimation of K. .
3.2.2 Parameter Estimation with Simultaneous
Estimation of K • • • • • • • •
3.3 A Function of Segmented Cubic Polynomials
3.3.1 Least-Squares Spline Functions
3.3.2 Construction of the Segmented Cubic
Polynomials • • •
• • • •
3.4 Comparison of Estimators • • • • • •
•
4.
Genera~
ESTIMATING
4.1
4.2
K
IN
.9-Z
=
dt
•
13
16
16
20
21
22
25
31
35
Ky
Estimation of K and Smoothing Operations
Conducted Separately • ••
••••
4.1.1 General Development
4.1.2 Numerical Example •
Simultaneous Estimation of K and the Parameters
of the Smoothing Function • • • •
4.2.1 General Theory
4.2.2 Numerical Examples
..
..
4
Equations
• • • • • • •
• • • •
Investigation of Observational Error Effect
2.2
36
36
45
51
51
55
•
v
~BLE
OF CONTENTS (continued)
.
Page
5.
GENERALIZATIONS OF THE SIMPLE CASE
5.1
Estimation of
5.1.1
5.1.2
5.2
5.3
6.
.
7.
...
.
•
K
1
and
K
2
in
.=
y
62
K Y + K
1
2
z
Estimation of K Following Initial 5-point
Polynomial Smoothing and Derivative
Estimation
• • • • • • • • •
E~timation of
K and K with Simultaneous
1
63
63
2
Smoothing •
• • • • • • • • •
Generalization to a System of Differential Equations
Generalization to a System of Differential Equations
in Which Some of the Coefficients Are Related or
Assume Known Values • • • • • • • •
• • • •
66
68
77
BIAS REDUCTION
83
6.1
6.2
83
General Considerations •
Examples of Reducing Biastotal
85
SUMMARY AND OVERVIEW OF OPEN PROBLEMS •
7.1
7.2
Summary • • • • • • • • • •
Overview of Open Problems
8.
LIST OF REFERENCES
112
9.
APPENDICES
116
9.1
9.2
116
118
Derivation of Equation (4.11)
Derivation of Equation (5.26)
•
1.
INTRODUCTION
Differential equations of the form
dYj _
dt
-
k
t K.. Yl.'
j
i=l Jl
= 1,
••• , L, 1
~
L~ k
(1.1)
are frequently used in the mathematical models associated with biological
systems.
For example, differential equations of this type often appear
in the models of drug disposition in the human body.
Certain models of
population dynamics also employ differential equations related to
(1.1).
Assuming the relevance of (1.1) to the true underlying biological
phenomena, interest has been traditionally focused on estimates of the
values of the
K..
Jl
For observed values
in most experimental applications of this model.
y.(t ) = y.(t ) + €.(t ) ,
1
m
1
m
1
m
where
E.
1
(t)
m
is a
K.. has been
Jl
accomplished in most of the existing literature by a two-step procedure:
random error and
m = 1, 2, ••• , n,
estimation of the
~
(1)
A solution function
y.(t)
or numerical methods.
for the
t = t
(2). Values of the
~
m=l
of (1.1) is derived by analytical
J
Such a function is defined at least
m
K..
Jl
are chosen to minimize the quantity
[y. (t ) - ; ; ; )J
J
m
J
m
2
",........
Y. (t ) denotes the value of
J m
of observations.
where
..
•
(1.2)
~
y.(t )
J m
for a given set
Estimates of the
K.. derived from minimization of (1.2) are usually
Jl
referred to as "least-squares" estimates.
Except in conjunction with certain enzyme kinetic models (Cleland,
1967), apparently little work has been done on estimating the
K••
Jl
•
.
2
The few examples of existing
without solving the differential equation.
methods appearing in the literature involve estimating the value of the
dye
derivative
it-
t
at the
m
from eXPerimental observations by such
methods as difference schemes (~.~., see Rescigno and Segre
and, by choosing the
IC . .
J~
n{·[k
I:
I:
m=1 .
. 1
~=
p.
9»,
to minimize
/'-..}2
IC .• Y.(t)] - y.(t ) .
J~
~
m
m
J
dy.
denotes the estimates of ~
where
(1961,
I
,
t=t
estimating the
m
IC..
J~
in a manner analogous to that employed in linear least-squares
approaches to multiple regression.
~I
dt
values
t=t
However, the estimation of the
from the discrete observations
m
absence of an analytical solution y.(t)
J
considerable care in the choice of
(1966),
method~
Y. (t )
J m
in the
has traditionally required
(See, for example, Joksch
who discusses the effects of random observational errors on the
derivative estimates derived from the least-squares fitting of an
approximation function to the observed values.)
The purpose of this thesis is to formalize a method of estimating
the
IC..
J~
in (1.1) which is based on minimization of a quantity, the
expression for which is related to a discrete version of the Sobolev
norm.
For several differential equations of simple form, some of the
properties of the derived estimators
..
•
IC •.
J~
are investigated by both
approximate analytical and Monte Carlo simulation methods.
To insure
relevance to investigations of mathematical models related to biological
phenomena, the number, n, of observations is limited and the distributions
•
·
3
associated with the observational errors,
E,
are chosen to permit study
under both constant variance and variance proportional to the
y.(t ).
m
l
A review of some of the literature involving estimation of the
coefficients of differential equations of particular biological
Chapter 3 contains definitions
significance is the subject of Chapter 2.
and construction methods for 5-point moving-arc polynomial smoothing and
derivative estimation and for a particular type of spline function
composed of cubic segments.
The Sobolev norm-type expression and
several parameters useful for comparing estimators are also introduced
in Chapter 3.
In Chapter
4, the estimator of
K
•
in the simple
differential equation
•
.<?1:
dt
is derived and investigated.
= K
1
Extensions of the method derived in
Chapter 4 include estimation of
K
1
~-K
dt - 1
K..
and the estimation of the
.<?1:
=
dt
dz
and
y+K
a
K
a
in
z
in the system
Jl
dt
Y
K
=K
11
21
Y + K
12
y+K
2a
z
{1.4)
z,
where there may be, but not necessarily be, restrictions on the
K.
l
•
These extensions and examples of their use constitute Chapters 5 and 6.
•
•
4
•
REVIEW 0 F LITERATORE
2.
Estimation of Coefficients of Differential Equations
2.1
Nearly all of the current methods of estimating the coefficients
of differential equations rely on either a knowledge of the form of the
analytic solution or on the existence of the values of a numerically
generated solution at the points of interest,
corresponding to experimental observations.
the points
~.~.,
In fact, in the area of
tracer kinetics, the methods of estimating the coefficients are
generally described as the estimation of the
A.1
("rate constants")
in the expression
m
•
=
f(t)
I: A. exp ( - A. t ) •
i=l 1
1
For homogeneous ordinary linear differential equations with
constant coefficients (whose solutions are essentially sums of
exponentials), the coefficients of the differential equations appear
as the
A.
1
or some function of these coefficients appear as the
In such cases, several methods of estimating the
A.
1
A. •
1
are available.
The simpl.est is the common "peeling-off" technique described in detail
by Perl
(1960).
tecb~ique
Cook and Taylor
(1971) describe a computerized peeling
for application to radioactive tracer efflux data and report
the results of simluation studies for two- and three-compartment
systems for different
A.
1
and various relative errors.
peeling process assumes that the
•
•
Ao
1
Since the
are sufficiently different to
provide a linear terminal segment, the observations must be carried out
far enough in time, which can be experimentally difficult •
•
5
a
...
For these models of radioactive tracer dynamics, Gardner {1963)
proposes a Fourier transform method which is described by Pizer et ale
(1969).
Use of this method requires interpolation and extrapolation
of the data to accommodate the integration schemes employed.
For
application to biological experiments with their large errors and
short duration, this method may fail.
other methods of estimating coefficients generally involve seeking
values of the coefficients which, for a given set of data, minimize
the sums of the squares of deviations
SS =
nt (y. _ y.)2
"
i=l l
l
•
where the
y.
l
are the observations and the
the solution (analytic or numerical).
are the values of
While not often specifically
stated, the analog computer procedures generate curves for a given
set of parameter values; each generated curve is compared with the
data points in an attempt to determine the "best fit", resulting
essentially in a minimization effort applied to the sums of squares.
The coefficients
K..
lJ
in the differential equation, say
dq.
k
---.=s:L - "
~ K . .q.
dt
lJ l
i=l
which is common in tracer kinetics, are usually represented by
variable potentiometers on the analog computer.
Varying the
K:.. ,
lJ
one attempts to generate a curve which fits the observations more
•
closely.
Heinmets (1970) discusses in detail and expounds the virtues
of the use of analog computers in biological model evaluation.
•
6
·
...
Analog computer notation and symbolism are used in such package
programs as the IBM Continuous System Model Program, which is used
by Parrish and Saila
(1970) to estimate the coefficients of a system
of Latka-Volterra competition-type differential equations aod a
modified Volterra predator-prey equation.
Evert and Randall
(1970),
encourage the use of the matrix approach to systems of differential
equations in the context of tracer kinetics and cite the value of the
Continuous System Model Program.
When an analytic solution to a differential equation (or a
system of differential equations) is not available or when the solution
is not linear in the parameters to be estimated, then a non-linear
•
regression scheme is frequently employed to determine estimates of the
parameters which minimizes the sums of squares of deviations.
Rosenbrock and Storey
(1966) and Swann (1969) present reviews of
current methods, most of which are iterative and employ a linear
approximation at every step.
Nearly all such procedures require
initial estimates of the parameters and can experience difficulty in
convergence.
Further, in the case of differential equations for which
analytic solutions are not available, numerical integration procedures
are a necessary part of the regression technique.
Cannon and Filmer
(1967) describe in rigorous detail some of the
mathematical properties of the estimates of the rate constants found
in chemical kinetic models.
(Cannon and Filmer,
•
The authors provide a numerical example
1968) in which the initial conditions and the
error rate are varied to demonstrate the behavior of their estimation
•
..
7
method.
Their original system of simultaneous differential equations
is solved by finite differences while their norm
is evaluated by the trapezoidal integration procedure.
Procedures for estimating the coefficients of (a set of) ordinary
differential equations without use of an integration scheme are
apparently rare, with the exception of an important class of enzyme
kinetic models.
Since most enzyme kinetic experiments are performed
by measuring initial reaction velocities at various initial substrate
concentrations, the estimation of the rate constants
K.
in the
l.
differential equation
...,.
K , K ,
1
:3
... )
may be accomplished by a least squares scheme if the velocity equation
is linear in the unknown rate constants or by a nonlinear technique if
the unknown rate constants appear nonlinearly.
"observed" values of
v
= ~;
are made from either a continuous
recording device or by fitting a function
the discrete observations on
In either case, the
(~.~.,
a parabola) through
P, of which there are a large number
(Cleland, 1967).
Whittle (1956) considers a time-dependent infection rate function,
A(t) , of wild rabbits in the integral equation
t
R( t) =
•
J
o
where
R(t)
t
exp
( J [A(V)
- ~(v)J} A(u)du
u
is the ratio of infected to healthy rabbits and
the death rate due to infection and assumed to be 0.45.
~(t)
is
He notes that
•
.
8
the above integral expression is a solution to the differential equation
=
dR(t) + [e(t) - A(t)] R(t)
dt
He proposes estimating
R{t.) ,
dR
dt
and estimating
J.
Metzler et ale
by difference quotients, eye-smoothing the
A(t)
A(t)
A(t) •
at discrete points by
= R' ( t)
+ @(t) R ( t )
1 + R(t)
(1965), in their work with non-steady state tracer
kinetics, propose a similar approach to estimating the transfer rate
function
B(t)
of sodium from the plasma to rumen of sheep in the
differential equation
dA
dt
:a
B(t) [A {t) - A:a(t)]
l
=
N (t)
2
where
A
l
and A
:a
are the specific activities of sodium in the plasma
and rumen, respectively, and
rumen.
N
is the concentration of sodium in the
2
The authors suggest that each of the
N , A , and A
2
l
be smoothed
:a
by a moving-arc polynomial scheme, for instance, from which
dA
:a
dt
caL
Then,
be estimated at the times at which observations were performed.
B~t)
can be estimated at the times of observation
t.
J.
by
N (t.) A '(t.)
B (t.) =
J.
•
:a
J.
:a
J.
Al (t.)
- A (t.)
J.
:a J.
In the context of drug kinetics, Martin
of determining
(1967) proposes a method
K, the rate constant for elimination of drug by all
•
9
k
routes, from measurements of unchanged drug in the urine.
Assuming
first-order kinetics, he derives the differential equation
dD
U
dt
where
time
D
K(D
um - Du )
is the cumulative amount of unchanged drug in the urine to
u
t
=
and
Dum
is the total cumulative amount of drug in the urine
after excretion is complete.
distributive phase,
For times
t
after the absorption and
K is estimated by considering the linear relation
which results from substitution of
6D
for the derivative.
dD
u
M
u
estimates
at the midpoint of the urine-collection interval
dt
6D
of
According to Martin, since
u
6t
6t , the values
dD
would have to be interpolated to yield estimates of
the end-points of interval
D
u
The author shows that, when the
dD
d't"""u
at
At, to coincide with the times of
observations of the quantities
decline of
u
~
is first-order, the substitution of
an error which would not exceed two percent even if
6D
u
At
At
would produce
were as large
as one half-life of the drug.
Rescigno and Segre (1961) note that the estimation of the rate
constants
•
where
K
1
and
K
2
in
Q is the constant concentration of a substance in an external
•
-
10
medium, can be accomplished by numerically estimating the derivative
dX(t) at
dt
t=t.
~
by a certain difference scheme and writing the
differential equation in the form
X = M- NX(t) = {l~X) [ MNJ
which is recognized to be a special case of the usual linear expression
y
in which
and
N
eo
Y= X ,
(hence,
"
K
1
= K Q ,
1
and
=
eo
+
eX
1.
e1
and
Then estimates of
= K
a
M
~ ) can be obtained in the customary fashion
a
for linear least squares models.
The potential value of this type of
approach is obvious if fairly accurate estimates of the derivatives can
be obtained.
In fact, Rosenbrock and Story
(1966) remark that, if we
could measure the derivatives directly, then estimation of the rate
constants would not involve the solutions to the differential equations;
however, the authors don't pursue their remark.
2.2
Investigation of Observational Error Effect
Although many authors have dealt with the techniques of estimation
of rate constants, and, hence, of the coefficients or functions of
coefficients of differential equations, relatively few authors have
reported the results of simulation studies in which the performance
of the various estimation schemes was investigated under specific data
error and rate constant magnitudes.
•
In general, these studies involve
simulated data which are constructed from hypothetical situations such
as could be expected in biological applications, where the number of
•
•
11
samples is limited and where the accuracy assumed is not normally found
in non-biological situations •
In the context of tracer kinetics, MYhill (1967) reports on studies
involving the sum of two exponentials with positive coefficients,
-A t
1.
f(t) = N e
1.
with
N , N , A , A >0
1.
:<I
1.
:<I
,
+ N e
:a
!.~.,
-A t
:a
for specified ratios of
N
1.
IN:<I
under the limitation of 11 and 31 equally spaced points.
and
A1
IA :<I
Using a
"valid least squares gaussian iterative" technique, MYhill found that
for 11 points and a ratio of
A
1.
IA :<I
=
2
the technique did not
converge for data sets with more than 1% error.
A similar analysis of
the tracer activity curve of a three-compartment steady-state open
system, which is represented by a sum of three exponentials with
positive coefficients, is reported by MYhill (1969).
In this analysis,
each point was weighted in order to reduce the differences in estimates
between the cases of percentage error and constant error.
In work similar to that of Myhill, Glass and deGarreta (1967)
report on error analyses in which a sum of two exponentials was
fitted to generated data with error using the Marquardt method and a
method based on the Newton-Raphson technique in which the weighting
of data may be incorporated.
Unfortunately, both the Myhill and the
Glass and deGarreta results are unavoidably restricted to monotone
decreasing functions.
Although Westlake (1971) did not conduct an extensive error
•
investigation, his report appears to be the only available which has
discussion on the effect of data error on a two-compartment open model
•
-
12
which is represented by the sum of three exponential terms
•
-A. t
A. e
1
1
in which the
A.
1
and the
A.1
are functions of the rate constants
and the volumes of distribution.
Such sums of exponentials are
distinctly not monotone decreasing in the drug kinetic case in
which the drug is orally administered.
Westlake discusses, by means
of an example, the effect of a constant error in the plasma
concentrations on the estimation of the rate constants (and the
functions of the rate constants) in the solution of the differential
equations for the plasma concentration.
Cook and Taylor (1971) give tabular examples of the performance
of their "peeling" computer routine in the estimation of rate
constants for two- and three-compartment systems in a tracer efflux
system•
•
•
-
13
3.
..
THE SOBOLEV NORM-TYPE EXPRESSION, SMOOTHING
AND DERIVATIVE ESTIMATION, AND ESTIMATOR COMPARISON
3.1 General Considerations Regarding
Use of a Sobolev Norm-type Expression
In this chapter and the next, methods are formulated to estimate
the constant coefficients of a certain class
o~
first-order ordinary
differential equations which are linear in the coefficients.
Although
the techniques may have more general application, the formulation will
be restricted to a rather simple class of differential equations.
Consider the differential equation
where
K is a constant
estimated.
(l.~.,
time-invariant) coefficient to be
Observed values of
by capital letters,
~.~.,
y
at times
t=t.
1.
will be denoted
y(t.) •
1.
Although most current techniques of coefficient estimation use
some scheme which seeks the value of the coefficient
K which
minimizes either the sums of squared deviations of a solution
(analytic or numerical) from the observed values or, as in the
example of Cannon and Filmer (1968), the integral of the squared
deviations, the estimation procedure proposed in this discussion seeks
to minimize the expression
1l =
•
n
2
I: {[y(t.) - f(t.)J
1.
1.=
.11.
. )
+ w[y(t.) - g(t1..)]Z
1.
which is somewhat analogous to the discrete version of the Sobolev
norm.
In equation (3.1),
.
y(t. )
1.
denotes an "observed value" of the
•
14
derivative
%t
points
of observation by means of the model
t.
~
expressed in terms of the observed values of
.
=
y(t.)
~
w
~
and
0
is a constant (usually
and
f(t.)
~
g(t.)
at the
K y(t.) ,
~
w
= 1)
with dimensions of time-squared,
are approximations to
~
y
y(t.)
~
and
.
y(t.)
~
obtained from the data by smoothing and derivative-approximation
procedures.
Then, in the current example,
ut = t.
([y(t.) - f(t.)J2 + w[K y(t.) - g(t.)J2}
~
~
~
~
~
and, not only an estimate of the unknown coefficient
any parameters in the functions
f
so as to minimize the value of
va.
shows that the
of the function
and
g
K, but also
will have to be chosen
However, the model as written
K directly determines the values of the derivative
y, so that it is natural to include in the selection
criteria the minimization of the squared-deviations of derivatives of
y , as well as the values of
Y itself.
Further, the incorporation of the differential equation itself
in the quantity (3.1) to be minimized and the appearance of
K
linearly in the differential equation permit
K
instead of a function of
estimation of
K, which often results when only a
solution curve, in which the
K does not appear alone or linearly,
is considered in the quantity to be minimized.
The functions
•
•
f
and
g
are arbitrary at this point, although
several of their desirable potential properties are obvious:
(1)
f
i
and
th
g
should be defined at each time
observation is made.
t. , when the
~
•
15
(2)
..
f
and
should have smoothing properties, 2:..~., their use
g
should reduce the effect of experimental error •
(3)
f
and
g
should be useful regardless of the "shape" of the
graph of the underlying data function.
(4)
g
(5)
Ideally,
should be reasonably insensitive to experimental error.
where
t
f
1
and
t
~
~
g
t
should be defined for all time,
t,
n
Such properties suggest a number of potential forms of the
functions
f
and
defined at the time
g.
For example,
t., i
1
= 1,
f
could be the point function
2, ••• , n,
which results from
application of a polynomial moving-average smoothing function and
g could be the value of the first derivative with respect to time
of the smoothing polynomial applied to the point
hand,
g
On the other
t.1. .
could be the point function defined at the
t.
1.
which
results from some numerical differentiation technique such as a
central differencing scheme.
Of course,
f
could be the polynomial
defined over the entire time domain of the data, a technique used
by Lewi et ale (1970) to represent radiochemical data.
These
authors recognized the limitations imposed by fitting one polynomial
to the entire set of discrete observations; for instance, a low
degree polynomial may poorly approximate the data while a polynomial
of high degree may better represent complex relationships in the data
at the expense of smoothing and may also introduce excessive variation
in the derivative estimates.
•
One important group of functions which
shares some of the desirable attributes of polynomials, which does
•
16
~
not require the fitting of one polynomial over the entire range of the
data but which is defined over all
t
such that
t
l
~
t
~
t
is a
n
set of spline functions.
In this dissertation
t
f(t)
is restricted to those functions'of
~.~.,
which are linear in their parameters,
defined at
t
(a) the function
which is derived from 5-point moving-arc smoothing
i
using either the cubic polynomial
p(t.)
~
a . + a .t. + a .t. 2 + a .t. 3
=
o~
l~
~
2~
~
2~
(3.2)
~
or the linear-hyperbolic function
q(t.)
~
=a
1
.t. + a . + a . -t
o~
~
l~
2~.
~
and (b) a spline-type function composed of cubic polynomial segments.
Similarly, the function
defined at
t.
g{t)
is restricted to (a) the functions
which are derived from the first derivative with
~
respect to time of the two smoothing functions
p{t)
and
q(t)
and
(b) the derivative of the segmented cubic polynomial.
3.2
5-point Moving-Arc Polynomial
Smoothing and Derivative Estimation
3.2.1
of
Smoothing and Derivative Estimation Independent of Estimation
K
Consider the
of the
•
•
y(t.), j
J
n
observations
= 3, 4, ... ,
y(t.), i = 1, 2, ••• , n.
~
Each
n-2 , can be replaced by the value of
the 5-point smoothing function {cubic or linear-hyperbolic) which is
obtained by the fitting of the smoothing function to the five points
y(t.), i
~
= j-2, j-l, j, j+l, j+2, by the method of linear
•
...
•
17
least-squares.
In the usual notation, the estimates of the
a.
-).
in
equation (3.2) can be computed
A
T
-).
-J -J
a. = (X. X.)
-1
T
X. y.
-J-J
where, for the cubic smoothing function,
J-a
t . at . 3
J+a
J+a
t '+
J a
1
Then, the smoothed estimates of
y*( t .)
=
J
...
3
t.
1
[ 1
y( t.)
J
y( t. )
J+a
can be computed
t.
J
t. 2
J
t. 3 ]
J
( X. TX. ) -1 X. Ty.
-J -J
-J-J
t.
J
t a
J
t
(X. TX .) - l X .T
-J -J
-J
s. Ty.
= -J
-J
where
s.
T
-J
=
[ 1
]
3
J
To obtain arbitrary smoothed values of the first two and last two
observations, the vectors
S
T
-1
•
t
-2
=
[ 1
t
T
s
-n-1
=
[ 1
t
=
[ 1
t
s
...
T
[ 1
s
-n
T
s
-1
T
s
T
1
t a
1
t
a
t a
a
t
n-1
n
t
;c
n-1
t a
n
s
-n-1
-2
t
t
1
and
s T are defined
-n
3
]
(X Tx )-l X T
-3 -3
-3
3
]
(X Tx )-l X T
a
3
n-l
3
n
T
]
-3 -3
]
-3
(X
Tx
)-l X T
-n-;c -n-a
-n-a
T
(X
Tx )-lX
-n-:;l -n-a
-n-a
•
•
18
other possibilities,
~.~.,
a lower degree polynomial, were employed to
estimate the first two points but were rejected as no better than the
make-shift method outlined here.
observations
y*
Then, the vector of smoothed
can be computed by !* =
T
S
0
-1
S
-2
! , where
~
T
T
S
-:3
0
S
-4
S =
(nxn)
T
T
s
-6
T
s
-n-2
0
s
-n-2
o
s
-n-1
T
T
s T
-n
y( t ) •
y(t n )
2
J.
Similarly, by defining the (lX5) vectors
d T
-1
=
[0
1
2t
d T
-2
T
d.
=
[0
1
2t
=
[0
1
=
[0
=
[0
-~
d
-n-1
•
d T
-n
T
]
(X Tx )-lX T
<3
]
(X TX )-lX T
-3 -3
-a
2t.
3t.~ 2
]
(X. TX. f1 X.,
T i=3,4, ••• , n-2
1
2t
3t
1
2t
1
3t
1
3t
2
~
n-1
n
2
2
n-1
3t 2
n
2]
]
-:3
-a
-~ -~
-3
-~
Tx
)-lX T
-n-2 -n-2
-n-2
(X Tx )-lX T
-n-a -n-.
-n-a
(X
•
19
a matrix
D can be constructed so that
estimates of
.9L1
dt t=t.
y* = D Y is a vector of
derived from the 5-point moving-arc cubic
l
smoothing.
In an obvious manner, matrices
Sand
D can be constructed
for 5-point moving averages using linear-hyperbolic functions or
any power-polynomial function.
Further, it is clearly not necessary
to use smoothing functions of the same degree for all
The problem of choosing the
,
f(t.)
l
the
g(t.)
l
t.l
,
and
.
K
to
minimize equation (3.1) under the specification of 5-point movingarc smoothing and subsequent derivative estimation results in
minimization of
(3.4)
where matrices
Sand
D were defined above.
From equation
(3.4),
y(t ) (including
i
t , t , ••• , t ) and for a
n
1
:a
it is evident that, for a given set of values of
specific knowledge of the time-sequence
given moving-arc smoothing polynomial,
of
u:a
in
(3.4) involves choosing
However, since the
y(t. )
l
!.~.,
a cubic, the minimization
K to minimize the last term,
are assumed to contain an additive
random error, it is possible to attempt to reduce the effect of this
random error by applying the smoothing matrix
involving
K Y in
S to the terms
(3.4) so that the expression to be minimized is
(3.6)
It will be shown in Chapter
4 that the minimization of (3.6) instead
•
20
of (3.5) often does result in an estimator of
variance and which is more subject to bias.
K which has greater
However, for completeness,
the discussion of Chapter 4 will involve equation (3.6) with
references to equation (3.5) being immediate with
S = I , the
identity matrix.
3.2.2
Parameter Estimation with Simultaneous Estimation of
The case of 5-point moving-arc cubic smoothing of
y(t.), i=l, ••• , n,
1.
coefficients
involves the estimation of
(a ., a ., a ., a.}
01.
11.
21.
n
values
sets of the
of equation (3.2).
21.
n
K
Similar to
the situation discussed in Section 3.2.1, minimization of the
Sobolev norm-type expression under the condition of simultaneous
estimation of the 4n parameters
representation of the
f(t.)
1.
a ..
J1.
and
K
requires the
by
a .
01.
a .
[1
t.1.
11.
a .
21.
a .
31.
However, in this case the vectors
a .
01.
a .
11.
a .
21.
a .
21
are not estimated by the usual regression techniques.
Instead, if
smoothing is not attempted for the first-two and last-two
y(t.) ,
1.
the specification of 5-point smoothing suggests the formulation
•
..
21
1 t
1 t
1
2
t2 t6
a
1
1
a
t2 t3
:a 2
0
a
a
a
a
1
yet )
a
13
a
a
yet )
03
2lii
yet )
3:il
fi
yet )
2
04
yet )
14
:3
24
yet )
034
6
---y(t n _)
a
o
a
a
1 t
a
n
yet
)
n-a
yet
).
n-:a
y(t _ )
n 1
yet n )
o,n-:a
1,n-:a
:a,n-:a
O3,n-:a
[5 (n-4 )Xl ]
[5(n-4)x(4n-4)]
of the form
!
~ =
y ,
where the symbol
-~
indicates approximation
in the sense of the minimum Sobolev norm-type expression.
It will be
necessary in Section 4.2 to compute the inverse of the matrix
! T!
+
!' T!',
of n-4 (4x4)
these
where
~,
~
denotes
matrices.
d~'
dt;
Since we assume
the task involves the inversion
t. ~ t.
1
J
for
i ~ j
inverses exist.
3.3
A Function of Segmented Cubic Polynomials
The case involving initial 5-point moving-arc smoothing and
•
•
derivative estimation (Section (3.2.1)) involves representation of
the point functions
f(t.)
1
and
g(t.) , of equation (3.1) by linear
1
•
22
combinations of the observations, where the coefficients are uniquely
determined by the time-pattern of the observations.
f( t. )
in Section 4.1 that, in the case where the
l
It will be shown
and
are
g(t.)
l
derived from segmented cubic polynomials, the minimization of
equation (3.1) involves the simultaneous estimation of the
the parameters which characterize the cubic segments.
K and
In the
following Section 3.3.1, we define spline functions and discuss
several criteria for spline functions from the literature as motivation of our own development of the segmented cubic polynomials as a
In Section 3.3.2, a complete description is provided
special case.
for the construction of the cubic segments.
The computing procedure
K and the parameters which
to simultaneously estimate the
characterize the cubic segments will be developed in Chapter
4.
...
3.3.1 Least-Squares Spline Functions
The following definition of spline function follows that of
Greville (1969).
Let
t , t , ••• , t
a
l
t , t , ••• , t
l
a
n
be a strictly increasing
A spline function of degree
sequence of real numbers.
knots
n
is a function
Set)
m with
satisfying the following
conditions:
(1)
In each interval (t., t. ), i == 0,1, ... , n,
l
l+l
t
=
_co
0
and
t
n+l
polynomial of degree
(2)
for
=
co ,
Set)
is given by some
l
m or less.
and its derivatives of order
Set)
where
1, 2,
... , m-l
are
lThe spline functions we construct will be of interest only
t s t ~ t •
l
n
•
23
(-=, =) ,~o~o,
continuous on
t. E (a,b),
is a function of the
....m-l
class
Given the
S(t)
~
n
o
points
(t., y(t.)), i = 1, 2, ..• , n, with each
l
l
Greville (1968) shows that for arbitrary
l
smoothest function
po in t s (~ . ~ . ,
k
of the class
f(t)
f(t.) = y(t.))
l
l
which t'fits" the
is a spline function of degree
t , t , .0., t
having abscissa values
C
k < n , the
l:3
n
2k-l
as knots, where the
n
"smoothest" function is defined as that function which minimizes
the integral
Sb [f(k)(t)]2
dt
0
a
It is noted that, in general, each of the adjoining polynomial arcs
..
must have equal ordinates and derivatives of orders
1, 2, ••• , m-l
at each knot.
Reinsch's (1967) scheme for smoothing by spline functions relaxes
the fitting requirement
and seeks a function
f(t.) = y(t.)
l
l
f(t)
which minimizes the integral
t
S n[f(2)(t)]2
t
f ( t)
among all functions
n
1::
i=l [
where
•
f(t)
S
dt
l
e
C2 (t , t )
l
such that
n
f(t.) - y(t.)]2
l
l
S
S
Q'.
l
is a given number and the
is a cubic spline with knots
Q'.
l
are weightso
t , 000, t
l
n
He shows that
and, hence, that the
•
24
polynomial {cubic) segments join at their endpoints and that
and
fh
are contlOnuous.
f
,
f
I
,
There f ore, th e a d
···
h
JOlnlng
segmen t save
equa1
ordinates and first derivatives at each knot.
DeBoor and Rice (1968a, 1968b) consider the case where, for the
given data pairs
(to, Y.),
1
1.
a polynomial spline function of degree
m is "least-square-fitted" on each of the
where the
~i
partition the interval
coincide with any of the observed
[a,b]
t-values.
k
intervals
(~
If
)
i' ~i+l '
but do not necessarily
The authors consider
both the case where the number and placement of the knots
~o
1.
are
specified and the case where the number of knots is fixed but their
placement is determined by attempts to seek a minimum (integral)
least-square
t
n
S
•
t
[f(t) - S(t)J2 dt
1.
which, due to the discrete nature of the data, is evaluated by a
numerical procedure,
~o~.,
trapezoidal sums.
An approximation function which is closely related to the
least-square splines described by deBoor and Rice (1968a, 1968b) is
the "smooth" continuous function composed of segments of cubic
polynomials which will be characterized by the following points:
1
(1)
The segments must join at the common knots.
(2)
The first derivatives (with respect to time) of any pair of
adjoining segments must be equal at their common knot.
•
lThis particular characterization is derived from a more general
formulation developed by D. C. Martin.
•
..
25
The whole function of such cubic segments can be fitted to
the points
(t., y(t.»
1.
using ordinary least-squares
1.
techniques.
(4)
The choice of the domain of each of the
k
segments is left
to the discretion of the user to encourage adaptation to the
vicissitudes of the experiments originating the data.
It is natural, in the use of this function to approximate data, that
the first derivative of the function be used to approximate the
derivative
~
dt
at the points of observation,
It will be necessary to note, for the development of Chapter
4,
that the construction method described in the next section provides
that the parameters uniquely representing a segmented cubic polynomial,
given a set of points and the sequence of knots, are, in fact, the
values of the function and the values of the first derivative of the
function at the specified knots.
by the letter
These parameters will be denoted
B .•
J
3.3.2 Construction of the Segmented Cubic Polynomials
The function
i = 1, 2, ••• , k,
f(t)
is composed of
k
cubic segments,
f. (t) ,
1.
and is constructed so that a pair of adjoining
segments have identical ordinate values and first derivative values
at the common knot or "join-point".
•
f
f. (t * )
if
t < t*
f. (t)
if
t E Ct.*
fk(t k* )
if
t
1.
f(t) =
That is,
0
1.
could be defined
0
1-1
> t*
k
, t.* ]
1.
•
26
*~ i=O, 1, ..• , k, t *0 ~ t 1 , t k* ~ tm)
(t.;
where
knots which partition the interval
[t ,t J
1
m
are the user-specified
for the case where the
m
observations
{(t., y(t.); i=l, 2, ... , m, t. E[t , t J, t.
< t.)
~
~
~
1
m
J-l
J
are given. 1
To guarantee uniqueness, we specify that
Consider, for the present development, the i
f. ( t) : ;: a.
~
Obviously,
~o
~
~l
..
[0,1 J •
~a
*J.
[to*
).-1 ,t.]
B,y the transformation
).3
t
T ::;::
the subset
~a
~a
).1
segment, the cubic
+ a. t + a. 't 2 + a. t 3
+ 2a. t + 3a. t 2
f. : ;: a.
th
2k + 2 ~ m •
-
t.*).-1
t.* - t *
).
i-l
of the domain of
f
is transformed onto
Therefore,
*~
f.(T)=b.
+ b.
~o
J.l
T + b. Ta + b. TIS , T E (O,lJ
).3
la
and
dfi
df*
i / dt
:
;
:
dT
dT
dt
=
a / *
*
(b.J.1 + 2b.J.a T + 3b.J.3 T) (t.). - t.),-1 ).
t.*
,
).-1
Letting
we define
B , j = 2i-l, ,2i,2i+l, 2i+2, as
j
follows:
B.
2J.-1
•
t <
*J.-1 ) = f~(T)1
J.
T=O
::;:: f. (t.
J.
:=
b.
1Forpurposes of this discussion
and for t > t *
k •
to*
J.o
f
could be left undefined for
•
27
df.
I
df.*
-- -2:.
dt
t=t.*
B.
21
=
1-1
dt
d// d,. IT=O
*1.
1',(")1
'1-'1
df.1
= dt
B2i+2
It=t~
1
= b.1.0
d1'.* dt
= d// d,.
I
T=1
= b.11.
+ b.
Is.1
+ b.
11
1a
+ b.
1G
=
In the more convenient matrix notation
B.
o
1
a1-1
0
o
o
o
b.
1
b.
11
=
B.
1
B.
:a 1+2
o
b.
1
o
o
o
o
s.
o
o
b.
..3
-28.
3
b.
2
s.1
-2
21.+1
1
1
1:01
b.
13
so
10
b.
1.1
=
12
1.3
Therefore, for the
Ct.*
1.-1
1.
1.
m
i
21.-1.
B.1
s.
1.
observations in the i
2
th
+:3
interVal
, t .*] ,
1.
1
,.
b.
1.0
1
b.
1).
•
B .
b.
12
1
T
m.
1.
y
m.).-1, +1
or
y
B .
m.
+1
1-1
21-1
"
B .
1-3"~ +2T~
J
J
(T .-2T~+T~)S.
J
J
J
1
3T~-2T~
J
J
21
(-"~+T~)8 .
J
J
1.
..
B .
»l+a
which is of the form
B i
a
t.B.
-1-1.
with the obvious notation,
y
-1
m.
1.-1
B i
a
B. =
-1.
Y
B.
a 1.+1
B.
21.+a
and
•
--
~.
-1.
denotes the (m.x4) matrix.
1
-fil.
=
1
Y
m.
1
+1
~.~.,
•
29
•
Now, the matrix
•
is constructed so that to approximate the vector
~
~
-1
B
0
(m x4)
1
B
Y
B
Y
2
~
-a
t B
Y
1
3
a
a
3
,..,
,..,
(m x4)
=
1
o
B
Y
m
ak+1iI
[mX(2k+2) ]
Remark 1.
If the above equation is treated as a least-squares fit,
then solving the above linear system of equations for the
•
1
yield
Of course, the predicted values of the observations are
The matrix
t'
~'
.!..~.
,
::::
(d~
d,.
1
Remark 2.
1
are easily computed.
= a.10
is
-2,.. +3"~ ]
(6,. .- 6,.~ )/ s .
1-4,. .+3"~
J
W;
-1
J
J
J
The original coefficients
f.(t)
~
/ dt)
d,. ij
the jth row of the submatrix
f( -6,. J. -6,.~)
/ s.
L
J
/I
Y
=
is, then,
-
•
would
B.
a ..
1J
of the i
+ a. t+ a. t lil + a. t
11
12
J
1
th
J
cubic
3
13
Since the relationship between the cubic
"
B
.
Y
•
30
1
coe£ficients before scaling,
a .. , and the coefficients after scaling,
1J
b .. , can be expressed by
1J
a.
10
1
*
-to Is.
1-1 1
t~ a/s.a
-to* 3/ S. 3
J.-1 J.
b.
J.o
a.
J.1
0
lis.1
a
-2t.* Is.
J.-1 J.
3t.*a / S. 3
J.-1 J.
b.
J.1
1-1
1
=
=
a.
1a
0
0
a.
J.3
0
0
where
S.
1
0
t.*J.. - t.*
J.-1
=
3
3t.*J.-1 Is.J.
lis.J. a
,
l/Si
C.b.
-J.-J.
b.
J.a
3
b.
J.3
and since
b.
J.o
1
0
0
0
B
b.
J.1
0
s.J.
0
0
B
b.
1a
-3
-2s.
J.
3
-soJ.
B
b.
13
2
-2
S.
B
ai-1
ai
D.
= -J.
=
S.
J.
B.
-J.
ai+1
J.
ai+a
then
I
I
a
a
I
C
10
,''
1
o
(4x4 )t,
I
I
=
I
1
B
0
:
Ca
J
---.-_ .. - - - --I
I
D
I
I
I
:(4x4)
:
'h. __ ..J
:
B
a
a
:
(4x4) :
----r---------l
D
I
:
I
3
:
:-------.,(4x4):
,-----
': Dk
o
o
: (4x4)
•
1
(4x4 ):
- - - - - - , - - - --I
11
I
D
(4kX4k;)
[4k.X(2k+2) ]
B
2k+2
•
31
3.4 Comparison of Estimators
Classically, a useful comparative measure of the suitability of
/I
an estimator has been the mean-squared error, which for estimator
of
/I
K is defined to be
/I
in which the quantity
estimators
e[(K-K)2J
K-e(K)
of
K
and which has the property
is defined to be the bias.
Of several
K, the one which has the smallest mean-squared
error if often said to be the "best".
Obviously, by definition, the variance and bias of the random
variable
/I
K are non-random parameters associated with the distribution
of the given estimator
/I
/I
K where the random variables of which
K is
a function have a particular, though, perhaps, unknown distribution.
..
/I
Frequently, the estimator
K cannot be expressed in a form readily
e( ~~)
amenable to application of analytic methods of determining
/I
var (K)
mation to
and
In such cases, it is often possible to devise an approxi/I
K from which approximations to
analytically determined.
/I
e(K)
/I
and
var(K)
The appropriateness of such approximations
is usually verified by simulation studies for values of
region of interest.
estimates of
/I
e(K)
In the event approximations to
and
for specific values of
can be
/I
var(K)
K.
/I
K within the
/I
K are not made,
are often found by simulation studies
In this context, the simulation studies
frequently involve the generation of artificial observations with
additive error from a known distribution.
For example, the generated
observations could be of the form
•
y(t.)
l
= ~,(t.)
l
+ E(t.)
l
•
32
where
€{t.) - h[O,v]
l
cov[€(t.),
€(t.)] =
l
J
and
°
v
with either
=
constant or
A
K
The mean,
"K from m sets of artificial observa-
of the
tions is used as an estimate of
value of
"K
e{K)
"
for error-free values,
"v=o
denoted by either
K
or
~.~.,
"Kexact
" = (K-K"v=o )
K-e(K)
for a particular
y(t.) = y(t.)
l
l
v.
The
will be
The total bias (biastotal) is
+
"
(K"v=o -e(K))
and, from a simulation study, can be approximated by
A
K- K =
...
Hereafter,
"v=o
K-K
"A
(K- "K
) + (K
- K) •
v=o
v=o
,which refers to the bias due-to-estimation
procedure, will be denoted by
biaSmethod' and
"K
V=O
A
-K
refers to the bias due to error in the observations as
which
bias
error
While the mean-squared error may represent a useful measure for
comparing several estimators, it fails to provide information concerning
the relative magnitudes of the bias and variance.
Since interest is
often centered on the bias of an estimator, it may be worthwhile to
consider the MSE and a quantity which relates the MSE and the bias.
Either of the ratios
or
•
*
~ =
·
a
b lastotal
MSE
•
...
33
are such quantities.
Trivially,
MSE
=
~a
II
1
var(K:)
*
~
~
It is obvious that
=
ll'a
_.::..~-
1 + ~a
is a measure of
standard deviation, while
which is attributable to
biastotal
in units of
~* is the fraction of the value of MSE
biastotal.
It is entirely possible that, of two estimators, the one with
the smaller mean-squared error might actually have the greater bias,
and, hence smaller variance, than the other estimator.
.
assuming the
II
K:
In fact,
values are distributed normally, it is possible that
the bias of the former estimator might be so large that the interval
e(~) ± 2Jvar(~)
would exclude the true
K: , or, equivalently, that
the ,interval bounded by
e(~)
2Jvar(~)-K
±
J~ar(~)
would exclude zero.
e(~)
However,
- K±
2Jvar(~) = _-_b_ia_s_t~o~t~a~l
±2
Jvar(~)
•
= -~
indicating that the value of
~
± 2
provides information concerning the
•
34
effect of the relative magnitudes of the
biaS
total
var(~)
and
in the construction of confidence intervals.
Obviously, there is interest in attempts to decrease the MSE and
~.
The value of MSE can be decreased by decreasing one of either
"
or
var(K)
by an amount greater than the other is
increased, ~.~., by accepting an increase in
(biastotal)S
as the
penalty for a greater decrease in the variance.
"
Since the biastotal (K)
is defined to be
" = biasme th0 d (K)
"
bias t o t a 1 (K)
"
+ bias error (K) ,
it would appear naively that decreasing either
IbiaSerror
"
(~) I
would result in decreasing
However, it will be shown in Chapter
the
biastotal
biastotal
and
bias
or
.
IbiaSmethodl
or
Ibias error I
On the other hand, in the estimators to
be discussed, an increase in
Ibiaserrorl
"
by an increase in variance (K).
•
(~) I
I
4 that, in some cases where
can be achieved when either
in later chapters •
Ibiastotal
( ~)
are of unlike signs, minimum
error
are judiciously increased.
Ibiasmethod
is generally accompanied
Individual cases will be discussed
•
35
4.
ESTIMATING
K IN ~
dt -- Ky
Consider the simple differential equation
*" =
in which the value of
observations,
and
£(t.)
K is unknown and is to be estimated from
The objective will be to minimize the
Sobolev norm-type expression
'lfl
=
I:
i=l
uy (t . ) - f (t . ) J2
J.
with respect to
K, where
Usually
g
f
and
.
n
J~ + w[y(t. )-g(t. )]2}
I: (Cy(t.~ )-f(t.)
J.
J.
J.
i=l
n
=
J.
f
and
+
~
are as defined in Chapter 3.
g
contain parameters which must be estimated so that
structure of the functions
4.1,
f
and
the estimation of
least-squares methods so that
y(t )
i
f
and
K by minimizing
and
f
g
~ = ~!
with
g
and
f
and
•
II
that this procedure yields an estimator,
to non-negligible bias.
II
Of special
li are
f =
S Y
D defined as moving-arc smoothing and
derivative estimation matrices as in Section 3.2.1.
"
is
are chosen by classical
by knowledge of the time sequence.
~
(4.2)
are completely specified for
interest is the case where the (nXl) vectors
and
K depends on the
g.
implemented after the parameters of
given set of
(4.2)
W[ I<Y ( t J.. ) -g ( t . ) Ja }
the theory associated with the estimation of
In Section
n
y(t.) = y(t ) + £(t.)
~
i
~
{y(t ), y(t ), ••• , y(t )} , where
1
a
n
is a random error.
J.
(4.1)
= Yo'
Ky(t), y(O)
K, of
K which is subject
For several true values of
K are investigated where the errors
£(t.)
J.
It will be shown
K, properties of
are distributed normally
•
36
with proportional standard deviation and with constant standard
deviation.
In Section 4.2, the estimation of
with the choice of the parameters of
minimization of (4.2).
K is implemented simultaneously
f
and
g
by means of the
Two particular forms of
f
and
g
are
investigated:
(i)
f
is constructed to be the point function resulting
from 5-point moving-arc cubic smoothing and
(ii)
f
f
,
::::: g •
is a continuous and differentiable function composed
of cubic segments and
f
,
::::: g •
Wherever MOnte Carlo simulation is conducted, the necessary normal
deviates are constructed by the built-in uniform random number
generator function RAND of the PL/C computer program compiler
(Blankinship, 1971).
Twelve such pseudorandom numbers were summed
and their theoretical mean was subtracted from the sum to yield the
pseudorandom normally distributed deviates.
All computer programs
were compiled and executed on either the IBM 360/75 or its successor,
the IBM 370/165, digital computers operated by the Triangle Universities
Computation Center, Research Triangle Bark, North Carolina.
4.1
Estimation of
K and
Smoothing Operations Conducted Separately
4.1.1
General Development
If
•
•
w is arbitrary in units of time-squared and if
f
and
g
are
the point functions derived from 5-point cubic or 5-point linear hyperbolic smoothing and derivative estimation, then, as described in
Section 3.2, the value of
K which minimizes
va
in equation (4.2)
•
37
•
is that value of
K which minimizes the last term of equation (4.2),
(4.3)
For the sake of generality, this discussion will consider the case
where
rather than
e~pression
(4.3)
is to be minimized.
Taking
setting
=0
and solving for
K, we have
Alternatively, if
~
=
K §l
£t = Ky
differential equation
the observed values are
y(t.)
is an approximation to the
where
= y(t.)
T
Y = (y(t 1 ) ••• y~tn )}
-
and if
+ E(t.) , then
1 1 1
or
DY
where
10
•
~
=
(~-K£)!.
=
K SY + DE-KSE
=
K
SY + 6
Considered as a regression model with errors
the above approximation admits a least-squares estimate of
that
K which minimizes
~,
K, namely
•
38
.
•
the same quantity as in
(4.4) .
One has that, upon substitution into equation
(4.5),
(4.6)
T T
.
T T
TT
Y.. §. De: + ! §. Q;y, + ! §. De:
+--------------T T
T T
T T
T T
¥. §.
~
+ Y:. §. Se: + ! §.
~
+ ! §. Se:
Since
is an estimate of
K under error-free observations,
!.~.,
! = Y:. ,
then
For the case in which !
•
(-1) bias
error
contains random error, i.e.,
!
rQ
is given by the expected value of the second and
•
•
39
third terms on the right-hand side of equation
(4.7),
namely
•
(4.8)
since
A
e(K) ,
To derive an analytic expression for
it would be
necessary to compute the expected value of expression
approach is complicated by the appearance of the
(4.8).
£(t.)
1.
This
in first-
and second-powers and in cross-products in the numerators and
denominators of the fractions involved.
However, the expression
1
-..,...-----------=-.
. . . .- - -.......-------.T T
T T
TT
l. §.
~
+
2~
§. S £ + ! §. S £
permits an approximation to
terms involving
to
•
(4.8)
£(t )
i
(-1) bias
error
to be formed in which the
appear only in the numerator.
is of the form
where, for practical purposes,
x
could be either
The approximation
•
40
•
(i)
1
l.TST
_ S
~
or
•
1
(ii)
yTST'2L
__
S
(iii)
T
i
Z~~
Since the expression being approximated is a factor in the complete
expression (4.8), the choice of either (i), (li) or
(iii) must be
investigated from two aspects:
(a)
The suitability of the approximation to the factor (4.9).
(b)
The suitability of the resulting approximation of
A
K,
as in equation {4.7).
This investigation involves computing the value of (i) and the means
and standard errors of the sample means of the quantities (4.9) and
A
(ii) and (iii) as well as those of the resulting estimates of
which were made using (4.9), (i), (ii) and (iii), where
S
=
K
I
and
D is the matrix defined in Section 3.2.1.for initial derivative
estimation by 5-point moving-arc cubics.
For each of
K = -0.1, -0.2,
-0.3, -0.4, -0.5, 200 sets of hypothetical observations
(y ( t ~. ); t.~ = 1, 2, ... , 15} were constructed, where
and
•
•
e(t i ) ~ h [0, ( ~~o y(ti))i] •
y(t.)
~
= 1000
-Kt.
..
The results shown in Table 4.1
indicate that approximation (ii) is satisfactory to approximate the
fraction {4.9) and that approximation (i) is not unreasonable •
1
e
.
Table 4.1
,
..
Simulation results for approximations to
1
·e
Entries in subcolumns 2 through 7 are means.
yTST SY
A
Var(K)
,where computed, are in parentheses.
1
approximation (i)
yTSTSY
approximation (ii)
approximation (iii)
---
fraction
10- 7
-0.1
2.3145
A
K
-0.1020
approximation
10- 7
2.3300
A
K
-0.1025
approximation
10- 7
2.3049
(0.0029)
-0.2
4.8707
-0.2076
8.2205
-0.2994
4.9305
-0.2092
4.8362
12.197
-0.4016
8.2222
-0.3033
8.1272
17.648
-0.4851
(0.0071)
-0.2075
-0.2993
12.255
-0.4070
11.994
-0.4010
4.8727
-0.4914
17.343
-0.4848
(0.0069)
-0.1020
-0.2076
(0.0048)
8.2212
-0.2996
(0.0056)
12.207
(0.0062)
17.183
A
K
(0.0029)
(0.0055)
(0.0064)
-0.5
2.3147
(0.0048)
(0.0056)
-0.4
-0.1019
approximat ion
10-7
(0.0029)
(0.0048)
-0.3
A
K
-0.4019
(0.0064)
17.650
-0.4855
(0.0071)
+:-
I-'
•
42
Using the approximation (ii),
of
1
the estimator for
II
II
K (equation
4.7)
can be approximated by
II
K• K
+ K
exact
exact
1
+--
(4.10)
T
z.9L
where
C
= STS
and
B
= ST n .
As in equation
third terms of the right-hand side of
of
(-1) bias
error
•
equ~tion
the second and
(4.10) are an estimate
Now the expected value of the approximation
(4.10) can be expressed in terms of moments of
orders.
(4.7),
E
of the first few
Although the tedious details of the deviation are reserved
for the appendix, the expected value of the approximation to
II
K
+
•
exact
T a
(l. Cy)
(4 t (t y.c .. )S var(E.)
i
j
J J1
1
+ t t (C .. B.. + C. . a + C.. C. .) va.r{ E.) var{ E. ) }
"
1
J
11 JJ
1J
J1 1J
1
J
II
K is
•
43
- (-l:--T)
s (2
C
¥.. .1.
+ 2
+
r: r:
i j
(I:
1
r: (r:
i
r:"
J
y.C
., )(I: y .B .. )
J J1 . J J1
J
var{ e:.)
1
y . C . . ) (r: y .B. .) var{ e: . )
J J1 j J 1J
1
j
(4.11)
(C .. B .. + C. .B. . + C .. B. .) var{ e: .) var ( e: . ) }
11 JJ
1J 1J
J1 1J
1
/I
K, where
An approximation of the variance of
by equation
J
/I
K is approximated
(4.10), would involve oomputations of e(!TceeTce)S which
arise from the form of
indicate that
TIT
;r §. Sy
..
e(~S). Since the results shown in Table 4.1
is not a poor ~pproximation of
1
/I
K could be approximated by
the estimator
t
/I • K
/I
K
+ /IK
exact
exact
which, when squared, would involve computation of
complicated procedure.
/I
Kia
::!:
, a less
Indeed,
/I
/I
2K s
2K
exact
exact
T
T
T
T
T
(2;r
f.!
+
!
Ce)
+
(I Be + ! .!?l + ! Be)
T
T
/Is
K
exact
l~
l~
+
•
e(!TQ!!TCe)
/Is
K
exact
(y"T~)S
T
T
S
(2y" .Q! + ! Ce:) -
/I
2K
exact
(IT~)S
T
T
(2y" Ce + ! c e )
•
44
where, as before,
and
Therefore,
e(~a)
_
approx
~a
"exact
2K a
_
exact
"exact
T
l.9X.
2K
+
1
"
J
[
K
+
I: B.. var ( E .) +
. 11
1
T
'l.9X.
exact
T
C
l. '::.fZ
:Ii
"exact
[~e~act
]
C
a [4 I: (I: c ..y.) a var( E • )
i
'l.'::.fZ
j
1J J
1
]
.. Ci .) var( E.) var ( e: . ) ]
[t t (C .. C .. + C.1J.C.1J. + CJ1J
1
J
'.
11 JJ
J
1
2K
T
('l. ~)
{ 2 I: (1: c .. y . )( I: B . y ) var( E. )
. . 1J J
ml m
1
1
J
m
a
(4.12)
(t. ci.y.)(t
J J
+ [2 t
.
J
1
"
m
B. y ) var(Ei)J}
1m m
2K
exact
(l.T ) a
9l
I: I: (C .. B .. + C.. B' + Cj.B .. ) var(E.) V\3.r(E.)
i j
1.1. JJ
1J 1 j
1 lJ
1
J
[t (t B .. y.)a var(E.) + 2 t (t Bj.y.)(t B.
.
.
J
1
+
t (I:
i
j
J1 J
1
"
1
J
1.
J
m
y ) vadE.)
1m m
B.. y.)a var (E. ) ]
1J J
1
t t (B .. B .. + B. .B. j + Bj . B. .) var ( E.) var( e: . )
"
11 JJ
1J 1
1 1J
1
J
1
•
J
1
•
II
e(K)
The ~pproximate value of
(4.11).
can be computed u~tng expression
II
Hence, the variance of
II
var(K)
K can be approximated by
II
..1
e(K a )
(4.12a)
approx
Of course, the presence of
non-norma~
errors would require the
re-writing of equation (4.12).
4.1.2
Numerical Example
II
K
In this section, some of the properties of the estimator
{equation 4.5) are investigated by means of constructing values
{y(t
to
l
),
y(t:a)' ••• , y(t
= i}
by the solution
y(t)
= 1000
e
~ = Ky , y(O) = 1000 , and generating sets of artificial data
= 1,
t.~
all cases
and
lS
ti
):
€
2, ••• , 15}
(t.)
,..., h( 0, v) , where
~
~
[P
y(t i
100
(i)
v
(ii)
v ;:: constant
(iii)
v
Sand
r
where
=0
)
t
0
y(t.)
~
v
= y(t.) +
~
e(t.)
•
~
In
is either
or
Qr
,
Dare (15X15) matrices constructed for 5-point moving-
arc cubic smoothing and derivative estimation, or
S;:: I
and
D is
as defined above.
To investigate
biasmethod'
~exact
is obtained by equation
(4.5), the expected value of the qpproximate
•
bias
error
by equation (4.10), and the approximate variance of
(4.12a).
is obtained
~ by equation
To investigate the validity of such approximations, a
Kt
•
46
Monte Carlo simulation study was conducted in which 200 sets of
artificial data were generated.
For a given value of
K, the
same pseudorandom deviates were used to construct the
E(t.) •
~
1\
The means and variances of the
K obtained by averaging over
the 200 sets are included in the accompanying tables as follows:
_[p y(t
v -
i ) ] ,
, p
100
= 10
v = 900
-S as
expected(appro~):
Table 4.2
expected(approx): Table 4.6
5-point
cubic
smoother
simulation:
Table 4.3
simulation:
- = -I
Table 4.7
expected(approx): Table 4.4
expected(approx): Table 4.8
simulation:
simulation:
S
Table 4.5
Table 4.9
These results indicate the following:
(1)
•
•
There is very little difference between the
1\
the
var(K)
from minimization of
and
(KSY - Dy)T (K§1 - DY)
Ii
K and between
(KY _ Dy)T (KY - DY)
-...
--
-
(2)
Biastotal' although obvious, is small compared to
(3)
For each estimator
tends to increase as
~ as a function of K,
IKI
Jvar(~)
IbiaSmethodl
increases, indicating increasingly
poor approximation of the exponential function by the cubic
function.
•
•
..
.;
;
Table 4.2
Estimates using approximations of
for proportional data error.
A
A
K
K
exact
e( K)
bias
m
p
and var(~) following initial 5-point moving-arc cubic smoothing
error
A sample size of 200 is ass~~ed.
bias
= 10.
bias
e
bias
var(K)
.Jvar(R)
Jvar(~)
A
t
MSE
S
s*
-0.1
-0.1000
-0.1023
0.0000
0.0023
0.0023
0.001738
.0.0)+17
0.0029
0.001743
0.0552
0.0030
-0.2
-0.1998
-0.2027
-0.0002
0.0027
0.005221
0.0723
0.0051
0.005228
0.0374
0.0014
-0.3
-0.2990
-0.3012
-0.0010
0.09 2 9
0.0022
0.0012
0.009943
0.0997
0.0071
0.009944
0.0120
0.0001
-0.4
-0.3966
-0.3975
-0.0034
0.0009
-0.0025
0.016197
0.1273
0.0090
0.016203
-0.0196
0.0004
-0.5
-0.4914
-0.4908
-0.0086
-0.0006
-0.0092
0.025650
0.1602
0.0113
0.025735
-0.0574
0.0033
Simulation results for the estimation of K following initial 5-point moving-arc cubic smoothing for
proportional data error. p = 10. n = 200.
Table 4.3
K
*.
A
K
exact
Ii
K
bias
m
bias
e
bias
t
•
var (K)
~var(~)
Jvar(R)
MSE
S
S*
-0.1
-0.1000
-0.1030
0.0000
0.0030
0.0030
0.001766
0.0420
0.0030
0.001775
0.0714
0.0051
-0.2
-0.1998
-0.2091
-0.0002
0.0093
0.0091
0.004695
0.0685
0.00)+8
0.004778
0.1328
0.0173
-0.3
-0.2990
-0.3011
-0.0010
0.0021
0.0011
0.006333
0.0796
0.0056
0.006334
0.02-38
0.0002
0.0034
0.008222
0.0907
0.0064
0.008234
0.0375
0.0014
-0.0131
0.010300
0.1015
0.0072
0.010472
-0.1291
0.0164
-0.4
-0.3966
-0.4034
-0.0034
0.0068
-0.5
-0.4914
-0.4869
-0.0086
-0.0045
+"
~
•
,
EstL~ates
A
A
K:
K:
using approximations of
A
bias
and var(K:)
error
5-point moving-arc cubics for proportional data error. p
Table 4.4
exact
e(K)
bias
m
bias
e
bias
following initial derivative
= 10.
A
t
..
vadK:)
e~timation
by
A sample size of 20C is assumed.
Jvar(~)
Jvar(~)
MSE
S
S*
-0.1
-0.1000
-0.1012
0.0000
0.0012
0.0012
0.001794
0.0424
0.0030
0.001795
0.0233
a.co08
-0.2
-0.1998
-0.2012
-0.0002
0.0014
0.0012
0.005237
0.0724
0.0051
0.005238
0.0166
0.0003
-0.3
-0.2990
-0.2995
-0.0010
0.0005
-0.0005
0.009998
0.1000
0.0071
0.009998
-0.0050
0.0000
-0.4
-0.3964
-0.3957
-0.0036
-0.0007
-0.0043
0.016274
0.1276
0.0090
0.016292
-0.0337
0.0011
-0.5
-0.4912
-0.4891
-0.0088
-0.0021
-0.0109
0.025812
0.1607
0.0114
0.025931
-0.0678
0.0046
Simulation results for the estimation of K following initial derivative estimation by 5-point
cubics for proportional data error. p = 10. n = 200.
Table 4.5
A
K
K
exact
-0.1
-0.1000
-;;:
K
-0.1020
bias
m
0.0000
bias
e
0.0020
bias
A
t
0.0020
vur(K)
0.001736
••
~oving-arc
~var(~)
Jv[).r(~)
0.0417
0.0029
0.004685
0.1117
0.0123
0.006268
-0.0076
0.0001
NeE
0.001740
S
s*
0.0 1;('.0
0.0023
-0.2
-0.1998
-0.2076
-0.0002
0.0078
0.0076
0.004627
0.0680
0.0048
-0.3
-0.2990
-0.2994
-0.0010
0.0004
-0.0006
0.006268
0.0792
0.0056
-0.4
-0.3964
-0.4016
-0.0036
0.0052
0.0016
0.008107
0.0900
0.0064
0.008no
0.0178
0.0003
-0.5
-0.4912
-0.4851
-0.0088
-0.0061
-0.0149
0.010097
0.1005
0.0071
0.010319
-0.1483
0.0215
.00-
co
•
•
~
~
Estimates using
h
h
K
exact
e(K)
approxin~tions
bias
m
of
h
bias
bias
e
bias
h
t
var (K)
)var(K)
Jvar
(~)
MSE
~*
S
-0.1
-0.1000
-0.0998
0.0000
-0.0002,
-0.0002
0.000224
0.0150
O.OOll
0.000224
-0.0134
0.0002
-0.2
-0.1998
-0.1991
-0.0002
-0.0007
-0.0009
0.000838
0.0289
0.0020
0.000838
-0.0311
0.0010
-0.3
-0.2990
-0.2971
-0.0010
-0.0019
-0.0029
0.002389
0.0489
0.0035
0.002398
-0.0593
0.0035
-0.4
-0.3966
-0.3929
-0.0034
-0.0037
-0.0071
0.005891
0.0768
0.0054
0.005941
-0.0925
0.0085
-0.5
-0.4914
-0. Lf849
-0.0086
-0.0065
-0.0151
0.013516
0.1163
0.0082
0.013744
-0.1299
0.0166
Simulation results for estimation of K follo\~ing initial 5-point moving-arc cubic smoothing and derivative
estimation for the case of constant error variance. v = 900. n = 200.
Table 4.7
K
••
and var( K) followbg initial 5-poir.t movJ.ng-arc cubic
error
smoothing and derivative esUmation for the case of constant data error variance. v = 900. A sample
size of 200 is assumed.
Table 4.6
K
Jl
-;;
h
K
exact
K
bias
m
0.0000
bias
e
-0.0001
bias
h
t
-0.0001
var(K)
0.000234
r-:T
"Vvad K )
,j'!ar(~)
0.0153
0.0011
0.00023 u
~lSE
..
';>
s*
-0.0065
0.0000
-0.1
-0.1000
-0.0999
-0.2
-0.1998
-0.2009
-0.0002
O.OOll
0.0009
0.000838
0.0289
0.0020
0. 0008 39
0.0311
0.0010
-0.3
-0.2990
-0.2973
-0.0010
-0.0017
-0.0027
0.001517
0.0389
0.0023
0.001524
-0.069~
0.0048
-0.4
-0.3966
-0.3948
-0.0034
-0.0018
-0.0052
0.002646
0.0514
0.0036
0.002673
-0.1011
0.0101
-0.5
-0.4914
-0.48 in
-0.0086
-0.0073
-0.0159
0.005312
0.0729
0.0052
0.005565
-0.2182
0.0454
0'
•
.
•
Estimates using approximations of
Table 4.8
error
moving-arc cubics for constant error variance.
A
K
bias
K
exact
e(~)
bias
m
bias
e
bias
and
v
•
A
var(K)
following initial
= 900.
deri\~tive
estimation by 5-point
A sample of 200 is assumed.
A
t
var(K)
)var(R)
Jvar(~)
~-1SE
"
':>
t::*
-0.1
-0.1000
-0.0997
0.0000
-0.0003
-0.0003
0.000217
0.0147
0.0010
0.000217
-0.0204
0.0004
-0.2
-0.1998
-0.1985
-0.0002
-0.0013
-0.0015
0.000850
0.0292
0.0021
0.000852
-0.0514
0.0026
-0.3
-0.2990
-0.2957
-0.0010
-0.0033
-0.0043
0.002382
0.0488
0.0035
0.002401
-0.0881
0.0077
-0.4
-0.3964
-0.3899
-0.0036
-0.0065
-0.0101
0.006048
0.0778
0.0055
0.006150
-0.1299
o.Cl66
-0.5
-0.4912
-0.4800
-0.0088
-0.0112
-0.0200
0.013500
0.1162
0.0082
o. O13~-·{iC
-0.1721
0.0283
Simulation results for estimation of K following initial derivative estimation by 5-point
for constant error variance. v = 900. n = 200.
Table 4.9
K
A
K
exact
A
K
bias
m
bias
e
bias
A
0
'.
~
var(K)
)var(R)
moving-~rc
Jvar(~)
MSE
S
cubics
s*
-0.1
-0.1000
-0.09'l7
0.0000
-0.0003
-0.0003
0.000232
0.0152
0.0011
0.000232
-0.0197
0.0004
-0.2
-0.1998
-0.2003
-0.0002
0.0005
0.0003
0.000826
0.0287
0.0020
0.000826
0.0104
0.0001
-0.3
-0.2990
-0.2959
-0.0010
-0.0031
-0.0041
0.001!,88
0.0386
0.0027
0.001504
-0.1063
0.0112
-0.4
-0.3964
-0.3919
-0.0036
-0.0045
-0.0081
0.002582
0.0508
0.0036
0.0026h8
-0.1594
0.0248
-0.5
-0.4912
-0.4791
-0.0088
-0.0121
-0.0209
0.005106
0.0715
0.0051
0.005543
-0.2925
0.0788
/
/-
'V1
0
•
51
4.2 Simultaneous Estimation of K and
'.
the Parameters of the Smoothing Function
i
(
4.2.1 General Theory
4.1, minimization of the Sobolev norm-type e4pression
In Section
(4.2» for a specified set of observations resulted in the
(equation
problem of minimizing
~ w [Ky(t.) _ g(t.)J2
~
~
i
K only, since the parameters associated with the
as a function of
function
il>
g
were determined by the choice of li1p;proximating polynomial
and by the time-sequence of observations.
smoothing function
•
function
g
~herefore,
coq,siq.ered,
f
and the
In this section, the
a~sociated
derivative-estimating
are assumed to be linear in their parameters,
for the type of polynomial functions,
and
g
ca,n be represented by
respective~y, where
~
and
~
f
II
discussion,
df
= g(t)
dt
-
so that
suggested by the notation.
~
~
f , to be
~13
and
.! I~
,
m ~ n , matrices and
are (mXp) ,
is the (pXl) vector of parameters.
~j'
In all cases in this
and
~I
'j/
are re 1a t e d , as
The Sobolev norm-type expression can
be written
(4.13)
where, as before,
elements,
1
n
and
Y is the vector of
K y(t.) , in our case evaluated with the observed values
~
of the variables.
•
yT = [y(t.) ..• y(t)]
Since
(4.13) can be expanded,
•
52
•
%t = Ky
so that, in the simple case
~
•
(4.14)
Taking
0(" )
oK
0(")
and
o!
and setting the
res~ting e~pressions
equal to zero yields
o( lfi)
oK
=
21<:wyTy - 2w~T_;r... IT!
(4.15)
..
and
(4.16)
..
These normal equations
(4.16) can be written
tTt + wtlTt l
.@"
_w!/T~
I
I
--
-
(pxp)
I
(pXl)
- - -----1-- --- .. - -
_wyT t ,
I
..
=
--
I
"
WyTy
I
I
I(
o
(lXl)
(4.17)
[(p+l)X(p+l) ]
.
•
which is of the form A y
=
~
Assuming A- 1
G
exists and letting
G
).l
I
1a
-,.--1 .... --
GIG
21
I
I
22
•
53
be the partitioned inverse of
~,
then
•
Since
~ =1
G iTy from (4.17), it fQllows that
1- -
It
K=
or, for
w
(4.18)
= 1(time-unit)2
,
(4.l9)
where
As in Section 4.1 (expressions (4,6) and (4.7)), this form of
permits the construction of an approximate expression for
Specifically, if I
It
K
=
bias
It
K
error
Z + ! ' then
(Z + ~)T~(Z + !)
= ------(Z + ~) T!(Z + .5)
•
•
(4.20)
•
As discussed in Section 3.4, since
then
K: ="K:
"
\=xact
+ "K:
exact
(4.21)
the last two terms of which are an estimate of
(-1)
. bias error •
Further, if it is assumed, as in Section 4.1, that
is an adequate approximation to
1
"K:
then
~
can be approximated by
exact
+ ~
exact
(4.22)
•
the second and third terms of which are an approximation to
Since (4.22) is linear in the E(t.) and in integer
~
error
, the expected value of the approximate
powers of the E(t.)
~
(-1) bias
•
55
(-1) bias error
(and, hence, of the
appro~imation
to
"K
by (4.22))
can be computed by equation (4.11), provided the distribution of
E
is known or assumed.
1
If
1
is an adequate approximation to
y"T!:l..
"K
tnen, by expression (4.12a), an approximation to the variance of
can be computed.
4.2.2
Numerical ExamRles
In this section, some of the properties
of
the estimator
~
(equation (4.19)) are investigated from sets of artificially constructed
observations
ti
= 1,
y(t.) •
~
2, ••• , 15
For
sever~l
values of
K , values of y(t.)
,
~
are qomputed blf
= 1000e
y(t )
i
-Kt
i
y(t ) = y(t ) + e(t )
i
i
i
e(t.)
~ h(O,v) , where either
~
and sets of observations
(i)
v
=
[p
y(t i )]
2
were generated for
or
100
(ii)
v = constant
(iii)
v
f 0
or
=0
Two pairs of functions
f
and
g;
are considered.
First,
f
is
the function composed of three cubic segments with knots at 0.5, 5.5,
10.5, and 15.5 time units.
•
In this case, the matrices
Section 4,2.1 are defined in Section 3.3 .
t
and
I'
of
•
56
The second function
f
to be considered is the function derived
from 5-point moving-arc cubic smoothing and derivative estimation as
described in Section 3.2.2.
In this case, however, the vector
! of
observations in equation (4.13) is the (55Xl) vector which appears
in equation (3.7), so that the matrices
E and
F of equation (4.19)
are of dimension (55X55).
By matching columns of
duplicate elements of the
y( t.)
y ,matrices
E*
and
which appear
1
~n
E and
F with the
the (55Xl) vector
F* of dimension (15X15) can be formed by
arithmetic addition of elements of
E and l
so that
,.
where
.
yT
is the vector of 15 observations
y(t ) .
i
It is important to note that, for a given value of
random deviates were generated to compute the
K, the same
e(t ) .
i
The tabular results are shown as follows:
v =
lp Y(
100
•
J·
ti )
, p
= 10
5-point
cubic
moving-arc
expected(approx): Table 4.10
3 cubic
expected(approx): Table
segments
simulation:
simulation:
Table 4.11
4.~2
Table 4.13
v
:;:
constant
= 900
expected(approx): Table 4.14
simulation:
Table 4.15
expected(approx): Table 4.16
simulation:
Table 4.17
•
,
,.
•
'.
A
Estimates using approximations of bias
and var(K) for simultaneous smoothing by 5-point moving-arc
error
cubics and estimativn of K for proportional error 'rariance. p = 10. A sample size of 200 is assQ~ed.
Table 4.10
A
A
A
K
exact
e(K)
bias
m
bias
-0.1
-0.1000
-0.0997
0.0000
-0.0003
-0.0003
-0.2
-0.1999
-0.1994
-0.0001
-0.0005
-0.0006
0.000099
0.000264
-0.3
-0.2995
-0.2986
-0.0005
-0.0009
-0.00:.4
0.000829
K
-0.4
-0.5
-0.3981
-0.4945
Table 4.11
K
exact
-0.1
-0.1000
-0.2
-0.1999
-0·3
-0.4
-0.5
-0.0019
-0.0055
-0.0013
-0.0020
bias
t
var (K)
-0.0032
0.002468
-0.0075
neg
Jvar(~)
Jvar(~)
0.0099
0.0162
0.0288
0.0007
a.OOll
0.0497
0.0020
0.0035
S
S*
0.000099
-0.0302
0.0009
0.000264
0.000831
-0.0369
-0.0486
0.0014
0.002478
-0.0644
0.001+1
liSE
Simulation results for simultaneous smoothing by 5-point moving-arc cubics and estimation of
proportional error variance. p = 10. n = 200.
A
K
-0.3968
-0.4925
e
;;
K
-0.0987
-0.2004
bias
m
0.0000
-0.0001
bias
e
bias
A
t
var(K)
-0.0013
0.0005
-0.0013
0.0('1)4
0.000090
0.000253
-0.2995
-0.2990
-0.0005
-0.0005
-0.0010
-0.3981
-0·3994
-0.0019
0.0013
-0.0006
0.000596
0.000811
-0.0108
0.001208
-0.4945
-0.4892
-0.0055
-0.0053
~var(~)
Jvar(~)
MSE
0.0024
K for
S
s*
0.0095
0.0159
0.0007
0.0011
0.000092
0.000253
-0.1370
0.0251
0.0184
0.0006
0~0244
0.0017
-0.0410
0.0017
0.0285
0.0020
0.000597
0.000811
-0.02ll
0.0004
0.0348
0.0025
0.001325
-0·3107
0.0880
V1
-..J
•
..
Estimates using approximations of
Table 4.12
and estimation of
A
A
A
bias
and var(K)
error
K for proportional error variance. p
e(K)
-0.1
-0.1000
-0.0997
-0.2
-0.1999
-0.1992
-0.3
-0.2991
-0.2978
-0.0054
-0.0121
bias
m
0.0000
bias
e
t
~var(~)
Jvar(I)
0.000230
0.0152
var(K)
NSE
S
s*
0.0011
0.000230
-0.0198
0.0004
-0.0001
-0.0007
-0.0008
0.000793
0.0282
0.0020
0.000794
-0.0284
0.0008
-0.0009
-0.0013
-0.0022
0.002151
0.0464
0.0035
0.002155
-0.0474
0.0022
0.005323
0.0730
0.0052
0.005352
-0.071+0
0.0054-
0.012774
0.1130
0.0080
0.012920
-0.1071
0.0113
-0.3968
-0.3946
-0.0032
-0.5
-0.4913
-0.4879
-0.0087
-0.0034
Simulation results for simultaneous smoothing by three cubic segments and estimation of
error variance. p = 10. n = 200.
Table 4.13
;:
A
K
exact
-0.1000
ass~med.
-0.0003
-0.4
-0.1
= 10. A sample size of 200 is
-0.0003
-0.0022
K
bias
for simultaneous smoothing by three cubic segments
A
K
exact
K
K
bias
-0.0988
0.0000
m
bias
e
-0.0012
bias
A
t
var(K)
Jvar{~)
Jvar(~)
K for proportional
MSE
-0.0012
0.000237
0.0154
0.0011
0.000238
S
s*
-0.0779
0.0060
-0.2
-0.1999
-0.2025
-0.0001
0.0026
0.0025
0.000757
0.0275
0.0019
0.000763
C.Oj09
0.0082
-0.3
-0.2991
-0.2974
-0.0009
-0.0017
-0.0026
0.001485
0.03,'35
0.OO~'7
0.001·'191
-0.0(,,75
0.0045
-0.4
-0.3968
-0.3982
-0.0032
0.0014
-0.0018
0.001874
0.0433
0.0031
0.001877
-0.0!}16
0.0017
-0.4913
-0.4841
-0.0087
-0.0072
-0.0159
0.002'130
0.0493
0.0035
0.002683
-0.3225
0.09 t12
-0.5
••
~
V1
C))
•
••
..
A
Estimates using approximations of . biaserror and ·...ar( K)
cubics and estimation of I': for constant error variance.
Table 4.14
A
A
I':
K
exact
e(l':)
-0.1
-0.1000
-0.0997
-0.2
-0.1999
-0.3
-0.2995
bias
m
bias
e
0.0000
-0.0003
-0.1981
-0.0001
-0.2934
-0.0005
bias
for simultaneous smoothing by 5-point moving-arc
v
A
var (K)
t
A sample size of 200 is assumed.
~var(~)
Jvar(I)
0.0020
0.0001
S
;*
0.000004
-0.1487
0.0216
~1SE
-0.0003
0.000004
-0.0018
-0.0019
neg
-0.0061
-0.0066
neg
0.0116
0.0008
0.000441
-1.5090
0.6949
0.0162
0.0011
0.001746
-2.3707
0.8490
-0.4
-0.3981
-0.3825
-0.0019
-0.0156
-0.0175
0.000134
-0.5
-0.4945
-0.4615
-0.0055
-0.0330
-0.0385
0.000264
Simulation results for simultaneous smoothing by 5-point moving-arc cubics and estimation of
error variance. v = 900. n = 200.
Table 4.15
-;:
A
K
= 900.
K
exact
K
bias
m
bias
e
bias
A
var(l':)
t
-_..
~var(K)
Jvar(~)
MSE
K
for constant
~
s*
_-->------~---
-0.1
-0.1000
-0.0992
0.0000
-0.0008
-0.0008
0.000019
0.0044
0.0003
0.000020
-0.1818
0.0325
-0.2
-0.1999
-0.1979
-0.0001
-0.0020
-0.0021
0.000108
0.01011
0.0007
0.COO1l2
-0.2019
0.0392
-0.3
-0.2995
-0.2933
-0.0005
-0.0062
-0.0067
0.000340
0.0184
0.0013
0.000385
-0.3641
0.1166
-0.4
-0.3981
-0.3837
-0.0019
-0.0144
-0.0163
0.000626
0.0250
0.0018
0.000892
-0.6520
0.2979
-0.5
-0.4945
-0.4589
-0.0055
-0.0356
-0.0411
0.001341
0.0366
0.0026
0.003030
-1.1222
0.5574
V1
\0
•
'.
I'
A
and var (K) for simultaneOlls smoothing by three cubic segments
Estimates using approximations of bias
error
and estimation of K for constant error variance. v = 900. A sample size of 2(:(; is assumed.
Table 4.16
K
A
A
K
exact
e(K)
bias
m
bias
e
bias
A
t
var (K)
~varC~)
JvarCI)
NSE
S
s*"
-0.1
-0.1000
-0.0997
0.0000
-0.0003
-0.0003
0.000029
0.0054
0.0004
0.000029
-0.0559
0.0031
-0.2
-0.1999
-0.1986
-0.0001
-0.0013
-0.0014
0.000077
0.0088
0.0006
0.000079
-0.1600
0.0250
0.0234
0.0017
0.000573
-0.204.8
0.0402
0.0471
0.0033
0.002367
-0.2568
0.0619
-0.3
-0.2991
-0.2952
-0.0009
-0.0039
-0.0048
-0.4
-0.3968
-0.3879
-0.0032
-0.0089
-0.0121
0.0005 4 9
0.002221
-0.5
-0.4913
-0.4737
-0.0087
-0.0176
-0.0263
neg
•
61
These results suggest the following:
(1)
The methods involving simultaneous smoothing, derivative
estimation and estimation of
with smaller
biastotal
K yield estimates of
K
and smaller variance than the
methods involving initial smoothing.
(2)
For the case of proportional variance, the method of
simultaneous 5-point moving-arc cubic smoothing and
estimation of
biaStotal
K yields estimates of
and smaller variance than the method involving
three cubic segments.
However, for the case of constant
variance, the method of cubic
slightly better estimates •
•
K with smaller
s~gments
appears to yield
•
62
•
5.
•
Two natural
(1)
GENERALIZATIONS OF THE SIMPLE CASE
of the work iq Chapter 4 are obvious:
e~tensions
application of the simple case
form~ation
to a single
differential equation of more than one term and (2)
the
~imple
application of
case formulation to a system of differential equations.
In Section 5.1, the estimation of
and
K
1
K
2
in the model equation
y(t) = K y(t) + K z(t)
1
Ii
is investigated where the function
f
in the Sobolev norm-type
expression (equation (3.1)) is either the ~oint function derived from
initial 5-point moving-arc polynomial smoothing, or the function
derived from
simultaneo~s
moving-arc polynomial smoothing and estima-
tion of the
K. , or the function derived from the simultaneous
~
fitting of cubic segments and estimation of the
g = f
K. , and where
l
l.
Section 5.2 includes a discussion of the estimation of the
elements of the
(2X2)
matrix
[~]
=
K in the sy~tem
[::
The formulation leading to
::]
[:]
is shown to closely parallel that of
the simple case.
Section 5.3 is devoted to a discussion of the estimators of
and
•
K
Iil
in the system
y = - K Y + K z
.
z
1,
a
K
1
•
63
•
Consider the single
•
K
Estimation of
5.1
and
1
in
K
a
differentia~
y =Ky+l<z
1
a
equation
dy(t) = I< yet) + K z(t)
dt
1
a
where
and
K
1
values
K
1
and
K
2
2
are constants to be estimated given observed
= y(t.)
l
y(t.)
l
= 1, .. .,
i
I<
n •
and
+ E(t.)
l
z(t.)
= z(t.) + '(t.) ,
l
l
l
As in the previous chapter,
mo~ing-arc
following 5-ppint
discqs~ion
of estimating
polynomial smoothing and
derivative estimation will be followed by discussion of simultaneous
smoothing and estimation of
...
an,d
I<
1
K
2
for the cases involving
cubic segments and 5-point moving-arc cubic smoothing •
5.1.1
Estimation of
Fol~owing
K
Smoothing and Derivative
Initial 5-point Polynomial
Estimat~on
For the differential equation (5.1) above, estimation of
and
I<
a
I<
1
by minimization of the Sobolev norm-type expression (3.1)
following initial 5-point polynomial moving-arc smoothing and
derivative estimation reduces to the problem of minimizing the
term
n
w t
•
(y(t ) - g(t ))2
i
h:l
of equation (3.1) or minimizing
T
w 5 5
-
-
= w(K )Sy
,-
1
th~
+ I< SZ - Dy)T (I< SY + I< SZ - DY)
2-
where, as defined in Section 3.2,
•
equivalent expression
-
1--;
S
2-
is either the smoothing matrix
associated with initial 5-point moving-arc polynomial smoothing or
S = I.
Expression (5.2) could have arisen independently of the
•
64
•
Sobolev norm-type expression if the differential equation (5.1) is
written in the approximation form
•
D~ = K S~ + K S z
-
1 -
= S
Cl
[~~J
[ KK:]
p
and, subsequently, in terms of the Qbserved
D Y
!
where
= ~ + ..§. , and
I(
1
and
K
~
[! ~J
~ = ! +
w ~*T~*
We find that, since
of
=S
[
! ' and
I(Kaj
we find
•
=0
,
Z, as
vJI.*.
•
is, in fact, expression (5.2), estimates
w !T! •
to minimize expression (5,2), we take
Settiqg
+
and
~* ~ ~ ! - (K 1 S E + K2~)
derived from minim~zing w
those derived from minimizing
I
e*To*
are identical to
•
65
or
II
.-
-1.
yT
K
1.
~T~ [y ~J
J:;:
II
ZT
K:
~
yTST SY
yTSTSZ
-1.
[:~
STD Y
- --
yTSTDy
---
-., -
---
ZTSTSY
~?£TSZ
---
=
---
ZTSTDy
STS is symmetric, it follows that ZTSTSY
---
Since
(5.4a)
:c
yTSTSZ
---
so
d
= det
rv~]
[l~TSY
=ZTSTSZyTSTSY
----
~T~TSZ
ZTSTSY
---
_ ZTSTSYZTSTSY
-----
Therefore,
II
ZTSTSZ
K
1.
:;::
II
K
---
1
d
_yTSTSZ
---
2
ZTSTSZyTsTDY _ ZTSTSyZTSTDY]
----
-----
=
[ _yTSTSZyTSTpy + yTSTSYZTSTDY
---....,-
However,
y(t.)
= y(t.) + E(t.)
1 1 1
leads to the
•
e~pression
1
d
-...-_--
and
Z(t.)
= z(t.) + '(t.) , which
1 1 1
•
66
"
K
1
T T
T
T
T
~ ~ .§.( zy - l!:.. ) .§. ~
a
=
,
+T T
T
. T
T
~ ~ £ (~ "1 y:!:. ) ~ ~
b
=
where, with
a =
a
+ b
exact
Q = ~T£
{~T~(zyT -yz T)Cy
_
"K: 1
} { (~+P T~
[(~+~)(~+.:) T
(~+.:)(:.+~) T ] ~(~+.:) }
_
{~T2.(~T_ YZT)~} {(~+.i)TQ. [(~+~)(~+!)T
_
(~+~)(~+~)T] ~(~+.:))
and
'"K:
A similar expression for
2
length of the expressions for the
The complexity and
is obvious.
quantiti~s
a
and
b
preclude
efficient investigation of analytical forms of the expected value of
the
bias
error
, as we did in Chapter 4, necessitating investigation
of bias qy Monte Carlo simulation with repeated estimation of
K:
2
1
and
from artificially constructed data.
5.1.2
Estimation of
Estimation of
1
K
1
K
and
K
and
yet)
•
K:
=
?
K
2
with Simultaneous Smoothing
in the differential equation
K yet) + K z(t)
1
2
by minimization of the Sobolev norm-type expression (equation (3.1))
where
f
is either the function composed of three cubic segments or
the function derived from simultaneous smoothing and derivative
•
67
estimation and concomitant estimation of the
K. , and where
1.
= f'
g
can be accomplished by setting the appropriate partial derivatives
equal to zero and solving the resulting normal equations for the
unknown parameters.
ITt + wt,T I ,
The normal equations in matrix form are
w!,T
X
,
wi 'TZ
-- - - - - - -
I
- - - - - - ----
B
a
,.
wyT I ,
wyT Z
wyTy
where
t
wyT Z
wzTt'
wZTZ
=
B
0
s
~
-
tTy
1
1\
I
I
/\
B
1
K-
0
:3
is the matrix associated with the particular function of.
segmented cubics or 5-point moving-arc cubic smoothing as defined in
Sections 3.2.2 and 3.3.
The partitioned form of the matrix in the
normal equations leads to a natural partitioning of the usual inverse,
"T
K
so that the vector
" ,
= (K
1
"K )
a
can be expressed as (for
time-unit-squared)
-1
"K1
=
"K2
=
•
where
l::]
F Y
w
=1
•
68
F ::': ~/(~Ti + ~/T~/)-l~T
-
-
--
-
r--
-
Paralleling the argument in Section 5.1.1,
"K1
-
"K
=
ZTE{ZyT _ YZT) F y
~T~(ZyT _ gT) ~ !
yTE(ZyT _ YZT) FY
=
=
~T~(ZyT _ YZ'r) E y
:3
anp., so
=~
where
for
a
and
C and
b
F
a
+~
b
lexact
are as in expre~sions (5.7) upon substituting
for
B.
A similar expression is obvious for
5.2 Generalization to a
Syste~
of
Diff~rential
~
"K:
:3
Equations
Consider the system of differential equations
~
dt -- K1 y(t) + K:3 z(t)
•
•
dz ::':
dt
K:
and assume that observations
3
y(t) +
K:
4
z(t)
(y(t.), z(t.):
1
1
i ::': 1, 2, ••. , n}
are
•
69
given where
y(t.)
= y(t.) + E(t.)
1 1 1
Z(t.)
= z(t.) + '(t.) .
1 1 1
and
It follows from (5.11) that
l
a
z
K
a
1
K
2
K
a
z
a
l
3
z
K
4
where
·T
Y
=
T=
~
aT
=
and similarly for
(~(t 1 ) ~(t 2 )
(y(t ) y(t )
1
2
a
(a
z
and
a )
z.
For the system (5.11), we define the Sobolev norm-type expression
.
U*2 = t [(y(t.) _ f (t.»2 + w(y(t.)
1
1
1
g (t.»2
1
1
1
+ (Z(t.) - f (t.»2 + w(Z(t.) - g (t.»2J
1
where
1
.
y(t.) = K y(t.) + K z(t.)
and
f (t.)
1
1
and
1
and
11121
If the values of
SY
2
f (t.)
2
1
SZ, respectively, where
~
.
Z(t.)
2
=
1
K y(t.) + K Z(t.) .
13141
are the elements of the vectors
is the matrix associated with
initial 5-point moving-arc polynomial smoothing, and if
g (t.)
2
•
•
1
are the derivative estimates
DY
and
g (t.)
1
1
and
DZ, respectively, where
D is the matrix associated with
S as defined in Section 3.2.1, then
minimization with respect to the
K.1
of
•
70
+
(z- - -SZ)T(Z- - -SZ)
can be considered as a problem
+ w(K Y + K Z - DZ)T(K Y + K Z - DZ)
3-
4-
-
3-
4-
-
minimization of
invo~ving
w(K Y + K Z - Dy)T(K Y t K Z - DY)
1-
since the terms of
2-
-
1-
2-
U*2
involving
-.
S are independent of the
K.
l
•
As in the univariate case, the details of constructing the estimators
II
K.l
will actually be derived from minimizing
w(K SY + K SZ - Dy)T(K SY + K SZ - DY)
1-
2-
with reapect to the
-
K.
l
1-
2--'
-
rather than minimization of expression
On the other hand, we can write equation
(5.12)
(5.14).
in the approximation
form
K
DY
S Y
S z
0
0
2
=
Dz
1
K
K
0
0
E.l
S z
3
K
4
or, in terms of
Y and
Z ,
K
S Y
DY
S Z
0
0
1
K
2
=
•
DZ
K
0
0
S y
S Z
3
K
4
+ 0
•
71
where
Sand
D are the smoothing and derivative estimation matrices,
respectively, associated with initial 5-point moving-arc polynomial
smoothing and where
K
Se:
0
S'
1
0
K
2
6 =
K
[::]
Se:
0
0
(3
S,
K
4
One can obtain least-squares-type estimates of the
with respect to the
K.
l
,
K.
l
by minimizing,
the expression
T
eTc
DY
SY
SZ
0
0
DZ
Q
0
SY
SZ
~J
=
. DY
DZ
-[
which is formally equivalent to expression (5.15) for
squared.
SZ
o
w = 1 time-unit-
Therefore, estimators derived from minimization of (5.15)
will be identical to those obtained from minimization of
eTc
be referred to as "least-squares" estimators in Section 5.3.
and
we have
.
•
I
g
=
and will
Letting
•
72
yT
ZT
aT
aT
aT
;T
~T
ZT
y
- g
z
Taking
oC§.T~)
oK
=
- 2
+ 2
yT aT
~T aT
;T ~T
~T ~T
yT aT
~T ;T
•
o<.~T.§.)
oK
gTg
aT ~T
~T ZT
-
setting
gT~
-
11
[~]
l:
-Z a
a y
= a , and solving for -K , one obtains
~J
11
K
•
73
-1
1
y1
0'1'
ZT OT
OT yT
OT ZT
"
K =
~~ [:
Z
0
0
Y
yT OT
ZT OT
~T yT
~]
OT ZT
gT~
[~]
(5.16)
which is similar to equation
.
y = Ky.
(4.5)
for the simple differential equation
However,
-1
yT OT
;T ~T
OT ~T
~T ;T
=
gTg
[~
Z 0
Y
0
:]
Z1' OT
_yT OT
1
~T ZT
~T _~T
d
so
Z'r
"K =
1
d
OT
_~T ~T
T ZT
0
ZT _~T
gTg
[~ -:
o
~
0]
-I
yT OT
;T OT
OT yT
OT ZT
gT~
[~J
.
(5.17)
•
•
74
After multiplication,
~
:::
1
11
K
:::;
2
11
K
4
-1d
_ yTSTSZyTSTDY]
· [yTSTSYZTSTDY
----- -----
-1
:::
-1
11
K
•
d
:::
3
J
-1
-
~T§?SYZT.§.TDY
-ZTSTSZyTSTDZ _ ZTSTSYZTSTDZ
__ -----
L- -
d
d
[~?QTSZyT.£TDY
J
• [yTSTSYZTSTDZ - yTSTSZyTSTDZJ
----------
where
Alternatively, the approximation to the system (5.11) can be written
(sy)T]
[ (SZ) T
or, in terms of the observed
!
and
For purposes of this development,
•
(DY
i DZ)
(SY
! SZ)
~
~,
more convenient form is
•
[ ::' . :"4]
+ 6*
•
75
where
6* -- (De:
K
K
K
K
l
g)
~) - (Se:
:3
4
:il
In the preceding discussion, the estimates of the
K.
as elements of
l
a vector were derived from minimization of the sum of the squares of
elements of the error vector, viz.,
manner, estimates of the
K.
T
6 6
= t 6~.
l
In an analogous
in the matrix formulation above can be
l
derived by considering minimization of the euclidean (or Frobenius)
matrix norm
t
where
i,j
(6 *.. ) -- 6* = (DY:DZ)
-,lJ
-
_.-
(SY ! SZ)
K
K
K
:a
I<:
l
3
4
However,
6* = [DY - (SYK
-l
-- [
+ SZK )
2
DZ
oj(-
(SYK
-:3
6*
6
-l
+ SZK ) ]
-
4
]
--cl
so
t 6..*2
lJ
* 6* + 6*T 6*
= 6
--l
-:L.
--cl
-.a
which is formally equivalent to (5.15) for
.
•
Therefore, the estimators
"1<:.
l
w= 1
time-unit-squared.
derived from minimization of (5.20) are
identical to those derived from minimization of the Sobolev norm-type
expression (5.15).
•
76
•
.
In summary, the estimators
"K1
"K2
"K
and
3
"K
in the system
4
(5.11) are identical for each of the following formulations in the case
of initial 5-point moving-arc polynomial smoothing and derivative
estimation:
Consideration of each equation of the system separately
(1)
as in Section 5.1(2)
Minimization of the Sobolev norm-type expression (5.13).
(3 )
Consideration of the system in vector fo rmula t io n (5.12).
(4)
Consideration of the system in matrix formulation (5.19).
For the case involving simultaneous smoothing, derivative
..
estimation and concomitant estimation of the
functions
f
1
by the vectors
, f
2
,gl'
and
ts, ~ , !'~,
at the
K. , the values of the
1
t.
1
can be represented
and !'y, respectively, where the
matrix ! , whose elements are functions of the
t. , and
1
~
and
y , which are vectors whose elements are parameters characterizing
the smoothing function, are defined as in Sections 3.2.2 and 3.3.
For 5-point moving-arc cubic smoothing, the
~
and
yare the
coefficients of cubic polynomials; for cubic-segment smoothing, the
~
and
yare the values of the function and the first derivative
at the chosen knots.
.
•
Equation (5.13) is, then, of the form
+ (~ -~)
T
.
T·
(~-~) + w(~ - !'y) (~- !'y)
•
77
Taking
o(tf )
oK.1
and
..
letting
w
=1
time-unit-
squared, and setting the partial derivatives equal to zero yields
normal equations of the form
~T~ +
~ ,T~
a
- -- - - _yT~,
.
_ZT~,
aT
aT
/
_~
a
~T~ + ~ ,T~
-
-
-
<
,Ty
_~
,Tz
a
_~/Ty
a
_~
,T
"
~Ty
y"
~TZ
~
z
a
a
aT
yTy
yTZ
a
a
"K1
aT
yTZ
ZTZ
a
a
"K2
a
a
a
y'I;y
yTZ
"K
a
a
a
yTZ
ZTZ
"K
a
-
-
.
I
-
_yT~,
_ZT~,
=
(3
4
a
Clearly, from the blocked sYmmetry of the submatrices of the partitioned
matrix, the estimates of the
K.
from this formulation are identical
1
to those derived from consideration of the two differential equations
separately, as in Section 5.1.2.
5.3 Generalization to a System of Differential Equations in
Which Some of the Coefficients Are Related or Assume Known Values
..
As a particular case of the general system of differential
equations (5.11), consider the system
•
y = -K Y + K z
1
z
=
2
y(a)
K z , z(a)
:;3
=a
= z0
•
78
in the general form
y
•
y
z
0
~1
0
~.,
=
z
where
For
l:l
n
z(t.)
l
-
"'1 -
-
I<:
0
~
1
.,
=
y
0
I<:
observed values
= z(t.)
l
(5.21)
~3
.,
z:
S4
~3 = 0,
~ 4 =-1<: .,
([y(t.), Z(t.)J:
l
+ '(t.),
i
l
= 1,
= y(t.)
l
y(t.)
l
l
2, •.• , n}
+ E(t.),
l
and for initial 5-point
moving-arc polynomial smoothing and derivative approximation represented
by matrices
S
and
~
, equation (5.21) can be written in the
approximation form
Dy
S Y
S Z
0
0
13
~.,
=
Dz
1
+ 6
(5. 23)
~3
0
S Z
S Y
0
S4
where
D E
S E
~~
0
S'
0
0
S E
S'
6 =
0
In this section, it will be shown that estimates of
I<:
•
.,
=
13
.,
1<:1 ~-el
and
by the least-squares method as defined in Section 5.2 after
adjustment for the linear restrictions (5.22) are identical to
•
79
estimates of
K
and
1.
K
from application of the same least-squares
2
methods to
Q
!] !
[-.e.
[D Z
--
0
.e.
~]
[K
-8 Z
l
]
+ 0*
(5.24)
K
--
~
where
Goldberger
(1964) proves that, if £ is the unrestricted least-
squares estimator of
in the general linear model, of which our
~
b * , the restricted least-
equation (5.23) is a special case, then
squares estimator of
is
~,
(5.25)
where, in our case,
x
and where
r = R b
=
8 Z
o
o
8 Y
defines the exact restriction on the
K. .
1.
In
the case under consideration, the restrictions (equations (5.22))
r = ~~
have the form
I'l
•
[:1
=
[: :]
0
1
I'2
1
0
1'3
1'4
•
80
It is shown in the appendix that the second term in equation
(5.25) is a 4Xl vector, the first two elements of which are
and
1
d d
1
d
d
1
_ yTCZZTCY
= 2yTCYZTCZ
-----
2
yTCYZTCZ _ yTCZyTCZ
= ----
From equation (5.18), the unrestricted estimators
b
1
= [_~TCZyTBY
band
1
2
bare
2
+ ~TCYZTBYJ. 1
d
2
(5.27)
b
2
= [_rTCYZTBY
+ rTCZyTBYJ. 1
d
2
The sum of equations (5.26) and (5.27) yield the restricted estimators
•
•
81
b
*1
-- (-2~?CZyTBY + .~?CZZ TBy
1
1:TCZZTBZ ) •
d
1
(5.28)
b
(_~?CyyTBY + rTCYZTBY _ rTCYZTBZ)
*2 =
1
d
On the other hand, the least-squares estimators
equation (5.24) can be computed by minimizing
and
verified that, writing
B
=
1
'I(" 1
and
6*T 6*.
'K" 2
in
It is easily
T
S D,
'I"( 1
1
=
d
'K" 2
The estimators of equation (5.29)
are identical to those derived by restricted least squares, equation
(5.28) •
The estimation of the
Ie
1
and
of the Sobolev norm-type expression
K
2
in this case by minimization
(5.13), which for simultaneous
smoothing, derivative estimation, and concomitant estimation of the
K.
1
,
is
(_I(
1
•
Y + K Z _ !/~)
2
+ (~ _
! y)T
involves normal equations of the form (for
w
(~ _ ! y)
=1
time-unit-squared)
•
82
~T~ +
I
~ ,T~,
I
a
I
_~
~,Ty
,T
~Ty
"
Z
~
~,TZ
Y.."
I
I
~T~ +
a
- -
-
-
- -
-
-
-
-
-
-
-
-
-
-
~ ,T~,
-
-
yT~,
aT
_ZT~,
ZT IR ,
-
-
I
I
I
- 1- -
I
I
I
I
I
I
a
-
-
-
-
-
-
-
-
-
- -
~TZ
=
yTy
_yT Z
"K1
a
_yTZ
T
2Z Z
"K2
a
The solution of these normal equations yields
2Z TEZ
"K1
y'I'EZ
yT FY
1
=
"K
yTEZ
2
yTEY
_ZT FY + ZTFZ
d
1
d
=
where
Except for the definition of the matrices
E and
~,
these estimators
are identical in form to those derived from initial smoothing in the
preceding discussion (equation (5.2~)) •
•
•
83
6.
BIAS REDUCTION
6.1 General Considerations
From the discussion in Chapter
simple differential equation
negligible
biastotal
reduce
. 1
t OlJa
bias
\biaSmethodl
biastotal
y(t)
4 on the estimation of K in the
=
Ky(t) , the presence of non-
in the estimates is obvious.
Attempts to
in this case naturally involve attempts to decrease
and/or
Ibias
error
I.
Although these two components of
have been treated separately computationally in previous
chapters, the fact that they are related is obvious from inspection
of the simulation sampling results of Sections 4.1.2 and 4.2.2.
is not surprising, then that reductions in
by changes in
bias
biasmethod
It
are accompanied
error
In general, three approaches to reduction of
considered in this chapter.
First, a function
f
biastotal
will be
(as in the Sobolev
nOIT·-type expression (3.1)) can be chosen which better approximates
the underlying data-generating function and its derivatives at the
points of observation.
For instance,
f
might be chosen to be the
simultaneous 5-point moving-arc cubic rather than the initial 5-point
cubic smoother.
This approach also includes such procedures as
eliminating the points of known or suspected poor fit or poor derivative approximation, such as eliminating the first two and/or the last
two points in the case of initial 5-point moving-arc polynomial
smoothing and derivative approximation.
•
For example, the omission of
the smoothing (or approximation in the case of error,-free values
y(t.)) and the derivative approximation of the first and second points
l
•
84
in the case of initial 5-point moving-arc cubic smoothing for
K := -0.5
(see the fifth row of Table 4.5) yields the following estimates of
points omitted
"Kexact
none
-0.4912
-0.0088
yet )
-0.5057
0.0057
y{t ), yet )
1
:a
-0.5017
o .oorr
1
bias
K:
method
The second approach involves the utilization of auxiliary
information.
For example, such information can appear in the form
of knowledge of a second differential equation which contains
coefficients in common with the first.
.
An example will appear
later in this chapter •
The third approach depends on a reduction in the error associated
with the observations.
situations where
It is to be noted, however, that, in the
biasmethod
and
bias
error
are of unlike signs
(as in initial 5-point moving-arc cubic smoothing for proportional
error variances, results of which appear in Tables 4.4 and 4.5), a
minimum
Remark 1.
biastotal
can occur for nonzero
Although the estimator of
p.
K resulting from minimizing
the expression (4.4)
where
Sand
D are the smoothing and derivative estimation matrices
associated with initial 5·-point moving-are cubic smoothing, would
•
I
intuitively yield less bias than the estimator derived from mini.mizing
•
the approximation and simulation investigations of' 8eetio:l 4.1.2
suggest that this conjecture is not generall.y true.
Remark 2,
In the above example of decreasing
biaG me th
. 0d
by the
omission of the smoothing and derivative estimation at the first two
points, the assumed underlying function is
Although
y(l)
= 606.53
y(t):::: 1000 e- 0 . 5t .
y(2):::: 367.88
and
are used to smooth the
third point, the onission of these values from the formula for the
estimation of
the sample.
K results in disregarding part of the information in
In this particular case, the disregarded information
corresponds to an important part of the graph of the underlying
function, namely, that part where the values are greatest, where the
function is changing most rapidly, and where the probable errors are
greatest •
•
6.2 Examples of Reducing Biastotal
In this section, the estimation of
and
K
1
K
2
is considered
for the model differential equation
y(t)
=-
(6.1)
K y(t) + K z(t)
1
2
as a single equation from the underlying system of differential
equations
.
y( t) -- - K1"y(t) + K2 z(t)
I
(6.2)
z(t)
"
•
-
K z(t)
:a
For purposes of simulation, first, values of
computed for
t.l = 1,2, ... , 15
•
y( t. )
l
and
z(L)
l
from the solution of the system
are
(6.2)
•
.....
86
y(t) = z
0
['2 \,] [e-',
z(t) =
e
t -e
-K 2
t]
-Kt
where
z
o
= 1000,
K
~
Z
0
= 0.21
of Section 5.1, estimates of
2
,and
K
1
and
1
= 0.20
K
K
Using the
procedur~s
are obtained for the
:3
following smoothing and derivative approximation schemes:
(i)
Initial 5-point moving-arc cubic smoothing and
derivative approximation followed by estimation
of
(ii)
K
1
and
K
2
Initial 5-po~nt moving-arc linear-hyperbolic
smoothing and derivative approximation followed
by estimation of
(iii)
and
K
1
K:
2
Simultaneous smoothing and derivative approximation
using three cubic segments with knots at 0.5 , 5.5 ,
10.5 , and 15.5 time-units and concomitant estimation
of
(iv)
K
1
and
K
:3
Simultaneous smoothtng and derivative approximation
using 5-point moving-arc cubics and concomitant
estimation of the
K
1
and
K
2
To investigate the properties of
•
in the presence of proportional data error, Monte Carlo simulation
was conducted in which the artificial observations,
•
87
y( t. )
1
and
z(t.)
= z(t.) + ,(t.)
1 1 1
were constructed where
and
At the risk of the possible inclusion of peculiar (~.!., unlikely)
samples, the same sets of random deviates were used for both
and
....
p
= 10
and for all four estimation procedures.
that such duplicity assists in elucidating the
error and modifications in the estimators.
It is hoped
effe~ts
Table 6.1
results of estimation from error-free values,
=5
p
y(t. )
of increased
displ~ys
and
1
the
z(t.)
,
1
and from the sampling investigation.
From the
II
values (Table 6.1~, it is app~rent that the
Kexact
estimator associated with the initial 5-point moving-arc linearhyperbolic function is much poorer than those associated with the
other smoothing and derivative estimation functions.
the original values
y( t.)
1
and
z(t.)
1
Comparison of
and their approximations and
of the true derivative values and their approximations reveals that
approximations of
t. = 1, 2, and 3.
1
•
y(t )
i
and
z(t.)
1
are relatively poor for
The results of estimation of
smoothed values and derivative approximations for
are shown in Table
6.2.
K
and
1
t
i
=
K
2
using
4, 5, ••. , 15
These results indicate that the omission of
•
Table 6.1
..
t
l
Simulation results for estimation of
I<
1
and
I<
a
in
y=-l<y+l<z
a
1
~.
treated as a single equation from systerr: 6.2
Smoothing methods are initial 5-point moving-arc cubic = I-C, initial 5-point ~oving-arc linear-hyperbolic = I-L-H ,
simultaneous 5-point moving arc cubic = S-C, and simultaneous three cubic segments = S-3-C. K1 = 0.20 , K = 0.21
2
n = 25.
;;
K
~var(K)
.jvar(~)
-0.0598
0.000751
0.0274
0.0055
0.0021
-0.0652
0.001087
0.0330
-0.0612
0.0027
-0.0587
0.002862
0.2737
-0.0673
0.0036
-0.0637
0.004180
0.1986
0.1893
0.0014
0.0093
0.0107
0.2085
0.1979
0.0015
0.0106
0.0121
0.1986
0.1795
0.0014
0.0191
0.0205
bias
bias
p
K
K
I-L-H
5
K
0.2612
0.2598
-0.0612
0.0014
0.2773
0.2752
-0.0673
0.2612
0.2585
0.2773
1
K
2
10
K
1
K
2
I-C
5
K
1
I<
2
10
K
1
~*
0.004327
-2.1825
0.8265
0.0066
0.005338
-1. 9758
0.7964
0.0535
0.0107
0.006307
0.5463
0.0647
0.0129
0.008238
-1.0972
-0.98 4 5
0.000814
0.0285
0.0057
0.OQ0928
0.3754
0.1233
0.001104
0.0332
0.0066
0.001249
0.3645
0.1172
0.003217
0.0567
0.0113
0.00363'7
0.3616
0.0132
0.004873
0.3424
0.1155
0.1048
.~
0.4926
0.0226
0.004362
0.1988
0.1944
0.0002
0.0044
0.0046
0.000084
0.0092
0.0018
0.000106
0.5000
0.2005
0.2085
0.2038
0.0015
0.0047
0.0062
0.000136
0.0117
0.0023
0.000175
0.5299
0.2200
0.1988
0.185 4
0.0002
0.0134
0.0136
0.000310
0.0176
0.0035
0.OOO!~95
0.772'(
0.3734
0.2085
0.1950
0.0015
0.0135
0.0150
0.00049!~
0.0222
0.0044
0.000719
0.6757
0.3129
K
0.1991
0.1~'~1
0.0009
0.0050
0.000099
0.0100
0.0020
0.000134
I<
0.2089
0.2036
O.OOll
0.0053
0.0059
0.0064
0.000178
0.0133
0.0027
0.000219
0.5900
0.4812
0.2593
0.1870
0.1991
0.1831
O.OOO~
0.0160
0.0169
0.000381
0.0195
0.0039
0.000667
0.8667
0.4283
0.2089
0.1921
O.OOll
0.0168
0.0179
0.000680
0.0261
0.0052
0.001000
0.6858
0.3204
K
1
K
1
:3
1
2
10
or
VSE
0.0311
I<
5
t
0.0015
2
s-c
e
0.1874
K
10
m
~
0.2085
2
""
exact
bias
0.0660
II;
S-3-C
,---:::-
var(l<)
~
meth~d
K
1
K
2
co
(Xl
•
•
Table 6.2
Simulation results of estimating
and
K
1
K
2
in
y '"
KY + Kz
1
2
smoothing and derivative estimation for restricted sample.
n = 25
A
0.000273
0.0165
0.0033
0.000314
0.3879
0.1303
0.0113
0.000645
0.0253
0.0051
0.000772
0.4h66
0.1651.1
-0.0069
0.0035
0.001086
0.0330
0.0066
0.001099
0.10G}
O.Oll1
-0.01 2 5
0.0054
0.002625
0.0512
0.0102
0.002654
0.1055
0.0110
0.1921
0.1987
0.0179
-0.0066
K
0.1896
0.1965
0.0104
K
0.1921
0.2046
0.0179
1
2
= 0.21
~*
0.0064
2
0.20 , K
a
~
-0.0040
K
'"
MSE
0.0104
1
l
Jvar(~)
0.1936
IC
K
~var(K)
0.1896
5
bias
m
following initial 5-point linear-hyperbolic
t i '" 4, 5, "', 15 •
A
K
K
10
A
K
exact
p
bias
e
bias
t
'.
~
1
var(K)
0'
\.0
•
-
smoothing and derivative approximation at those points of poor fit
does result in reduced
biasmethod
and reduced
increased
bias
of opposite sign.
since the
biasmethod
biastotal
is reduced as the errQr rate,
error
and
biai'
error
I-
var(K)
but in
These results also show that,
are af unlike signs, the
p, increases.
As expected,
I-
the
var(K)
increases by approximately a factor of four when the error
variance is doubled.
in MSE, although
th~
This increase is also reflected in the increase
marked decrease in the values of
that the proportion of MSE which is due to
Although in a real
experiment~l
biastotal
~*
indicate
decreases.
situation it might be inappropriate
to consider the incomplete model consisting of one equation (6.1) when,
in fact, the complete model is known to be tne system (6.2), the
example presently under consideration (equations (6.1) and (6.2)) is
appropriate for demonstrating use of the method of
bias to tal
reduc-
tion incorporating a second differential equation with coefficients
in common with the original model equation.
example, estimation of
I(
1
and
K
2
For purposes of this
in the system (6.2) is conducted
utilizing the first procedure discussed in Section 5.3, which involves
the use of initial 5-point moving-arc cubic smoothing and initial
5-point moving-arc linear-hyperbolic smoothing.
estimation methods, and for each
For each of the
p; 5, 10 , the same sets of
random deviates are used to construct the errors
E(t.)
~
and
The sampling results for both methods are shown in Table 6.3.
'(t.) .
~
These
results indicate that, in the case of both smoothing functions, the
•
biasmethod
is reduced in comparison with the single equation
estimation (see Table 6.1), although the reduction is most drastic
•
..
•
Simulation results for estimation of
Table 6.3
K
1
and
K
2
•
\
in system 6.2 following initial smoothing and derivative estimation.
Smoothing methods are 5-point moving-arc linear-hyperbolic
K = 0.21 .
n = 25
= I-L-H
and 5-point moving-arc cubic
= I-C.
K1
= 0.20
,
2
method
p
K
I-L-H
5
K
1
K
A
K
exact
K
1
K
2
I-C
5
K
1
bias
m
bias
e
bias
A
t
var(K)
Jvar(~)
JV9.r(~)
~1SE
S
~*
0.2356
-0.0097
-0.0259
-0.0356
0.000731
0.0270
0.0054
0.001998
..;1.3185
0.6343
0.2184
0.2504
-0.0084
-0.0320
-0.0404
0.000904
0.0310
0.0060
0.002537
-1.3422
0.61+34
0.2097
0.2184
0.2431
-0.0097
-0.0334
-0.0431
0.002910
0.004768
-0.7996
0.3896
-0.0084
-0.0395
-0.0479
0.003695
0.0539
0.0608
0.0108
0.2579
0.0122
0.00~989
-0.7873
0.3831
0.1995
0.2036
0.0005
-0.0041
-0.0036
0.000502
0.0224
0.0045
O.C00515
-0.1607
0.0252
0.0049
0.000612
-0.1016
0.0102
0.2094
0.2125
0.0006
-0.0031
-0.0025
0.000605
0.0246
K
0.1995
0.2078
0.0005
-0.0083
-0.0078
0.002027
0.0450
0.0090
0.0020e8
-0.1733
0.0291
K
0.2094
0.2160
0.0006
-0.0066
-0.0060
0.002484
0.0498
0.0100
0.002520
-0.1205
0.01 113
K
2
10
K
0.2097
2
10
A
1
2
\D
f-'
•
92
in the linear-hyperbolic case.
bias
error
It is apparent, however, that the
increases in the linear-hyperbolic case while it decreases
for the cubic smqothing function.
Estimation of
K
and
1
~
~
6.2 by simultaneous
in the system
smoothing and concomitant estimation of the
K.
l
as discussed in
Section 5.3 results in estimates with insignificant
biastotal
and greatly reduced variance compared to single equation estimation.
The results of Monte
Ca~lo
simulation using the same sets of random
deviates as for the case involving initia+ smoothing (Table
appear in Table
•
6.4 .
6.3)
•
,
•
Table 6.4
Simulation results for simultaneous smoothing and estimation of
5-point moving-arc cubics
A
A
and
three cubic segments
1
and
= S-3-C.
A
K
1
K
2
in system 6.2
= 0.20
, K
:3
= 0.21.
~r-T
0.000062
-0.2078
0.0413
-0.1910
0.0350
0.000222
-0.0738
0.0054
0.0035
0.000309
-0.1322
0.0171
0.0055
0.0011
0.000030
0.0000
0.0000
0.0069
0.0014
0.000048
-0.1014
0.0102
0.0112
0.0022
0.000129
0.1875
0.0342
0.0138
0.0028
0.000191
0.0000
0.0000
K
S-3-C
5
K
0.1997
0.2016
0.0003
-0.0019
-0.0016
0.000059
0.0077
0.0015
K
0.20g5
0.2117
0.0005
-0.0022
-0.0017
0.000080
0.0089
0.0018
K
0.1997
0.2011
0.0003
-0.0014
-0.0011
0.000221
0.0149
0.0030
K
0.2095
0.2123
0.0005
-0.0028
-0.0023
0.000304
0.0174
K
0.1998
0.2000
0.0002
-0.0002
0.0000
0.000030
K
0.2097
0.2107
0.0003
-0.0010
-0.0007
·0.000047
K
0.1998
0.1979
0.0002
0.0019
0.0021
0.000124
K
0.2097
0.2100
0.0003
-0.0003
0.0000
0.000191
2
10
1
-
= 25.
0.000083
K
1
n
~*
p
K
Smoothing metr-ods are
~
method
exac
t
= S-C
K
•
bias
bias
bias
var(K)
'l/var(K)
·JvarCk)
m
t
e '
l>ll3E
:3
s-c
5
1
2
10
1
:3
\.D
'-"
•
7.
SUMMlI.RY AND OVERVIEW OF OPEN PROBLEMS
7.1
Summary
Consider a differential equation of the form
~-Ky+KZ
dt - 1
2
Given observed values
discrete times
t
1
,t
y(t.)
l
2
and
, ... , t
n
Z(t.)
l
of
y(t.)
l
1
z(t.)
l
at
,one is tempted to smooth the
data by fitting an approximating polynomial in
yT = [y( t ),
and
... ,
t
to the
y( t ) ]
n
ZT = [Z (t ), ... , Z(t ) ]
n
1
by a linear regression scheme, to estimate the derivatives
~
dt
t=t.
l
by computing the derivative of the approximating polynomial at the
t
i
' and to proceed to estimate the
K
i
by the usual regression
procedure applied to the model
y
•
by minimizing
T
~ ~
[ r*,:~*J
, where
Y*
+
and
Z
*
c
are the smoothed
This method often gives unsatisfactory estimates of the
•
the possibly poor approximation of the
y( t. )
l
Y and
K.
l
due to
from the observed
Z
•
95
values and the subsequent relatively poorer estimates of the
derivatives.
In this thesis, a method is proposed which is based on the
minimization of a quantity, the expression for which is related to
the discrete version of the Sobolev norm, namely
~)
where
f
and
g
are vectors, the elements of which represent the
values of the approximating and derivative estimating functions at
the
t.
l
and where
•
y
= [! zJ
[::J
Obviously, this procedure incorporates simultaneously the approximation of the
y(t.)
(or smoothing of the
l
y(t.)
l
in the case of data
with error), the estimation of the derivatives
~
dt
and the estimation of the
~
K
i
t=t.
l
By
suit~ble
f
and
, the naive regression-type approach is a special case of this
more general method.
g = f'
and
In each case discussed in this thesis, we have
f is a function which is linear in its parameters.
to the linearity of the parameters of
the coefficients
•
definition of
of equations .
f , and so, of
Due
g, and of
K. , the problem reduces to solving a linear system
l
•
96
In this thesis, three forms of differential equations are
considered in detail:
(1)
9X
dt
(2)
~==
dt
K Y
dy
dt
K Y + K z
1.
li1
== Ky
1.
+
K z
2
The general theory of estimating the coefficients is developed for
each and
the
K. •
1.
examples are includeq for certain representative values of
The approximation function
f
is either a 5-point moving-
arc cubic polynomial, a 5-point moving-arc linear-hyperbolic function,
or a recently-developed function composed of
joine~
cubic segments.
The numerical results, which include Monte Carlo simulation of
each of the above
d~fferential
equations with additive normally
distributed errOr, the standard deviation of which is either constant
or proportional to the
y(t.) , indicate that the minimization of
1.
tf yields substantially improved estimates of the
K.
1.
in terms
of less bias and less variance than the minimiz<:!-tion of the above
The estimation of the coefficients of a differential equation
by minimization of the Sobolev norm-type expression has several
..
advantages over other current methods.
know either the
•
ana~ytic
a numerical solution.
It is not necessary to
solution, if it exists, or the values of
Initial estimates of the parameters and
coefficients are not required, as they are in most iterative schemes.
•
97
Further, use of this method avoids, for the above described type, the
frequent convergence problems associated with iterative schemes.
The
procedure does not require a specific time-interval-sequence of
observations; in particular, it does not require equally-spaced
observations as do some transform methods.
On the other hand, disadvantages include the need for observations
on each variable in the differential equation at identical times.
Although the procedure is
comp~tationally
uncomplicated, the choice
of smoothing and derivative estimating functions is a potential
source of poor estimation,
•
7.2
Overview of Open Problems
This work suggests several general areas in which further
research is possible, including the followiqg:
(1)
Investigation of other choices of smoothing and derivative
estimation functions.
(2)
Extension of the basic method to different forms of the
model differential equation,
(3)
Investigation of minimizing a weighted Sobolev norm-type
expression.
Each of the above areas will be considered in more detail, although
their inter-relationships prevent
separa~e
and exclusive treatments.
In this thesis, we have considered two of the most common types
of smoothing and
11
•
d~rivative
polynomial function and (2)
estimation functiQns, (1)
the moving-arc
the continuous function defined over the
entire time-span of the data, an example of which is the function
composed of joined cubic segments.
Obviously, neither type of function
•
98
completely smoothed the observations and neither type provided exact
estimates of the derivatives.
This was hardly unexpected, since
polynomial approximations to exponential functions are not exact.
However, the use of approximating polynomials does involve the
assumption that the polynomial is a "suitable" approximating function
to the underlying data-generating function at least over the span
of fit.
Intuitively, the term "suitable" is associated with such
equally-undefined notions as "not distorting the data", whicn, for
example, can be interpreted as yieldiJ;1g values "very close" to those of
t:: underlying function.
Further, when derivative estimation is
accomplished by means of computing the first time-derivative of the
•
approximating function, the scope of meaning of "suitable" is
enlarged to include such features as preservation of monotonicity,
the presence of maxima (or minima), and the existence of asymptotes.
This type of derivative estimation has been iJ;1vestigated by many
authors (for example, se~ Carpahan, et ~., 1969), nearly all of
whom warn that small errors in the approximating function tend to
be magnified in differentiation.
One shoQld, then, strive for a balance on one hand of faithfulness to the underlying function and, on the other hand, of
reduction, by smoothing, of peculiarities in the observations which
are present due to error.
It is with these goals in mind that the
minimization of a Sobolev norm-type ex~ression (equation (3.1)) was
•
proposed •
Specifically, in the case of moving-arc polynomial approximations,
•
consideration
nomial~
mu~t
be given to the degree of the approximating poly-
the span of the arG, and the treatment of the first and last
•
99
groups of observations.
Most literature dealing with choosing the
appropriate degree for the approximating polynomial is based on
applications to physical experiments with many more observations than
we have assumed to be
availabl~
(~.g., see Savitzky and Golay,
from biological experimentation
1964).
Unfortunately, even the
number of points considered in one span is frequently larger than
the total number available from one biological experiment.
the case in the paper by Luers and Wenning
Such is
(1971) who discuss
derivative estimation by least-squares polynomial fitting of either
cubics or straight lines as a function of the
one span.
•
.
n~ber
of points in
Apparently, when relatively few observations are
available, the choice of both degree and number of points in a span
is as much art as science and is based mostly on the intuition and
experience of the experimenter.
In some biological applications,
knowledge of initial values suggests a particular smoothing and/or
derivative estimation function,
For the case
0::('
approximation by segmented polynomials, choices
involving the number and degree of segments and placement of the
knots must be made.
Some work has been done on choosing knots by
least-squares criteria, as in the papers by deBoor and Rice
and Hudson
(1968)
(1966), although in most cases it is still necessary for
the user to choose the knots, as Fuller
(1969) did, using the data
on which he reports.
Two potentially profitable directions for further research
include (1) .considerations regarding choice of norm and approximating
•
functions guided by graphical display and interactive computing
devices, which enable the user to make instantaneous decisions
•
100
(see LaFata and Rosen, 1970) and (2)
the approximating function, an
diversification of the form of
~xample
of which is the exponential
spline function developed by Spath (1969) to avoid undesirable
inflection pointE.
Another problem which would be worthy of more research involves
the parameter
w in the Sobolev norm-type expression (3.1).
the case
the
wh~re
K
i
For
are estimated following initial moving-arc
smoothing and derivative estimation, the value of
w is arbitrary.
However, in the cases involving simultaneous smoothing, derivative
estimation, and estimation of the
K ,
i
w has been set equal to
one time-unit-squared in t):lis thl;!sis without explanation.
This
parttcular choioe was made both on the basis of intuition and on
..
the basis of severa+ numerical studies which suggest that this
particular value is, in fact, reasonable.
The first such study
was one in which the single equation model (6.1) from system (6.2)
was investigated for
of the
K.
l
K
0,20 and
c
1
= 0.21 .
K
2
The estimation
was accomplished by simultaneous smoothing, derivative
estimation and estimation of the
t = 0.5 , 5 1 5
with knots at
Section 5.1.2.
K
i
using three cubic segments
10.5,
In this study,
and
15.5, as described in
w in the Sobolev norm-type
expression was allowed to vary,_ The results are shown in Table 7.1
and include
II
tfs = t.
K
lexact
l
.2'
lfd~ = w t [y(t.) - g(t.)J
.
l
•
l
l
2
~
and
suggest that the minimum value of
\f
tf
=
Cr(t.)
- f(t.)J2
l
l
u: + ~
These results
occurs for
time-unit-squared, at least for the values of
w equals one
w considered.
•
101
Table 7.1
Values of Sobolev norm-type expression as a function
of w • K = 0.20 and K = 0.21 •
1
,
2
0&
"K1
w
0.25
0.50
0.75
0.90
1.00
1.10
1.25
1.50
2.00
3.00
"K2
exact
0.1981
0.1984
0.1986
0.1987
0.1988
0.1988
0.1989
0.1989
0.1989
0.1991
1?
s
~
1?
2.586
2.936
3.197
3.319
3.390
3.454
3.539
3.655
3.655
4.042
13.226
12.243
11.816
11.666
11.591
11. 530
11.458
11.373
11.373
11.184
15.812
15.179
15.013
14.985
14.981
14.984
14.997
15.028
15.101
15.226
exact
0.2076
0.2081
0.2083
0.2084
0.2085
0.2085
0.2086
0.2086
0.2086
0.2088
In a second study, errorifree values of
y(t)
and
z(t)
were computed
for the same model equation as in the above example but for the values
of
K
1
and
K
and the time-sequence of observations associated with
a
the example included by Metzler (1969) in the descriptive manual for
the computer program NONLIN (see Table 7.2).
K
<3
Estimates of
K
l
and
were computed simultaneously with smoothing and derivative
t = 0.5 , 5.5
estimation by three cubic segments with knots at
20.5,
and
30.5
time-units.
The values of
"K
,
lexact
u: '
~,
values of
•
•
at
and
va ,
which are displayed in Table 7.3 for several
w, suggest that the minimum value of
w equals one
ti~e-unit-squared.
to investigate the influence of
"Ki
for data with errors.
tf
does occur
It would clearly be worthwhile
w on the quality of the estimators
•
102
Table 7.2
Values of
y(t.)
1
system (6.2) for
K = 0.1155.
2
....
•
1
from solution of
100 , K1 = 0.05775 ,
(From Metzler, 1969)
z
=
o
y( t.)
z(t.)
1.0
2.0
3.0
4.0
5.0
10.0
15.0
20.0
25.0
30.0
10.59
19.44
26.75
32.74
37.58
49.25
48.74
43.16
36.07
29.11
89.09
79.37
70.72
63.00
56.13
31.51
17.68
Table 7.3
0.25
0.5
0.75
0·9
1.0
1.1
1.25
1.5
2.0
3.0
z(t.)
t.
1
w
and
"K1
1
1
9·93
5.57
3.13
Values of Sobolev norm-type expression as a function
K = 0.05775 ,
K = 0.1155 .
of w
1
2
"K
exact
2exact
0.05661
0.05677
0.05687
0.05691
0.05693
0.05695
0.05697
0.05700
0.05703
0.05710
0.1147
0.1147
0 ..n47
0.1148
0.1148
0.1148
0.1148
0.1148
0.1147
0.1147
l?
~
if
0.007516
0.012562
0.016662
0.018746
0.020015
0.021204
0.022864
0.025374
0.0 29758
0.037194
0.081135
0.067005
0.060308
0.057770
0.056432
0.055298
0.053882
0.052049
0.049517
0.046485
0.088651
0.079567
0.076970
0.076516
0.076447
0.076502
0.076746
0.077423
0.079275
0.083679
s
•
'.
103
One generalization of the role of
w as a weighting coefficient
lead q to the following Sobolev norm-type expression
•
if
w
=
[(r _ !)T
.
T
(I - ~) ]
If we let the submatrices
w..
lJ
equal
w11
W
w21
W
V..
[~E-::-J
12
22
-1
where the
lJ
V ..
lJ
are
the submatrices of the variance-covariance matrix
v11
v =
V
lz
,
.'V
-----f-----
V
21:
ZZ
,
associated with the vector
then, for the case involving the model equation
observations
the
e(t.)
1
y(t.) = y(t.) +
e(t.) ,
y = Ky
i = 1, 2, ••• , n,
1 1 1
are independently distributed, and where
dimension (nXl), the matrix
W
11
where
Y is of
is diagonal with elements
(w11 )'i
= var(y(t.»)-l
1
1
•
Therefore,
w
: K-1W
11
•
with
,-
-- ---
K-1W
:
•
11
-.' - - - -
-
: K- 2 W
11 I
I
11
-
•
104
If we assume that
var(y(t.)) =
0
1
2
i = 1, 2, ••• , n , then
,
I
--
- - -,- - - - -1I I -2
I K
I
K
(~:
I
- ~)
I
The Sobolev norm-type expression minimized in this thesis is,
.
in the above form for the case
y
= Ky
,
I
t?
=
[ (~:
(~
!)T
~)TJ
I
I
I
I
(~
0
----1-----
- !)
-------
I
0
I
I
wI
(~
-
-
~)
I
which suggests that our expression, in the sense of this particular
case of variance-covariance weighting,. ignores correlations between
the
Y and the
Kr, and which suggests the role assumed by the
with dimensions identical to those of
K- 2
•
In fact, setting
w
w
equal to one time-unit-squared, as we have done, is tantamount to
designating
var(r)
=
I
and
var(Kr)
=
I.
Obviously, the development of estimators of
K by minimizing
a Sobolev norm-type expression with a weighting scheme is deserving
of future study.
One important aspect of such a study must be the
choosing of the weighting matrices.
Although the inverses of the
covariance matrices are used in the usual linear least-squares
techniques to insure minimum variance of the estimators for the
•
parameters of the model, it is not clear that use of the covariance
matrices in the Sobolev norm-type expression for the estimation of
parameters in differential systems will result in minimum variance
•
105
estimates of the
even exist.
K..
l
does not
Even if the inverses of the partitioned covariance matrices
are used as weighting matrices, the values of
must be known or estimated.
var{Y( t. ))
Y(t.)
l
1.
and of
Such estimation, if required, would
possibly be accomplished by an iterative scheme.
where the
V- 1
In fact, in the above example,
Further, for cases
are normally distributed and where the vector
Y
in the Sobolev norm-type expression is of dimension (mXl), where
m> n
as in simultaneous 5.. point moving-arc smoothing, derivative
estimation, and estimation of
K, the computation of the inverse
of the covariance matrices would probably require use of a form of
the generalized inverse.
A second major area of potential future interest involves the
extension of the basic method,
2:..~.,
minimization of a Sobolev
norm-type express.ion, to other forms of the model differential
equations.
It is our intention to include in this category both
the way the variables occur in the differential equation and the
form of the coefficients.
Perhaps, the simplest example is the
differential equation of the type
dy(t)
dt
K z(t)
which is often incorporated in models of the excretion phase of
pharmacokinetic studies (Cummings and Martin, 1964), where
..
•
is the amount of a drug metabolite in the body and
z(t)
y(t)
is the
amount of drug in the body.
Assuming that observed values of each
of the variables
are available at discrete times
y
and
z
t. ,
l .
then the Sobolev norm-type expression to be minimized is of the form
•
106
...
.,
from which the following normal equations are derived
~T~ +
- - - .-
W
- - - -
_wZT~
- I
t
t
1
- --I
I
I
"
-w ~/TZ
=
"
wZTZ
K
in the case
of the
0
yields
an expression which is similar to equations
"K
~Ty
~
----------
1
"K
Solving for
I
~/T~,
3l
= Ky
dt
•
(4.5) and (4.18) for
Obviously, the derivation of the estimate
K (equation 7.1) is straightforward; the investigation of the
properties of
"K
is made easier by the assumption of additive and
independent errors
e(t.)
l
and
'(t.)
l
associated with
y(t.)
l
and
z(t.)
, respectively.
l
A second example is the well-known system
proposed by Volterra in his study of fish populations (see Gael
(1971)).
al.
Depending on whether or not relationships exist between the
K. , estimates of the
K.
l
•
~
in Sections
the product
l
5.1 or 5.3
yTZ
= (x:
can be derived from the methods discussed
It is the presence of non-linear errors from
+
~)T(~ +
£)
which calls for further investigation.
•
.
107
A third related example which involves both the presence of
variables non-linearly and a non-linear relationship between the
K.1
is the differential equation
~
dt
= K Y
1
K
where
1
K =
2
, which Gause (1934) applied in his famous work on
K
2
yeast growth, where
y
is the size (in volume units) of a pure
culture of Saccharomyces cerevisiae.
From his observations, which
are shown in Table 7.4, Gause estimated
K ,the maximum population
2
size, by visual methods to be 13.0 volume-units and
"coefficient of geometr ic increase",
on
to be 0.21827.
t
the
K
1
by linear regre s sion of log [K ~ y]
2
Since equation (7.2) is linear in
K
K
K
and
1
~
, it is of interest to obtain estimates of
2
and hence, of
K
1
expression (3.1).
and
1
1
K
2
K ,by minimization of the Sobolev norm-type
2
K.
1
by one cubic segment with knots at
and 40 time-units, the estimates of
obtained.
and
K
For simultaneous smoothing, derivative estimation,
and estimation of the
•
and
1
K = 0.184
1
and
K
2
t = 6
13.28 are
This particular choice of knots, although it eliminates
consideration of the observations at
t = 48
and 53 time-units, is
justified by knowledge of the behavior of the cubic polynomial in
•
108
...
approximating a group of observations near the horizontal asymptote.
R by regressing
It might be worthwhile to compare the quality of
•
K2
log
[
-
y]
on
t
2
with our method.
Y
Table 7.4
Observations reported by Gause (1934) for the
growth of Saccharomyces cerevisiae.
time, hrs.
population size (volume units)
6
0.37
16
8.87
24
10.66
29
12.50
40
13.27
48
12.87
53
12.70
As a preliminary demonstration inspired by Gause's data, values
of
K
y(t)
=
2
were computed from equation (7.2) for
13.0 , y(6)
step-size two.
=
0.4
For the eight constructed values of
K
=
3
method.
•
and
and
K
and
1
above simultaneous procedure with
p
1
0.25 ,
by a fourth-order Runge-Kutta scheme with
in Table 7.5, estimates of
t = 5.5 , 27.0,
K
cubic segments with knots at
48.5 time-units.
5, estimates of
K
1
and
shown
were obtained by the,
2
t~o
y(t)
From artificially constructed
K
2
were computed by the same
The results are displayed in Table 7.6 and exhibit a strong
suggestion of significant bias in
"K1
,and, therefore, in
"K
2
Interest in models of the type (7.2) sho~ld probably initially
•
109
•
center on the effect of
square~
error terms, that is, on comparing the
influence of either squaring the smoothed
•
y( t. )
or smoothing the
l
y(t.):3 .
~
Table 7.5
Values of
of
y(t.)
computed by numerical integration
~
-YJ.
K
~:::Ky_:3~_
[
dt
1
K
K ::: 0.4 ,
1
K
:3
::: 13.0 .
:3
time
riJJ.
6
12
18
24
30
36
42
48
0.4
1.62
5.06
9.63
12.06
12.78
12.95
12·99
..
Table 7.6
Estimates of
K
and
1
K
:3
in
~ _ K
dt -
lY
Y]
K :3
[ ~K--
:3
Entries are means and standard deviations (in
parentheses) of samples of size n. K ::: 0.25
1
and K ::: 13.0 •
:3
II
p
n
o
3
20
5
25
•
•
K
1
0.2417
0.2358
(0.0143)
0.2208
(0.0271)
II
K
:3
13.087
13.160
(0.3350)
13.390
(0.5053)
•
110
•
The final example of the types of differential equations suggested
for future investigation is of the form
•
~=
dt
K
K
1
2
Y
which was suggested from empirical considerations in a study of water
vapor sorption in wood (Kelly and Hart, 1970).
in the estimation of the
K.
1
One possible approach
involves the use of transformations to
yield an equation of the form
log
~
dt
=
log K + K log Y
1
2
Another suggested area of further research is motivated by two
facts:
(1)
Sigmoid curves are often found in studies relating drug
dose and physiological response.
(2)
Certain differential equations are known to have solutions,
the graphs of which are sigmoid curves.
In many experimental situations, an estimate is sought either for the
dose which is lethal to 50 percent of the experimental subjects,
the so-called
LD
50
~.~.,
, or for the dose which induces a 50 percent-of-
Most current methods of
50
estimating these parameters rely on some scheme which purports to
maximum response,
~.~.,
the so-called
ED
linearize the dose-response relationship to permit the application of
simple linear regression techniques.
•
Besides often lacking theoretical
foundations, the basis of these methods frequently puts restrictions
on the inclusion of observations associated with 0 percent and 100
•
III
•
•
percent responses.
The estimation techniques advanced in this thesis
have potential value in deriving estimators of the dose-response
parameters, since, in many interesting cases, these parameters are
functions of the coefficients of the differential equations whose
solutions are sigmoid curves.
The final recommended area of further research is that which
would compare the methods of this thesis with those presently available
under the general title of non-linear regression procedures, which
involve minimizing the sum of squared deviations between either
analytical or numerical solutions to the differential equations and
the observed values •
•
•
.'
112
8.
LIST OF REFERENCES
Blankinship, S. (ed.) 1971. The User's Guide to PL/C. Document
Nwnber LSR-095-1, Triangle Universities Computation Center,
Research Triangle Park, North Carolina.
deBoor, C. and J. Rice. 1968. Least Square Cubic Spline Approximation
I: Fixed Knots. Report CSD TR 20, Computer Science Department,
Purdue University, Lafayette, Indiana.
deBoor, C. and J. Rice. 1968. Least Square Cubic Spline Approximation
II: Variable Knots. Report CSD TR 21, Computer Science Department, Purdue University, Lafayette, Indiana.
Cannon, J. R. and D. L. Filmer. 1967. The Determination of Unknown
Parameters in Analytic Systems of Ordinary Differential Equations.
SIAM Journal in Applied Mathematics 15:799-809.
,
Cannon, J. R. and D. L. Filmer. 1968. A Numerical Experiment on the
Determination of Unknown Parameters in an Analytic System of
Ordinary Differential Equations. Mathematical Biosciences
3:267-274.
Carnahan, B., H. A. Luther, and J. O. Wilkes. 1969. Applied Nwnerical
Methods. John Wiley and Sons, Inc., New York, New York.
Cleland, W. W. 1967. The Statistical Analysis of Enzyme Kinetic
Data, pp. 1-32. In F. F. Nord (ed.), Advances in Enzymology,
Volume 29.
Cook, D. A. and G. S. Taylor. 1971. The Use of the APL/360 System
in Pharmacology: A Computer Assisted Analysis of Efflux Data.
Computers and Biomedical Research 4:157-166.
Cummings, A. J. and B. K. Martin. 1964. Drug Elimination by
Simultaneous First-Order and Zero-Order Processes. Nature
202 :779-'780.
Evert, C. F. and M. J. Randall. '1970. Formulation and Computation
of Compartment Models. Journal of Pharmaceutical Sciences
59:403-409.
•
•
Fuller, W. A. 1969. Grafted Polynomials as Approximating Functions.
The Australian Journal of Agricultural Economics 13:35-46.
(Reprint Series Nwnber 255, Statistical Laboratory, Iowa State
University of Science and Technology, Ames, Iowa.)
Gardner, D. G. 1963. Resolution of Multi-Component Exponential
Decay Curves Using Fourier Transforms, pp. 195-203. In
Multi-Compartment Analysis of Tracer Experiments, Annals of
the New York Academy of Sciences, Volwne 108, Article 1.
•
113
Ga~se,
G. F. 1934. The Struggle for Existence.
Wilkins Company, Baltimore, Maryland.
The Williams and
Glass, H. I. and A. C. deGarreta. 1967. Quantitative Analysis of
Exponential Curve Fitting for Biological Applications. Physics
in Medicine a.nd Biology 12:379-3.38.
Goel, N. S., S. C. Maitra, and E.W. Montrol1. 1971. Nonlinear
Models of Interacting Populations. Academic Press, Inc.,
New York, New York.
Goldberger, A. S. 1964. Econometric Theory.
Inc., New York, New York.
John Wiley and Sons,
Greville, T. N. E. 1968. Data Fitting by Spline Functions.
Mathematics Research Center Technical Summary Report Number 893,
The University of Wisconsin, Madison, Wisconsin.
,
Greville, T. N. E. 1969. Introduction to Spline Functions, pp. 1-35.
In T. N. E. Greville (ed.), Theory and Applications of Spline
Functions. Academic Press, New York, New York.
Heinmets, F. 1970. Quantitative Cellular Biology: An Approach to
the Quantitative Analysis of Life Processes. Marcel Dekker,
New York, New York.
..
Hudson, D. J. 1966. Fitting Segmented Curves Whose Join Points
Have 1- Be Estimated. Journal of the American Statistical
Association 61:1097-1129.
,wksch, H. C. 1966. Random Errors of Derivatives Obtained from
Least Squares Approximations to Empirical Functions. SIAM
Review 8:47-56.
Kelly, M. W. and C. A. Hart. 1970. Water Vapor Sorption Rates by
Wood Cell Walls. Wood and Fiber 1:270-282.
LaFata, P. and J. B. Rosen. 1970. An Interactive Display for'
Approximation by Linear Programming. Communications of the
ACM 13:651-659.
Lewi, P. J., A. E. F. Chandler, D. G. VanRiel, P. J. Somers,
F. T. N. A11ewijn, J. G. H. Dony, and P. A. J. Janssen. 1970.
Computer Procedures for Evaluating the Results of Studies on
the Distribution and Metabolism of Drugs in the Body'.
Arzneimittel-Forschung 20:684-693.
Luers, J. K. and R. H. Wenning. 1971. Polynomial Smoothing Linear VS Cubic. Technometrics 13:589-600.
•
Martin, B. K. 1967. Drug Urinary Excretion Data: Some Aspects
Concerning the Interpretation. British Journal of Pharmacology
and Chemotherapy 29:181-193.
•
114
•
c. M. j969. A:j'isers ManuaJ, 1'0[' NONLIN. Technical Report
Numter 7292/69/7292/005, 'l'he Upjohn Company, Kalamazoo, Michigan.
MctzleL'~
Metzler, C. M., G. Matrone, and H. L. Lucas, J~. 1965. Estimation
of' 'l'ranspcrt Rates by Radioisotope Stlldies of J.'iro!J.~s t;eady-State
S;;rstems. Institute of Statistics Mimeo Sel-ies Number )+46,
North Carolina State University) Raleigh, North Ce.:c'olina.
Myhi11~
J. 196'7. Investigation of the Effect of Data. Error in the
Analysis of Biological I'race:..~ Data. Biophysical Joun'!.al
'7: 903-911.
l'1~rhil~tj J.
1969. Ir~vestigatiorl of the Effect of Data Error in the
Analysis of Biologica.l 'I'I-acer Data from a 'l'hree·-Compartment
System. Journal of Theoretical Bi.ology 23:218-231.
Parrish, J. D. and S. B. 8aila. 19'70. Interspecific Competition,
Predation and Species Di.versity. J'ournal of Theoretical
Biology 27:207-220.
,
Perl, W. 1960. A Method for Curve-Fitting by Exponential Functions.
IGter'national Journal of Applied Radiation and Isotopes
8:211-222.
Pizer, S. M., A. B. Ashare, A. B. Callahan, and G. L. Brow:J.el1. 1969.
Fourier Transform Analysis of Tracer Data, pp. 105-129. In
F. Heinmets (ed.), Concepts and Models in Biomathematics:-Simulation Techniques and Methods. Marcel Dekker, New York,
New York.
Reinsch, C. E. 1967. Smoothing by Spline Funct.ions,
Mathematik 10:177-183.
Numerische
Rescigno, A. arid G. Segre.1961. La Cinetica dei Far-mad e dei
Traccianti Radioattivi. Editore Boringhieri Societa per Azioni,
'l'or ino, via Brofferio 3.
English translation, 1966, P. Ariotti
(translator), Drug and Tracer Kinetics. Blaisdell Publishing
Company, Waltham, Massachusetts.
Rosenbrock, H. H. and C. Storey'. 1966. Corr~putat.ioGal Techniques for
Chemical Engic.eers. Pergamon Press, Oxford, England.
Savi.tzky, A. c..od M. J'. E. Golay. 1961.:.. Smoothing and Differentiation
of Data by Simplified. Least Squa:C'es Procedures. A.ile.ly'tical
Chemistry 36:1627-1639.
Spath, H. 1969.
4: 225-·233.
•
Exponential Splir:·e Interpolatior~.
Computing
[3wanr..., W. H. 1969. A Survey of' Non~Linear' Optimization 'l'echniques.
FEBS :L,etters, Volume 2 Supplement:S39~S55 .
1
•
115
•
Westlake, W. J. 19'71. Problems Associated with Analysis of Pharmacokinetic Models. Journal of Pharmaceutical Sciences 60:882-885.
Whittle, P. 1956. The Estimation of Age-Specific Infection Rates
from a Curve of Relative Infection. Biometrics 12:154-162.
t
•
•
116
9.
9.1
APPENDICES
Derivation of Equation (4.11)
The right-hand side of equation (4.10) is of the form
+
,
Since the
e(t i )
=
ei
are assumed to be independent, it follows tqat
the expected value of the second term is
[~e~ac~1 e
y"'CY
[~exac~ 1
::::
yT CY
= -
l~:;::t J
(~
T
1
T
Ce) +--e (~ Be)
yTCY
(~~
~
C.. var( e.) +_1_
i
"
1J
11
Now,
•
1
e
-· T C
Tc )2
(2--~
~- + ~ ~
c.. e.1 eJ.)
1J
1
e
+-yT cy
yTcy
(~~ B..
"
lJ
~ B..
.
1
11
1J
e. e .)
var( e.)
1
1
J
(9.2)
•
117
But
and
4Y TCe:Y TCE ::: 4[E e: :<3
+
j
T
T
e: Ce: E Ce:
- -
-
=
terms i.n
E.E
l
o
J
,
i
t-
j
,
E E E E E E E e:. C C.
km lJ
i j m k k m J l
0
E E e:.:<3 e: 2
i k l
k
C..
II
0
C + E E e:':<3
kk
i j
l
E. 2
C.. C..
J
lJ
lJ
+ E E Eo:<3 e: :<3 C . C' - 2 E E. 4 C.. 2
kl l kk
i k l
l
II
+
terms in odd powers of the
E.
l
from which it follows that
,
and
e
T
T
(~~ Ce:)
= E E
. k
l
+
c..
C
n kk
E E Ck,C' k
i
k
l
l
var ( E. ) var( E ) + E E
k
l
. .
l
•
First,
c.lJ.2
var ( E • ) var ( E. )
1
J
var(El.)var(E k )
The expected value of the fourth term of
similar manner.
J
(9.1)
is computed in a
•
118
Now,
e,
(2XTCEyTBe:) == 2
2
t (t
,
J
k
l
t (E
.
k
l
Ck'Yk)(I: B ..Y.) var(e: )
j
J
. lJ l
C . Y )( E B, ,Y ,) var( Ii: • )
kl k
, lJ J
l
J
and
e,
'I'
T
(~ Ce:e: Be:) ==
+ E E C. ,B. , var( e: . ) var( e: , )
,
l
+
,
.
J
Et
i k
lJ lJ
l
J
Ck,B'k var(e:l·)var(e: )
k
l
l
The sum of expressions (9.2) through (9.7), where each is
weighted as in expression (9.1), yields the desired re1ation~hip
(4.11) .
•
it follows that
'.
119
•
(~T~)-l
ZTCZ
_ZT CY
0
0
_yT CZ
yTCY
0
0
1
==
0
0
ZT CZ
0
0
_yT CZ
_ZTCY
yTCY
'Ci
2
Therefore,
~ (~~I'~) -1~ T
=
,
[~
1
:::
d2
0
1
1
0
~J
(~T~)-l
1
~ ~TCZ
_",TCY
_yT CZ
2yTCY
0
0
0
1
1
0
0
1
. 1d2
and
[~(~TK) -l RTJ
Tcy
-1
==
l2:f
ZT CY
1
(~T~)-l~T[~(~T!)-l~TJ
-1
1
- d
2
•
ZTCZ
0
T
-y CZ
0
yT CY
- CZ
-
_yT CZ
ZT CY
yT CY
ZT
_yTCZZTCY
==
l~l
--yTCYZ'l'CY
--
d
.d
2
1
lCy
r
lczJ.
ZTCY
ZT CZ
_yTCZZTCZ
---
yTCYZTCZ
--
1
d
1
d
2
d
1
120
»
However,
~TCZ;yTBX _ ~TCYZTBY
r - Rb
[~] [~
=
0
1
1
0
~]
yTCYZTBY _ yTCZyTBY
---
---
ZTCZyTBZ _ ZTCYZTBZ
---
---
1
~
I?CYZ T~ _ I?CZyTBZ
o
. -1d
=
o
so
_yTCZZTCY
---
=
1
d d
1
yTCYZTCY
---
_yTCZZTCZ
-~-
yTCYZTCZ
---
2
and the indicat~d product is exactly equation (5.26) .
•.
•
,
2