0021
9?00/81,0101
CO?
$O?.o(I/O
THE USE OF OPTIMALLY REGULARIZED
FOURIER
SERIES FOR ESTIMATING HIGHER-ORDER
DERIVATIVES OF NOISY BIOMECHANICAL DATA*
H. HATZE
National Research Institute for Mathematical
Sciences,
CSIR, P.O. Box 395, Pretoria 0001, South Africa
Abstract Optimally regularized (filtered) Fourier series can be used most effectively for estimating higherorder derivatives of noisy data sequences, such as occur in biomechanical
investigations.
As the optimal
filtering process takes place in the derivative domain, use being made of the statistical information contained
in the periodogram
of the data sequence, no numerical differentiation
as such takes place, and all the
problems known to be associated with such a procedure are avoided. The optimally filtered approximation
to the function itself is obtained by the same technique. The algorithm is based on known results from time
series and approximation
theory and offers several advantages when compared with other techniques: the
optimally filtered approximations
of the function and all its required derivatives are obtained in a single run;
compact analytical Fourier representations
of the filtered function and the derivatives are available, allowing
interpolation
between data points and further analytical
treatment
of the function;
all the desirable
properties of Fourier approximations
such as periodicity, end point continuity, etc., can be utilized; if fast
Fourier transforms are used, the algorithm is computationally
far more efficient than other techniques.
spline approximation
of the data sequence. A wrong
guess results in grossly over- or undersmoothed
derivative estimates.
Direct application of orthogonal
Chebyshev polynomial fits have been shown (Zernicke et al., 1976;
Pezzack et al., 1977) to produce highly oversmoothed
and unrealistic second derivatives.
Jackson (1979) has recently presrnted a method of
choosing the order of a polynomial or Fourier series
such that reasonable
estimates of the higher-order
derivatives
of a data sequence are obtained.
The
average discrepancy (residual) between the data and a
polynomial (Fourier series) fit of given order is determined as a function of this order, and the first and
second derivatives of this function are calculated from
linear interpolations
between function values. If the
absolute value of the second derivative has been less
than a prescribed level for three consecutive orders of
approximation
the cutoff point is deemed to have
occurred, thus determining the order of the approximation. The “prescribed level” mentioned above is,
however, not clearly defined. It is based on the
discontinuous
behaviour
of the second derivative,
which itself was constructed
from finite differences
applied to an inherently continuous residual function.
Hence it is not clear what meaning the discontinuities
of the second derivative and thus the “prescribed level”
actually have. It thus appears that the choice of the
order of the polynomial or Fourier approximation
is
somewhat arbitrary. It is not possible to assess the
validity of this method since Jackson does not support
his claim of good derivative approximations
by experimental evidence.
In this paper we shall outline a different algorithm
for obtaining higher-order
derivatives of noisy obser-
INTRODUCTION
It is notoriously difficult to compute the higher-order
derivatives of a sequence of experimental observations
contaminated
with noise. Yet such estimates of derivatives are required in many areas where it is difficult
or impossible to obtain this information
by direct
measurement,
but where the history of the displacement process whose derivatives are required can be
observed relatively easily. One such area is the analysis
of displacement
data obtained in the study of human
motion.
A number of papers dealing with this topic have
appeared recently in the biomechanics literature. Zernicke et al. (1976) have proposed the use of cubic
splines, an approach which has been shown (Wood
and Jennings, 1979) to yield less accurate results than
the use of quintic splines. Pezzack et al. (1977)
advocate a technique where the raw data are first
smoothed by a second-order,
recursive Butterworth
digital filter and then differentiated
by using a firstorder finite-difference method. Their computed second
derivative agrees well with the directly measured
acceleration record. However, a disadvantage
of this
technique is that the user has to decide on the cutoff
frequency of the filter after repeatedly comparing the
computed filtered second derivative with the finitedifference differentiation
of the unfiltered data. A
similar disadvantage
is inherent in the method of
Soudan and Dierckx (1979) where the user has to
guess, by trial and error, a smoothing factor S for the
*Received 26 October 1979: in revised form 9 May 1980.
13
H. HATZE
14
vations. The approach is based on the regularization
procedure of Cullum (1971) and its extension by
Anderssen and Bloomfield (1974), which references
should be consulted for the detailed theoretical background. A brief account of the method, when used to
estimate first derivatives, was given by Laurie (1975).
We shall give a short description of the method when
applied to estimates of higher-order derivatives, and
demonstrate its effectiveness by obtaining an estimate
of the second derivative of the displacement data of
Pezzack et at. (1977). A complete derivation of the
computational algorithm is given in Hatze (1979b).
observed process {gk} are used to determine the best
possible filter function $‘)(&,m). Further details can be
found in Anderssen et al. (1974) and in Hatze (1979b).
Since {gk}has been approximated by a Fourier sine
series, the second, first and zero-th optimally regularized (filtered) derivative approximations are (for
0 < t < I) respectively given by
N-l
f@‘(t)
=
C
-
(ltj/I)’
c:_‘)
-
y,)/Z + c
sin(njt/Z),
(5)
j=l
N-l
f(l)(t)
=
(yN
(nj/Z)c$rrcos(njt/Z),
(6)
j=l
OPTIMALLY
REGULARIZED
FOURIER
APPROXIMATION
N-l
SERIES
f(t)
=
Y,
+
(Ye - Y&/Z + C c$“sin(xjr/Z),
(7)
j=l
The method requires a sufficiently large number
(N + 1) of observations {y,; k = 0,1,.. ., IV} to have
been made at equidistant time points tk = k6, k
= 0, 1,. . ., N, tL+ i - tl, p 6, such that little or no detail
of the underlying function f(t) has been lost in the
sampling process over the total sampling time I. It also
requires the transformed (detrended) data, defined by
& 4 Y, - Ye _ K!$
k,
k=O,l,...,
N,
(1)
to be considered a weakly stationary stochastic process, an assumption which is valid for data sequences
observed in motion analysis and many other biomechanical investigations. Because of the transformation (l), go = gN = 0, which permits an odd extension of {gr) and hence a representation in a Fourier
sine series.
Under these assumptions it is possible to represent
by Fourier integrals the transformed underlying function G(t) (which is assumed n times differentiable), its
n derivatives, the (with G(r) uncorrelated) observation
errors
Ed4 g, - G(Q),
rk = kA = k/N = tJ1,
(2)
and gk, k = 0, 1, . . ., N. An approximate expression can
then be derived for the optimal nth derivative which
minimizes the mean squared error. This expression has
the form
/$’ =
_,,F<,
(i~JA)Q”‘(w,)
exp(ike+), (3)
I.
where oj = 2nj!2N (since the period of the periodically
extended data is 2N), the complex Fourier coefficients
are given by
g, = (2N)-’
c
gt exp( - iio&
(4)
-7I<o,C*
and p)(wj) denote the optimal filter weights for the nth
derivative. The function P”)(wj) can be approximated
by a function l,(“)(&,m, wj) which, for given m, depends
on the optimal value &of the regularization parameter
a. This optimal value of a can be found by minimizing a
certain logarithmic function containing the periodogram of the data. Hence the statistical properties of the
where cy’ = g&$“‘@,2), and I = N6 is the total
sampling time of the data sequence. If consistency
among derivatives is required, then the filtering process can be applied to the highest derivative only [so
that c$” = cy’ in (5) -(7)] and all lower derivatives may
be obtained by integration. Oversmoothing of lowerorder derivatives can occur in this situation. This can,
however, easily be checked by comparing the filtered
function values with the raw data.
NUMERICAL
RESULTS
The above algorithm was coded in ANSI FORTRAN IV and executed on a CDC CYBER 174. For
ease of comparison with previously published results
we have chosen as input data the angular displacement
history provided by Pezzack et al. (1977). The data
length (N + 1) is 142, and the sampling time interval 6
is 0.0201 seconds. The starting time t, has been set
equal to zero.
Figure 1 shows the raw data of the angular displacement (continuous line) and the corresponding secondorder filtered Fourier approximation (crosses). Figure
2 depicts the measured angular acceleration (continuous line) and the corresponding optimally filtered
Fourier approximation
of the second derivative
(crosses), while in Figs. 3 and 4 we demonstrate that
the present method can be applied successfully to data
sets which are severely contaminated by noise (details
in the legends of these figures).
In the present example we have put c!) = cy’ in
(5)-(7), i.e. optimal filtering was only applied to the
second derivative in order to maintain consistency
amongf@)(t), f(‘)(t), and f(t). As can be seen from Fig.
I this procedure does not result in oversmoothing of
f 0).
DISCUSSION
AND CONCLUSION
It can be seen from Figs. 1 and 2 that the present
method yields satisfactory approximations
of the
angular displacement and acceleration data, but SO do
the techniques proposed by Pezzack et al. (1977),
Wood and Jennings (1979), and Soudan and Dierckx
Estimating
higher-order
15
derivatives
Angular displacement
32or
24O-
2 ooD
e
GOi.
.z
I120-
oeo-
040.
OOOL
L
I
080
I
I 20
,
I
I 60
I1
200
I,
240
Time,
Fig. I. Raw data
approximation
I
280
<I
I
320
I
360
11
400
I
40
set
of angular displacement
(continuous
line) and corresponding
second-order
(crosses). Data are from film analysis of horizontal elbow flexion (Pezzack
filtered Fourier
et al., 1977).
Angular acceleration
1200 7
8
0
-
900
-
600
-
3.00
;
cm000
e
D
,o
0
5300
-
-
z
-6OO-
msoo-
-1200 -
I
I
080
120
1
,
160
200
,
1,
,
,
I
240
280
320
360
Time,
I
I
400
1
‘
0
set
Fig. 2. Analog accelerometer output (from Pezzack et al., 1977) ofangular acceleration ofelbow flexion (continuous
line) and corresponding
optimally regularized Fourier approximation
of second derivative (crosses).
16
H.
HATZE
Angulardisplacement
3.20 -
2.80
-
240
-
080
120
160
200
2.40
Time,
2.80
320
3 60
4.00
J
4.40
set
Fig. 3. Pezzack’s (1977) raw data (dots) contaminated
by normally distributed
random noise, obtained from the
random-noise
generating
IMSL-routine
GGNML,
and corresponding
second-order
filtered Fourier approximation (continuous line). The standard deviation of the random noise is 0.05, thus introducing
an average error of
+ 5 % into the data set.
(1979). The present algorithm has, however, several
unique advantages : first, the algorithm automatically
selects the optimal jltering window for a given data
sequence in a single computation, thus obviating the
need for repeated trial-and-error comparison of filtered and unfiltered derivatives, as required by
Pezzack’s, and Soudan and Dierckx’s techniques.
Secondly, equations
(5)-(7) represent compact
analytical expressions for the optimally filtered approximations of the function f(t) and its derivatives.
These expressions exhibit all the desirable properties
of Fourier representations such as periodicity, continuity at the end points of the range for all derivatives,
etc. ; properties which polynomials and hence splines
do not possess. Interpolation between data points, i.e.
at t # k6, k = 0, 1,. . ., N, 0 < t < I, is of course also
possible for f(t) and all its derivatives, and the
possibilities of an analytical treatment of f’“‘(t), n
= 0,1,2, are greatly enhanced by the Fourier representations of these functions. Thirdly, optimally filtered derivatives are computed directly, thereby effectively eliminating the process of numerical differentiation and all its associated problems. Fourthly, the
algorithm is computationally far more efficient than
comparable techniques, if fast Fourier transforms are
used. The execution time for the program ORFOS
executing the present algorithm on a CDC CYBER
174 digital computer was 1.93 s for the present input
data sequence, which computing time included that
required to find the optima1 value of the second-order
regularization parameter tl. For comparison, a run on
the same computer, using the IMSL-routine ICSVKU
for cubic spline least-square approximation of the
given data, required 86.4 s execution time, without any
derivative values being computed. If quintic splines
were used and derivatives computed it is safe to
estimate that the present algorithm would be at least
50 times more efficient computationally. This does not
take into account the additional computational cost
incurred by the trial-and-error repetitions required by
Soudan and Dierckx’s method.
The program also makes it possible to compute the
optima1 filtering of each of the derivatives (5)-(7)
separately, and in this case prints out the three sets of
optimally filtered Fourier coefficients cy), c$l), cy’, j
= 0 , . . ., N, in addition to the smoothed function values
f(*)(t,J, f(l)(t,J, f(‘)(t,J, k = 0, 1,.. . , N. Consistency
among derivatives is, however, obviously not maintained with this option.
PRACTICAL
IMPLEMENTATION
METHOD
OF THE
In this section we shall summarize those characteris-
Estimating
higher-order
derivatives
17
Angular acceleration
-6.00
-900
-12.00
Time,
set
Fig. 4. Optimally
regularized
contaminated
data set (dots)
approximation
Fourier approximation
of the second derivative,
obtained
from the noiseshown in Fig. 3. Note the good agreement between the present derivative
and the measured angular acceleration displayed in Fig. 2 (continuous line).
tics of the method with are important for its practical
implementation.
The
method
is applicable
to
any
noisecontaminated
multidimensional
sequence of experimental observations,
which have been made at equidistant time points t,, k = O,l,. .., N. The observations
may be linear or angular displacement coordinates of
body parts, or forces, torques, etc. It is also possible to
process simultaneously
more than one data sequence
(i.e. a multidimensional
data sequence) such as result
from observations of human body motions with many
degrees of freedom. As pointed out above, the method
requires a sufficiently large number of observations,
such that no detail of the underlying function f (t) has
been lost in the sampling process over the total sampling
period. Unfortunately,
no detailed quantification
is
possible since this minimum number of observations
depends on the structure of the function f(t). Obviously, comparatively
smooth functions need fewer
sampling points than functions which exhibit erratic
behaviour. If doubt about the functional behaviour
exists, the use of a larger number of sampling points is
advisable.
Regarding
the computed
results, the computer
program ORFOS provides for two options:
With Option I, the optimal filtering process is applied
to each derivative (including the zero-th) individually,
resulting in diferent Fourier coefficients cy’, cl”, and
I$‘). Hence Equations (6) and (5) cannot be obtained
from (7) by differentiation.
This means that consistency among derivatives is not maintained with this
option.
However, it is often desirable to maintain consistency among derivatives to allow for an analytical
treatment of the data sequences, i.e. (5) and (6) should
result from (7) by direct differentiation.
This is
achieved by using Option 2, in which case the optimal
filtering process is applied to the second derivative
only, and the lower-order derivatives are obtained by
integration. With this option, however, oversmoothing
of the first and zero-th derivatives may occur in some
cases. The check for this is extremely easy: the user
runs the program first with Option 1, and then with
Option 2. A comparison
of the results immediately
reveals whether or not significant oversmoothing
in
the lower-order derivatives has occurred with Option
2.
Finally, we should like to reiterate that the program
is designed to process simultaneously
multidimensional data sequences such as result from observations
ofhuman body motions with many degrees of freedom.
For the author’s 17-segment model having 21 (configurational) degrees of freedom (Hatze, 1979a), the
evaluation of the optimally filtered second derivatives,
H. HATZE
18
and the corresponding
first and zero-order derivatives
for 137 sampling points of an observed long-jump
take-off phase required only 28.64 s execution time for
all the 21 data sequences. However, when compared
with the 0.515 s computer time needed for the program
SEMCI (Hatze, 1979a) to obtain the whole set of body
segment parameters
(principal moments of inertia,
masses, centroids, etc.) for the 17-segment model from
a battery of 242 anthropometric
input data, this
computational
investment appears considerable.
The source listing of the computer program ORFOS
executing the present algorithm is available from the
author upon request.
Acknowledgement - I thank J. Geyer
with the programming of the algorithm.
for assistance
REFERENCES
Anderssen, R. S. and Bloomfield, P. (1974) Numerical
differentiation procedures for non-exact data. Numer.
Math. 22, 157-182.
Cullum, J. (1971) Numerical differentiation and regularization. SIAM J. Numer. Anal. 8, 254-265.
Hatze, H. (1979a) A modelfor the computational determination
of parameter values of anthropomorphic segments. CSIR
Techn. Report TWISK 79, Pretoria.
Hatze, H. (1979b) An algorithm for computing higher order
derivatiues of noisy experimental data sequences. CSIR
Techn. Report TWISK 123, Pretoria.
Jackson, K. M. (1979) Fitting of mathematical functions to
biomechanical data. IEEE Tr. Biomed. Eng. BME-26, (2),
122-124.
Laurie, D. P. (1975) Numerical dz$erentiation of experimental
data. CSIR Special Report WISK 168, Pretoria.
Pezzack, J. C., Norman, R. W. and Winter, D. A. (1977) An
assessment of derivative determining techniques used for
motion analvsis. J. Biomechanics 10.377-382.
Soudan, K. and Dierckx, P. (1979) Cal&nation of derivatives
and Fourier coefficients of human motion data, while using
saline functions. J. Biomechanics 12. 21-26.
Wdod, G. A. and Jennings, L. S. (1979) On the use of sphne
functions ior data smoothing. J. Biomechanics 12,477-479.
Zernicke, R. F., Caldwell, G. and Roberts, E. M. (1976)
Fitting biomechanical data with cubic sphne functions.
Res. Q. 47, (l), 9-19.