34th INTERNATIONAL CONFERENCE ON
PRODUCTION ENGINEERING
28. - 30. September 2011, Niš, Serbia
University of Niš, Faculty of Mechanical Engineering
PREDICTION OF THE NONLINEAR STRUCTURAL BEHAVIOUR BY DIGITAL
RECURRENT NEURAL NETWORK
Vesna RANKOVIĆ1, Nenad GRUJOVIĆ1, Dejan DIVAC2, Nikola MILIVOJEVIĆ2, Konstantinos
PAPANIKOLOPOULOS3, Jelena BOROTA1
1
Faculty of Mechanical Engineering, University of Kragujevac, Sestre Janjić 6, Kragujevac, Serbia
2
Jaroslav Černi Institute for the Development of Water Resources, Jaroslava Černog 80, Belgrade, Serbia
3
Institute of Structural Analysis and Seismic Research, National Technical University of Athens, Iroon Polytechneiou 5,
GR-157 80 Zographou, Athens, Greece
[email protected], [email protected], [email protected], [email protected], [email protected]
[email protected]
Abstract: The dynamical systems contain nonlinear relations which are difficult to model with conventional techniques.
Nonlinear models are needed for system analysis, optimization, simulation and diagnosis of nonlinear systems. In
recent years, computational-intelligence techniques such as neural networks, fuzzy logic and combined neuro-fuzzy
systems algorithms have become very effective tools to identification nonlinear plants. The problem of the identification
consists of choosing an identification model and adjusting the parameters such that the response of the model
approximates the response of the real system to the same input. This paper investigates the identification nonlinear
system by Digital Recurrent Neural Network (DRNN). A dynamic backpropagation algorithm is employed to adapt
weights and biases of the DRNN. Mathematical model based on experimental data is developed. Results of simulations
show that the application of the DRN to identification of complex nonlinear structural behaviour gives satisfactory
results.
Key words: identification, structural behaviour, digital recurrent network
1. INTRODUCTION
Nonlinear system identification and prediction is a
complex task. All the processes in nature are nonlinear. In
large number of processes, the nonlinearities are not
prominent, so their behavior can be described by the
linear model. In the linear systems theory there exist a
large number of methods that can be applied for obtaining
the linear model of processes. The nonlinear model must
be chosen when the nonlinearity is strongly exhibited.
Neural network modelling and identification from
experimental data are effective tools for approximation of
uncertain nonlinear dynamic system. Neural networks are
classified into two major categories: feedforward and
recurrent. Most of publications in nonlinear system
identification use feedforward networks, for example
multilayer perceptrons, [1]. The feedforward neural
networks trained with a standard back-propagation
algorithm can be used for the identification of nonlinear
dynamic systems [2]. Chen [3] explained neural networkbased method for determining the dynamic characteristic
parameters of structures from field measurement data.
Conventional back-propagation is used to train the neural
network. However, the conventional back-propagation
algorithm has the problems of local minima and slow rate
of convergence.
An improvement to the back-propagation algorithm based
on the use of an independent, adaptive learning rate
parameter for each weight with adaptable nonlinear
function is presented in [4]. Adaptive time delay neural
network structures was utilized in [5], for nonlinear
system identification. Nonlinear system on-line
identification via dynamic neural networks is studied in
[6]. Reference [7] studied the modeling and prediction of
NARMAX- (nonlinear autoregressive moving average
with exogenous inputs) model-based time series using the
fuzzy neural network methodology.
This paper investigates the identification nonlinear
structural behaviour system by digital recurrent neural
network. Dynamic backpropagation algorithm is used to
adapt weights and biases.
2. METHODS FOR THE IDENTIFICATION
OF NONLINEAR DYNAMIC SYSTEMS
Different methods have been developed in the literature
for nonlinear system identification. These methods use a
parameterized model. The parameters are updated to
minimize an output identification error.
ym  k   f m   k  , 
(1)
ym  k  is the output of the model,   k  is the
regression vector and  is the parameter vector.
Depending on the choice of the regressors in   k  ,
different models can be derived:
NFIR (Nonlinear Finite Impulse Response) model –
where
  k   u  k 1 , u  k  2 ,..., u  k  nu   ,
where nu denotes the maximum lag of the input.
NARX (Nonlinear AutoRegressive with eXogenous
inputs) model:
  k    u  k  1 , u  k  2  ,..., u  k  nu  ,
y  k  1 , y  k  2  ,..., y  k  n y 

,
NARMAX (Nonlinear AutoRegressive Moving Average
with eXogenous inputs) model:
  k    u  k  1 , u  k  2  ,..., u  k  nu  ,
(2)
i 1
where nH is the number of hidden nodes and:
ne is the
  k    u  k  1 , u  k  2  ,..., u  k  nu  ,

in the hidden layer and the output node; bi represents the
biased weight for i-th hidden neuron and b is a biased
weight for the output neuron.
The output of the network is:
nH
e  k  1 , e  k  2  ,..., e  k  ne  
ym  k  1 , ym  k  2  ,..., ym  k  n y 
node. Then the summed signal at a node activates a
nonlinear function. The hidden neurons activation
function is the hyperbolic tangent sigmoid function. In
Fig. 2, i represents the weight that connects the node i
ym  k    i i  b
y  k  1 , y  k  2  ,..., y  k  n y  ,
where e  k  is the prediction error and
maximum lag of the error.
NOE (Nonlinear Output Error) model-
multiplied by weights  yij and summed at each hidden
i 
eni  e ni
eni  e ni
(3)
nu
ny
j 1
j 1
ni   u  k  j  uij   ym  k  j   ym  bi (4)
ij
.
The difference between the output of the plant y  k  and
NBJ (Nonlinear Box-Jenkins) model- uses all four
regressor types.
In this paper, NOE model (Fig. 1) is used for
representation of nonlinear systems.
the output of the network ym  k  is called the prediction
error:
e  k   y  k   ym  k 
(5)
Fig. 1. The general block schema of the NOE model
This error is used to adjust the weights and biases in the
network via the minimization of the following function:
3. DRN NEURAL NETWORK FOR
1 the NOE model 2
NONLINEAR SYSTEM Fig.
IDENTIFICATION
1. The general block schema of
Fig. 2 is an example of a DRN. The output of the network
is feedback to its input. This is a realization of the NOE
model. The output of the network is a function not only of
the weights, biases, and network input, but also of the
outputs of the network at previous points in time. In [7]
dynamic backpropagation algorithm is used to adapt
weights and biases.
DRN network is composed of a nonlinear hidden layer
and a linear output layer. The inputs u  k  1 ,
u  k  2 ,…, u  k  n u  are multiplied by weights uij ,
outputs
ym  k 1 ,

ym  k  2  ,…, ym k  n y

are
   y  k   ym  k  
2
(6)
Using the gradient decent, the weight and bias updating
rules can be described as:
u  k  1  u  k   
ij
y
mij
ij

uij
 k  1   y  k   
mij
bi  k  1  bi  k   

bi
(7)

 ymij
(8)
(9)
b  k  1  b  k   

b
(10)
  k   u1  k 1 , u2  k 1 , ym  k 1 
2 j
is the season varying
365
between 0 and 2 , j represents the number of days since
January 1st.
A data set includes 783 data samples. The available set of
data was divided into two sections as training and test set.
The MATLAB Neural Network Toolbox is applied for the
implementation of the digital recurrent network network.
Different DRNN models were constructed and tested in
order to determine the optimum number of neurons in the
hidden layer. The two-layer network with a tan-sigmoid
transfer function at the hidden layer and a linear transfer
function at the output layer were used. The optimal
network size was selected from the one which resulted in
maximum correlation coefficient for the training and test
sets, Table 1. Based on Table 1, it was concluded that the
optimal number of hidden neurons is 27. The total
number of the parameters of DRN network is 136. In the
where u1 is water level, u2 
where:

 e ym

 e ym
;
;


uij ym uij  yij ym  yij
  e ym   e ym


;
bi ym bi b ym b
where the superscript e indicates an explicit derivative,
not accounting for indirect effects through time.
ym
ym
y m
ym
The terms
,
,
and
must be
b
uij
 yij
bi
propagated forward through time, [8].
Fig. 2. Digital Recurrent Network
4. SIMULATION RESULTS
The major objective of the study presented in this paper is
to construct DRNN model to predict the radial
displacement of arch dam. The DRNN model was
developed and tested using experimental data which are
collected during eleven years.
It is considered the behaviour of nonlinear dynamic
system with two input and one output.
The model input vector is defined by:
learning processes, the weights (108) of the neural
networks were adapted as well as the biases (28).
Table 1. Correlation coefficient for the training and test
sets.
DRNN- 3-20-1 3-24-1 3-27-1 3-30-1
structure
Training 0.889
0.936
0.974
0.966
Test
0.877
0.943
0.972
0.965
Figure 3 shows the measured and models computed
values in training +test set.
[2] NARENDRA, K.S., PARTHASARTHY, K. (1990)
Identification and control of dynamical systems using
Fig. 3. The measured and modelled values in trening+test set.
5. CONCLUSION
[3]
The dynamical systems contain nonlinear relations which
are difficult to model with conventional techniques. In
this paper, DRN has been successfully applied to
unknown nonlinear system identification and modelling.
The real-data set was used to demonstrate the
effectiveness of the proposed approach. Comparing the
modelled values by DRNN with the experimental data
indicates that soft computing model provides accurate
results. In the designing of neural network model, the
problem is how to determine an optimal architecture of
network.
The determination of the values of nu and ny is an open
[4]
[5]
[6]
question. Large time lags result in better prediction of the
NN. However, large nu and ny also result in large
[7]
number of parameters (weights and biases) that need to
be adapted. In considered example, satisfactory results
were obtained for nu1  nu2  ny  1 .
[8]
REFERENCES
[1] YU, W., (2004) Nonlinear system identification using
discrete-time recurrent neural networks with stable
learning algorithms, Information Sciences, Vol. 158
No. 1, pp 131-147.
neural networks, IEEE Trans. Neural Networks, Vol.
1, No. 1, pp 4-27.
CHEN, C.H. (2005) Structural identification from
field measurement data using a neural network,
Smart materials and structures, Vol. 14, No. 3, pp
S104-S115.
GUPTA, P., SINHA, N. K. (1999) An improved
approach for nonlinear system identification using
neural networks, Journal of the Franklin Institute,
Vol. 336, No. 4, pp 721-734.
YAZDIZADEH, A., KHORASANI, K. (2002)
Adaptive time delay neural network structures for
nonlinear system identification, Neurocomputing,
Vol. 47, No. 1-4, pp 207-240.
YU, W., LI, X. (2001) Some New Results on System
Identification with Dynamic Neural Networks, IEEE
Transactions on Neural Networks, Vol. 12, No. 2, pp
GAO, Y., ER, M.J. (2005) NARMAX time series
model prediction: feedforward and recurrent fuzzy
neural network approaches, Fuzzy Sets and Systems,
Vol. 150, No. 2, pp 331-350.
HAGAN, M., JESUS, O. D., SCHULTZ, R. (1999)
Training Recurrent Networks for Filtering and
Control, Chapter 11 of Recurrent Neural Networks:
Design and Applications, L.R. Medsker and L.C.
Jain, Eds., CRC Press, pp 325-354.
ACKNOWLEDGMENT: The part of this research is
supported by Ministry of Science in Serbia, Grants
III41007 and TR37013.
CORRESPONDENCE
Vesna Ranković, Associate Professor, PhD
Department for Applied Mechanics and Automatic
Control, Faculty of Mechanical Engineering, University
of Kragujevac, Sestre Janjić 6, Kragujevac, Serbia
[email protected]
Nenad Grujović, Full Professor, PhD
Department for Applied Mechanics and Automatic
Control, Faculty of Mechanical Engineering, University
of Kragujevac, Sestre Janjić 6, Kragujevac, Serbia
[email protected]
Dejan Divac, Senior Research Associate, PhD
Institute for Development of Water Resources "Jaroslav
Černi", 80 Jaroslava Černog St., 11226
Beli Potok, Serbia
[email protected]
Nikola Milivojević, Research Associate, PhD
Institute for Development of Water Resources "Jaroslav
Černi", 80 Jaroslava Černog St., 11226
Beli Potok, Serbia
[email protected]
Konstantinos Papanikolopoulos, Research Associate,
M.Sc.
Institute of Structural Analysis and Seismic Research,
National Technical University of Athens, Iroon
Polytechneiou 5, GR-157 80 Zographou, Athens, Greece
[email protected]
Jelena Borota, B.Sc
Faculty of Mechanical Engineering, University of
Kragujevac, Sestre Janjić 6, Kragujevac, Serbia
[email protected]