IEEE Region 10Conference. T e m
-
11th 13th November, 1992
Melbourne. Australia
ADAPTIVE TRAINING OF ARTIFICIAL NEURAL NETWORK
S.A.
Khaparde
Parnerkar
N.S. Hiremath
Indian Institute of Technology
Bombay-400 076, India
A.
B.J. Sheshaprasad
university College of
Central Queensland
Rockhampton Q 4702
Australia
Abstract
Adaptive training of neural network for non-stationary process
is reported in the framework of multilayer perceptron model using
Back Propagation (BP) algorithm.
The error introduced by small
changes in system parameters is reflected to adapt the changes in
the converged weight matrix.
The error is minimized using
constrained optimization method like Gradient Projection Method
(GPM).
The method is applied for harmonic prediction in voltage
waveform. The results for sample system are discussed.
I.
Introduction
Multilayered preceptron model has established itself as a efEeck
ive tool for large variety of applications owing to its BP error
algorithm [l].
Occurrences of slowly varying non-stationary
processes are common in real life. Ref. [21 reports how adaptation
of neural network can incorporate such process, without training
the network all over again.
The overall error is minimized using
Reduced Gradient Method (RGM). The objective function is augmented
using linearization process.
The appgication reported in Ref. [2]
is for forecasting the load which exhibits adaptive nature.
In this report Gradient Projection Method (GPM) is employed for
error minimization. BP algorithm has been reported for harmonic
The adaptive nature of the prediction is explored
prediction [ 3 ] .
and comparison of RGM and GPM is reported here [4,5].
0-7803-W9-2192
$3.00 0 1992 IEEE
525
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 4, 2008 at 06:25 from IEEE Xplore. Restrictions apply.
11.
Problem Formulation
Consider a 2-layer network in N patterns using BP algorithm,
where dimension of x(i) and d(i) is assumed as I and 1 respectively.
Let h be the number of neurons in hidden layer. Let W be the weight
matrix between the input and the hidden layer and V be the vector
between the hidden layer and the output layer.
For a given input, the corresponding output produced is given
by
constitutes the output of the hidden layer with the sigmoidal
function f (x) = l/(l+e-x 1.
W(N) and V(N) are the weight matrices
and are obtained by minimizing the error
U
(3)
The problem can be defined as
Given W(N) and V(N), the N data s_e s and x(N+l), d(N+l
mine W(N+1) and V(N+l) such that E(N+l) is minimized.
, deter-
W(N+l) = W(N) + O W
V(N+1) = V(N) + AV
Y(N+1) = f(VT(N)u + AVTu)
defining ,
N
A E =~
N
Z X%
j
k b jk
N
hwjk
+Z
k=l
bVk
bVk
526
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 4, 2008 at 06:25 from IEEE Xplore. Restrictions apply.
(7)
and constraint equation
c l = zT a
where,
Z
111.
=
[AWvec
A?IT
GPM Algorithm [SI
GPM employs the projection of- vf(Zi) given by eq. (7) on
the constraint eq. (8). The algorithm is:
0.
1.
2.
Start with initial feasible solution 2
Set i = 1
Evaluate the projection matrix Pi
pi = I
-
G ( G ~~ 1 - lG~
where G = gradient of the constraint given by eq. (8)
3.
Find the search direction Si as
Si
= -Pi vF(Zi)
4.
If IISilI
5.
Update
6.
Go to 2.
fE
then stop
c
2
as
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 4, 2008 at 06:25 from IEEE Xplore. Restrictions apply.
IV.
Application and Results [ 6 ]
The training sets are generated by adding known harmonics
amplitudes to fundamental signal.
16 sample points on voltage
waveform are considered in one cycle. Large number of training sets
over entire operating range are considered.
( N + 1 ) th pattern is
for adaptation lying outside the operating range. Various values
of gain and momentum terms were attempted.
Accuracy of RGM and GPM method was tested against training the
network with ( N + 1 ) patterns afresh.
The mean error for RGM
0.89 x 10”
and for GPM, 1.9 x lo’*.
The
errors
are
within
specified limits. The
results showing the harmonic amplitude
obtained by different methods are displayed on the graph in Figure
shown below.
0.80
COMPARISON BETWEEN DIFFERENT METHODS
-
.e---
D
4
I
I
I
-
0.60
-
- c
z
C
z
3
n
a 0.40 I
k-
CL
0
W
0
’>
2 0.20 f
C D -
A
2
a
B
PESULTS
RESULTS
RESULTS
RESULTS
WITH
WITH
WITH
WITH
TRAINING 6 PAlTERNS
TRAINING 7 PATTERNS
GRADIENT PROJECTION MRHOD(GPM:
REDUCED GRADIENT METHOD(RGM)
1
0.00
1
0
I
I
I
I
I
I
I
I
I
1
I
I
2
I
I
1
I
I
1
1
4
NO
1
1
I
I
OF PATTERNS
1
1
%
I
I
I
6
I
----*
I
I
r
I
r
I
1
11
8
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 4, 2008 at 06:25 from IEEE Xplore. Restrictions apply.
v.
Conclusions
As predicted in ref. [2] it is confirmed that RGM performs
better than GPM. The feasibility of adapting the network for nonstationary process is established. This concept is closely related
with the learning theory where the network would respond to varying
environment automatically. However, it is observed that repetition
of process increases the error.
References
1.
D. Rumelhart, G.E. Hinton and R.J. Williams, "Learning Internal
Representation by Error Back Propagation," Parallel Distributed
Processing: Explorations in the Microstructure of Cognition,
Vol. I, MIT Press, 1986.
2.
D.C. Park, M.A. El-Sharkawi and Robert J. Marks, "An Adaptively
Trained Neural Network," IEEE Transactions on Neural Network,
V o l . 2, NO. 3 , May 1991, pp 334-345.
3.
R.K. Hartana and G.G. Richards, "Harmonic Source Monitoring and
and Identification Using Neural Networks," IEEE Transactions on
Power Systems, V o l . 5, No. 4, November, 1990.
4.
S.S.
5.
Richard L. Fox, "Optimization Methods for Engineering Design,"
Addison-Wesley Publishing Comppny, f971.
6.
Abhay Parnerkar, '"Adaptive Training of Artificial Neural Network
to Predict Harmonics in Power System," M.Tech Dissertation, E.E
Department, IIT Bombay, Feb. 1992.
Rao, "Optimisation Theory and Applications," Wiley Eastern
Limited, 1984.
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 4, 2008 at 06:25 from IEEE Xplore. Restrictions apply.