Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
Chapter 5
Adaptive Neuro Fuzzy Inference System
5.1 Introduction
Adaptive Neuro Fuzzy Inference System (ANFIS) is a combination of fuzzy systems and
neural network. In ANFIS, neural network is used to decide parameters of the fuzzy
system. Necessity for manual optimization of the fuzzy system parameters is mainly
removed by ANFIS.
The system parameters are adjusted automatically with use of Neural network. e.g.…
adjusting the membership functions bounds, leading to improved performance
without operator intervention.
“The neuro fuzzy system with the learning capability of neural network and with the
benefits of the rule-base fuzzy system can improve the performance considerably and
can provide a mechanism to incorporate past observations into the classification
process.
In neural network the training basically builds the system. However, using a neuro
fuzzy scheme, the system is built by fuzzy logic definitions and is then refined using
neural network training algorithms.” [21]
Advantages of ANFIS:

To define the behaviour of a complex system. ANFIS improves fuzzy if-then
rules. It does not require prior human expertise

Larger choice of membership functions to use.

It uses membership functions

It has very fast convergence time.

Desired dataset to approximate
51 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
Both neural network and fuzzy logic are universal estimators. They can estimate any
function to any prescribed accuracy, if sufficient hidden neurons and fuzzy rules are
available. Back-propagation and Gradient descent algorithms are used to adjust the
parameters of membership functions (fuzzy sets) and the weights of defuzzification
(neural networks) for fuzzy neural networks. For updating the parameters, ANFIS
applies this two techniques.
The ANFIS is a FIS implemented in the framework of an adaptive fuzzy neural network.
In ANFIS, the knowledge representation of a FIS is combined with the learning power
of ANNs.
The basic objective of ANFIS is to integrate the features of fuzzy systems and neural
network. The advantage of fuzzy is that prior knowledge is denoted into a set of
constraints to decrease the optimization research space. The adaptation of back
propagation to structured network so as to automate fuzzy control parametric tuning
is utilized from Neural Network. ANFIS employs gradient descent algorithm to finetune premise parameters which define membership functions. For consequent
parameters that describe the coefficients of each equation, Least squares method is
used by ANFIS to identify them. This approach combines gradient descent algorithm
and least-squares method, thus called hybrid learning method. To achieve good
generalization of unseen data, the size of the training data set must be as equivalent
atleast as the number of adjustable parameters in ANFIS.
The neurons in ANFIS have different structures.
• The Membership function is described by parameterized soft trapezoids
(Generalized Bell Functions).
• Functions are linear regressions and multiplication with w, that is, normalized
weights ω, and Output (Algebraic Sum).
• The Normalization is by Sum and arithmetic division.
• The rules are differentiable T-norm usually product.
52 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
5.2 ANFIS Architecture
Implementation of Fuzzy rules in Neural Network -FNN
The adaptive neuro fuzzy inference system (ANFIS) architecture is represented by Sugeno
fuzzy models.
Considering 2 inputs x and y and 1 output z. For “first order Sugeno fuzzy model”, a rule set
with 2 fuzzy “if then rules are:
Rule 1: if x is A1 and y is B1 then f1=p1x + q1y + r1
Rule 2: if x is A2 and y is B2 then f2=p2x + q2y + r2
Reasoning mechanism for Sugeno model, the corresponding neuro fuzzy system where nodes
at same layer performs the similar functions”. [Adaptive Neuro Inference systems – J. S. R.
Jang and M. Mizutani]
Layer1: In this layer, each node i is an adaptive node with a node function
O1,i = µAi (x), for i=1,2…
O1,i = µBi-2 (y), for i=1,2…
Node i is given the inputs x (or y) and Ai or (Bi-2) is a linguistic label (“small or large”) related
with this node. The membership grade Oi of fuzzy set A (=A1 or A2, B1 or B2) and it mentions
the degree to which the given input x (or y) satisfies the quantifier A. The membership
function for A can be very suitable parameterized membership function like generalized
function
µAi (x) =
1
|𝑥−𝑐𝑖|2b
1+ 𝑎𝑖
Here (ai, bi, ci) is the parameter set. As the values of these parameters change the function
varies accordingly this exhibiting various forms of membership for fuzzy set.
Parameters in this layer are called premise parameters.
53 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
Layer 2: The multiplication of all received signals is generated as an output by each node in
this layer which is a fixed node is labelled ∏.
O2,i = wi = µAi (x) * µBi (y) , i = 1,2…
Each node output denotes the strength of a rule. In general any other T-norm operators that
perform fuzzy AND can be used as the node function in this layer.
Layer 3: Each node in this layer is a fixed node labeled N. The ith node calculates the ratio of
the ith rules firing strength to the sum of all rules firing strengths:
𝑤𝑖
O3,i = 𝑤𝑖
̅̅̅ = 𝑤1+𝑤2 ; i=1,2,..
Outputs of this function are used for convenience which are known as normalized firing
strengths
Layer 4: Each node i in this layer is an adaptive node with a node function
O4, i = 𝑤𝑖
̅̅̅𝑓𝑖 = 𝑤𝑖
̅̅̅ ( pix + qiy + ri)
Where 𝑤𝑖
̅̅̅ is a normalized firing strength from layer 3 and {pi, qi and ri} are the parameter set
of this node. Parameters in this layer are known as “consequent parameters”.
Layer 5: The single node in this layer is a fixed node labeled ∑, which calculates the overall
output as the addition of all incoming signals :
Overall output = O5,i = ∑𝑖 𝑤𝑖
̅̅̅𝑓𝑖 =
54 | P a g e
∑𝑖 𝑤𝑖𝑓𝑖
∑𝑖 𝑤𝑖
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
Layer 1
Layer2
Layer 3
Layer4
Layer5
A1
∏
X
w1
̅̅̅
𝑤𝑖
A2
∑
B1
∏
y
w1
̅̅̅̅
𝑤2
B2
Figure 5.1 first order Sugeno Fuzzy Model
Here there are two adaptive layers in this ANFIS architecture, namely the first and the
fourth layer. In the first layer, there are three modifiable parameters {ai, bi, ci}, which
are related to the input membership functions. These parameters are the called
“premise parameters”. In the fourth layer, there are also three modifiable parameters
{pi, qi, ri}, pertaining to the first order polynomial. These parameters are called
“consequent parameters” (Jang, 1992, 1993).
5.3 Implementation of ANFIS
Classification using ANFIS
Input Phase
clear
n_trainingdata=input('\nEnter the name of Training Data files (only prefix, without
neither number nor extension): ','s');
n_checkingdata=input('\nEnter the name of Checking Data file (only prefix, without
neither number nor extension): ','s');
55 | P a g e
f
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
n_outputdata=input('\nEnter the name of Output file (only prefix, without neither
number nor extension): ','s');
%===================================================================
%INPUTS
>> Enter name of Training Data files (only prefix, without neither number nor
extension): f1trnData
Enter name of Checking Data file (only prefix, without neither number nor extension):
f1ChkData
Enter name of Output file (only prefix, without neither number nor extension):
f1OutData
%=============================================================
epoch=input('\nNumber of epoch for training process: ');
numMFs=input('\nNumber of membership function: ');
%=============================================================
%INPUT
Number of epoch for training process: 3
Number of membership function: 3
%=============================================================
inmftype=str2mat('gbellmf','trimf','pimf');
% str2mat converts the string values in matrix form
outmftype=('linear');
extension='.dat';
56 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
% strcat combine two strings together
axis=strcat(n_trainingdata,extension);
axis1=strcat(n_checkingdata,extension);
axis2=strcat(n_outputdata,extension);
%========================================================
axis =
f1trnData.dat
>> axis1
axis1 =
f1chkData.dat
>> axis2
axis2 =
f1OutData.dat
%=======================================================
% Loading Training data
ldtraining=load(axis);
ldchecking=load(axis1);
ldoutput=load(axis2);
data=[ldtraining ldoutput];
>> data
data =
1174
1175
1237
1
1238
1239
1240
1
1176
1177
1179
1
1747
1177
1178
1
1747
1748
1749
2
1686
1687
1688
2
57 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
1813
1814
1815
2
1691
1692
1750
2
932
992
993
3
868
869
870
3
865
866
992
993
994
3
2278
2279
2342
4
2535
2537
2598
4
2406
2407
2408
4
2533
2534
2535
4
1646
1647
1648
5
1582
1583
1645
5
1583
1645
1646
5
1644
1645
1646
5
867
3
figure(1);
subplot(211);
bar(data);
title('Training data');
subplot(212);
bar(ldchecking);
title('Checking data');
legend('input1','input2','input3','output');
58 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
Figure 5.2 : Bar charts describing Training Data and Checking Data
% Classification/Fuzzy Inference System Phase to get the FIS
fis=genfis1(data, numMFs, inmftype, outmftype);
%=================================================
>>
fis =
name: 'anfis'
type: 'sugeno'
59 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
andMethod: 'prod'
orMethod: 'max'
defuzzMethod: 'wtaver'
impMethod: 'prod'
aggMethod: 'max'
input: [1x3 struct]
output: [1x1 struct]
rule: [1x27 struct]
%================================================================
figure(2);
subplot(311);
plotmf(fis,'input',1);
subplot(312);
plotmf(fis,'input',2);
subplot(313);
plotmf(fis,'input',3);
60 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
Figure 5.3 : Degree of Membership for Input1 , Input2 and Input3
% Anfis Model Training Phase
% Recording time for ANFIS training
tic;
[fistraining, errortraining, stepssize, checkfis,errorchecking]=anfis (data,fis,epoch,[1 1
1 1],ldchecking);
61 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
%=========================================================
ANFIS info:
Number of nodes: 78
Number of linear parameters: 108
Number of nonlinear parameters: 30
Total number of parameters: 138
Number of training data pairs: 20
Number of checking data pairs: 20
Number of fuzzy rules: 27
Warning: number of data is smaller than number of modifiable parameters
Start training ANFIS ...
1
0.00245377
0.00245377
2
0.000860652 0.000860652
3
0.00259725
0.00259725
Designated epoch number reached --> ANFIS training completed at epoch 3.
%=========================================================
figure(3)
subplot(311);
plotmf(fistraining,'input',1);
subplot(312);
plotmf(fistraining,'input',2);
subplot(313);
62 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
plotmf(fistraining,'input',3);
time=toc;
Total_time_for_ANFIS_training=time;
fprintf('\n<<<<<<<<<<<<<<<<<<<<WARNING!!!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>')
fprintf('\nResults above show second column is training error')
fprintf('\nand third column is checking error')
fprintf('\n<<<<<<<<<<<<<<<<<<<<WARNING!!!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>')
fprintf('\n')
Figure 5.4 Degree of Membership of input1, Input2 and Input3
63 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
% Display Error for Training and Checking
final_training_error=errortraining(epoch,1);
final_checking_error=errorchecking(epoch,1);
maximum_training_error=max(errortraining);
minimum_training_error=min(errortraining);
average_training_error=(maximum_training_error+minimum_training_error)/2;
maximum_checking_error=max(errorchecking);
minimum_checking_error=min(errorchecking);
average_checking_error=(maximum_checking_error+minimum_checking_error)/2;
average_global_error=(average_training_error+average_checking_error)/2;
>> final_training_error
final_training_error =
0.0026
>> maximum_training_error
maximum_training_error =
0.0026
>> minimum_training_error
minimum_training_error = 8.6065e-004
>> average_training_error
average_training_error =
0.0017
>> maximum_checking_error
64 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
maximum_checking_error =
0.0026
>> minimum_checking_error
minimum_checking_error = 8.6065e-004
>> average_checking_error
average_checking_error =
0.0017
>> average_global_error
average_global_error =
0.0017
% Save the trained FIS
saving=input('\nSave the Trained FIS, eg: yes=1 or no=0?= ');
if saving==1;
writefis(fistraining);
else saving=2;
end;
% Application Phase
%Evaluate the fis based on new set of data (application data)
name_applicationdata=input('\nEnter
name
of
Application
Data
files
(onlyprefix,without neither number nor extension): ','s');
expected_result=input('\nEnter name of Expected Data Result files (onlyprefix,without
neither number nor extension): ','s');
applicationdata=strcat(name_applicationdata,extension);
65 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
%======================================
% applicationdata = appdata.dat
%======================================
axis3=strcat(expected_result,extension);
loading=load(applicationdata);
out=evalfis(loading,fistraining);
output=round(out);
>> axis3
axis3 = outdata.dat
>> loading
loading =
1174
1175
1237
1238
1239
1240
1176
1177
1179
1747
1177
1178
1747
1748
1749
1686
1687
1688
1813
1814
1815
1691
1692
1750
932
992
993
868
869
870
865
866
867
992
993
994
66 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
2278
2279
2342
2535
2537
2598
2406
2407
2408
2533
2534
2535
1646
1647
1648
1582
1583
1645
1583
1645
1646
1644
1645
1646
>> out
out =
0.9997
0.9998
0.9998
0.9999
2.0024
2.0015
2.0013
2.0016
2.9999
3.0001
2.9999
3.0000
4.0003
3.9997
4.0000
4.0001
67 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
5.0013
4.9994
4.9998
5.0000
>> output
output =
1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
5
5
5
5
% Detection Phase
target=load(axis3);
number=0;
[a,b]=size(output);
[c,d]=size(target);
if a~=c
68 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
error('Mismatched input and output data matrices');
end
for i=1:size(output)
if output(i,1)==target(i,1);
disp('Correct')
elseif output(i,1)~=target(i,1);
disp('InCorrect')
number=number+1;
end
end
Total_Misclassified=number;
Figure 5.5: Anfis Model Structure
69 | P a g e
Image Processing for Movement Detection and Face Recognition using Fuzzy Neural Network
Figure 5.6 : Rule Editor for Anfis
70 | P a g e