ONE APPROACH TO FUZZY EXPERT SYSTEMS
CONSTRUCTION.
Dmitry A. Kropotov, Dmitry P. Vetrov.
Russia, Moscow, 119991
Dorodnicyn Computing Centre of the Russian Academy of Sciences.
FUZZY LOGIC IS…
Fuzzy logic is based on the theory of fuzzy sets, proposed by Zadeh in 1965. The main idea of this theory is generalization
of classical sets theory for the continuous case. This means that the objects may partly belong to the sets with different
n
degree of membership. In other words fuzzy set can be associated with their membership function  : R  [0,1]. It is
easy to define the generalized logical operations, like, for example, conjunction and disjunction, over the fuzzy sets as
minimum and maximum of membership functions. Fuzzy implication can be defined by different ways, but in present
report, we use Mamdani scheme by taking minimum of membership functions from the sumption and using it as a
belonging degree to the result set of the rule.
Average temperature
 A ( x)
1
1
0
2
5
Normal
Low
High
Crop
x
0
Crisp set “Between 2 and 5”
1
Below
average
Average
Above
average
0
1
Fuzzy
sets
Qualitative relations
0
Knowledge base
Feature
values
5
2
x
Good
Rich
centner per hectare
mm
0
Mamdani’s implication scheme for the rule
“IF Average temperature is Normal AND Precipitations are Average THEN Crop is Good”
Fuzzy set “Approx. between 2 and 5”
Fuzzy
set
Normal
MIN
Precipitations
 A ( x)
Poor
1
t, C
Forecast
… A GOOD WAY OF CONSTRUCTING EXPERT SYSTEMS.
The main advantage of fuzzy logic is its ability to operate in natural terms for a user. By defining linguistic variables, one
may construct fuzzy rules. Fuzzy expert system works by interpreting input data as linguistic variables, implicating the
Fuzzyficator
Defuzzyficator
Fuzzy rules
necessary fuzzy rules and then defuzzyfying the result. But THERE APPEAR
TWO BOTTLENECKS IN
BUILDING FUZZY EXPERT SYSTEMS:
Principal scheme of fuzzy expert system.
HOW TO OBTAIN THE SHAPES OF FUZZY SETS…
It is usually difficult to form the exact shapes of fuzzy sets for the expert. It is not obvious “how fuzzy” they should be. In
order to solve this problem we use so-called (a, b)-parameterization. Let the shapes of membership functions belong to the
parameterized family of isosceles trapeziums. The location of fuzzy set is defined by the approximate borders (ai , ai 1 )
which are part of the partition of numerical axis, that can be easily assigned by the user. In this case a–parameter can be
interpreted as a crossing degree of the sets, and b–parameter shows the “fuzziness” of set. To reduce the number of
coefficients to be optimized, we use an assumption that pair (a, b) is responsible for the properties of the whole
feature rather than a single set. In this case we may find the coefficients by solving optimization task on the learning
sample.
 ( x)
1
a2
a1
0
Representation - the rate of objects in the
sumption of the rule.
c1  d1
a1
a2
b1  0
c2
b2 
d2
d 2  c2
a3  a2
a3
c3
b3 
x
d3
d 3  c3
a4  a3
a4
(a,b) – parameterization of fuzzy sets’ shapes
Effectiveness - the rate of objects from the
sumption that satisfy the rule.
Each rule can be represented as a point in effectiveness/representation plane.
SUCH SYSTEM CAN BE USED EITHER FOR FORECASTING…
… AND HOW TO GET THE NECESSARY FUZZY RULES.
In many areas one hasn’t enough knowledge about the process being researched in order to form linguistic rules. If there
are some precedents with known forecasts, the necessary rules may be generated automatically. The proposed algorithm
is based on the two notions: representation and effectiveness of the rule. The first shows the rate of objects being
involved in the consideration, while the second shows the rate of objects that satisfy the rule. The more representation and
effectiveness are, the better is the rule. In fact, we want to increase the effectiveness at least to some threshold, holding
the representation above the predefined level. To do this, we fuse (restrict) several rules to one of higher order, by
conjuncting their sumptions, until we exhaust the set of potential rules. During such process the representation becomes
lower, but the effectiveness may become higher. All rules that have both representation and effectiveness higher than
corresponding thresholds are accepted; rules that are not enough representative are rejected; and all other rules are used
for further fusion
To forecast continuous values, we find (a, b) coefficients by using least squares method. In fact we just minimize the sum
of squared deviations from the correct answer. And to calculate the forecasted variable according to the given number of
fuzzy rules, use the centre of gravity defuzzification method.
This mode was used to predict the places of football teams in Russian Championship according to the tournament table
(won scores were excluded) and for forecasting magnetic field oscillations in cavities of accelerating klystrons (DESY,
Hamburg). The results received by described method (program ExSys) were compared with linear regression for football
and Matlab fuzzy logic toolbox for cavities. It appeared that in some cases fuzzy logic works better than linear regression
even for such simple and obviously linear tasks like finding the place of the team by the tournament table. As for Matlab
toolbox, the system tends to overfit to the learning data even after the use of independent precedents, which had to
prevent overtraining.
Classification
methods
Melanoma
(super small
sample)
Phoneme
(big sample)
Liver
(average sample,
many classes)
MLP
65.6
78.2
77.5
LDF
59.4
77.4
77.5
TA
62.5
65.5
65.7
LM
50.0
77.2
79.3
SVM
56.3
76.4
83.1
QNN
62.5
84.7
80.3
ExSys
66.6
77.5
76.5
Russian football championship.
Sum of squared deviations for linear regression is 38.056, for ExSys is 21.936.
Melanoma – 48/32 objects, 33 features, 3 classes.
Phoneme – 2200/1404 objects, 6 features, 2 classes.
Liver – 170/150 objects, 8 features, 7 classes.
Oscillations of magnetic field amplitude.
Sum of squared deviations for MatLab is 0.405, for ExSys is 0.274.
… OR CLASSIFICATION.
If one needs to predict the value which belong to the finite set (i.e. classification task), the quality functional is just the rate
of misclassified objects. And as defuzzification method, we use defuzzification by mode. In other words the object is
classified to the set with the maximal value of membership function.
The system in such mode was tested on several tasks and was compared with numerous recognition methods – linear
Fisher discriminant (LDF), q-nearest neighbors (QNN), test algorithm (TA), committee of hyperplanes (LM), support vector
machines (SVM) and multilayer perceptron (MLP). The results of work are presented in the table.
Results of classifications
THEORY OF STATISTICAL SOLUTIONS IS USED TO FIND THE NECESSARY
THRESHOLDS AND PREVENT OVERFITTING.
It’s clear that changing representation threshold, we may thus regulate the number of discovered fuzzy rules and hence
the time of rule generation. Unfortunately we can’t do the same directly with the effectiveness threshold. Making it too low
forces computer to generate a lot of “parasitic” rules, which contain no useful information. As a result the expert system
suffers from overtraining and degrades greatly. To avoid this, the apparatus of mathematical statistics was applied. This
allowed to define the effectiveness threshold by finding the upper bound of confidence interval after fixing the rate of
“parasitic” rules we would like to exclude. By varying the level of confidence we get different thresholds and may regulate
the number of discovered rules. To reduce the time of rule generation, the lower effectiveness bound is calculated. If the
effectiveness of the rule if less than this bound, it cannot be raised to the demanded level without decreasing
representation value too low (lower than corresponding threshold). With these remarks, the effectiveness/representation
plane becomes as shown on the figure.
Fuzzy Expert System
Low and upper bounds for effectiveness
THE PROPOSED SYSTEM CAN BE USED IN MULTIPLE MODES…
Feature partition
Fuzzy sets
Fuzzy rules
Optimization
Auto partitioning
Auto generation
Manual definition
Manual partitioning
Manual input
Depending on what the expert can do, the system described above, may work in different modes. When there is no prior
information about the task, all necessary steps can be done automatically. The user only needs to input the number of
fuzzy sets to be generated for each variable and to give them names. If needed, expert can define the approximate
borders of sets himself. The rules are either generated according to the learning table or/and entered by the expert. He can
also change the sumptions or weights of some rules if necessary. The shapes of sets are either set or found as a result of
optimization procedure. The modes, that system support, may vary from the most autonomous way, to the case, when all
fuzzy sets and rules are known for the expert. In the last case system acts as classical fuzzy controller.
Different modes of ExSys
… NOT ONLY FOR FORECASTING, BUT ALSO FOR UNDERSTANDING THE NATURE
OF PROCESS.
Using rule generation option, we may not only use the discovered rules for further forecasting, but also examine them for
understanding the nature of the process being researched. Here are some rules extracted from the football tournament
table.
Such knowledge can be extremely useful if some of known features can be managed. Knowing the influence they have on
the hidden value, we may effectively control the process by adjusting the features that are available for changing.
IF Dropped goals are Not many AND Losses are Few THEN Rank is High
IF Wins are Not few AND Scored goals are Many AND Draws are Some THEN Rank is
Very high
IF Scored goals are Few THEN Rank is Low
IF Dropped goals are Not few AND Draws are Many THEN Rank is Medium
Examples of generated fuzzy rules for football ranks predictions