Intelligent Diagnosis
Systems
Meir Kalech
Course Outline
1.
Intelligent diagnosis systems
2.
Model-based diagnosis: basics and definitions
3.
Resolution theorem prover
4.
Diagnosis of multiple faults: general
diagnosis engine GDE
5.
Assumption-based truth maintenance system
6.
Measurements to differentiating diagnosis
Course Outline cont.
7.
Diagnosis with behavior modes
8.
Self-Configuring Systems
9.
Model-based diagnosis as a constraints
satisfaction problem
10.
Diagnosis of Discrete Event Systems
11.
Diagnosis of distributed systems
12.
Diagnosis of multi-agent systems
Today Outline
1.
What is a diagnosis?
2.
Expert systems
3.
Model-based systems
4.
Case Based Reasoning (CBR)
5.
Inductive learning
6.
Probabilistic reasoning
What is a diagnosis?
Diagnosis system identifies the reason for a problem by
examining observed symptoms.
Requirement: a knowledge of the diagnosed system

Given by experts

Learned by AI techniques
Examples for diagnosis domains:

Disease diagnosis

Identification of software and hardware problems

Troubleshooting of electrical and mechanical systems

Fault detection and diagnosis of planning
Example
Doesn’
t
start!!!
1. Ignition
2. battery
3. both
Example
What
should
I do?
¬battery
¬headlights
------------Check
headlights
Example

Components:





Battery
Bulbs (headlight)
Wiper motor
Ignition
Possible observation:



Headlights work/don’t
Engine starts/doesn’t
Wipers work/don’t
Diagnosis objectives

How to infer consequent diagnosis from
observation?

How to model the relationship between
symptoms and diagnosis?

How to diagnose faults from the model?
Different approaches
These objectives are differently treated by
different approaches, for instance:

Model-based diagnosis must have precise
model of the system (i.e. car).

Expert systems treat incomplete model
(medical diagnosis).

Case-based reasoning represents the
knowledge in a repository of cases.
Outline
1.
What is a diagnosis?
2.
Expert systems
3.
Model-based systems
4.
Case Based Reasoning (CBR)
5.
Inductive learning
6.
Probabilistic reasoning
Expert Systems

Knowledge is represented via rules

Rules express what happens when certain
conditions met:


If….Then statements

Implications A → B
Rules tell expert system what to do in
certain circumstances
Knowledge Representation

Recommendations



Directives



Take set of inputs and output advice
Example:
If alarm AND smoke
then tell people go outside
Take set of inputs and output direct action
Example:
If smoke AND fire
then go outside
Relations


Take set of inputs and output information
Example:
If temperature below 32°
then weather is cold
Rule-based Systems

Rule-based/Production system




Use rules to provide recommendations/diagnoses
Use rules to determine course of action
Use rules to solve a particular problem
Consists of:



Knowledge base – database of rules
Database of facts
Interpreter/ inference engine
Architecture
16
Elements of an Expert System

User interface – mechanism by which user and system
communicate.

Exploration facility – explains reasoning of expert
system to user.

Working memory – global database of facts used by
rules.

Inference engine – makes inferences deciding which
rules are satisfied and prioritizing.
17
Inference engine
Conclusions derived using deduction

Forward chaining – using deduction from set of
antecedents

Backward chaining – starts with conclusion and
works towards logical set of antecedents
Forward Chaining

Data-driven reasoning



Starts with data set and moves towards conclusion
When all antecedents matched, rule is triggered
and conclusion added to facts database
Conflict resolution

Occurs when more than one conclusion deduced
from facts
Conflict resolution:
1. Priority resolution
 Each
rule given a priority level
 Rule with highest priority triggered

Example:


IF patient has pain
THEN prescribe painkillers (priority 10)
If patient has chest pain
THEN treat for heart disease (priority 100)
Conflict resolution:
2. Longest-matching strategy

Conclusion deduced from longest rule triggered

Example:


If patient has pain
THEN prescribe painkillers
If patient has chest pain
AND patient is over 60
AND patient has history of heart conditions
THEN take to emergency room
Conflict resolution:
3. Most recent match

Rule that matches facts most recently added
to database fired

Example:


If patient has pain
THEN prescribe aspirin (entered 10/17/1975)
If patient has pain
THEN prescribe acetomenophin (entered
11/1/2000)
Backward Chaining
Goal-driven reasoning

Starts with a conclusion/hypothesis and moves to
show hypothesis can be reached from rules and
facts in database

Used in formulating plans
FC vs BC

Forward chaining appropriate when:



Set of facts available but conclusion unknown
Many possible conclusions
Backward chaining appropriate when:


Few possible conclusions but many possible facts
Possible facts consist many not relevant to
conclusion
FC vs BC Example

Rules:
1.
2.
3.
4.
5.
6.


A^B →C
A
→D
C^D →E
B^E^F→G
A^E →H
D^E^H→I
Facts:
1. A
2. B
3. F
Goal = H
Forward Chaining
Facts
Rules triggered
Rules fired
A,B,F
1,2
1
A,B,C,F
2
2
A,B,C,D,F
3
3
A,B,C,D,E,F
4,5
4
A,B,C,D,E,F,G
5
5
A,B,C,D,E,F,G,H
Goal Reached!
FC vs BC Example

Rules:
1.
2.
3.
4.
5.
6.


A^B →C
A
→D
C^D →E
B^E^F→G
A^E →H
D^E^H→I
Facts:
1. A
2. B
3. F
Goal = H
Backward Chaining
Facts
A,B,F
A,B,F
A,B,F
A,B,C,F
A,B,C,D,F
Goals
H
E
C,D
D
Matching Rules
5
3
1
2
STOP!
Example: rule-based diagnosis
Forward chaining
Working memory:
1. Headlights don’t work
2. Faulty bulbs and/or battery
3. Faulty battery
Rules fire:
1
2
No rule matches
No further matched
Rule 2rules
is close:

faulty battery Querying the user:
Status of engine
Response: Engine doesn’t start
Problem!!!
Working memory:
Rules fire:
1. Headlights don’t work
2. Faulty bulbs and/or battery
3. Faulty battery
4. Wipers work
BUT!
By adding this fact…
No further matched rules

faulty battery
faulty ignition
AND
faulty bulbs
Is not covered by the
knowledge system
1
2
The Limitations of Rules

The success of rule-based expert systems is
due to several factors:



They can mimic some human problem-solving
strategies
Rules are a part of everyday life, so people can
relate to them
However, a significant limitation is the
knowledge elicitation bottleneck



Experts may be unable to articulate their expertise
Heuristic knowledge is particularly difficult
Experts may be too busy…
Challenges

The knowledge must be covered by the
experts.

Representing the knowledge to efficient rules.

Chose the appropriate inference mechanism.

Diagnosing multiple faulty components.
Outline
1.
What is a diagnosis?
2.
Expert systems
3.
Model-based systems
4.
Case Based Reasoning (CBR)
5.
Inductive learning
6.
Probabilistic reasoning
Model-based systems

Expert systems – complete model is unavailable
(medical diagnosis)

Model-based systems – the physical principles are
largely known (automobile diagnosis)

Formulating rules to capture the causal functional
knowledge.

Describing the properly working components. For
instance:
if ok(battery) AND ok(ignition) THEN start(engine)

If engine doesn’t start then infer by truth maintenance
system (TMS) that battery or ignition are faulty.
Scenario – example for ATMS reasoning

A goes to a birthday party of B.

B is a woman.

All women love flowers.

A decides to give flowers to B.

C tells A that B is allergic to flowers.

A buys to B a gum.
Some logic…

C = common knowledge (facts & rules)

A = assumptions

D = C  A (DataBase)

Explanation for p: minimal E' ⊆ A: E'  C ⊨ p

Suppose a new fact q. since D contains assumptions
that may contradict q, we should identify them.

We find all explanation for ¬q in D: {E'1,E'2…E'n}.

Then we find a minimal set H such that:
∀E'i∈{E'1,E'2…E'n}: H ∩ E'i != {} (hitting set).

H: the minimal set of assumption that contradict q.
Example for MBD
Example for MBD

Observations (facts):




¬works(headlights) ∧ ¬starts(engine) ∧ works(wipers)
we should find explanations to works(headlights), and
starts(engine).
Explanations:

E’1(works(headlights)) =
{ok(battery),ok(bulbs)}

E’2(starts(engine))
{ok(battery),ok(ignition)}
=
Diagnosis:
 ¬works(wipers)

H={ok(battery)}

H={ok(bulbs),ok(ignition)}
MBD - drawbacks

It is not always feasible to build an accurate
model

Inferring the diagnosis is computationally
intractable in complex systems.

Improvements:

Less accurate model

Less accurate diagnosis
Outline
1.
What is a diagnosis?
2.
Expert systems
3.
Model-based systems
4.
Case Based Reasoning (CBR)
5.
Inductive learning
6.
Probabilistic reasoning
Another Way We Solve Problems?

By remembering how we solved a similar
problem in the past

This is Case Based Reasoning (CBR)



memory-based problem-solving
re-using past experiences
Experts often find it easier to relate stories
about past cases rather than to formulate rules
Databases

Database technology would seem ideally
suited to the task of retrieving known
solutions to problems

Databases are excellent at finding exact
matches…

But are poor at near or fuzzy matches
The CBR Cycle
Solution
Review
Retain
Adapt
Retrieve
Similar
New
Problem
R4 Cycle

Retrieve the cases from the case-base whose
problem is most similar to the new problem.

Reuse the solutions from the retrieved cases to
create a proposed solution for the new problem.

Revise the proposed solution to take account of
the problem differences between the new problem
and the problems in the retrieved cases.

Retain the new problem and its revised solution
as a new case for the case-base if appropriate.
CBR Assumption(s)

The main assumption is that:

Similar problems have similar solutions:


e.g., an aspirin can be taken for any mild pain
Two other assumptions:

The world is a regular place: what holds true today
will probably hold true tomorrow


(e.g., if you have a headache, you take aspirin, because it
has always helped)
Situations repeat: if they do not, there is no point
in remembering them

(e.g., it helps to remember how you found a parking space
near that restaurant)
Problems We Solve This Way

Medicine


Law



English/US law depends on precedence
case histories are consulted
Management


doctor remembers previous patients, especially
for rare combinations of symptoms
decisions are often based on past rulings
Financial

performance is predicted by past results
Good / Bad Applications for CBR


Classification tasks (good for CBR)

Diagnosis - what type of fault is this?

Prediction / estimation - what happened when we saw
this pattern before?
Synthesis tasks (harder for CBR)

Engineering Design

Planning

Scheduling
Retrieval by indices
DB of
3 cases:
Scoring
function:
Retrieval by indices
Degree of
match is
the sum
of score:
New DB:
Car Diagnosis Example

Symptoms are observed



Engine does not start
Battery voltage = 7v
Goal


Cause of failure: flat battery
Repair strategy: charge battery
Car Diagnosis Case
Each case describes one diagnostic
situation

Described by a list of features
Contains a list of feature values

Problem



Case 1



Symptom: headlight does not work
Car: Ford Mondeo
Year: 2001
Solution


Diagnosis: headlight fuse blown
Repair: replace headlight fuse
Feature



Value
Battery: 10.4v
Headlights: undamaged
HeadlightSwitch: on
Car Diagnosis Case-Base

A collection of independent cases

Problem
Case 1




Solution



Case 2
Diagnosis: headlight fuse blown
Repair: replace headlight fuse
Problem




 Battery: 10.4v
Symptom: headlight does not work
 Headlights: undamaged
Car: Ford Mondeo
 HeadlightSwitch: on
Year: 2001
Battery: 9.5v
 Headlights: surface
Symptom: headlight does not work
damage
Car: Ford Ka
 HeadlightSwitch: on
Year: 2003
Solution


Diagnosis: defective bulb
Repair: replace headlight

Case Representation

Depends on problem domain

Flat structure

A list of feature values (car diagnosis example)
 Easy

to store and retrieve
Specialised representations

Graphs - nodes and arcs

Plans - partially ordered set of actions

Object-oriented - objects (instances of classes)
 More
difficult to store and retrieve
Case Representation

Object-oriented representation: A case is
a set of objects


An object is described by a set of features
Classes are arranged in a hierarchy

Relations between objects


Car
(e.g. part-of)
Combine similarities of parts Brakes
Engine Transmission
Ignition System Fuel Injection
Coil
Spark Plug
Colour: dark grey
Gap: 1.2mm
New Car Diagnosis Problem
A new problem is a case without a solution part
 Not all problem features must be known

same for cases
 Problem
New




 Battery: 9.2v
Symptom: brakelight does not work
 Headlights: undamaged
Car: Ford Fiesta
 HeadlightSwitch: ?
Year: 1997
Feature
Value
Calculating Case Similarity
Similarity(problem,case) = weighted sum of
Similarityf(problem,case) for all features f
 High importance features have large weight
 symptom,
battery, headlights
 weight = 6

Low importance features have low weight
 car,
year
 weight = 1

Case similarity =


w1*s1 + w2 * s2 + …… + wn*sn
w1 + w2 + …… + wn
si is similarity of ith feature
wi is weight of ith feature
New Problem and Case 1
Similarity
Symptom: brakelight does not work 0.8
New Problem






Car: Ford Fiesta
Year: 1997
Battery: 9.2v
Headlights: undamaged
HeadlightSwitch: ?
weight = 6
1
Problem


0.6
0.6
0.9
1.0




Symptom: headlight does not
work
Car: Ford Mondeo
Year: 2001
Battery: 10.4v
Headlights: undamaged
HeadlightSwitch: on
Solution



Case 1
Diagnosis: headlight fuse blown
Repair: replace headlight fuse
Similarity(New, Case 1) =
6*0.8 + 1*0.6 + 1*0.6 + 6*0.9 + 6*1
6+1+1+6+6
= 17.4 / 20 = 0.87
New Problem and Case 2
Similarity
Symptom: brakelight does not work 0.8
New Problem






Car: Ford Fiesta
Year: 1997
Battery: 9.2v
Headlights: undamaged
HeadlightSwitch: ?
weight = 6
1
Problem

0.8
0.4
0.975
0.0





Symptom: headlight does not
work
Car: Ford Ka
Year: 2003
Battery: 9.5v
Headlights: surface damage
HeadlightSwitch: on
Solution



Case 2
Diagnosis: defective bulb
Repair: replace headlight
Similarity(New, Case 2) =
6*0.8 + 1*0.8 + 1*0.4 + 6*0.975 + 6*0
6+1+1+6+6
= 11.85 / 20 = 0.59
Reuse Solution from Case 1
Problem

New Problem







Symptom: brakelight does not
work
Car: Ford Fiesta
Year: 1997
Battery: 9.2v

Headlights: undamaged
HeadlightSwitch: ?
Symptom: headlight does not
work
…
Solution


Diagnosis: headlight fuse blown
Repair: replace headlight fuse
Solution to New Problem



Case 1
Diagnosis: headlight fuse blown
Repair: replace headlight fuse
After Adaptation


Diagnosis: brakelight fuse blown
Repair: replace brakelight fuse
Characteristics and drawbacks


Characteristics
1.
No accurate model
2.
No rule-based representation, But: repository of
cases.
Drawbacks:
1.
2.
High computational cost of:

Retrieval.

DB organization.
Not guarantee to provide complete diagnosis
Outline
1.
What is a diagnosis?
2.
Expert systems
3.
Model-based systems
4.
Case Based Reasoning (CBR)
5.
Inductive learning
6.
Probabilistic reasoning
Inductive learning systems

Previous approaches try to model the system

Diagnosis inference based on the model

In inductive learning approach the model is
learned from examples.

Diagnosis inference based on the relationships
between symptoms and diagnoses.

The system induces an appropriate set of
classification rules.
Formally:
(Ik,Ck)
Ik: instance like
vector of attributes
Ck∈C (C: set of
possible classes)
In the domain of diagnosis:
Ik: observed symptoms
Ck∈C (C: set of possible diagnosis)
Challenge: to learn an accurate description of a
class from a representative set of examples.
Training set
In this representation multiple faults are not addressed
For some cases multiple faults could be addressed by
Sequentially isolating single faults
Multiple fault - training set
In this representation we need much larger training set
Generalization

Generalization is the ability of the inductive
learning system to classify new instance.

Factors:

Size of training examples.

The distribution of the training examples.

The representation of the instances and classes.
Decision tree (ID3)

ID3 algorithm provides a way to learn classification
rules represented by decision tree.

Leaf nodes: represent the classes (diagnoses).

Internal nodes: represent the attributes (symptoms).
ID3 algorithm

The goal of ID3 is to generate a decision tree as small
as possible.

This will require fewer attributes tests.

Recursively select attributes that yield the maximum
information gain.
ID3 algorithm
n1 and n2 are the number of examples in the training
set of class 1 and class 2 (should be generalized to n3…)
The expected information from attribute A to be the root:
Where are the number of examples of the training set
belong to class 1 and class 2 and have a value of Ai.
ID3 algorithm
The information gained by testing the value of attribute
A is:
Recursively, at each step of the tree, ID3 computes the
information gain of the untested attributes and chooses
the attribute with the maximum information gain.
Example: ID3 algorithm (based
on TS in page 59)


3 attributes: Engine, Headlights, Wipers
4 classes:
1.
2.
3.
4.


Faulty
Faulty
Faulty
Faulty
Bulbs
Battery
Wiper motor
Ignition
The distribution of the examples are (1,1,2,2)
Equation 1:
Example: ID3 algorithm (based
on TS in page 59)

Equation 2:

Equation 3:
Example: ID3 algorithm (based
on TS in page 59)


Much compact tree than the one in page 61.
We can construct rules for expert system.
Outline
1.
What is a diagnosis?
2.
Expert systems
3.
Model-based systems
4.
Case Based Reasoning (CBR)
5.
Inductive learning
6.
Probabilistic reasoning
Probabilistic Reasoning

Once we do not know the truth value of an attribute
(symptom) for certain, we can model its uncertainty.

The training set uses for getting the distributions of
the data.

The chosen class (fault) is the most likely given the
attribute values.
Probabilistic Reasoning - Notation

X={X1, X2 …Xn}: set of observations (symptoms)
variables


Each of Xi takes a value from: X={x1,x2,…,xn}




Example: Headlights, engine and wipers.
Example: X2 - engine takes the value “starts”.
P(X): the probability that the random variables in X
take the values X.
P(xi): the probability that the random variable Xi takes
the value xi.
{C1, C1,…, Cm}: set of classes (possible faults).

Example: Faulty bulbs, faulty battery, faulty motor, faulty
Model

Given a particular input X, by using Bayes rule we can
calculate the likelihood of each class Ci:
(6)
where:
(7)

Now, the classification of X can be performed by
choosing the class Ck that maximize the likelihood:
Model

For the random variables X1, X2 ,X3 that observed
taken the values x1,x2,x3, Eq.6 can be represented:
(8)

We should calculate the conditional and unconditional
probabilities:

For large number of attributes it is intractable.
Model

Assumption: conditional independence of the
observations (symptoms):

Bayes rule for m classes involving 3 symptoms is:
(9)
Example – based on TS in page 59

Based on the training set we can infer (P(Ci)):

For Eq.9 we need to calculate the conditional
probabilities of each one of the symptoms.

For instance:

Given faulty bulbs, the probability that headlights will not work
is 1: P(¬H|¬Ba)

P(¬E|¬Ba): since there is no causal relationship between
engine and Bulbs we can say that: P(¬E|¬Ba)=P(¬E)=3/6=0.5
Example – based on TS in page 59
Given a certain input: wipers work (W), headlights don’t
work (¬H), engine doesn’t start (¬E), the probability
for faulty Bulbs (¬Bu):
(10)
Example – based on TS in page 59

The probabilities for other faulty classes:

Faulty Ignition is the diagnosis with the highest
probability.
Conclusion


Diagnosis is a classification problem, where:

symptoms = attributes and

faults = classes.
We learned several approaches:
1.
Expert systems.
2.
Model-based diagnosis.
3.
Case-based reasoning.
4.
Inductive learning systems.
5.
Probabilistic reasoning.