Components of Problem Solving and Types of Problems ∗
Joost Breuker †
University of Amsterdam
Department of Social Science Informatics
Roeterstraat 15
NL–1018 WB Amsterdam, the Netherlands
email [email protected]
tel (31) 20 525 3494/6798/6789
fax (31) 20 525 6896
Abstract
A typology of problems is presented that is used for indexing and accessing reusable problem solving components in a library that supports the CommonKADS methodology for building knowledge based systems. Eight types of problems, such as planning, assessment etc.,
are distinguished, and their dependencies are explained. These dependencies suggest that the
typology is to be viewed as a “suite” rather than the usual taxonomy of “generic tasks”. Developing the suite has lead to some new insights and elaborations of [Newell and Simon, 1972]’s
theory for modeling problem solving.
• Tasks are distinguished from problem definitions. A tasks is constructed by finding
and configuring problem solving methods (PSMs), which are suitable for solving the
(well-) defined problem. Tasks and PSMs therefore have a one to one correspondence
([O’Hara and Shadbolt, 1993]), while there is a one to many corresponce between a
problem definition (type) and PSMs.
• Three phases are proposed that turn spontaneous, ill-defined problems into well-defined
ones, respectively. into problem solving tasks.
• A complete solution consists of three components: a case model, an argument structure
and a conclusion. The conclusion is a sub-part of both other components.
• Tasks (PSMs) package recurring chains of dependent types of problems in variable
ways.
• The availability of behavioural models, or of structural/behavioural models in a domain
determines to a large extent which types of problems can be posed and solved.
∗
From: L. Steels, W. Van de Velde & G. Schreiber (eds) A Future for Knowledge Acquisition, Proceedings of the
EKAW-94, Springer-Verlag, Berlin, 1994, p 118 – 136
†
The research reported is partially funded by the Esprit programme of the European Commission, P-5248, KADS-II.
I would like to thank André Valente for commenting on this paper. A different, and far more elaborate version of the
framework presented this paper can be found in [Breuker, 1994b]
1
1
Introduction
In the CommonKADS methodology for constructing knowledge based systems, a Library of
reusable problem solving components takes a central role in top-down support of knowledge acquisition [Valente et al., 1993, Breuker and de Velde, 1994]. It performs the same role as the KADS-1
library of “interpretation models”, but differs in content and structure [Breuker et al., 1987]. The
new Library’s content no longer consists of only highly abstract inference structures — one for
each type of task — but of linked components of different grain-size levels, allowing for refinement by selection and combination to fit the application domain.
The components come in a large variety. They may range from a full generic model that
needs only to be instantiated by domain terms, to a simple inference function. The components are expressed in CML, the Conceptual Modeling Language [Wielinga et al., 1993],
and can be semi-automatically translated into M L2 , the formal language of CommonKADS
[van Harmelen and Balder, 1993]. The most abstract description of a component is a function.
A function has input- and output roles.
This Library has two divisions: the Task division, where problem solving methods (PSMs) are
stored, and the Domain division, containing reusable ontologies. The Domain and Task divisions are linked by features (assumptions), representing interactions between PSMs and domain knowledge. In this paper, we are concerned with the Task division, and more precisely
with one of its major accesses: a typology of problems. This typology differs in two major
respects from current approaches, such as “role limiting methods” [McDermott, 1988], Generic
Task [Chandrasekaran, 1983], [Chandrasekaran and Johnson, 1993], [Clancey, 1985]’s taxonomy,
or the KADS-1 one [Breuker et al., 1987, Schreiber et al., 1993]. First, the typology is about “well
defined” problems, and not about tasks. As will be explained below, a task consists of a problem
definition and a configuration of PSMs. The problem definition is used to find an appropriate PSM
in the Library. Second, the typology is not a taxonomy (see Fig. 1), but a “suite”, consisting of
rational dependencies in the breakdown of any problem. See Fig. 7 for what is meant by a suite.
It is based on dependencies between problems instead of a classification of their properties. What
started as a revising the taxonomy of KADS-1 (Fig. 1), ended in redefining many taken for granted
concepts such as task, solution, problem space, etc. To a large extent, this paper can be taken
as revisiting Newell & Simon’s classical problem solving paradigm that is still prevailing in AI
[Newell and Simon, 1972]. The results of this exploration are rather elaborations of this paradigm,
than complete revisions or an alternative view. Some of these results are new. For instance, it
appears that the solutions are not simple answers to questions, but complex objects, consisting of
two strongly related parts or sides: a case model and an argument structure. This double side of
a solution is also reflected in problem solving methods (PSMs) and the their “generate & test”
structure.
2
- assessment
- monitoring
- classification
system
analysis Q
Q
s prediction
Q
- diagnosis
- repair
- system
modification
- remedy
basic task
- system
synthesis
- elements
- configuration
- scheduling
- structure
- refinement
design
- s&e
- transformational
- none
- planning
- open endeddesign
design
Figure 1: Taxonomy of “basic tasks”: the revised KADS-1 taxonomy of generic tasks [Aamodt et
al., 1992, App.C.]
2
Problems and Tasks
The classical definition is that problems are some conflict between a current state and a goal state
[Newell and Simon, 1972]. However, many (everyday life) problems are not so obviously goal
related:
An old lady wants to visit her friend in a neighbouring village. She takes her car, but
halfway the engine stops after some hesitations. On the side of the road she tries to
restart the engine, but to no avail.
In this story, the goal is to visit a friend, and one may assume that the old lady’s plan to do so may
take a multitude of intermediary subgoals, but none of these is in conflict with the current state
of the malfunctioning engine. The term “goal” would become highly inflated if it should cover
any expected state, including those assumed states as subsumed under the ‘frame problem’. The
old lady’s assumption about a correctly functioning car is violated, when she perceives that there
is no longer a correspondence between her foot’s pressure on the accelerator and the car’s speed.
Therefore, it is more appropriate to speak about norm states than about goal states. Identifying
a discrepancy between a current state and a norm or expected state leaves us with an ill defined
problem. The discrepancy itself does not say anything in which direction a solution should be
found.
Problems are well-defined when there is a simple test to conclude that a solution is a solution. This
view stated by [McCarthy, 1956] is at the basis of Newel & Simon’s description of these problems by their problem space [Newell and Simon, 1972, p. 72-78]. This notion has been widely
accepted, but further discussions on this theme are rare. [de Velde, 1988] elaborates the notion
3
of problem space, as a triple Ω =< P, S, solution >, where P is the set of problems, S is the
set of solutions, and solution is the relation that should hold between both sets. This definition
explains the competence in problem solving, because it focuses on the relation between input (P )
and output (S). The meaning of this relation (solution) is generally that a solution (s) satisfies
some problem (p). A diagnosis solution, consisting of a faulty component that explains the malfunctioning of some device, is an example. Besides ‘satisfies’, ‘is-consistent-with’ may be an
interpretation of the solution relation (see section 3). From a performance perspective, the solution set P may not be known, because the problem space has not been completely explored,
or the set may be infinitely large. Moreover, it takes a rather strong interpretation of McCarthy’s
prescript of testability. Of course the simplest and most complete test to see whether a particular
solution s is a solution is by finding it as a member of the set of possible solutions S. However, the
test may imply no more and no less than whether some attributes are part of the solution s. From
a performance perspective, requirements on solutions are available rather than sets of solutions.
Therefore it appears here that a problem is defined by some abstract description of a solution,
rather than some (enumerable or typical) set of solutions.
When the solution is given in abstracto, problem solving is a process of refinement of an abstract
solution with the help of more specific, but still generic domain knowledge, and of ‘filling’ the
structure of the solution with the specific data from the current state. [Clancey, 1985] was the first
to propose this combination of abstraction and refinement as the major operators in the heuristic
classification method. [de Velde, 1988] took this view of Clancey, which is blurred by so many
other interesting observations, in a more generic and principled way, and showed that all problem
solving can be characterized by abstraction and refinement procedures, which come in two major
but mixable flavours: classification and construction. We will return to this issue in the last section
of this paper.
A problem may be well-defined but that does not make it a task. For instance, the old lady may
define her problem as one that should lead to an outcome that is some repair of her car. Note that
many alternative problem definitions may have occurred. She might have, for instance, decided to
find alternative means for transportation, or to give up her current plan. The difference between
a well-defined problem and a task looks rather subtle. A problem can be solved, and a task is
executed. The distinction is hardly noteworthy if we see in solving a problem the controlled
execution of problem solving methods. However, with task we also mean something that can
be repeated, that can be distributed and that contains a plan, i.e. a configuration of PSMs. A task
differs therefore from a problem, because in a task the problem solving methods are known, and for
a problem these still have to be found. Of course, in a more dynamic view where problem solving
consists of defining new subproblems and solving these, the distinction is less sharp. However, in
knowledge engineering the problems to be automated are so routine that they are tasks, rather than
problems for human agents. People may complain that they have problems; not that they have
tasks at their work. This means that problems are deviations and complications, and tasks are not.
Going from an ill-defined problem to a task can be described as an ontogenesis of three steps (see
Fig 3 for a summary).
Problem Identification. The output of the first stage is a conflict between a current state and a
norm state: these two states are inputs to problem identification. The conflict is a “sponta4
neous” problem: it has the role of a ‘complaint’. The complaint is the inducement or trigger
for a (well defined) problem, a p in Van de Velde’s formula. The conflict is not the cause of
the problem. Finding the cause of a problem is a problem by itself (diagnosis, or reconstruction). The conflict describes the difference between what is the case, and what should have
been the case. What should have been the case, i.e. the norm, may have two different meanings. The first one is an expectation, i.e. a state that is predictable on the basis of knowledge
about the world. The malfunctioning car of the old lady is an example. The second type
of norm is a desire, i.e about states in the world that ought to be the case. The distinction
between these types of norms is of importance, because different types of inference are
involved (e.g. [Sergot and Jones, 1992], [Breuker, 1993], [Valente and Breuker, 1994]).
Problem Definition. This second step takes as its input the conflict, and produces as its output
some abstract solution, or, as we will see in the next section, a generic conclusion. A problem definition turns a spontaneous problem into a well-defined one. It establishes the problem space, Ω. A problem definition is a necessary step to provide sense to a problem. For
instance, the old lady of our story may decide to have first her car diagnosed and repaired,
and may then continue her journey. An alternative is, that she may have a taxi pick her up
and have a garage take care of her car. Or she may give up the goal — visiting her friend —
altogether. Problem definition is a planning activity, which is guided by the dependencies
between abstract solutions (= problem types: see Fig. 7). The sense of a problem definition
is the fact that the chain of the potential solutions in the plan aim at a goal.
Task Specification. Well-defined problems change into tasks by finding or constructing appropriate PSMs. In organizations, recurring well-defined problems are planned as tasks, and
distributed over agents and other resources. These scheduled tasks have often lost their relation with spontaneous problems, which means that tasks have not necessarily some violated
norm as their direct origin. 1 . Task specification has as its inputs a defined problem, and
one or more PSMs which should solve the problem. The controlled configuration (decomposition) of PSMs forms the ‘task-body’, while the problem definition is its ‘header’. Fig. 2
shows the structure of task knowledge. The “assumptions” slot of the task structure refers
both to the required data, and to the required (generic) domain knowledge.
task
HH
H
)
problem HH
H task-body (PSMs)
j
definition
type
HH
Z
~
Z
)
HH
j
?
control
specification
=
assumptions
additional
structure
?
roles
data
generic
BBN
- obligatory
solution
control
task
- optional
elements
goal Figure 2: Structure of task knowledge
1
Indirectly they have, as the goals of a task are the operationalization of some ‘desire’
5
PSMs
goal
- task specification
problem type
- task
HH
j data
problem definition
conflict
normproblem identification
current state
Figure 3: Three steps in going from problem to task
3
What’s in a Solution?
Problems are defined by their generic solutions. This is also the philosophy behind the various
taxonomies of problem/task types refered to in Sec. 1. Newell & Simon observe that the term
solution may mean different things [Newell and Simon, 1972, p. 76]. They distinguish between
solution-objects, solution-paths, and solution-action sequences . The first is what we normally
would see as a solution: some outcome or conclusion of the reasoning. For instance, the solutionobject in diagnosis is the set of faulty components. Solution paths take the reasoning itself as its
objective. In finding a proof in a geometry problem we are not so much interested in the outcome
— that was already given or hypothesized — but in the way it can be argued. Solution-action
sequences are plans or instructions that lead to the required solutions: they are plans, or rather:
problem solving methods (PSMs).
In later literature one will find no explicit elaboration on this observation, but rather varied descriptions of what solutions consist of. A good example can be found in the work of Clancey. In
[Clancey, 1985] he shows that in analytic types of problems, the solution is part of the knowledge
base, i.e. the solution is to be found by classification rather than by construction. For instance, in
medical diagnosis, the solution may be the name of one or more known diseases. The solution is
here the conclusion of applying a PSM, and coincides with Newell & Simon’s “solution object”.
However, in later publications, notably [Clancey, 1992], he convincingly argues that this is too
meager a view, and that a solution is rather constructed as a situation specific, or case model (cf.
[Steels, 1990]). At the end of the article (sec. 7) the emphasis is no longer on modeling, but on
the way evidence supports the case model components. This is a “process model relational structure”, which “can be interpreted like a proof”. From these three distinctive aspects of solutions of
problems, it is possible to synthesize the notion of a complete solution to a problem. We propose
that a complete solution has three components. (1) The case model represents the understanding
of the problem. (2) The conclusion is the answer to the question posed by the problem definition,
while the supportive evidence forms (3) the argument structure that states what evidence supports
the conclusion (see Fig 4).
The conclusion is a part both of the argument structure, and of the case model. From the perspective of the argument structure, a conclusion is the “sentence” or theorem which is supported by all
the evidence. Evidence may be empirical (data, requirements), or based on common knowledge,
6
complete
solution
Z
Z
Z
Z
case
model
conclusion
argument
structure
Figure 4: Components of a solution
which may range from simple facts to as-valid-accepted structures or procedures of argument.
From the perspective of a case model, the conclusion is that part of the case model which is of
interest (input) in solving a next problem (see Sec. 6 for an explanation). The argument structure
is not independent of the case model. It refers to facts or entities in the case model. The argument
structure is linked to the case model by making references to its behaviours and/or components.
Complete solutions are rare, or rarely made explicit, because one may be interested in only one
component (e.g. conclusion, argument strcucture), or because it is assumed to be known by the
‘user’ (e.g. (large parts of the) case model). The roles of these three components of a solution with
respect to communication to the user are the following :
Explanation: The CASE MODEL has the role to explain the data.
Justification: The ARGUMENT STRUCTURE justifies the conclusion.
Result: The CONCLUSION is an input for another problem definition, or (sub)task. This (sub)task
may be executed by another agent than the one who produced the solution (see also
[Breuker and de Greef, 1993])
3.1
Problem Type and Components
The way the conclusion covers the other two components is not fixed, but depends on the problem
type. For instance, if the problem is to find a proof, the conclusion coincides fully with the argument structure. In design, the conclusion — some structure — covers completely the case model,
i.e. there is no distinction between these two components. 2 The argument structure consists of
the set of explicitely satisfied requirements. For diagnosis problems, the conclusion consists of the
identification of one or more faulty components, while the case model includes also the correct
ones and their structural description [de Kleer et al., 1992]. Here the argument structure is built up
from the tests performed to show that the faulty components are indeed faulty, and the correct ones
correct — or can still be assumed to be working correctly. It should be noted that the argument
structure is not simply the trace of some problem solving process: it is largely its restructuring
in terms of argument relations between case model components and evidence. Assessment is
concluded by a decision class or metric attributed to a case model [Valente and Löckhoff, 1994].
The argument structure for the complete solution of this type of problems is very thin. It suf2
In Fig 4 the dashed box ‘conclusion’ should then cover the box ‘case model’ completely.
7
fices to show that the terms used in the norms of the decision class are applicable to the case
[Breuker, 1993].
Usually the case model is described as a situation specific version of generic domain models
(e.g. [Steels, 1990]). Similarly, the argument structure can be viewed as a problem specific version
of the rationale of a PSM (see [de Velde, 1993] about the “competence” of a PSM). The rationale
of a PSM describes how and under what assumptions a method yields valid results.
3.2
Problem Solving Methods and Components
A good example of the different contributions to the solution components is provided by PSMs
for solving diagnosis problems of devices. The conclusion of such a diagnosis consists of the
set of abnormal component(s). This set is the only relevant one for the dependent problem: the
repair of the device. However, some of these PSMs may as well be aimed at the construction of
a more complete case model by identifying the normal components as well. Requirements on the
completeness of the solution has direct consequences for the cost-effectiveness of a PSM. It is
cheaper to identify only faulty components, instead of also the correctly working ones, because
a single deviant value is sufficient evidence for a fault, whereas establishing the correct working
of a component may involve checking its full behavioural repertory. Minimal solutions may be
tractable, where a slightly more complete solution may easily make the problem too complex
[de Kleer et al., 1992].
PSMs may also differ whether they reach the conclusion by modeling or by argumentation. This
can be illustrated by the two major approaches in model based diagnose. 3
In consistency based diagnosis (e.g. GDE [deKleer and Williams, 1987]), the abnormal components are identified when their behaviour is inconsistent with the model of their correct behaviour.
These conclusions are reached in a deductive way, and primarily by argumentation. The behaviour
of the smallest set of remaining components that cannot be explained by models for normal components “must be” abnormal components. The price payed is that we have an incomplete case-model,
because we do not know what the faulty behaviour of the abnormal components is. Note also that
the other components are assumed to be working properly, because no falsifying evidence has
been obtained in testing. Therefore, also the argument structure is incomplete, but the conclusion
— the set of abnormal components — is completely justified.
In abductive diagnosis, the abnormal components are not only identified, but they are modeled as
well. Generic models of faulty behaviour are matched against the data to obtain explanations (covers). The case model, therefore, explains the abnormal components [Console and Torasso, 1991].
Abductive PSMs can only be justified by assuming the domain to be a closed world. It is easy to
see how consistency based diagnosis can be extended by the use of fault models. In this way abductive and consistency based approaches are complementary (e.g. [Console and Torasso, 1991],
[de Kleer et al., 1992]).
3
We will use the convention to denote tasks or methods by a verb — e.g. (to) diagnose —, while problem types will
be indicated by nouns, e.g. diagnosis. Sometimes, this convention is hard to maintainfor PSMs, because of the use of
adjectives to denote a specific method, e.g. in heuristic classification.
8
4
Problem Types
Problems can be characterized by their (minimal) solutions, i.e. their generic conclusions. A
generic conclusion is an abstract description of an object that covers the set of conclusions. The
generic conclusions can be thought of as types, and there exists also a fair consensus on a number
of these types. Diagnosis, design, planning are well established ones. We will identify another five
types: assessment, monitoring, modeling, assignment (including scheduling and configuration)
and prediction. We do so, not only because some of these (assignment, monitoring) are still the
largest common denominators in the various typologies proposed for knowledge acquisition (see
also [Breuker, 1994]), but also because these additional types fill up holes in the systematic view
we will develop in the next section. These definitions are still tentative and will be the object for
further study, in particular by formalization. For some problem types formalizations can be found
in the literature: [de Kleer et al., 1992] for diagnosis; [Poeck and Gappa, 1993] for assignment.
Modeling is concerned with the identification of what is a system and what is its environment,
or more precise: what is its interface with the environment. This interface is consists of
behaviour (input/output state dependencies): a behavioural model. The model may be a
partial one, as in functional specifications.
Design has as its conclusion a structure of components and their connections. The components
may be objects or processes, physical or symbolic.
Planning delivers sequences of causally/teleologically connected states (behaviour) in the world,
which may be filled by actions. If the final state(s) are not intended future states (goal states),
but states in the past, the planning problem is called reconstruction.
Assignment distributes (additional) elements (components, actions) over a structure. If the structure is a plan, the assignment problem is generally called scheduling. If the structure is a
design the problem is often called configuration.
Prediction delivers the resulting states of a system over time, starting with an initial state. When
one derives an initial state from some output states one may speak of postdiction.
Monitoring yields a discrepancy between a predicted state and an observed state.
Assessment provides a measurement that classifies behaviour according to norms.
Diagnosis finds components or structures which conflict with their behavioural model or design.
Table 1 summarizes the typology.
5
A Framework for Analyzing Tasks
By finding relevant distinctive properties, these generic conclusions may be categorized in a taxonomy. The advantage of a taxonomy is that it provides coherence (abstraction; exclusion). However, for a typology of problems, a single taxonomy appears to work as a Procrustian bed. There
9
major type
synthesis
modification
analysis
type of problem
modeling
design
planning/reconstruction
assignment (scheduling, configuration)
prediction
monitoring
diagnosis
assessment
generic conclusion
behavioural model
structure of elements
sequence of actions
distribution/assignments
state of system
discrepant states
faulty element
class/grade attribution
Table 1: Type of problem and corresponding generic conclusions
is a wide consensus on the first major distinction, i.e. between synthesis and analysis problems.
[Clancey, 1985] gives a most eloquent view on this difference by arguing that analysis problems
deal with operations on known systems — “models of systems in the world” — while in synthesis problems there is not yet a model-of-a-system available. However, beyond this distinction,
further refinements become problematic, because distinctive properties represent different views.
For instance, synthesis problems can be distinguished by the number and nature of dimensions of
the generic conclusion. A plan is one dimensional, over ‘time’. A design is multi-dimensional
and generally involves ‘space’. Moreover, these problem types may also be distinguished by the
nature of internal requirements, i.e. whether the elements and/or the structure of the system to
be designed or planned are given, or have to be found (see Fig. 1). A third problem with these
taxonomies is that when they go deeper than one ply, consistency is easily lost.
Therefore, we have looked for another principle to arrive at a collection of problem types, which
has the same properties as taxonomies (exclusion, covering), but not the problems that are the
consequence of ‘multiple inheritance’. These have been found in looking at the the way problems
are related to one another by the fact that the results of one problem are necessary to define a
dependent problem. These dependencies, i.e. the generic conclusions, between well-defined problems are a more consistent organizer of problem types than a taxonomy, and have moreover the
advantage of a practical analytic tool for identifying configurations of problems and tasks, as they
occur in real life.
5.1
Some Typical Tasks and Their Problems
The suggestion to look rather for the dependencies between problems as they are packed in a
task, than for a set of distinctive properties of generic conclusions that could serve as the basis
for a taxonomy, comes from experiences with the KADS-1 taxonomy of generic tasks ( see e.g.
[Breuker et al., 1987, Ch. 8, Real Life Tasks]). In real-life applications, a problem type hardly
ever comes alone, but rather in chains that reflect the fact that the conclusion of one problem is a
required input to solve a next problem. These dependencies between problem types — i.e. their
generic conclusions — can be easily overlooked because task decompositions (PSMs) may break
up the task in sub-tasks which may cover more than one problem, and one problem may be solved
by more than one subtask. In the next sections we will provide evidence for the view that by
distinguishing the problems from the tasks, common and rational dependencies between problems
10
become visible.
Problems in Diagnose Tasks
The minimal solution of a problem is a conclusion. Conclusions may be needed to resolve a next
problem. The conclusion of a diagnosis problem — a set of faulty components — is a necessary
input for a repair task. The repair is dependent on the results of a diagnosis problem. In well
organized, routine problem solving, the problems and their dependencies are packed into tasks
and task decompositions. This packing is provided by a PSM, and each PSM may pack problems
in a different way. Here we will discuss some examples from diagnose tasks.
A very simple version of a diagnose task is exemplified by the a common practice in finding faulty
computer components by replacing the boards in a systematic way. This replacement is a repair,
and can be conceived as a very simple assignment problem in which a new component is assigned
to the position of a failing or suspect component. In this simple task, the diagnosis and assignment
problems are configured in such a way, that repairs are used to test diagnosis hypotheses, and
the solutions to the diagnosis problem are used as hypotheses. This alternation between simple
hypothesis generation and test-by-repair stops, when a repair yields a correctly functioning device
(see Fig 5).
diagnose-task (2)
"
generate/
discriminate
hypotheses
"
"
evidence
hypothesis
l
l
l
-
test
hypothesis
assignment
diagnosis
Figure 5: Decomposition of a simple diagnose task with implied problem types (in italics). The
test-task consists of solving a repair problem
However, this configuration will only work under specific conditions (cost of replacement, easily
decomposable device etc.) and assumptions (single fault). In most other practical diagnose tasks
the solution of repair problems is postponed until this task has been definitely concluded, for
instance, when a single fault has been identified.
In the diagnose-by-repair task there is still a one to one correspondence between the problems to
be solved and the subtasks. The generate-hypotheses subtask covers the diagnosis problem, and
the repair subtask stands for solving the assignment problem. Another, more common structure
of a diagnose task with a different set and distribution of problems is the following (see Fig 6).
A diagnosis problem can only exist, when some malfunction or complaint has been observed.
A malfunction or complaint is a discrepancy between what a system should do according to its
design/model and what is observed. Stating how the system should behave — given some input
— is the result of some prediction. This prediction problem can be solved by derivation from
11
structural/behavioural models, or by instantiating a behavioural model (e.g. a causal model, or
an associational model based on empirical correlates). However, in this diagnose task one is not
free to choose from either option. If the diagnose task indeed implies a real diagnosis it should
lead to pointing out which components in the structure are the cause of the complaint. It should
take a structural model as its road map, and behaviour models of the components as input for
prediction, or testing. The requirements of the diagnosis problem dominate the dependent test
subtasks. A combination of prediction and monitoring of the actual behaviour of the components
of a device provides the primary ingredients for a test task. The test task, or rather the conclusion
of the monitoring problem in this task, may result in a discrepancy. This discrepancy may play two
roles. The first one is the input-role for a diagnosis problem, as a complaint. In this way a diagnose
task can be triggered, because a fault has been monitored. The second role is an ‘evidence’ role,
and is also an input to a diagnosis problem. In that case, the test has become a proper subtask in a
diagnose task, rather than a ‘front-end’ task to a diagnose task. This distinction can be expressed
in the control over the test (sub)task and diagnose task. In the first role, the test task is followed by
a diagnose task; in the second role, the test task is a subtask and is used in an iterative way to refine
the hypothesized diagnoses (output of the diagnosis problem). The conclusions of the diagnosis
problem take the role of hypotheses. The diagnosis problem does not have a single solution, but all
solutions. However, the diagnose task uses the hypothesis generation and empirical testing cycle
— a PSM — to bring this number down to a minimal set that explains the malfunction. It is the
control over these problems that constitutes the task and provides roles to the conclusions of these
problems. Note that the generic nature of the problems remains the same, while for each iteration
the specific problem to be solved is a different one.
diagnose-task (1)
"
l
"
"
l
l
generate/
evidence/complaint test
discriminate
- hypothesis
hypotheses
hypothesis
"
@
"
"
diagnosis
@
"
@
- compare
predict XXX observe XX
X :
prediction
monitoring
- dependency
Figure 6: Decomposition of a typical diagnose task with implied problem types (in italics)
In Fig 6 the relation between this diagnose task decomposition, and the implied problems is presented.
Whatever the packing of problems in these diagnose tasks (and its monitoring “front-end”, respectively assignment (repair) “back-end”), a common chain of dependencies between the types of
problems can be abstracted:
prediction → monitor → diagnosis → assignment
Other types of problems may enter the diagnose task as well, in particular in providing evidence
12
(test roles). We have already mentioned the assignment problem, that can be used in two roles:
as a test, and as a “back-end” to the diagnose task proper. However, more complicated versions
of diagnose may occur. For instance, when observations are difficult or impossible to obtain in
a direct way, test may be performed by reconfiguring a system. For instance, a cheap test for
stereo-audio equipment is to switch channels, when the complaint is in one of the channels. The
configuration problem that is solved is not part of a test, and does not feed in one of the problems
of the test, but it changes the diagnosis problem, because the system description of the device has
been changed. This dependency with the configuration problem can be simply added to the chain
above (we will use here the standard term assignment for configuration):
assignment → prediction → monitor → diagnosis → assignment
Another type of problem that can be observed in real life diagnose tasks is planning. The sequence
(or tree) of hypothesis testing can be governed by a plan or schedule e.g. as to maximize the
effectiveness and minimize costs of tests (cf. MOLGEN, [Stefik, 1981]). The chain of problems
is as follows:
planning/assignment → prediction → monitor → diagnosis → assignment
Model and Design Tasks; Levels of Understanding
Before problems with a system can be identified and diagnosed, the system has to be understood.
This means that a (case)model has to be constructed. Generic versions of such a model may already
be available, thus reducing the model task to an instantiation or adaptation of this generic model
of a system. For instance, we may know the global structure and behaviour of amplifiers, which
we may use to diagnose a particular type and exemplar. Understanding is not an all or nothing
process. For all practical purposes, a system can be understood at three levels:
• By classification: the properties of a system are known, which distinguish a type of system
from another type. For instance, a taxonomy of bacterial diseases is a very simple model of
malfunctions of the body caused by bacteria.
• By understanding its behaviour, i.e. by modeling how changes in the environment, i.e. input
to the system, affect the state of the system.
• By understanding its working. This implies not only to have a model of its behaviour,
but also a description of what causes this behaviour. There are two major types of causal
models:
– The causal model describes the behaviour of a model by intermediary, internal processes or states of a system. For instance, in describing diseases in terms of processes
(syndromes), intermediary states such as “immunosuppressed” may provide explanations to account for bacterial diseases which normally would not interfere with the
normal processes in the body.
13
– The deepest form of understanding consists of relating the causal model to a (physical)
structural description that explains by its connections how causes flow through a system. The components of such a system are viewed as subsystems, for each of which a
behavioural model may be available, for which again a structural description may be
used for more detailed understanding. An example of a structural model is the wiring
diagram of an electrical circuit.
When we use the term causal model, we mean in general only the first type of model, while
the one that has a mapping between causal chains and physical connections or components
is more precisely denoted by the term structural/behavioural model.
A structural model of a system is the result of solving a design problem. In the construction of
artifacts (devices), the design problem is solved by a design task. 4 However, we use the term
to model, when it concerns the design of causal or structural models of natural systems. Both
model and design tasks appear to produce the same kinds of models, and there are more striking
similarities. A design or a model is tested. A design is tested against its functional requirements,
i.e. against the behaviour that it is intended to exhibit in interacting with its environment (including
a user), while an empirical model is tested against its predicted behaviour. These tests look exactly
the same as the tests performed in diagnose task; the difference is that this testing is not focussed on
explaining a particular discrepancy, but may imply the full behavioural repertoire of the designed
or observed system.
There is a growing literature on design and model tasks. A typical example of the PSMs for
these tasks can be found in the work of Chandrasekaran. As a first decomposition of these
tasks, the following subtasks are identified: “Propose, Verify, Critique, Modify (a design)”
[Chandrasekaran, 1990]. Other versions of the same decomposition can be found in the “propose & revise” PSM for configure tasks [Marcus and McDermott, 1988], or the “generate, test &
debug” paradigm of [Simmons, 1992] for plan and reconstruct tasks. The Propose subtask is the
only task that contains the design problem. The Verify, Critique and Modify subtasks represent
process control cycles, which check whether the current state of the design is sufficiently close to
(a subset) of the requirements. Therefore, the design task appears to imply the following chain of
problems, where Verify implies monitoring, Critique diagnosis and Modify an assignment:
design → prediction → monitor → diagnosis → assignment
A design problem is solved by the construction of a structural model. The nature of the structure may be explicitly required — internal requirement —, or may be part of a design option.
The structure is created to satisfy functional/behavioural (external) requirements. These are the
result of solving a modeling problem. Within a model/design task one may often find the design
problem and the next dependent assignment problem quite entangled, because the design problem
delivers the structure, while the assignment fills it with elements. Because of interaction problems
[Simmons, 1992], it is far more efficient to solve both problems in the same refinement cycles. A
similar weaving is found in schedule tasks where the planning and assignment problems are often
4
Here we intend the verb “to design”; not the substantive “design” which we use for problems.
14
solved in a combined way [Valente, 1994, Sundin, 1994]. The same can be the case for diagnose
and repair (assignment) [Friedrich, 1993]. Therefore, the full chain implied in model or design
tasks looks as follows:
modeling → design → assignment → prediction → monitor → diagnosis →
assignment/design
Similar analyses as those for diagnose, model and design tasks can be performed for other kinds
of tasks. In [Breuker, 1994], other tasks are described. For instance, in process control and device
manipulation one finds the follwing chains:
planning/assignment → prediction → monitor → diagnosis → assignment
planning/assignment → prediction → assessment → assignment
Similar chains are to be found in test, evaluate and remedy tasks. and diagnose tasks which are
aimed at the reconstruction of actions that have lead to a failure.
6
A Suite of Problem Types
These chains of problems which are identified in a large variety of tasks exhibit congruent patterns,
which can be integrated into a graph: a suite of problem types (see Fig. 7).
synthesis modification
analysis
behavioural view (system*environment)
assessment
planning
>
assignment prediction
HH
3
j
H
@
R
@
modelling
Q
s
Q
-
design
monitoring - diagnosis-
structural view (system)
Figure 7: A suite of problem types, dependencies and views
The graph can be divided into two overlapping perspectives: a behavioural and a structural one.
A behavioural perspective is taken when its is not possible or feasible to control a system by
encapsulating causal effects into physical structures, or to know how causal effects ‘propagate’
through the physical structure. In that case we have to plan or assess actions. By planning we
15
make a spatio-temporal arrangement that enables causal propagation for a specific situation. For
instance, a travel plan (schedule) consists of local and temporal contingencies (“connections”) and
actions (components). Therefore, the perspectives are relative, and the graph may be collapsed
into a single chain. However, we think that the split is highly relevant, because it reflects two ways
to control our world: by material technology where we pack causality in (increasingly complex)
devices, which leads to mechanization and automation, and by enabling chains of actions: the
latter being the default option.
In a very abstract sense we can now explain the ontogenesis of problems and their packing into
tasks by PSMs, as a tracing — mainly backtracking — along the suite. Spontaneous, ill-defined
problems are identified by solving (permanent) monitoring or assessment problems. The spontaneous problem is an indication that somewhere earlier in a dependent problem a solution is no
longer valid. This backtracking in problem definition has a minor or a major scope over the suite,
dependent on the nature of available domain knowledge and information (data) in the situation.
The minor type of problem definition consists of a refinement of the conflict (ill defined problem)
to allow a minimal, local change — modification — to a system (repair) or a course of action
(remedy): both modifications involve an assignment problem. The required refinement may be
obtained by solving a diagnosis problem. In diagnosis faulty components instead of the faulty
system are identified. Refinement may also be obtained by assessing the conflict, e.g. in terms
of classes of malfunctions. If the problem cannot be refined further or cannot be solved by assignment (repair; remedy), more drastic problem solving is required, i.e. one has to start at the
modeling entry of the suite.
In this way, a problem definition may consist of more than a single type of problem. As we have
seen in Sec 5.1, a task covers in general more than one problem type. This is for two different
reasons or rationales. The first one is based upon the fact that more than one dependent problem
may have to be solved, as in the case of a diagnosis and an assignment (repair). We have seen
the packing of combinations of problems by a single method/task in particular for reasons of
efficiency (see e.g. [Friedrich, 1993]). However, we have also found other chains of problems. In
all those cases these problems are packed into tasks because of the different roles that are involved
in arriving at a complete solution. The rationale for these additional tasks is not that they are
intermediary steps towards some goal, but that they constitute a test or evaluation cycle that builds
the argument structure of a solution. Despite the fact that this control cycle is never a main task, but
a rather universal subtask, it also follows the dependencies in the suite. For a further elaboration
of the consequences of the reappearence of the ‘generate & test’ paradigm, the reader is refered to
[Breuker, 1994].
7
Conclusions
These new conceptualizations of the elements of solutions, and the problem typology have a number of advances:
• By distinguishing problem definitions from tasks a smaller, less variant set of types can
be identified. Tasks and their decompositions hide the problems they are to solve. More16
over, the names and ‘real life’ appearances of tasks such as ‘operation control’, ‘(medical)
diagnosis’, ‘trouble shooting’, ‘verification’ etc. obscure the their commonalities and distinctive features. The typology of problems is at least more parsimonious and appears thus
far sufficient to characterize the problems that are solved by (real life) tasks.
• It is assumed that this parsimonious typology also provides a more strict classification of
problem solving methods (PSMs). Ideally, for each problem type exclusive PSMs should
be identified. There is not yet enough evidence to warrant such a claim, but we have at
least shown that by disentangling problem types from (sub)tasks we get a better view of
what problem(s) a PSM is supposed to solve, and what task-goals it is aimed to accomplish. For instance, it can be understood why many PSMs for medical diagnose are not
diagnostic methods at all, but rather aimed at assessment (normative classification) (e.g.
[Clancey, 1985]).
• The view that a complete solution to a problem has two major components or ‘sides of the
same coin’ — a case model and an argument structure — clarifies why some PSMs justify
conclusions (partial solutions) — e.g. tests which check observations against hypotheses
— and other PSMs focus on a coherent interpretation — e.g. causal abduction. More in
general, the ‘generate and test’ paradigm that is observed in all complex tasks reflects the
separation of the construction of the case model and of the argument structure.
• Conclusions are that part of a case model and/or argument structure, that answer the question
implied by a problem. Conclusions are necessary inputs to solve a ‘next’, related problem,
and for that reason sufficient as a minimal solution to a problem.
• The pattern of dependencies between problem types forms a “suite”, i.e. rational sequences
of conclusions, which start in principle with modeling, but may be triggered by monitoring
or assessing (one’s own) activities. The suite avoids the inconsistencies in (single view)
taxonomies of problem, or task typologies proposed in the literature. Moreover, it provides
guidance for understanding the structure of real life tasks.
• The genesis of problems and tasks may be modeled by three steps:
– A problem is identified as the discrepancy between some norm(al) state and an actual
state.
– The ill-defined problem is turned into a well-defined one, i.e. one for which a generic
conclusion is found. The problem type suite reflects the typical (sequences of) generic
conclusions.
– In task-specification for a well-defined problem appropriate problem solving methods
(PSMs) are found or constructed.
Besides these practical consequences of the rather theoretical exercise presented here, the more
specific application is of course in the Common KADS library tools, as currently under development. Further research is going on, aimed at a formalization of the problem types to assess the
detailed structure of the problem types (generic conclusions) and the consistency of the dependencies.
17
References
[Aamodt et al., 1992] Aamodt, A., Bredeweg, B., Breuker, J., Duursma, C., Löckenhoff, C.,
Orsvarn, K., Top, J., Valente, A., and van der Velde, W. (1992). The CommonKADS Library. Deliverable 1.3. KADS-II/T1.3/VUB/TR/005/1.0, KADS Consortium, Free University,
Brussels.
[Breuker, 1993] Breuker, J. (1993). Modelling Artificial Legal Reasoning. In Aussenac, N., Boy,
G., Gaines, B., Linster, M., Ganascia, J., and Kodratoff, Y., editors, Knowledge Acquisition for
Knowledge Based Systems, Proceedings of the EKAW-93, pages 66 – 78, Berlin. Springer.
[Breuker, 1994] Breuker, J. (1994). A suite of problem types. In Breuker, J. and de Velde, W. V.,
editors, The CommonKADS Library for Expertise Modeling. IOS-Press, Amsterdam.
[Breuker and de Greef, 1993] Breuker, J. and de Greef, P. (1993). Modelling system-user cooperation. In Schreiber, G., Wielinga, B., and Breuker, J., editors, KADS: a Principled Approach
to Knowledge Engineering, pages 47 – 70. Academic Press, London.
[Breuker and de Velde, 1994] Breuker, J. and de Velde, W. V., editors (1994). The CommonKADS
Library for Expertise Modeling. IOS-Press, Amsterdam.
[Breuker et al., 1987] Breuker, J., Wielinga, B., van Someren, M., de Hoog, R., Bredeweg, B.,
Wielemaker, J., Billault, J.-P., Davoodi, M., and S, H. (1987). Model Driven Knowledge Acquisition: Interpretation Models. Esprit P1098 KADS A1, University of Amsterdam.
[Chandrasekaran, 1983] Chandrasekaran, B. (1983). Towards a Taxonomy of Problem Solving
Tasks. AI Magazine, 4(1):9 – 17.
[Chandrasekaran, 1990] Chandrasekaran, B. (1990). Design problem solving: a task analysis. AI
Magazine, pages 59–71.
[Chandrasekaran and Johnson, 1993] Chandrasekaran, B. and Johnson, T. (1993). Generic tasks
and task structures. In David, J.-M., Krivine, J.-P., and Simmons, R., editors, Second Generation Expert Systems, pages 232 – 272. Springer, Berlin.
[Clancey, 1985] Clancey, W. (1985). Heuristic classification. Artificial Intelligence, 27:289–350.
[Clancey, 1992] Clancey, W. (1992). Model construction operators. Artificial Intelligence, 53:1–
115.
[Console and Torasso, 1991] Console, L. and Torasso, P. (1991). A spectrum of definitions of
model based diagnosis. Computational Intelligence, 7(3):133 – 141. extension of paper from
the ECCAI-90 Proceedings.
[de Kleer et al., 1992] de Kleer, J., Mackworth, A., and Reiter, R. (1992). Characterizing diagnoses and systems. Artificial Intelligence, 56(2–3):197 – 222.
[de Velde, 1988] de Velde, W. V. (1988). Inference structure as a basis for problem solving. In
Kodratoff, Y., editor, Proceedings of the 8th European Conference on AI, pages 202 – 207,
London. Pitman.
18
[de Velde, 1993] de Velde, W. V. (1993). Issues in knowledge level modelling. In David, J.-M.,
Krivine, J.-M., and Simmons, R., editors, Second Generation Expert Systems, pages 211 – 231.
Springer, Berlin.
[deKleer and Williams, 1987] deKleer, J. and Williams, B. (1987). Diagnosing multiple faults.
Artificial Intelligence, 32:97–130.
[Friedrich, 1993] Friedrich, G. (1993). Model-based diagnosis and repair. AI Communications,
6(3/4):187 – 206.
[Marcus and McDermott, 1988] Marcus, S. and McDermott, J., editors (1988).
Knowledge Acquisition for Expert Systems. Kluwer, Reading MA.
Automating
[McCarthy, 1956] McCarthy, J. (1956). The inversion of functions defined by Turing machines.
Automata Studies, Annals of Mathematical Studies, 34:177 – 181.
[McDermott, 1988] McDermott, J. (1988). Preliminary steps towards a taxonomy of problemsolving methods. In Marcus, S., editor, Automating Knowledge Acquisition for Expert Systems,
pages 225–255. Kluwer Academic Publishers, The Netherlands.
[Newell and Simon, 1972] Newell, A. and Simon, H. (1972). Human Problem Soving. Prentice
Hall, Englewood Cliffs, NJ.
[O’Hara and Shadbolt, 1993] O’Hara, K. and Shadbolt, N. (1993).
Knowledge Acquisition, 5(4):449 – 481.
Locating generic tasks.
[Poeck and Gappa, 1993] Poeck, K. and Gappa, U. (1993). Making role-limiting shells more
flexible. In Aussenac, N., Boy, G., Gaines, B., Linster, M., Ganascia, J., and Kodratoff, Y.,
editors, Knowledge Acquisition for Knowledge Based Systems, Proceedings of the EKAW-93,
pages 103 – 122, Berlin. Springer.
[Schreiber et al., 1993] Schreiber, G., Wielinga, B., and Breuker, J. (1993). KADS: a Principled
Approach to Knowledge Engineering. Academic Press, London.
[Sergot and Jones, 1992] Sergot, M. and Jones, A. (1992). Deontic Logic in the Representation
of Law. Artificial Intelligence and Law, 1(1):45 — 64.
[Simmons, 1992] Simmons, R. (1992). The role of associational and causal reasoning in problem
solving. Artificial Intelligence, 53(2–3):159–207.
[Steels, 1990] Steels, L. (1990). Components of expertise. AI Magazine, 11(2):28–49.
[Stefik, 1981] Stefik, M. (1981). Planning with constraints (MOLGEN: Part 1). Artificial Intelligence, 16:111 – 140.
[Sundin, 1994] Sundin, U. (1994). Assignment and scheduling. In Breuker, J. and de Velde,
W. V., editors, The CommonKADS Library for Expertise Modeling. IOS-Press, Amsterdam.
[Valente, 1994] Valente, A. (1994). Modeling components for planning problems. In Breuker, J.
and de Velde, W. V., editors, The CommonKADS Library for Expertise Modeling. IOS-Press,
Amsterdam.
19
[Valente and Breuker, 1994] Valente, A. and Breuker, J. (1994). A commonsense theory about
normative systems. In Breuker, J., editor, Proceedings of the ECAI Workshop on Artificial
Normative Systems. ECAI.
[Valente et al., 1993] Valente, A., Breuker, J., and Bredeweg, B. (1993). Integrating modelling
approaches in the Common KADS library. In Sloman, A., Hogg, D., Humphreys, G., Ramsay,
A., and Partridge, D., editors, Prospects for Artificial Intelligence, Proceedings of the AISB-93,
pages 121 – 130. IOS Press, Amsterdam.
[Valente and Löckhoff, 1994] Valente, A. and Löckhoff, C. (1994). Assessment. In Breuker, J.
and de Velde, W. V., editors, The CommonKADS Library for Expertise Modeling. IOS-Press,
Amsterdam.
[van Harmelen and Balder, 1993] van Harmelen, F. and Balder, J. (1993). M L2 : a formal language for KADS models of expertise. In Schreiber, G., Wielinga, B., and Breuker, J., editors,
KADS, a principled approach to knowledge based system development, pages 169 – 201. Academic Press.
[Wielinga et al., 1993] Wielinga, B., de Velde, W. V., Schreiber, G., and Akkermans, H. (1993).
Expertise model definition document. Esprit P5248 KADS-II/M2/UvA/026/1.1, University of
Amsterdam. date May 24 1993.
20