Medical Images and Clinical Decisions
John Fox1*, Paul Taylor2
1Advanced
2Centre
Computation Laboratory, Imperial Cancer Research Fund, London,WC2A 3PX
for Health Informatics and Multiprofessional Education, UCL, London, N19 5NF
Abstract. Most imaging studies are performed to provide information used in making decisions about the
management of patients. We have developed a generic design for clinical decision support systems that can
exploit information obtained from image processing. The design is based on a formal model of decisionmaking. The model has been used to guide the implementation of CADMIUM, a prototype radiology
workstation. CADMIUM has successfully been used to improve the decision-making of trained film-readers
interpreting mammograms.
1 Introduction
When we speak of understanding medical images we can mean different things. Basic research on image
understanding tends to yield techniques which are very general but may be of limited practical value in specific
applications which are aimed at interpretation in terms of domain-specific objects or events. On the other hand
research on specific domains may not result in application-independent principles. This paper introduces a
framework for medical imaging systems which lies between these two extremes.
Most medical imaging is carried out to support decision-making: to understand the requirements for systems
which assist in the interpretation of images, we must understand medical decision-making. Medical decision
theory has been strongly influenced by mathematical decision analysis, which adopts probabilistic inference and
quantification of utilities as its basic mathematical operations. Medical decision-making is not well described by
an expected-utility model because the process of medical decision-making involves much more than cost-benefit
analysis and weighing of evidence. In the absence of a clear understanding of the clinical decision process it is
difficult to develop a general model of when and how imaging technology should be used.
Consider the range of activities carried out in making clinical decisions, a few of which are illustrated in
Figure 1. These include: recognising that a decision is required, determining what information is relevant to
a decision, establishing the options that we must decide between, establishing criteria for choosing among
them and making a choice.
History
taking
Tests and
investigations
(including
imaging)
Physical
examination
Defining the clinical
problem
Classify
risks
Select
tests
Make
diagnosis
Select
therapy
Refer
to
specialist
Cancel/
Abort
procedure
Figure 1: some of the many tasks carried out in medicine, including some of the decisions that clinicians have to take
(bottom). If we are to provide medical professionals with practical tools these must reflect the variety and
complexity of medical procedures.
*
Author for correspondence, [email protected]
In the next section we present a model of decision-making, which acknowledges the complexity of
medical decisions, and consider how this model may be used to guide the use of image processing in
medical applications.
2 A model of medical decision making and patient management
We represent decision-making as a symbolic reasoning process (formalised elsewhere in terms of nonclassical inference procedures [1]). The “domino model” in Figure 2 gives a schematic view of this process.
Suppose a patient has a palpable breast lump. As a possible abnormality this implies a clinical problem: to
establish the cause, or diagnosis of the lump. In Figure 2 this process of recognising a problem and an
associated decision is represented by the arrow labelled (1). Using general medical knowledge we may
now deduce that possible causes of a breast lump include, say, cancer and benign cyst, and record these as
possible solutions (2). Using information in the patient record (e.g. the patient is elderly) and knowledge
about the kinds of information which are relevant in making diagnosis decisions, we construct arguments
for and against each alternative diagnosis (3). An example of an appropriate argument might be that: "when
looking for arguments in favour of a disease consider any symptoms or signs in the patient record which are
known to be associated with each possible disease".
1
Medical
problems
2
Patient
record
Clinical
actions
4
6
F
i
g
Alternative
Medical
Pros and
u
solutions
procedures
5
cons
r
3
e
3
Figure 2: The domino model of the clinical processes of decision making and patient management. The
:
nodes of the domino can be thought of as symbolic data structures and arrows as reasoning
T
mechanisms.
At some point enough information may be available to commit to one diagnosis candidate (4). Suppose it is
decided that the patient is suffering from cancer. This suggests another problem (arrow 1 again), how
should we treat the cancer? As before, a set of candidate solutions is inferred from our medical knowledge
(2), though this time the candidates are medical procedures, such as chemotherapy, radiotherapy or surgery.
Once again arguments are constructed for and against the different options (3) and, in due course, the
software proposes to commit to a particular therapeutic procedure (5).
Clinical procedures often involve a number of steps. These tasks (e.g. taking blood samples or giving
medication) must be scheduled. The scheduling process (6) uses general knowledge of temporal and logical
constraints, and situation-specific factors such as the patient's condition and the availability of resources.
3 Imaging and decision support systems: CADMIUM
The CADMIUM (Computer Aided Decision-Making and Image Understanding in Medicine) project is
exploring the integration of decision support and image interpretation to bring imaging closer to the care
process. The CADMIUM workstation implements the majority of the functions summarised in the domino
model, functions for decision-making (suggesting hypotheses or medical procedures, constructing arguments
and making recommendations) and process management (maintaining a plan of tasks, scheduling and executing
tasks when appropriate), in the context of an application in mammography.
Figure 3 shows a screen from CADMIUM. The main window shows a mammogram which is being
interpreted. The bottom left window is a view of the patient record which contains information about the
patient’s history and other clinical information. The record is also used to store the current state of the protocol
being followed.
CADMIUM’s decision making functions are used in conjunction with a medical knowledge base that
contains information about diseases and their manifestations and which is used to suggest options for
radiological decisions and construct lines of reasoning about them. The knowledge base also includes
knowledge about clinical protocols, used to schedule clinical tasks required by the protocol.
Figure 3: Screen of the CADMIUM workstation.
The application in Figure 3 is concerned with the differential diagnosis of microcalcifications (small flecks of
calcium detected on mammograms which may have a benign or malignant interpretation). It makes use of
knowledge stored in the knowledge base about the depiction of calcifications and their properties, and about a
range of image processing operators which have been used successfully in the classification of
microcalcifications.
Executing the symbolic decision procedure leads to the proposal of candidate diagnoses and the construction
of arguments supporting these hypothetical diagnoses. One such argument is that certain benign calcifications, if
they appear clustered, tend to have a rounded cluster shape. CADMIUM can determine if a mammogram
contains the evidence that would support this argument: rules describing a detection task invoke the appropriate
operators to detect microcalcifications, and rules describing a classification task invoke appropriate operators to
determine if calcifications are clustered and if the cluster is round. At the top right of Figure 3 are two small
windows showing the results of the detection, clustering and shape analysis operators.
Once all the possible candidates and arguments have been considered and all relevant evidence obtained, the
image analysis results are summarised. The microcalcification cluster data have been used in the construction of
arguments for and against further investigation, and the window at the bottom right in Figure 3 shows some of
these arguments.
The symbolic decision model used in CADMIUM is described in detail elsewhere [2]. It consists of a set of
logical rules that describe decision-making at a very general level: rules for proposing decision options,
constructing arguments about the options, and for extracting and interpreting relevant information. The model
extends the standard domino decision procedure with rules that define three classes of image interpretation
tasks: detection, classification and measurement.
A pilot study has shown that this kind of system can significantly improve the performance of trained filmreaders interpreting mammograms [3]. Another prototype has been developed for the use of abdominal CT in
the staging and assessment of childhood abdominal tumours.
4 Conclusions and future work
The use of image processing and interpretation needs to reflect not only the image modalities but also the
clinical context, and particularly the kind of decision that is being made when imaging studies are considered.
The domino model provides a useful way of understanding this clinical context. It is sufficiently general to
illuminate the functions of medical image interpretation tasks and sufficiently specific to guide the organisation
of practical decision support software.
Imaging may be used in various parts of the decision-making and patient management processes, ranging
from initial detection of a possible abnormality and determination of its clinical classification to selection of
treatment. Figure 4 shows the domino model again, this time with each reasoning process annotated with some
of the kinds of image interpretation function that can contribute to the process.
Medical
problems
Feature detection
and measurement
Hypothesis
generation
Alternative
solutions
Patient
record
Image
capture
Image
processing
control
Decision
taking
Symbolic feature
abstraction
Pros and
cons
Clinical
actions
Decision
taking
Medical
procedures
Figure 4: The integration of decision making and image interpretation
The CADMIUM workstation brings together research into clinical decision support systems and ideas about
combining image processing and symbolic inference. Much of this work, notably the domino model, has been
developed formally, though the overall organisation of CADMIUM itself is more of an experiment. While
experience with this approach appears promising, being flexible and extensible, there is scope for more work in:



understanding the relationships between signal-processing and symbolic reasoning.
establishing a firm ontology of image interpretation tasks
clarifying appropriate methods for managing uncertainty at the image and interpretation levels
These and other topics will be addressed in future work.
References
1. S. Das, J. Fox, D. Elsdon and P. Hammond. A general architecture for an intelligent agent, Journal of Experimental and
Theoretical Artificial Intelligence (in press).
2. P. Taylor, J. Fox and A. Todd-Pokropek. A model for integrating image processing into decision aids for diagnostic
radiology, Artificial Intelligence in Medicine, 9, 205-225, 1997.
3. P. Taylor. Computer Aided Decision-Making and Image Understanding in Medicine (PhD Thesis, in preparation).