Role and Experience Determine Decision Support
Interface Requirements in a Neonatal Intensive Care
Environment.
Gary Ewing1, Yvonne Freer2, Robert Logie3, Jim Hunter1, Neil McIntosh2,
Sue Rudkin3 and Lindsey Ferguson2
1. Department of Computing Science, King’s College, University of Aberdeen, Aberdeen, UK
2. Neonatal Intensive Care Unit, Simpson Centre for Reproductive Health, New Royal Infirmary, Edinburgh,
UK
3. Department of Psychology, King’s College, University of Aberdeen, Aberdeen, UK
Abstract
The aim of this paper is to describe a novel approach to the analysis of data obtained
from card-sorting experiments. The card-sorting experiments were performed as a part
of the initial phase of a project, called NEONATE, which has an aim to develop
decision support tools for the neonatal intensive care environment.
Physical card-sorts were performed using clinical “action” and patient “descriptor”
words. Thirty-two staff (8 junior nurses, 8 senior nurses, 8 junior doctors and 8 senior
doctors) participated in the actions cards sorts and 32 staff (of similar classifications)
participated in separate descriptors card-sorting experiments. The card-sorts were
replicated for all staff in the descriptor experiments, but only for nurses during the
action card-sorts.
The card-sort data were analysed using conventional cluster analysis to produce treediagrams or dendrograms. Differences were shown in the way various classes of staff
with different levels of experience mentally map clinical concepts. Clinical actions
were grouped more loosely by nurses and by those with less experience with a
polarisation between senior doctors and junior nurses. Descriptors were classed more
definitively and similarly by nurses and senior doctors but in a less structured way and
quite differently by junior doctors.
This paper presents a graphical method of summarising the differences in the card-sort
data for the various staff categories. It was shown that concepts are used differently by
various staff groups in a neonatal unit and this may impact on the effectiveness of
computerised decision aids unless it is explored during their development.
Keywords: card-sorts, cluster analysis, user interface, neo-natal intensive care,
clinical terminology, clinical roles
Journal of Biomedical Informatics, Vol 36, pp 240-249 (2003)
1
1 INTRODUCTION
The modern intensive care unit is an environment that requires medical and nursing
staff to deal with large amounts of, and many different types of, information in
making clinical decisions. It has been shown that just displaying these data in their
raw form does not of itself lead to improved patient care [1]. The work reported in
this paper formed part of the initial effort in an ongoing project, NEONATE, to
develop decision support for clinical staff (doctors and nurses) in a neonatal
intensive care environment—Neonatal Intensive Care Unit (NICU) at the Simpson
Centre for Reproductive Health, Edinburgh. The component of this initial phase,
which is reported here, focussed on developing a concise lexicon of terms used by
clinical staff during clinical practice, and further, the use of these lexicons as the
basis for card-sorting experiments designed to elucidate the way clinical staff
mentally organise those terms. This information was used to design the userinterface for a software tool to allow a trained observer (research nurse) to record (in
a standardised fashion on a computer database) the clinical activities of doctors and
nurses as a data gathering exercise for the NEONATE project. (Detailed
physiological data such as heart rate, blood pressure, are automatically collected by
the NICU’s computerised monitoring system.)
The standard way of analysing card-sort experiments is by performing cluster
analysis in order to generate tree-diagrams (or dendrograms) as a graphical
representation of the relationships between the concepts under study. A cluster exists
when two or more concepts (cards) are grouped together (deemed similar).
However, the dendrograms generated by cluster analysis of card-sort data are often
very complex and difficult to interpret. Even though, a visual analysis of treediagrams may possibly provide useful insight into how people mentally organise
concepts, it is tedious and does not provide a quantitative measure of the complexity
or order implicit in the dendrogram. The work reported here aims to partly fill this
gap by providing a simple graphical summary of the difference in the information
carried by various dendrograms that are being compared.
Using these tools, this study assesses how nursing and medical staff use language
and concepts in a neonatal unit. We speculated that within any staff group there
would be different degrees of knowledge and experience, which would result in
different needs (in terms of user-interfaces for clinical decision support). Words and
concepts might be used differently because of these differences.
2 METHODOLOGY
The scope of this section is to outline the methodology employed in the data
gathering, performance and analysis of the card-sorting experiments.
NICU staff were classified as senior nurses (SN), senior doctors (SD), junior nurses
(JN) and junior doctors (JD), in order to delineate their roles in the unit and the
vocabularies they use to categorise neonatal data they obtained by observation and
physical means The nurses were categorised as senior or junior depending on their
years working in a neonatal unit. The medical staff were categorised by their
experience working in a neonatal unit. The level at which staff were categorized,
2
whether junior or senior, was decided upon by the senior clinical staff involved in
the project.
2.1 Clinical Lexicons
In order to elicit lexicons for both patient “descriptors” and clinical “actions” a
research psychologist interviewed medical and nursing staff with various levels of
experience. Four hundred and nineteen actions and 520 descriptors were offered by
participating staff. Senior medical and nursing staff subsequently reviewed these lists
for consistency, and to remove synonyms and singletons (single words used by only
one member of staff). The derived actions lexicon contains 51 terms, while the
descriptors lexicon contains 166 terms (see section 4.1 for details).
2.2 Card-Sorting Experiments
Following the interviews, we carried out card-sorting experiments using the two
lexicons to represent the concepts to be sorted. Card-sorting was used as an
elicitation technique because “Concept Sorting” is a well-known technique, and
studies in Cognitive Psychology and related fields [2] [3] have shown it to be
effective and very efficient.
Thirty-two subjects consisting of, 8 junior nurses, 8 senior nurses, 8 junior doctors
and 8 senior doctors, participated in the actions card sorts and thirty-two staff (of
similar levels) participated in separate descriptors card-sorting experiments. There
was considerable overlap1 of staff in the two groups (of 32), but the actions and
descriptors experiments were held several months apart. At least one week after the
initial sessions, the card-sorts were replicated for all staff in the descriptor
experiments, and for nurses only in the actions experiments. Each actions cardsorting session took about an hour to complete on average, while each descriptors
card-sort session required about 1.5 hours to complete.
The actual card-sorting procedure that we asked subjects to perform is illustrated in
Figure 1. During a session, each subject was presented with a physical pile of cards,
with each card containing a term from the appropriate lexicon. These (level-0) cards
were marked on the back with a bar-code (3 of 9 code), containing the term on the
front of the card, and a unique identifying alphanumeric code on the reverse side. We
also produced cards that were blank on one side and bar-coded on the other, again
with a unique identifying alphanumeric code, to be used as indicated in the next
paragraph.
Each subject was asked to sort the cards into piles of “similar” cards, without any
prompting as to how many piles to create or what attributes to use to sort the cards.
Once satisfied, the subject was asked to name each group (without restriction) and
these names were written by the experimenter on to (level-1) blank cards. The
experimenter then entered the names and codes of the cards, within their sorted
groups, into a computer database by means of a bar-code scanner. This saved
considerable time and minimised errors in data entry. The subject was then asked to
sort the cards containing the names of the groups that had just been created into
1
Many of the staff that participated in the card-sorts were also involved in the interviews to establish
the lexicons. However, there was several months gap between the interviews and the card-sorts.
3
higher groups; i.e. groups of groups. Again, the names of the groups were written by
the experimenter on to (level-2) blank cards. This process continued until the subject
was satisfied with the result.
Figure 1 Card-sorting procedure.
2.3 Analysis of the Card-Sort Experiments
Within the context of the work described in this paper, cluster analysis calculates the
strength (distance) of the perceived relationships between card-pairs, and displays
these relationships graphically (dendrograms).
We performed hierarchical agglomerative cluster analysis of the card-sort data using
a free software package called EZCalc [4], which was designed to be used with its
companion software, EZSort, that facilitates computer based card-sorting
experiments. As we did physical sorts (though semi-automated), the data files
needed to be formatted2 and pre-processed to allow use by EZCalc. The preprocessing also checked for consistency within the card-sort data files. As EZCalc
used arbitrary weighting for second level groups (the highest level of hierarchical
grouping it could handle) in the card-sort, we only used first level sort data for our
analysis. We employed the average linkage method of cluster analysis, as this
provides a good compromise between the extremes of other methods [5].
Using in-house software, we carried out further analysis of the card-sort data. This
software produced distance matrices, which quantified how often all the possible
pairs of cards were grouped together by each class of subjects. More specifically, a
2
Into the format generated by EZSort
4
similarity matrix was produced from the frequencies that two concepts appeared in
the same pile, which were then normalized by the number of sorters that put those
concepts together; e.g. if two cards were always put in the same pile, the matrix cell
would contain 1. The distance matrix is obtained by subtracting each cell value in the
similarity matrix from 1; e.g. if two cards were always put in the same pile their
distance value would be 03, or if they were put in the same pile by say 80% of the
subjects the distance value would be 0.2.
Based on the distance matrices (which form the basis of the dendrograms) further
graphical summaries were produced, and are described in section 3, while the
outcomes of interpreting these graphs are presented in section 4.
3 DEVELOPMENT OF A GRAPHICAL SUMMARY OF
RELATIVE INFORMATION IN DENDROGRAMS
In this section we will derive a graphical method for summarising and comparing the
card-sort data for the four staff groups. It is shown later in this section that the graphs
that are developed are analogous to cumulative frequency distribution plots. (This is
a prelude to the development of quantitative measure of the amount of “information”
displayed by different dendrograms under comparison, to be reported in a
forthcoming paper.)
As described in section 2.3, one of the products of the card-sorting experimental data
were distance matrices, which were symmetrical (about the major diagonal) and the
elements had values between zero and unity. Since distance matrices are
symmetrical, we converted them to triangular matrices thus:
D=
d11
d12
0
d 22
0
0
d1N
.
(1)
d NN
Since each of the elements of the distance matrix (dpq) have values within the range
zero to one, they can be considered as random variables with associated probability
values. However, since all of the distance values in the matrix do not sum to 1; i.e.
N
N
p =1 q =1
d pq ≠ 1,
(2)
the elements of the distance matrix as a whole do not represent a probability
distribution. To overcome this difficulty we developed an analogue to a cumulative
probability distribution by the following means.
3
The principal diagonal of the distance matrix contains distances between each card with itself. This
cannot occur in a card-sort where it would make no sense, and therefore the distance is set to 1
(implying irrelevance).
5
Each element dpq of the distance matrix D represents the joint probability P(xp,xq);
i.e. the (postriori) probability that the two cards (xp and xq) appear together in the
same pile4 in a card-sort. We developed a cumulative histogram of the number (fk) of
dpq within an increasingly larger “bucket” (of size ), thus:
fk =
i.e.
fk + 1
fk
d pq < τ
∀d pq ∈ D,
d pq ≥ τ
(3)
is a monotonically increasing threshold value such that 0.0 ≤ τ ≤ 1.0 , while
was increased in discrete steps of δ : δ < min
N
N
N
N
p =1 q =1 p ′=1 q ′=1
| d pq − d p′q′ |, (i.e. δ is
smaller than the smallest difference between any two distances in the matrix, D) and
fk is the number of dpq with a value < at the kth incremental increase in value of τ.
The fk were normalised to a range 0.0 ≤ fˆ ≤ 1.0 by dividing by
k
N!
N ( N − 1)
=
, the number of possible combinations of different pairs of
2( N − 2)!
2
cards from a stack of N cards. A stylised plot of fˆ is shown in Figure 2 (real plots
k
are shown in section 4.4). This graph is analogous to the plot of a cumulative
probability distribution (F), as the y-axis represents:
Fτ (d pq ) = P (d pq ≤ τ );
(4)
(i.e. the probability that the distance between any card pair is less than some distance
value d = τ , and the x-axis is the distance (d).
Figure 2 serves as a summary of a distance matrix, and in fact the dendrogram(s)
produced from it. The straight-line graph depicted in Figure 2 represents the plot
expected when the distance matrix is completely random and uniformly distributed.
The other curves in Figure 2 represent different degrees of order in two hypothetical
distance matrices.
4
Deemed similar in some sense by experimental subjects.
6
Figure 2 The graph above is a stylised plot of the link between how close or how distant pairs
of cards (actions or descriptors) are to one another and how often that particular distance
occurs (as a fraction of the total number of card-pairs). The shape of the graphs summarises the
degree and location of branching (structure) in the dendrogram. The straight line (random)
represents the case when any card-pair distance is equally likely (i.e. blindfolded sort).
In the next section, actual data derived from the card-sorting experiments for both
action and descriptor words is presented.
4 RESULTS
Firstly, the results of the cluster analysis are presented and secondly, the cumulative
probability graphs are displayed and discussed. These results are presented firstly for
actions and secondly for descriptors within each sub-section.
4.1 Development of Lexicons
When developing the “action” lexicon, senior nurses gave 134 separate action
expressions, junior nurses 99, senior doctors 75 and junior doctors 108–a total of 416
but by eradicating duplicates between groups this was reduced to 193 different
actions. Similarly when developing the “descriptor” lexicon, senior nurses gave 520
separate descriptor terms, junior nurses 258, senior doctors 461 and junior doctors
358–a total of 1796 which were reduced to 520 by eradicating duplicates. The
discarding of terms used by only a single individual and the amalgamation of the
synonyms reduced the total number of different actions to 51 and the total number of
different descriptors to 166.
The large set of descriptors was divided into 7 major groups (by senior medical and
nursing staff)—feeding, crying, sleep, movement-muscle tone, skin, size-weightshape, and bowel-urine. Shown in Table 1 are examples of the descriptor terms for
the “movement-muscle tone” subset and synonyms that were elicited at interview.
7
The results of the lexicon development were used to design the user-interface of a
data collection tool (called BabyWatch, [6]) used by a research nurse to collect
observational data on patients and clinical intervention. A page of the user-interface
for BabyWatch is shown in Figure 3 displaying the descriptors lexicon.
MOVEMENT/MUSCLE
TONE
FLOPPY
HYPERTONIC
JITTERY
ACTIVE
IRRITABLE
LETHARGIC
GOOD TONE
RESPONSIVE
WRIGGLY
UNRESPONSIVE
FITS
BACK ARCHING
SYNONYMS
hypotonic, poor tone, flaccid, limp, flat, poor movement,
weak movement
stiff, rigid, poor tone, tense, increased tone
jerky, twitchy, tremulous
vigorous
agitated, restless, jumpy
sleepy, drowsy, inert, inactive, not moving, lying still
normal tone, handles well, movement appropriate for
gestational age, normal movement, appropriate response,
handling well tolerated, symmetrical movement, relaxed,
normal posture
active on handling
squirming
non-reactive, not responding
convulsions, tonic, clonic, cycling movement
opisthotonic
Table 1 Column 1 shows the descriptor terms for baby movement/muscle tone labels on sort
cards. Column 2 shows synonyms that were yielded during lexicon elicitation interviews.
8
Figure 3 A part of the user-interface for the BabyWatch data collection tool.
4.2 Consistency of the Card-Sorting Results
As indicated in section 2.2 all the card-sorts were replicated for nurses in the
experiments.
The initial and replicated distance matrices were converted into vectors by
concatenating the rows of the distance matrices into one column per matrix of
distance values.
These vectors were statistically compared using the both Wilcoxon Signed Ranks
Test (as a nonparametric alternative to the t-test, since plots of the data indicated they
were not normally distributed).
The results of the statistical tests for the actions data show, with very high
confidence (p < 0.00001) for both junior and senior nurse data, that initial and
replicate data come from the same probability distribution; e.g. the results between
separate experiments for the same staff class are consistent. This is also reinforced
by correlation (Pearson) tests: junior nurses (r=0.91, p < 0.0001), senior nurses
(r=0.92, p < 0.0001).
9
4.3 Dendrograms
As described in section 2.3, cluster analysis was performed on the processed cardsorting data and dendrograms were produced for the various classes of staff. This has
yielded some interesting results, which are briefly discussed here.
4.3.1 Dendrograms—Actions
Shown in Figure 4 is a section of the dendrogram derived from the actions card-sorts
for junior nurses, while Figure 5 displays a section of the dendrogram derived from
the actions card-sorts for senior doctors. Both of these sections of dendrograms show
some common terms of interest from the actions lexicon, and are now briefly
discussed.
In appearance, the clustering of action cards appeared more variable than with
descriptor cards (see section 4.3.2). Senior doctors created 7 clusters out of 14 items,
junior doctors 6 clusters out of 13 items, senior nurses 7 clusters out of 21 items and
junior nurses 10 clusters out of 24 items.
The only cluster that was common to all groups and all participants were the actions
of giving incubator oxygen and giving nasal oxygen. The 2 nursing groups showed
commonality on 5 occasions in clustering items such as intravenous fluid
management, oxygen delivery, gavage feeding, comfort strategies and skin care. The
2 doctor groups showed similarity in 2 clusters; delivery of oxygen and skin care.
There is a difference in structure of the dendograms across the different staff groups,
and an extreme difference in structure between junior nurses and senior doctors. In
the case of the latter, there is evident a much richer structure with more groups (more
discerning) than is evident for the former, who formed large groupings with little
discernment. For example, junior nurses did not tend to place the “Biophysical
Observation” card with the “Examine Baby” card (whereas the other three staff
groups (SN, JD, SD) grouped these closely together) nor do they seem to be helped
much by the monitor-“Observe Baby” was not grouped with “Biophysical
Observation”.
It is clear from the dendrograms that the various groups of staff within the NICU
interpret and categorise data differently. The grouping of data appeared to be
associated with particular professional practice. For example the actions related to
artificial ventilation were, in general, grouped the same for senior and junior nurses;
senior doctors had a similar group but omitted the management of the ventilator
humidifier - this was grouped with issues of equipment safety. Like senior doctors,
junior doctors similarly clustered actions related to artificial ventilation but in their
minds, humidifier management and equipment safety were associated with routine
nursing care. These variations in representation may correspond to differences in
knowledge and or professional role and responsibilities. The dendrograms of the
junior nurses and of the senior doctors displayed the greatest difference of the 4 staff
groups.
10
Figure 4 Section of the “Actions” dendrogram for junior nurses.
Figure 5 Section of the “Actions” dendrogram for senior doctors.
4.3.2 Dendrograms—Descriptors
Displayed in Figure 6 is a section of the dendrogram derived from the descriptors
card-sorts for junior nurses, while Figure 7 displays a section of the dendrogram
derived from the descriptors card-sorts for junior doctors.
Senior doctors created 22 clusters out of 72 descriptors, junior doctors 28 clusters out
of 71 descriptors,, senior nurses 24 clusters out of 65 descriptors, and junior nurses
28 clusters out of 85 descriptors. Except for junior doctors, the other 3 groups
showed remarkable similarity in clustering data both within each group and also
between groups. On no occasion did all staff groups cluster the same descriptors,
together into a single entity. However, on 57 occasions, 2 or more descriptors were
clustered together by 2 or more staff groups. For example:
11
“skin perfusion”, “shutdown” and “poor capillary return” were always clustered
together by all staff, but in addition junior doctors included “mottled” and “poor
colour”, and junior nurses included “blue”, “mottled” and “poor colour”.
Other similarities were associated with describing stools, urine, sleep wake state,
levels of consciousness, and skin quality.
Figure 6 Section of the “Descriptors” dendrogram for junior nurses.
Figure 7 Section of the “Descriptors” dendrogram for junior doctors.
4.4 Dendrograms Summary Graphs
We produced graphs of the form shown in Figure 2 from the distance matrices
derived from the card-sorting experiments. These graphs provide a summary of the
structure of the dendrograms that are produced from cluster analysis of the card-sort
data. The closer each curve on the graph is to the straight diagonal, then the less
consistent are the responses across members of a staff group. If any of the groups
shown in the graph had data lying along the straight diagonal, this would suggest that
12
there was no agreement among members of the group as to how the cards should be
sorted, and they would do equally well if they all sorted the cards without looking at
them. Thus, the area under the curve is related to the level of agreement, in cardsorting, among members of a group. Also, the actual shape of a curve indicates the
level of branching of the associated dendrogram for various distance values; e.g. the
trajectory of a curve remaining shallow and then rapidly increasing after a distance,
d, would indicate that most branching (in the associated dendrogram) is occurring at
a distance greater than d.
4.4.1 Dendrograms Summary Graphs—Actions.
Consider Figure 8, which shows the graphs of the proportion of card-pairs with
distances greater than each particular value of the x-axis. Visual inspection of Figure
8 reveals that the junior nurses have the ‘highest’ line and are closest to the straight
diagonal, while the senior doctors have a line that is further away from the diagonal.
This means that the senior doctors are much more consistent in how they sort the
action cards than are the junior nurses. However, all groups are well below the
diagonal, showing some degree of agreement in the way they organise their
knowledge of the set of actions that they were asked to sort. This agrees with what is
seen on the actual dendrograms, and intuitively makes sense in light of the actual
roles of the various staff classes (see section 5 for further discussion).
Figure 8 The graph above shows the link between how close or how distant pairs of actions are
perceived to be to one another and how often that particular distance occurs.
13
4.4.2 Dendrogram Summary Graphs—Descriptors
The summary dendrogram graphs for all 4 staff categories is shown in Figure 9.
Perusal of Figure 9 indicates that the degree of structure in the dendrograms
(consistency in card-sort results within a group) increases for staff classes of junior
nurses, senior nurses and senior doctors is very similar, while showing that the
dendrogram for junior doctors is less structured than for the other staff categories.
This is quite different to the results for the actions graphs.
Figure 9 The graph above shows the link between how close or how distant pairs of descriptors
are perceived to be to one another and how often that particular distance occurs.
5 DISCUSSION
The purpose of carrying out the card sorts was to find out how different groups of
people conceptualise knowledge. We are able to show that staff composed of
different roles and experience conceptualises their knowledge in different ways.
Junior doctors lack of agreement with the other three groups when sorting the
‘descriptor’ cards may reflect a lack of knowledge. However many of the terms used
are specific to newborn care, and in particular to the unit in which the study was
conducted. It is more likely that the difference seen in the junior doctors’ clustering
is an indication of the short time that they had spent on the unit and therefore not
having the opportunity to acquire the vernacular. The action cards create a more
complex picture. There is less agreement between the professional groups and it may
be that the differences are a reflection of the nature of the actions themselves. On the
whole, the actions represent nursing activities. The way these are organised depend
not only on their relationships to one another but also to expertise, the theory or
14
model that the individual practitioner uses in cue acquisition and competing goals
when combining actions [7] [8] [9].
Knowing that staff conceptualise knowledge in different ways raises a number of
questions.
The decision making process has been classified by both the construction of
data and complexity of the task itself [10]. Therefore, how do the differences
seen in the current study in clustering information impact on decision making?
Where there is blurring of boundaries in professional role and function, what are
the consequences of the differences seen in the current study on decisionmaking processes and outcome?
In specialised areas where there is considerable data acquisition and influence
on reasoning and decision making, as in a neonatal intensive care unit, how may
the differences seen in the current study influence the development of computer
assisted decision tools?
Offredy [11] has shown that although processes in decision-making are different
between nurse practitioners and general practitioners, outcomes as measured in
accuracy of diagnosis, are similar. This situation is however different from that
within the neonatal unit as not all staff within the unit are functioning at the level of
expert and practitioners would not necessarily be confronted with multiple and
complex information that is readily available in the NICU. Computerised systems are
seen as one way in which information can be assimilated to assist decision-making.
It is argued, however, that utilisation of such a system is dependent upon the
interrelation of the system’s development, implementation and functioning with the
skilled and pragmatically oriented work of various health professionals [12].
6 SUMMARY AND CONCLUSION
We have described card-sorting experiments designed to elicit knowledge, from
nursing and medical staff in a Neonatal Intensive Care Unit, about how they
mentally map clinical concepts. These experiments produced data on how the
subjects group concepts based on some notion of similarity. Often, this card-sort data
is not processed further [4], but if processed further, the usual mode of analysis is
cluster analysis, which is applied to the data to produce dendrograms. These
dendrograms give useful insight into how people mentally organise concepts, but are
often complex and tedious to analyse, and are not easily amenable to inclusion in
computer algorithms.
We have shown here, the feasibility of using a graphical method to summarise the
information implicit to dendrograms (we hope to publish in this journal a follow on
paper describing a numerical summary method based on information theory.). The
graphical summary method allows direct visual comparison of different (but related)
dendrograms. This facilitates a quick analysis of the difference in structure of
dendrograms under consideration, and in our study facilitated the comparison of the
dendrograms for the various staff groups.
Our results show that different staff groups have different needs from a decision
support tool on a NICU. We have previously shown that the different staff groups
have differing abilities at interpreting trended physiological information [13]. This is
15
not surprising as nurses and doctors are educated very differently. Reliance for data
interpretation falls heavily on nursing staff that are constantly at the bedside in an
ICU, usually with a patient nurse ratio of near 1:1.
We would suggest that the development of decision support tools, particularly in an
IC environment requires an understanding of the cognitive background of the staff
that will use the support. It is crucial that the language and concepts are identical in
the group developing and using the tool. These systems are developed with the
knowledge of the designer whose knowledge and experience base will be different
from the front line staff that are to use them.
Acknowledgements
This work was undertaken as part of the NEONATE project; we are grateful to both
the UK ESRC and EPSRC for providing funding under the People at the Centre of
Communication and Information Technologies (PACCIT) program. We also
acknowledge the input of our co-workers on that project: Peter Badger and Alan
Maxwell.
References
1. Alberdi E, Gilhooley K, Hunter J, Logie R, Lyon AJ, McIntosh N et al.
Computerisation and decision making in neonatal intensive care: a cognitive
engineering investigation.. J Clin Monit Comput 2000;16:85-94.
2. Burton AM, Shadbolt NR, Rugg G , Hedgecock AP. The Efficacy Of
knowledge Acquisition Techniques: A Comparison Across Domains and
Levels Of Expertise. Knowledge Acquisition 1990; 2:167.
3. Johnson P. Human-Computer Interaction: Psychology, Task Analysis
Software Engineering; 1992.
4. Dong JM, Shirley M , Waldo P. A User Input and Analysis Tool for
Information Architecture 1999.
5. Aldenderfer MS , Blashfield RK. Cluster Analysis: Sage Publications Inc.;;
1990.
6. Hunter J BabyWatch User Manual, University of Aberdeen Technical Report
TR0206, 2002.
7. Field P. The Impact of Nursing Theory on the Clinical Decision Making
Process. Journal of Advanced Nursing 1987; 12:563.
8. Thompson C. A Conceptual Treadmill: The Need for 'middle Ground' in
Clinical Decision Making Theory in Nursing. Journal of Advanced Nursing
1999; 35:1222.
9. Greenwood J, Sullivan J, Spence K , McDonald M. Nursing Scripts and the
Organisazational Influences on Critical Thinking: A Report of a Study of
16
Neonatal Nurses' Critical Reasoning. Journal of Advanced Nursing 2000;
31:1106.
10. Hammond KR. Toward Increasing Competence of Thought in Public Policy
Formation: Westview;; 1978.
11. Offredy M. Decision-Making in Primary Care: Outcomes from a Study Using
Patient Scenarios. Journal of Advanced Nursing 2002; 40:532.
12. Berg M. Patient Care Information Systems and Health Care Work: A
Sociotechnical Approach. International Journal of Medical Informatics 1999;
55:87.
13. Alberdi E, Beche JC, Gilhooley KJ, Hunter J, Logie R , Lyon Aea. Decision
Support in the Neonatal Intensive Care Unit: Expertise Differences in the
Interpretation of Monitored Physiological Data. Engineering Psychology and
Cognitive Ergonomics 1999; 3:397.
17