International Journal for Quality in Health Care 2004; Volume 16, Number 5: pp. 407–416
10.1093/intqhc/mzh064
Review Article
A systematic review of computer-based
patient record systems and quality of
care: more randomized clinical trials
or a broader approach?
CYRILLE DELPIERRE1,2, LISE CUZIN1, JUDITH FILLAUX1, MURIEL ALVAREZ1, PATRICE MASSIP1
AND THIERRY LANG2
1
Centre Hospitalo Universitaire Purpan, Service des Maladies Infectieuses et Tropicales, Toulouse, 2INSERM, U558,
Épidémiologie et Santé Publique, Toulouse, France
Abstract
Purpose. To analyse the impact of computer-based patient record systems (CBPRS) on medical practice, quality of care, and
user and patient satisfaction.
Data sources. Manual and electronic search of the Medline, Cochrane, and Embase databases.
Study selection. Selected articles were published from 2000 to March 2003. CBPRS was defined as computer software
designed to be used by clinicians as a direct aid in clinical decision making. To be included, the systems should have recorded
patient characteristics and offered online advice, or information or reminders specific to clinicians during the consultation.
Data extraction. Keywords used for the search were: electronic record, informatic record, electronic medical record, electronic patient record, patient order entry, computer-based patient system, clinical decision support systems, and evaluation.
Results. Twenty-six articles were selected. Use of a CBPRS was perceived favourably by physicians, with studies of satisfaction
being mainly positive. A positive impact of CBPRS on preventive care was observed in all three studies where this criterion was
examined. The 12 studies evaluating the impact on medical practice and guidelines compliance showed that positive experiences were as frequent as experiences showing no benefit. None of the six studies analysing the impact of CBPRS on patient
outcomes reported any benefit.
Conclusions. CBPRS increased user and patient satisfaction, which might lead to significant improvements in medical care
practices. However, the studies on the impact of CBPRS on patient outcomes and quality of care were not conclusive. Alternative approaches considering social, cultural, and organizational factors may be needed to evaluate the usefulness of
CBPRS.
Keywords: computer-based patient record system, evaluation, medical practice, patient outcomes, quality of care, randomized
controlled trials, systematic review
The evolution of medicine during the three past decades is
characterized by a paradox. Medical information needed for
clinical decision making has increased; however, the organization and accessibility of health data are still poor, resulting in
inappropriate decisions and medical errors [1,2]. To increase
the accessibility and management of medical information, the
use of informatic tools has been promoted [3]. First used for
management and administrative purposes, computer-based
patient record systems (CBPRS) have been developed to collect and synthesize medical information [2].
Many experiences have been published regarding general
practice [4], surgery [5], cardiology [6], psychiatry [7], or HIV
infection [8,9] based on electronic medical records. These
systems have common characteristics [10,11]. Medical data are
conceptually organized as the patient’s paper medical record,
and synthesize clinical and therapeutic patient history. CBPRS
Address reprint requests to C. Delpierre, Centre Hospitalo Universitaire Purpan, Service des Maladies Infectieuses et Tropicales,
Toulouse, France. E-mail: [email protected] or [email protected]
International Journal for Quality in Health Care vol. 16 no. 5
© International Society for Quality in Health Care and Oxford University Press 2004; all rights reserved
407
C. Delpierre et al.
are designed to be used directly by physicians during consultation and to provide online information and messages to help
physicians in their practice.
The aim of CBPRS is to offer support in medical decision
making, to increase coordination between different health
care providers, and to promote the use of guidelines, thereby
improving global quality of care [12,13]. Development of
CBPRS in a care unit has even been proposed as a criterion of
quality [14,15]. The aim of CBPRS is also: to improve the
speed of retrieval of medical records, allowing many persons
to have simultaneous access to the same medical record; to
improve data confidentiality while tracing who has accessed
it; and finally to collect routine data [16–18].
During the past three decades, the introduction and development of CBPRS in the health sector has followed the
development of data processing [19]. However, few systems
have been evaluated with regard to their impact on clinical
performance and patient outcomes [2,20–22]. Results from
the few studies that evaluated CBPRS were not concordant.
A systematic review that analysed 28 randomized controlled
trials (RCTs) from 1983 to 1992 concluded that compliance
with guidelines increased; however, patient outcomes were
unchanged [20]. Another systematic review, which analysed
25 studies from 1992 to 1998, confirmed the positive effect of
these tools’ compliance with guidelines, but was inconclusive
on patient outcomes and quality of care due to lack of data
[13]. The last systematic review, which analysed studies until
1999, confirmed these results and concluded that new
approaches were needed to evaluate the impact of these tools
on medical practice and quality of care [23]. Several studies
since then have tried to fill this gap. Our objective was to
carry out a systematic review of studies analysing the impact
of CBPRS on medical practice, quality of care, and user and
patient satisfaction, using papers published since 2000.
Methods
Study selection
Studies published between January 2000 and March 2003 were
identified by searching the US National library of Medicine
Medline electronic bibliographic database, and the electronic
Cochrane and Embase databases. A manual search of studies
most frequently cited in automatically selected studies was also
performed. The keywords used for the search were electronic
record, informatic record, electronic medical record, electronic
patient record, patient order entry, computer-based patient
system, clinical decision support systems, and evaluation.
To be selected, studies had to pertain to an evaluation of
the impact of CBPRS on medical practice and/or quality of
care and/or user satisfaction and/or patient satisfaction.
CBPRS was defined as follows: ‘computer software designed
to be used by a clinician involved in patient care as a direct aid
to clinical decision making. Patient data were recorded into the
system. Patient-specific information in the form of assessments
or recommendations, or alerts or reminders was presented to
the clinician during the consultation’.
408
Descriptive studies and analyses concerning the quality of
CBPRS, such as accuracy or completeness of data, were not
included. No criteria with respect to trial methodology were
defined.
Results
Thirty-nine articles were identified at the end of this process.
After careful analysis, 13 of them were excluded: eight did not
evaluate the impact of CBPRS on clinician performance and
five did not meet the criteria for the definition of a CBPRS.
Eventually, 26 articles were selected, corresponding to 25 studies.
One of them was published in two papers [24,25]. Selected
studies are presented in Tables 1–4.
Study designs
Several study designs were used: 10 were RCTs [24–32], 11
were before–after studies [33–43], three were cross-sectional
studies [44–46], and two were based on qualitative interviews
[10,47].
Criteria of judgment
Sixteen studies attempted to assess the process of care. Five
studies used patient outcomes as a criterion of judgment. Fifteen
studies used satisfaction among users and/or patients as the
endpoint.
Process of care
Length of the consultation. Six studies analysed the impact of
CBPRS on the length of the consultation [31,33,37,39,41,45].
An increase in this length was observed in three studies
[31,33,45]. This increase was found to range between 2.2 and
9.3 minutes per patient [31,33]. Makoul et al. [45] found discordant results according to the variety of consultation. Initial
visits had a mean length of 35 minutes when using CBPRS
compared with 25 minutes without (P < 0.05). No difference
was observed during subsequent visits. The length of the
consultation was found to be reduced in one study [41] and
unchanged in another study [37].
Content of the consultation. Three studies analysed the impact of
CBPRS on the content of the consultation [31,33,45]. Some
variables, such as questions regarding mental health or behaviour such as tobacco consumption were more frequently
recorded as items of the CBPRS than when these tools were
not used [33,45]. Clinicians were significantly more likely to
address routine health care maintenance topics, including psychosocial issues [relative risk (RR) = 1.4] and smoking at home
(RR = 15.6) in a paediatric primary care practice using CBPRS
than in one using paper records [33].
Clinical performance
Twelve studies analysed modification of medical practice [25–
30,33–36,40,48]. Three studies analysed the impact of CBPRS
CDSS
CDSS
CPOE
EMR
EMR
In-house
developed
Commercial
In-house
developed
In-house
developed
Commercial
Commercial
Eccles et al.
(2002) [27]
Hetlevik et al.
(2000) [28]
McKinley et al.
(2001) [29]
Montgomery et al. In-house
(2000) [30]
developed
In-house
developed
Dexter et al.
(2001) [26]
Overhage et al.
(2001) [31]
Rollman et al.
(2001) [24]
Rollman et al.
(2002) [25]
Domain
Judgment criteria
Results
Commercial
CDSS
Preventive care
for hospitalized
patients
Rates of use of four preventive
therapies: pneumococcal
vaccination; influenza
vaccination; prophylactic
heparin; and prophylactic
aspirin at discharge
Compliance with the guidelines
Significant increase in
the rate of delivery of
such therapies (P < 0.001)
2 years: 1 year before Management of
No change in the
and 1 year after the asthma and angina
management of
intervention
asthma and angina
1034 patients, 18 months
Treatment of
Fractions of patients without
No significant change
53 physicians
diabetes mellitus
registration; mean group
in doctors’ behaviour
in general practice
differences for the same variables or in patient outcome
67 patients
63 months
Trauma intensive
Physician acceptance; survival;
Positive effects on FiO2
care unit
length of ICU stay; morbidity;
and P plateau; survival,
barotrauma
ICU length of stay,
morbidity, barotrauma
not significantly different
No benefit to absolute
Percentage of patients with a
2 years
Management of
614 patients,
risk reduction or blood
5-year cardiovascular risk ≥10%;
hypertension in
74 general
pressure control
systolic blood pressure; diastolic
primary care
practitioners,
blood pressure; prescribing
11 nurses
of cardiovascular drugs
Physician time utilization;
Little extra time (2.2
744 patients,
3 different days
Physician time
physician opinion
minutes per patient);
34 physicians
over a 16-months
utilization in
patient care improved
period
ambulatory primary
care internal
medicine practice
216 patients,
20 months
Management of
Response to alert; agreement
Change in management
16 physicians
major depression
with diagnosis; treatment
of major depression
200 primary
Two phases:
Management of
Clinical outcomes and care
Little differential impact
care patients,
3 and 6 months
major depression
processes for patients with
on clinical outcomes or
17 physicians
major depression
on process measures
60 general
11 months
Primary care
Average number of blood tests
20% fewer tests on average
practitioners
ordered per order form per
practice
2276 patients
6371 patients, 18 months
202 physicians
CPOE, computer patient order entry; CDSS, computer decision support system; EMR, electronic medical record; ICU, intensive care unit.
Van Wijk et al.
(2001) [48]
CDSS
CDSS
CPOE
...........................................................................................................................................................................................................................................................................................................
Nature of the Name of Participants (n) Length of trial
tested system system
Authors
Table 1 Randomized controlled trials studies
Computer-based patient record systems
409
410
EMR
EMR
Commercial
In-house
developed
In-house
developed
In-house
developed
Commercial
Commercial
Adams et al.
(2003) [33]
Bansal et al.
(2001) [34]
Bouaud et al.
(2001) [35]
Durieux et al.
(2000) [36]
Gadd et al.
(2000) [37]
Gadd et al.
(2001) [38]
Mikulich et al. In-house
(2001) [39]
developed
Domain
Judgment criteria
Results
EMR
CDSS
CDSS
CPOE
EMR
2 months
pre-intervention
period, 6 months
post-intervention
period
Three 10 week
intervention
periods, four
10 week control
periods, 4 week
washout
28 physicians Physicians’ and
patients’
interview before
implementation,
after 6 months,
and at 1 year
170 physicians Physicians’
interview before
implementation
and after 6 months
142 physicians, 3 years
789 patients
1971 patients
Not
12 weeks: 5 predocumented
intervention
and 7 postintervention
13 physicians, 4 months
127 patients
1221 patients
Physician attitudes
Physical medicine
and rehabilitation
continued
Increase in compliance with
guidelines. Considerable
heterogeneity in belief
among complaints
Physician concerns regarding
implementation of an
EMR in an outpatient
clinic; patient satisfaction
Physical medicine
and rehabilitation
Physician attitudes; family
medicine; patient
complaints
Significant decrease in
overall optimism with
regard to EMR
Proportion of appropriate
prescriptions ordered for
anticoagulation
Prevention of venous
thromboembolism
in orthopedic
surgery department
Management of
chronic back pain
Significant increase in physician
compliance with guidelines
(85% versus 61.4%, P < 10−4)
Significant increase in
guidelines compliance
with CDSS (94.9% versus
82.8%, P < 10−4);
73% reduction in risk of
inappropriate prescription
Significant negative effects on:
patient privacy; overall quality
of health care; physician
autonomy
Treatment decision; physician
prescribing behaviour
Health history; risk assessment; Benefits of EMR on
clinical practice;
physical examination;
high satisfaction
anticipatory guidance;
immunizations; hearing;
vision; lead; anaemia;
tuberculosis testing;
users’ satisfaction
Number of arterial blood gas
No significant change
Management of
breast cancer
Intensive care unit
Paediatric primary
care
...........................................................................................................................................................................................................................................................................................................
Nature of the Name of Participants (n) Length of trial
tested system system
Authors
Table 2 Before–after studies
C. Delpierre et al.
CDSS
CDSS
Commercial
In-house
developed
Penrod et al.
(2001) [46]
Rocha et al.
(2001) [40]
Rousseau et al. Commercial
(2003) [32]
Sanders et al. In-house
(2001) [42]
developed
Domain
Judgment criteria
Results
In-house
developed
CPOE
6 months
Community- and
academic-based
primary care
47 physicians, 2 years
6 nurses
Not
9-week control
documented
period, 8-week
intervention
period
Not
4-week study
documented
periods before
and after
interventions
Hospital medical
practice
Management of
asthma and angina
Neuroradiology
imaging
Not
Evaluation before
Management of
documented
implementation
nosocomial
and after 6 months
infections
34 physicians
EMR, electronic medical record; CPOE, computer patient order entry; CDSS, computer decision support system.
Teich et al.
(2001) [43]
CPOE
EMR
Prescribing practices;
compliance with formulary
and prophylactic heparin
use guidelines
Clinician ordering patterns
User satisfaction
Attitudes of academic-based
and community-based
primary care physicians;
factors influencing their
attitudes
Physician satisfaction;
number of suggestions
to treat
No significant change in
treatment strategies;
significant increase in
physicians’ satisfaction
Negative perceptions
by physicians
Change in the distribution
of tests ordered; significant
increase in compliance
with guidelines
Increased frequency of use
of H2 blocker; increased
rate of ordering prophylactic
heparin; decreased rates of
excessive dosing; increased
appropriateness for frequency
of ondansetron use
Community-based physicians
more satisfied by EMR
...........................................................................................................................................................................................................................................................................................................
Nature of the Name of Participants (n) Length of trial
tested system system
Authors
Table 2 continued
Computer-based patient record systems
411
C. Delpierre et al.
Table 3 Cross-sectional studies
Authors
Nature of the Name of
tested system system
Participants (n) Domain
Judgment criteria
Results
Laerum et al.
(2001) [44]
Commercial
EMR
Makoul et al.
(2001) [45]
Commercial
EMR
Moderate
227 physicians General clinical Proportion of
utilization;
respondents using
tasks in
significant
the electronic
Norwegian
increase
system; physician
hospitals
in satisfaction
satisfaction
score
Physician–patient
More active role
204 patients,
Internal
communication
in clarifying
6 physicians
medicine
information;
faculty
difficulty
practice
focusing attention
on other aspects
of patient
communication
36 physicians Internal
Usability; learnability; Significant
medicine
efficiency; physician increase in
physician
satisfaction
satisfaction
........................................................................................................................................................................................................................
Rodriguez et al. Unknown
(2002) [41]
Graphicalbased
electronic
patient
record
EMR, electronic medical record.
Table 4 Qualitative studies
Authors
Nature of the
tested system
Name of
system
Participants (n)
Lee et al.
(2002) [47]
Unknown
12 nurses
Matsumura et al.
(2001) [10]
In-house
developed
Computerized
nursing care
plan system
EMR
Domain
Judgment
criteria
Results
Nursing
satisfaction
Ambivalent
feelings
Physician
satisfaction
Usefulness
for patient
history
........................................................................................................................................................................................................................
20 in-patients
Respiratory
intensive
care units
Hospital
medical
practice
EMR, electronic medical record.
on preventive care [26,28,33], and an improvement was
observed in all of them. Dexter et al. [26] analysed pneumococcal and influenza vaccination, and prophylactic heparin
and aspirin use. An increase was observed in vaccination rates
from 0.8% to 35.8% for pneumococcal vaccination, and from
1% to 51.4% for influenza vaccination. Likewise, prophylactic
heparin and aspirin prescription at discharge increased from
18.9% to 32.2%, and from 27.6% to 36.4%, respectively.
Eight studies analysed compliance with guidelines [25,
27–29,35,36,48]. An increase in the application of guidelines
was observed in three studies of the management of breast
cancer, venous thromboembolism, and blood test ordering
[35,36, 48]. This increase was 14% (85% versus 62%, respectively; P < 10−4) with respect to breast cancer guidelines and
12% (95% versus 83%, respectively; P < 0.001) with respect
to venous thromboembolism guidelines [35,36]. In a study of
412
blood test ordering, general practitioners who used CBPRS
requested 20% fewer blood tests [48]. Results regarding the
management of acute respiratory distress syndrome showed
that 95% of computer-generated instructions were followed
[29]. However, compliance with guidelines on the management
of asthma, angina, diabetes mellitus, and major depression was
not found to increase [25,27,28].
Nine studies analysed the impact of CBPRS on treatment
prescriptions [25,27,30,34–36,40,43,48]. Prescription rates concordant with recommendations increased by 31% for chemotherapy prescriptions and by 61% for prescriptions for venous
thromboembolism. Accordingly, the error rate decreased
[35,36]. Improvements in appropriateness for prescription of
drugs with CBPRS was reported in one study [43]. No significant evolution could be observed in the remaining five studies
[25,27,30,34,40].
Computer-based patient record systems
Patient outcomes
Five studies used patient outcomes as a criterion of judgment
[25,27–30]. The results were not improved for cardiovascular
risk [28,30], survival, length of hospitalization, or acute respiratory distress syndrome [29]. However a significant decrease
of 2.3 mmHg diastolic pressure was observed in one study
[28]. In two studies, outcomes were assessed by the patients.
No benefit with respect to the management of asthma and
angina, or major depression was observed [25,27].
User and patient satisfaction
Physician satisfaction. Thirteen studies analysed physician perception for CBPRS use [10,28,31–33,37–41,44–46]. Physician
satisfaction with the system was reported in nine studies
[10,31,33,39–41,44–46]. The positive points reported were
improved knowledge of patients’ medical history, better
medical examination, and improvement in quality of care. In
four studies, physicians were asked if they wanted to continue
working with CBPRS: all studies reported positive responses
[31,39,40,46]. Nevertheless, four studies raised some points of
dissatisfaction [28,32,37,38]. CBPRS was perceived as a physical barrier [33,37,38,45] that could have a negative impact on
the patient–physician relationship [37,38,46], particularly by
reducing eye-to-eye contact [33]. Concerns were expressed
about data confidentiality [37–39], personal and professional
privacy, bug management [32,38,44], and the additional work
for physicians [28,31,32]. Factors influencing perception of the
system were its characteristics, such as its interface and ease of
use, as well as users’ characteristics. Perception of CBPRS was
better for community-based physicians than for academicbased physicians. Physicians accustomed to working with a
computer had a better perception of CBPRS [46].
Nurse satisfaction. Three studies analysed nurse satisfaction
with the introduction of CBPRS in their practice [32,40,47]. A
global increase in satisfaction was observed in two studies
[32,40]. In another study, reduced administrative work and
increased accessibility to care protocols, especially for young
nurses, were reported. Negative points were the lack of flexibility of these tools, the loss of nurses’ judgment, and the
additional workload [47]. Nevertheless, nurses stated their
desire to continue working with an informatic system in all
three studies.
Patient satisfaction. Two studies analysed patient satisfaction
[37,49], and patients were found to have a positive opinion of
CBPRS in both. A mean score of 4.6 on a general satisfaction
scale of 0–5 was reported [37]. Patients did not report a
reduction in eye-to-eye contact with the use of CBPRS, and
found medical visits more effective. However, fear concerning data confidentiality was observed in both studies.
Discussion
Use of a CBPRS seems to be perceived favorably by physicians, with user satisfaction being mainly positive despite
some limits and concerns regarding its use [31,39,40,46]. The
main constraints reported were the physical barrier and the
impact on the patient–physician relationship that could result
from the use of CBPRS during the consultation. Technical
characteristics of the system and knowledge of data processing by physicians were criteria related to satisfaction with, and
thus implementation of, CBPRS [50]. Future research is needed
considering qualitative factors that could influence the use of
CBPRS [7,51].
Physicians wanted to continue working with CBPRS once
such tools were in place in their wards [31,39,40,46], as once
exposed, physicians felt a need for an information system. To
improve the quality of CBPRS it is essential that the quality of
data collected is certified and controlled [52]. Furthermore, to
prepare for technical problems with the computer it is advisable always to keep a paper print-out of patient records [53].
Only two studies analysed patient satisfaction. Generally,
patients were satisfied and did not complain about changes to
their relationship with their physicians [37]. Their greatest fear
concerned data confidentiality. It is therefore essential to
reassure patients about the confidentiality of their data. Physicians must be able to explain how data confidentiality can be
maintained with the system they use. Additional patientfocused studies are thus necessary to explore their feelings
about the use of CBPRS in consultation rooms and its consequences with respect to the patient–physician relationship.
The impact of CBPRS on medical practice was more balanced. A clear positive impact of CBPRS on preventive care
was noted. This finding is consistent with other systematic
reviews [13,20,54]. Improvements in medical practice and the
adoption of guidelines was less certain. Positive experiences
were as frequent as experiences showing no benefit. In studies of arterial hypertension and major depression [25,30,55–
58], there was no improvement in medical practice and
compliance with guidelines. The recent systematic review of
Kaushal et al. showed that CBPRS could decrease prescription
error, although most studies were inconclusive [59]. Possible
reasons for these inconclusive findings were underuse of the
tool, inadequate study design, and lack of study power [28].
Only six studies analysed the impact of the use of CBPRS
on patient outcomes and did not show any benefit of CBPRS.
In a systematic review [13], 25 RCTs evaluating the contribution of CBPRS to medical practice were analysed. Only eight
of these studies used patient outcomes as a criterion of judgment. The relationship between the process of care and health
outcomes might explain why improved outcomes are difficult
to relate to the implementation of CBPRS [60].
Users of CBPRS acknowledge the usefulness of such tools,
whereas the results of clinical performance or patient outcomes are not always conclusive. This paradox draws the
attention to the criteria and methodologies used in various
studies. RCTs are usually considered the methodological reference [61]. However, this methodological approach has
several limits in evaluating the impact of CBPRS on medical
practice. In order to organize RCTs and select a criterion of
judgment, tracers have to be chosen within complex and
multifactorial diseases (e.g. hypertension, depression). Moreover, even if positive results are found in some narrow clinical
fields, generalization to the more complex clinical situations is
413
C. Delpierre et al.
generally difficult [62]. Controlled conditions required by
RCTs do not reflect the diversity of medical practices and
working conditions. Due to these limitations, RCTs do not
offer a sound enough methodology to evaluate the contribution of CBPRS in medical practice, and its impact on clinical
performance and patient outcomes [61,63,64].
Two systematic reviews [23,65] describing methodological
approaches to the evaluation of systems designed to improve
medical decision making emphasized the lack of diversity in the
design of studies that evaluate these tools. This phenomenon
could lead to lack of conclusive results. Alternatives approaches
centered on social, cultural, and organizational factors are thus
advocated. Structure of care (general practice, hospital), organization of the ward, and the relationships between various
health professionals seem to be key factors to consider when
evaluating the impact of CBPRS on medical practice [65].
It is recognized that systematic review may be subject to a
possible selection bias. Studies presenting negative findings
are indeed published less often [13]. Nevertheless, a high proportion of articles showing no effect of CBPRS on quality of
care were found [25,27,28]. A wide range of keywords to find
systems with the same characteristics had to be used. A well
defined operating definition of CBPRS, sharing the same
characteristics, could facilitate future research and produce
more exhaustivity [66].
Conclusion
This systematic review, based on 25 papers published between
2000 and 2003, has contributed to a better understanding of
the relationship between the use of CBPRS and medical practice. Increased satisfaction of users and patients was noted,
which could lead to significant changes in medical practice.
However, as in previous reviews, the impact of CBPRS on
medical practice and quality of care was not well demonstrated. It is noteworthy that most of the studies did not
include qualitative factors such as characteristics of the disease and the tool, the ward in which it is developed, and the
relationship between various health care professionals, which
can impact upon the use of CBPRS.
A broad review including all the factors that could influence the success or failure of the use of CBPRS in medical
practice is indicated in the future.
References
5. Castelden W, Lawrence-Brown M, Lam H, McLoughlin B,
Thompson D, Lopez J. The hollywood surgical-audit programme:
a computer-based discharge and data-collection system for surgical audit. Med J Aust 1988; 149: 70–74.
6. Rossi R, Every N. A computerised intervention to decrease the
use of calcium channel blockers in hypertension. J Gen Intern Med
1997; 12: 672–678.
7. Lee F, Teich J, Spurr C, Bates D. Implementation of physician
order entry: user satisfaction and self-reported usage patterns.
J Am Med Inform Assoc 1996; 3: 42–55.
8. Pugliese P, Cuzin L, Enel P et al. Nadis 2000, développement
d’un dossier médical informatisé pour les patients infectés par
VIH, VHB et VHC. Presse Med 2003; 32: 299–303.
9. Safran C, Rind D, Davis R et al. Guidelines for management of
HIV infection with computer-based patient’s record. Lancet
1995; 346: 341–346.
10. Matsumura Y, Kuwata S, Kusuoka H et al. Dynamic viewer of
medical events in electronic medical record. Medinfo 2001; 10:
648–652.
11. Sujansky W. The benefits and challenges of an electronic medical record: much more than a ‘word-processed’ patient chart.
West J Med 1998; 169: 176–183.
12. Pringle M. Using computers to take patient histories. Br Med
J 1988; 297: 697–698.
13. Shiffman R, Liaw Y, Brandt C, Corb G. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc 1999; 6: 104–114.
14. Safran C. Electronic medical record: a decade of experience.
J Am Med Assoc 2001; 285: 1766.
15. Tsai C, Starren J. Patient participation in electronic medical
records. J Am Med Assoc 2001; 285: 1765.
16. Barrows R, Clayton P. Privacy, confidentiality, and electronic
medical records. J Am Med Inform Assoc 1996; 3: 139–148.
17. Gillies J, Holt A. Anxious about electronic health records? No
need to be. NZ Med J 2003; 116: U604.
18. Rind D, Kohane I, Szolovits P, Safran C, Chueh H, Barnett G.
Maintaining the confidentiality of medical records shared over
the Internet and the World Wide Web. Ann Intern Med 1997; 127:
138–141.
19. Benson T. Why general practitioners use computers and hospital
doctors do not. Br Med J 2002; 325: 1090–1093.
20. Johnston M, Langton K, Haynes R, Mathieu A. Effects of
computer-based clinical decision support systems on clinician
performance and patient outcome: a critical appraisal of research.
Ann Intern Med 1994; 120: 135–142.
1. TRW uses AI tools to control system problems. Aviat Week Space
Technol 1986: 79–81.
21. Wyatt J, Spiegelhalter D. Evaluating medical expert systems:
what to test and how? Med Inform 1990; 15: 205–217.
2. Tierney W. Improving clinical decisions and outcomes with
information: a review. Int J Med Informat 2001; 62: 1–9.
22. Sim I, Gorman P, Greenes R et al. Clinical decision support
systems for the practice of evidence-based medicine. J Am Med
Assoc 2001; 8: 527–534.
3. Bleich H, Beckley R, Horowitz G et al. Clinical computing in a
teaching hospital. N Engl J Med 1985; 312: 756–764.
4. Walton R, Gierl C, Yudkin P et al. Evaluation of computer support for prescribing (CAPSULE) using simulated cases. Br Med J
1997; 315: 791–795.
414
23. Kaplan B. Evaluating informatics applications—clinical decision
support systems literature review. Int J Med Inform 2001; 64: 15–37.
24. Rollman B, Hanusa B, Gilbert T, Lowe H, Kapoor W, Schulberg H.
The electronic medical record. A randomized trial of its impact
Computer-based patient record systems
on primary care physicians’ initial management of major depression. Arch Intern Med 2001; 161: 189–197.
25. Rollman B, Hanusa B, Lowe H, Gilbert T, Kapoor W, Schulberg H.
A randomized trial using computerized decision support to
improve treatment of major depression in primary care. J Gen
Intern Med 2002; 17: 493–503.
26. Dexter P, Perkins S, Overhage J, Maharry K, Kohler R,
McDonald C. A computerized reminder system to increase the
use of preventive care for hospitalized patients. N Engl J Med
2001; 345: 965–970.
27. Eccles M, McColl E, Steen N et al. Effect of computerised
evidence based guidelines on management of asthma and angina
in adults in primary care: cluster randomised controlled trial.
Br Med J 2002; 325: 941.
28. Hetlevik I, Holmen J, Kruger O, Kristensen P, Iversen H,
Furuseth K. Implementing clinical guidelines in the treatment of
diabetes mellitus in general practice. Evaluation of effort,
process, and patient outcome related to implementation of a
computerized based decision support system. Int J Technol Assess
Health Care 2000; 16: 210–227.
29. McKinley B, Moore F, Sailors R et al. Computerized decision
support for mechanical ventilation of trauma induced ARDS:
results of a randomized clinical trial. J Trauma 2001; 50:
415–424.
30. Montgomery A, Fahey T, Peters T, MacIntosh C, Sharp D.
Evaluation of computer based clinical decision support system
and risk chart for management of hypertension in primary care:
randomised controlled trial. Br Med J 2000; 320: 686–690.
31. Overhage J, Perkins S, Tierney W, McDonald C. Contolled trial
of direct physician order entry: effects on physicians’ time utilization in ambulatory primary care internal medicine practices.
J Am Med Inform Assoc 2001; 8: 361–371.
32. Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M.
Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. Br Med J
2003; 326: 314–322.
33. Adams W, Mann A, Bauchner H. Use of an electronical medical
record improves quality of urban pediatric primary care. Pediatrics
2003; 111: 626–632.
34. Bansal P, Aronsky D, Talbert D, Miller R. A computer based
intervention on the appropriate use of arterial blood gas. Proc
AMIA Symp 2001: 32–36.
35. Bouaud J, Seroussi B, Antoine E, Zelek L, Spielmann M. A
before after study using OncoDoc, a guideline based decision
support system on breast cancer management: impact upon
physician prescribing behaviour. Medinfo 2001; 10: 420–424.
36. Durieux P, Nizard R, Ravaud P, Mounier N, Lepage E. A clinical decision support system for prevention of venous thromboembolism: effect on physician behaviour. J Am Med Assoc
2000; 283: 2816–2821.
39. Mikulich V, Yi-Ching A, Steinfeldt J, Schriger D. Implementation of clinical guidelines through an electronic medical record:
physician usage, satisfaction and assessment. Int J Med Informat
2001; 63: 169–178.
40. Rocha B, Christenson J, Evans R, Gardner R. Clinicians’
response to computerized detection of infections. J Am Med
Inform Assoc 2001; 8: 117–125.
41. Rodriguez N, Murillo V, Borges J, Ortiz J, Sands D. A usability
study of physicians interaction with a paper-based patient record
system and a graphical-based electronic patient record system.
Proc AMIA Symp 2002: 667–671.
42. Sanders D, Miller R. The effects on clinician ordering patterns
of a computerized decision support system for neuroradiology
imaging studies. Proc AMIA Symp 2001: 583–587.
43. Teich J, Merchia P, Schmiz J, Kuperman G, Spurr C, Bates D.
Effects of computerized physician order entry on prescribing
practices. Arch Intern Med 2000; 160: 2741–2747.
44. Laerum H, Ellingsen G, Faxvaag A. Doctor’s use of electronic
medical records systems in hospitals: cross sectional survey.
Br Med J 2001; 323: 1344–1348.
45. Makoul G, Curry R, Tang P. The use of electronic medical
records: communication patterns in outpatient encounters. J Am
Med Inform Assoc 2001; 8: 610–615.
46. Penrod L, Gadd C. Attitudes of academic-based and communitybased physicians regarding EMR use during outpatient encounters.
Proc AMIA Symp 2001: 528–532.
47. Lee T, Yeh C, Ho L. Application of a computerized nursing care
plan system in one hospital: experience of ICU nurses in
Taiwan. J Adv Nurs 2002; 39: 61–67.
48. Van Wijk M, Van der Lei J, Mosseveld M, Bohnen A, Van Bemmel J.
Assessment of decision support for blood test ordering in primary
care: a randomized trial. Ann Intern Med 2001; 134: 274–281.
49. Goldberg H, Morales A, Gottlieb L, Meador L, Safran C.
Reinventing patient-centered computing for the twenty-first
century. Medinfo 2001; 10: 1455–1458.
50. Murff H, Kannry J. Physician satisfaction with two order entry
systems. J Am Med Inform Assoc 2001; 8: 499–509.
51. Loomis G, Ries J, Saywell RJ, Thakker N. If electronical medical
records are so great, why aren’t family physicians using them?
J Fam Pract 2002; 51: 636–641.
52. Thiru K, Hassey A, Sullivan F. Systematic review of scope and
quality of electronic patient record data in primary care. Br Med
J 2003; 326: 1070.
53. Kilbridge P. Computer crash—lessons from a system failure.
N Engl J Med 2003; 348: 881–882.
54. Sullivan F, Mitchell E. Has general practitioner computing made
a difference to patient care? A systematic review of published
reports. Br Med J 1995; 311: 848–852.
37. Gadd C, Penrod L. Dichotomy between physicians’ and patients’
attitudes regarding EMR use during outpatient encounters. Proc
AMIA Symp 2001: 275–279.
55. Hunt D, Haynes S, Hanna S, Smith K. Effects of a computerbased clinical decision support systems on physician performance and patients outcomes: a systematic review. J Am Med Assoc
1998; 280: 1339–1346.
38. Gadd C, Penrod L. Assessing physician attitudes regarding use
of an outpatient EMR: a longitudinal, multi-practice study. Proc
AMIA Symp 2001: 194–198.
56. Magruder-Habib K, Zung W, Feussner J. Improving physicians’
recognition and treatment of depression in general medical care.
Results from a randomized clinical trial. Med Care 1990; 28: 239–250.
415
C. Delpierre et al.
57. Montgomery A, Fahey T. A systematic review of the use of
computers in the management of hypertension. J Epidemiol Commun Health 1998; 52: 520–525.
58. Shapiro S, German P, Skinner E. An experiment to change
detection and management of mental morbidity in primary care.
Med Care 1987; 25: 327–339.
59. Kaushal R, Shojania K, Bates D. Effects of computerized physician order entry and clinical decision support systems on medication safety. Arch Intern Med 2003; 163: 1409–1416.
60. Mant J, Hicks N. Detecting differences in quality of care: the
sensitivity of measures of process and outcome in treating acute
myocardial infarction. Br Med J 1995; 311: 793–796.
outpatient setting: implementation barriers to outcome evaluation. J Am Med Inform Assoc 1996; 3: 340–348.
63. Forsythe D, Buchanan B. Broadening our approach to evaluating medical information systems. Proc Annu Symp Comput Appl
Med Care 1991: 8–12.
64. Mongerson P. Patient’s perspective of medical informatics. J Am
Med Inform Assoc 1995; 2: 79–84.
65. Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for
methodological pluralism. Int J Med Inform 2000; 64: 39–56.
66. Hogan W, Wagner M. Accuracy of data in computer-based
patient records. J Am Med Inform Assoc 1997; 4: 342–355.
61. Heathfield H, Pitty D, Hanka R. Evaluating information technology in health care: barrriers and challenges. Br Med J 1998;
316: 1959–1961.
62. Rotman B, Sullivan A, McDonald T et al. A randomized controlled trial of a computer-based physician workstation in an
416
Accepted for publication 25 May 2004