CLINICAL INFORMATION SYSTEMS:
ACHIEVING THE VISION
February 2002
By Brian Raymond and Cynthia Dold
About the Kaiser Permanente Institute for Health Policy
Mission Statement
To advance understanding of key health policy issues and to advocate, in concert with others as
appropriate, health policy that will improve health and the manner in which health care and
financing systems serve Americans.
Goals
The Institute's Goals are to:
• Identify significant long-term health policy issues;
• Organize internal and external resources to analyze such policies;
• Improve understanding and recommend actions; and
• Build coalitions to shape and influence policy.
Emphasis is placed on developing political alternatives and exploring their implications, building on
the experience of the largest privately organized health care delivery system in the United States.
Kaiser Permanente Institute for Health Policy
One Kaiser Plaza
Oakland, CA 94612
Additional copies of this document are available in the Kaiser Permanente
Institute for Health Policy web site at www.kp.org/ihp
NOTICE
This document builds on the concepts prepared for and discussed at a meetings held in October 2001
in which many stakeholders expressed their own views and those of their organizations. The Kaiser
Permanente Institute for Health Policy has studied the issues discussed at the meeting and has reviewed
the relevant literature, and based on these considerations has developed the policy recommendations
set forth in this paper. This paper is not meant to imply that there was consensus on any given issue.
In fact, there was a wide range of opinions. The following report represents the views of the Kaiser
Permanente Institute for Health Policy and does not necessarily present the views of individual
participants nor the organizations that they represent.
The Authors wish to thank Robert M. Crane, director, Kaiser Permanente Institute for Health Policy,
Andy Wiesenthal, MD, Associate Executive Director, The Permanente Federation, and Dean Sittig,
PhD, Senior Informatics Researcher, Kaiser Permanente for their guidence and input in the preparation
of this paper.
Clinical Information Systems: Achieving the Vision
I.
Executive Summary
1
II.
Introduction
1
III.
CIS: A Bridge to a New Health Care Paradigm
2
IV.
What is CIS?
5
V.
Evidence of Clinical Information System Value
7
VI.
Roadblocks
13
VII.
Need for a National Health Care Information Infrastructure
16
VIII. Policy Recommendations
17
IX.
20
Conclusion
Appendix
Notes
Meeting Participant List
21
23
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
I.
return on these investments. A select few larger,
integrated delivery systems in the U.S. have uniquely
positioned themselves to take advantage of CIS and
have demonstrated the value of these systems in care
delivery.
Executive Summary
Clinical information systems (CIS) and other
information management tools hold the promise of
improving the quality and safety of patient care and
increasing the efficiency of health care personnel.
However, the health care industry lags behind other
sectors in information technology investment and,
with few exceptions, has not fully benefited from the
information revolution.
Progress toward broader use of clinical information
technology in health care will not occur in the
absence of sound public policy. To this end, the
Kaiser Permanente Institute for Health Policy set
forth the following policy recommendations:
The practice of medicine, which largely uses paperbased records and is heavily dependent on unaided
recall, is ripe for change. The limitations of the 20th
century health care system are such that the old
medical care paradigm is less viable and the
emergence of a new way of practicing medicine is
almost inevitable. Evidence-based medicine is the
foundation for an emerging paradigm in medicine
that calls for new tools to enable improved quality
and outcomes. Furthermore, a variety of system
enhancements can prevent many of the medical errors
and adverse events that currently plague our health
care delivery system. Clinical information systems
can be a bridge to this new paradigm by providing the
fastest route to evidence-based medical knowledge
coupled with more complete and accurate patient data
and diagnostic and treatment information.
Recommendation 1: The federal government should
provide leadership to encourage development of a
standard clinical vocabulary, standards for the
exchange of clinical information, and other standards
for interoperability as they emerge.
Recommendation 2: State and federal privacy policy
should avoid establishing barriers to the legitimate
development and use of clinical information
technology.
Recommendation 3: The cost of health information
technologies should be shared among those who
benefit from them. Public investment is needed to
encourage the adoption of these important
technologies.
Recommendation 4: Research and development
focused on implementation and effective use of
health information technologies should be
encouraged and supported.
The growing body of evidence of the value of CIS
shows that clinical applications such as computerized
reminders, automated order entry systems, and
decision support tools can enable significant
improvement in preventive health services, disease
management, and patient safety. The evidence also
suggests that clinical information systems provide
practitioners with timely and comprehensive
information, which leads to improved care. Health
promotion and personalized care are also major tenets
of the clinical information system value proposition.
II.
Introduction
A review of the literature supports the conclusion that
Developments in Information Technology (IT) within
the past forty years have enabled some organizations
to integrate clinical information with medical care
delivery in ways that can promote safer, more
efficient, patient-centric health care. Clinical
information technologies can support outcomes
management, drug interaction checking, order entry,
and electronic capture of vital signs and clinical notes.
These advances in health care IT could pave the way
A variety of barriers stand between health care
providers and the implementation of clinical
information systems. Among the most significant of
these are the high capital and operating costs,
competing priorities for limited resources, cultural
issues, and the difficulty of defining and capturing a
1
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
for early intervention in disease processes,
improvement in quality, reduction of medical errors,
improved care management, administrative efficiency,
and increased patient satisfaction.
American Hospital Association, the Medical Records
Institute, the Center for Health Care Quality at the
University of Missouri, Cedars-Sinai Health System,
and the U.S. Agency for Healthcare Research and
Quality. (See Appendix A for a list of meeting
participants.)
From the late 1950’s forward, health care visionaries
have anticipated an information technology
revolution within medical care delivery that would
transform the health care industry in a manner similar
to those seen in the finance and retail sectors.
Pioneers of health informatics, such as Morris Collen,
MD, Homer R. Warner, MD, PhD, Octo Barnett,
MD, and Lawrence Weed, MD, among others,
formulated the vision for reforming the chaos in
modern medicine through the use of information
technology.1,2 3
The meeting focused on contributing to the growing
body of evidence regarding the benefits that can be
achieved from clinical information systems,
identifying the barriers and enablers related to
implementation, and identifying the components of a
policy agenda focused on encouraging investments by
other health care organizations in clinical IT.
This paper begins by describing a paradigm shift
toward the practice of evidence-based medicine and
the movement away from the traditional paper-based
system that relies heavily on the unaided mind. The
symptoms that suggest that the traditional medical
paradigm is not well suited for the 21st century are
also explored. Next, clinical information systems are
defined and the evidence of the value of these
systems is summarized. The following section
identifies the many obstacles to clinical information
technology usage and benefit realization. The paper
concludes with policy recommendations to expedite
the expansion of clinical information systems in U.S.
health care.
In 1991, the Institute of Medicine’s Committee on
Improving the Patient Record set a goal to make the
computer-based patient record a standard technology
in health care by 2001.4 Yet, to date the clinical IT
revolution has eluded much of the U.S. health care
system and the high expectations of the visionaries
remain largely unfulfilled. Meanwhile, a variety of
significant barriers still stand in the way of the
overdue promise of clinical information technology.
Although this concern has been discussed in a variety
of forums over the past several years, neither
straightforward solutions nor clear plans of action
have emerged.
This document provides an overview of the issues
surrounding the use of clinical information
technology in the U.S. health care system and
synthesizes the discussions at a meeting regarding
“The Benefits of Clinical Information Systems,”
sponsored by the Institute of Medicine and the Kaiser
Permanente Institute for Health Policy on October 4
and 5, 2001. Participants in the meeting included
representatives of some of the early adopters and
pioneers of clinical information systems, including
Beth Israel Deaconess Hospital, Brigham and
Women’s Hospital, Kaiser Permanente, LDS
Hospital, Mayo Clinic, Palo Alto Medical Foundation,
Queen’s Medical Center, and the Veterans
Administration. Other participating organizations
included the Health Technology Center, the
III.
CIS: A Bridge to a New Health Care
Paradigm
The practice of medicine, which combines a
constantly evolving scientific knowledge base with
clinical experience, human values, and human
intuition, is undergoing fundamental changes. The
cornerstone for what has been described as a
paradigm shift in medicine is the widely held belief
that the evidence-based practice of medicine can lead
to improvement in patient outcomes. In addition, the
demands on the health care system are shifting from
acute to chronic care and consumers are beginning to
play a much more active role in their health care.
Over the past fifty years, evidence-based medicine has
provided an explicit framework of scientifically
2
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
• consumer expectations for improved care and
service are rising.
validated information for medical decision making.
Evidence-based medicine supports the
implementation of cutting-edge care management
programs and can lead to improvements in patient
outcomes and cost-effective care. Yet despite an
exponential increase in the biomedical knowledge
base and revolutionary advances in technology, the
health care industry continues to rely on a framework
for the distribution of new information regarding the
practice of medicine that has changed little over the
last century. Balas and Boren found that “an average
of about 17 years is required for new knowledge
generated in randomized, controlled clinical trials to
be incorporated into practice, and even then its
application is highly uneven.” 5
Whereas the traditional medical paradigm assumes
that a physician’s initial medical school training and
experiences provide a sufficient foundation for
diagnosis and treatment, evaluation of new tests and
procedures, and development of clinical practice
guidelines6, the new evidence-based paradigm
comprises a different set of assumptions. For
example, one of the main tenants of evidence-based
medicine is that clinicians should use unbiased studies
to increase their confidence in the usefulness of
clinical therapies and diagnostic tests. Likewise, an
evidence-based orientation is necessary to evaluate
and apply the medical literature effectively.7
The practice of medicine has grown almost
unmanageably complex. The limitations of the 20th
century health care system are such that the old
medical care paradigm is less viable and the
emergence of a new way of practicing medicine is
almost inevitable. A variety of signs suggest that the
traditional medical paradigm is not well suited for the
21st century:
A. The Paper-Based System Supporting
Clinical Care Is Increasingly Non-Viable
Clinical decision making should be driven by pointof-care information accessed by providers in realtime. Traditional paper-based information systems
are no longer an acceptable long-term option for the
changing demands of health care delivery settings.
Paper-based information storage and retrieval systems
currently have high failure rates8 that can lead to
duplication of service, delays in treatment, increased
length of stay, and increased risk of medical errors
caused by missing or inaccessible data and inefficient
manual processes.9 The limitations of the paperbased system are numerous:
• Providers too frequently have access to only a
portion of a patient’s information because medical
histories are spread across multiple information
silos (i.e., doctor’s offices, hospitals, pharmacies,
health plans, and insurance companies).
Researchers have estimated that from 10 percent to
81 percent of the time, physicians do not find
patient information that has been previously
recorded and belongs in the medical record.10
• In large institutions, the single copy of a patient’s
paper-based record may not be available to others
who need it while a clinician finishes
documentation of an encounter.11
• When a chart is not available, health professionals
must provide medical care without the benefit of
medical history. They document the encounter and
hope that these notes eventually make it to the
patient’s chart—an outcome that is not always the
case, especially in large institutions.12
• the paper-based system supporting clinical care is
increasingly non-viable;
• human memory-based medicine is increasingly
unreliable;
• The structure of paper-based records makes it
difficult to locate essential information within the
chart. Paper-based systems allow free text
documentation, often resulting in disorganized,
non-integrated content that is not useful to others.
• clinical data capture has become a business
imperative; and
3
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
• Not all of the knowledge loaded in the minds of the
professionals is retained.
• Handwritten text entries are not always legible.
• After many years, a patient’s records can increase to
multiple volumes, forcing a distinction between
current and historic information and sending “old
records” and source documents to long-term
storage facilities.13
• Much of the knowledge that professionals retain
becomes obsolete, and there is no assurance that
that they will learn new knowledge relevant to their
patient's problems. In other words, they don't know
when they will get their "upgrade".
A recommendation of the Institute of Medicine
(IOM) for improving patient records dating back to
1991 called for the computer-based patient record to
become the standard for patient medical records by
2001.14 The IOM recommendation has not been
realized and the shortcomings of paper-based systems
that it identified ten years ago persist and are
escalating.
• Even with that reliable knowledge that it retains and
the incredible integration prowess of the human
mind, a doctor's unaided mind cannot reliably
integrate all that knowledge with the infinite variety
of patient data, to identify and systematically assess
all diagnostic or treatment options within the
soonest possible time.
• Faced with this clinical knowledge overload, doctors
tend to fall back on clinical judgment and heuristics
(global intuitive assessment), rather than organized
investigation and correlation of patient findings
with diagnostic and treatment options. Cognitive
psychology has shown that experts' global
judgments are inferior to judgments based on
thorough analysis of data. 17
B. Human Memory-Based Medicine Is
Increasingly Unreliable
The information-intensity of health care is increasing
rapidly. New clinical practice guidelines, research
findings, pharmaceuticals, and medical devices surface
daily. Practitioners are bombarded with changing
clinical decision factors and are challenged to stay
abreast of an increasing knowledge base in their areas
of expertise. So much information is now being
published that a practitioner does not have the time
to read the latest information. In 1966 just over 100
articles were published each year from randomized
controlled trials; in 1995 nearly 10,000 articles were
published annually.15 The current practice of
medicine relies heavily on the unaided mind to recall a
great amount of detailed knowledge, a process that
has repeatedly been shown to be unreliable,16 to the
detriment of all stakeholders nonetheless.
The advocates of clinical IT contend that automated
information management tools are increasingly
necessary as medical knowledge proliferates.
Likewise, clinical IT is needed to lend cognitive aid to
practitioners as they integrate the large body of
general medical knowledge with the complex data on
unique, individual patients.18
C. Clinical Data Capture Has Become A
Business Imperative
As the costs of health care continue to rise,
purchasers are increasingly impatient with the health
care industry’s inability to accurately account for its
expenses. While other industries have developed
extensive cost data and detailed accounting systems,
the health care industry is finding it increasingly
difficult to evade responsibility for the capture of
clinically derived information that is necessary for a
One physician described "voltage drops" in the
transmission of medical knowledge as it is transferred
from the source to the physician's brain to its final
application in patient care delivery. He observes:
• Only a portion of medical knowledge is ever loaded
into the minds of professionals.
4
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
wide variety of purposes. Patient data required for
reimbursement is often extracted from fragmented
information in the patient’s medical record.
Automated clinical data capture is a means of
reducing back-room overhead, increasing cash flow,
and achieving claims accuracy and turnaround time
standards that are required contractually in third-party
payment arrangements.
As consumers learn more about clinical information
technology they will appreciate that it can contribute
to improvements in the safety and quality of their
care, while increasing opportunities for them to
partner with providers in their health. Like other
advances in medical technology, clinical IT will raise
consumer expectations of what is possible and what
should be made available to them. Developments
that are most likely to raise consumer expectations for
the use of IT in the health care delivery system are
personal experience, word of mouth, and exposure
through the media. The omnipresent use of
information systems that support service delivery in
other sectors will also fuel these expectations.
Health care organizations are also beginning to feel
market pressure to invest in clinical data capture as a
growing number of provider organizations begin to
offer consumers online services such as appointment
scheduling, prescription refills and results reporting.19
Disease management programs, which require
stratification of patients with chronic diseases by risk
to achieve cost-effective outcomes, are becoming
increasingly dependent on clinical information
systems. In addition, organizations that do not invest
in systems that offer real-time access to clinical
information may find themselves at a competitive
disadvantage when it comes to provider recruitment.
There is a growing consensus that clinical information
systems can be the bridge to a new health care
paradigm. Clinical information systems offer new
opportunities to further improve the quality and
safety of care by insuring that the most recent
information is available for clinical decision making,
improving evidence-based guideline compliance, and
enhancing communication among providers. The
information management tools within CIS can drive
the changes toward an evidence-based framework for
the evaluation and treatment of patients. Although
clinical information systems are not an absolute
requirement for quality health care delivery, it is
difficult to envision addressing the limitations and
challenges of the 21st century health care system
without these information management tools.
Health care organizations that are not implementing
clinical information systems may soon be viewed as
having a material weakness in comparison to
institutions that are using CIS. This material weakness
impairs the fulfillment of their organization’s mission,
compromises the effective management of
organizational resources, and deprives the public of a
level of quality of care that they might find elsewhere.
D. Consumer Expectations For Improved
Care And Service Are Rising
IV.
Consistent with the trend toward health care
consumerism, individuals are looking for customized
care that fits their lifestyles and health care needs.
Information technology tools can facilitate the
delivery of convenient and personalized care.
Technologies such as the electronic patient record
give providers a better understanding of their
patient’s medical history, health status, allergies,
vaccination history, and personal preferences, which
leads to more appropriate care, better compliance,
and improved clinical outcomes.
“Clinical Information System” (CIS) is an umbrella
term that has been applied to a broad range of clinical
information technology. Several other terms are used
to describe information systems that support the
delivery of health care (e.g., electronic medical record
system, health information system, and computerbased patient record system). For the purpose of this
paper we use the term clinical information system to
refer broadly to various configurations of clinical
application components. In the past, these systems
have typically been clinically oriented homegrown
5
What is CIS?
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
applications on disparate platforms that has evolved
over time—not a single, seamless, integrated
application. Organizations seeking flexibility and
customization have combined application
components incrementally over time.
applications on legacy platforms that were used
primarily by larger hospitals and health care provider
organizations and focused on the information needs
of practitioners.
Because clinical information systems have taken
different evolutionary paths in various organizations,
a single definition of a clinical information system is
elusive. Tang and McDonald20 offer useful
definitions for the computer-based patient record, an
essential element of many clinical information
systems, and for the computer-based patient record
system:
Many of the rewards of clinical information
technology can be harvested without a full-scale
clinical information system that integrates multiple
applications around an electronic medical record. For
example, stand-alone applications such as results
reporting or computerized physician order entry offer
some of the same opportunities to improve care as
the more sophisticated clinical information systems
with electronic medical record functionality. Brigham
and Women’s Hospital in Boston has a long history
of CIS benefit realization in the area of medication
prescribing with a system that does not feature a
complete electronic medical record.
“A computer-based patient record (CPR) is a repository of
electronically maintained information about an individual’s
lifetime health status and health care, stored such that it can
serve the multiple legitimate users of the record.”
“A computer-based patient record system adds informationmanagement tools to provide clinical reminders and alerts,
linkages with knowledge sources for health-care decision
support, and analysis of aggregate data. A CPR system
extends the usefulness of patient data by applying information
management tools to the data.”
Nevertheless, a core source of automated clinical data,
such as an electronic medical record, that feeds
decision and communication support capabilities
helps to maximize the benefits of other system tools.
Furthermore, system components have a synergistic
effect—the value of multiple integrated components
is greater than the sum of the individual parts.
The Institute of Medicine (IOM) Committee on
Improving the Patient Record used the following
definition of a clinical information system during its
work:
The use of IT applications in health care is rapidly
evolving beyond what in the past has been considered
a clinical information system. Health care IT now
encompasses new tools and health services that are
delivered or enhanced through the Internet and other
advanced networking technologies, including
telemedicine, wireless handheld applications, and
home monitoring. Speech recognition tools hold the
promise of reducing transcription costs. New patientfocused tools target the growing ranks of empowered
health care consumers. These include patient selfmanagement applications, interactive health
promotion programs, on-line access to trusted
medical knowledge, and personal health records that
are owned by consumers.23 Many of the new
products and services are component-based and offthe-shelf rather than home grown.
“A system dedicated to collecting, storing, manipulating, and
making available clinical information important to the delivery
of patient care.”21
The IOM Committee acknowledged that the central
focus of a clinical information system is clinical data,
rather than financial or billing information. Clinical
information systems may be limited in their scope to a
single area of clinical information (e.g., pharmacy
data), or they may be comprehensive and cover
virtually every aspect of patient care.22
Clinical information systems typically include one or
more of the system components identified below in
Table 1. Within many organizations a clinical
information system is a patchwork of clinical
6
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
Table 1. Components that are often featured in Clinical Information Systems
Application
Practitioner Order Entry
§
Laboratory Management
System
§
Pharmacy Management System
§
Diagnostic Image Management
System
§
Referral Management System
Description
A means to assist providers in completing clinical tasks such as ordering laboratory tests,
prescription drugs, diagnostic imaging, or consult requests. Decision support and alerts
are typically integrated into order entry capabilities. 24
Integrated View of Patient Data
(e.g., Electronic Medical Record)
Documentation Management
A repository of information about patients that presents an appropriate view of patient
information to health care providers.25
Clinical Decision Support
Administrative Data
Integrated Communication
Support
Access to Knowledge Resources
A means, either using coded data entry or free-text input, to allow practitioners to record
the actions they have taken in diagnosing, managing and treating the patient. Such an
application could collect data such as nursing notes, physician progress notes, or even the
medication administration record, for example.
Alerts based on current data from the electronic medical record, evidence-based practice
guidelines, or more complex artificial intelligence engines for diagnostic support are
provided at the time the clinician is formulating an assessment of the patient’s condition
and making ordering decisions.26
Access to various types of administrative data such as admission, discharge and transfer
records, surgery schedules, demographic data, room assignments, etc. These data are
used to track patient movement and are often needed to allow the accurate generation and
delivery of clinical alerts and reminders.
Tools that positively impact the effectiveness and efficiency of communication among
team members to affect the continuity of patient care handoffs between multiple
providers.27
Online information such as reference materials or journal articles in the context of a
specific patient at the time decisions or orders are being made.28
of the published studies regarding the use of
information technology applications in the delivery of
health care.
This paper, however, takes a more narrow focus and
examines the use of practitioner point-of-care tools
that have supported the delivery of medical care within
hospitals or outpatient settings. As the emerging IT
applications take hold in the industry, the term clinical
information system becomes an increasingly
inadequate description of these technologies. A new
term is likely to develop to reflect this evolution.
V.
Following are the highlights from this review:
• A majority of the studies in circulation fall under the
category of quality, health outcomes and safety.
• The research demonstrates repeatedly that reminder
systems and order entry systems are extremely
effective in improving preventive health and disease
management guideline compliance, particularly in an
outpatient setting.
Evidence of Clinical Information
System Value
Since the 1960’s, researchers from universities, health
care providers and the federal government have
attempted to demonstrate the value of clinical
information systems. To gain a better understanding
of the body of evidence about CIS, the Kaiser
Permanente Institute for Health Policy reviewed many
• There is ample evidence that clinical information
tools can improve prescription drug administration
and patient safety through improved drug dosing,
7
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
reduction in adverse drug interactions, and more
appropriate utilization.
The following is a sample of CIS benefits that have
been documented in the literature as well as
unpublished benefits reported by participants at the
October 2001 meeting “The Benefits of Clinical
Information Systems”, sponsored by the Institute of
Medicine and the Kaiser Permanente Institute for
Health Policy.
• We confirmed what other reviews of literature have
found—a number of articles demonstrate benefit
related to specific aspects of clinical information
systems, but very few provide compelling evidence.29
• Very few studies contain substantial documentation
of improvements in productivity, efficiency, service,
or major cost savings in non-clinical areas.
However, studies looking at adverse drug
interactions and improved formulary usage have
demonstrated cost savings.
A. Improved Quality, Outcomes and Safety
Increase in Preventive Health Guideline
Compliance
The positive effect of computerized reminder systems
on preventive health guideline compliance is one of
the most studied aspects of CIS.34, 35, 36 McDonald et al,
found that physicians who were exposed to reminder
messages were twice as likely to provide preventive
care as those that were not. Reminder messages
improve the likelihood physicians will act on their
intentions.37 Following are some additional examples
of how computerized reminders have demonstrated a
positive impact on preventive health services:
Researchers continue to have difficulty finding
evidence of benefits for a number of reasons:
• Process changes associated with IT implementation
and human variability make it difficult to attribute
positive outcomes solely to CIS. Likewise, decisions
about patient care are ultimately made by physicians
and the patients themselves.
• Much of the research in the area has been performed
at individual sites on "home grown" systems, making
it difficult to generalize the results.
Health Risk Assessment. Health screening guidelines
are often overlooked in primary care settings.
Computerized protocols encourage physician
compliance with recommended guidelines. Physicians
who utilized a computerized protocol increased the
rate of recording blood pressure by 34%.38 In another
study, a computer-based health maintenance tracking
system significantly improved provider compliance
with 8 of 11 recommended procedures at a cost of
only 78 cents per patient per year.39
• Most studies have focused on individual
computerized processes in isolation30very few
studies look at entire care delivery systems. It is
likely that clinical and economic benefits will be
more easily demonstrated after systems have been
fully implemented and additional processes have
been computerized.31
Immunizations and Vaccinations. One in four
toddlers remains underimmunized despite national
campaigns to improve vaccination coverage such as
Healthy People 2000.40 Computer-generated letters
that identify children overdue for immunizations have
proven effective especially among HMO enrolled
populations.41 Fifty percent of parents who received a
computer-generated reminder letter reported
vaccinations for their children, whereas parents who
did not receive a reminder letter reported no
vaccinations.42 Other studies have found that patients
The published evidence regarding CIS benefits fall
into three broad categories32, 33:
• Improved quality, outcomes, and safety;
• Improved efficiency, productivity, and cost
reduction; and
• Improved service and satisfaction.
8
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
of physicians who received computer-generated
reminders were more likely to receive influenza
vaccinations than patients of physicians who did not.43
experimental group but in only 25 % of patients in the
control group.58
Asthma. Asthma is a common health problem that
affects 14.5 million Americans,59 creating a minimum
of 9.5 million office visits annually.60 A computerized
questionnaire that indicates the likelihood and severity
of asthma has been validated as a new approach to
asthma diagnosis. The program holds the potential to
ease the burden of information gathering and disease
detection on general practitioners.61 In a separate
study, a computer-generated reminder system was
found to improve patient management of the disease,
as well as reduce asthma related hospitalizations by
60% and emergency department visits by 50%.62
Cancer Screening and Diagnosis. Cancer is the second
leading cause of death in the U.S., accounting for
approximately 23% of all deaths.44 Computerized
reminders and prompts in an outpatient setting have
been found to have a significant effect on cancer
prevention activities such as the performance of stool
occult-blood test, rectal examination, cervical cancer
screening, pelvic examination, breast examination,
smoking assessment, smoking cessation counseling,
dietary assessment, and dietary counseling.45, 46, 47, 48, 49, 50,
51, 52
Improved Disease Management
Unstable Angina. Partial thromboplastin time, or
PTT, is a common screening test used to monitor
heparin therapy. Common indicators for heparin are
unstable angina and deep vein thrombrosis. At
Queen’s Medical Center in Honolulu an automated
heparin protocol was used to reduce the average time
to reach therapeutic PTT levels—96% within 24 hour
with the automated protocol vs. 14% within 24 hours
without the computer assisted protocol.63
Clinical information systems have been shown to have
a positive impact on care management of the following
diseases:
Diabetes. Diabetes, the 7th leading cause of death in
the United States53, requires a high degree of care
management. Clinical information systems have been
useful in improving care to diabetes patients by
stratifying at-risk patients, improving physician
compliance with guidelines and improving drug
administration.54, 55 A study at Duke University
documented a two-fold increase in clinician guideline
compliance with a computerized decision support
system.56 Kaiser Permanente in Ohio utilized an
automated medical record and computer prompt
system to stratify patients by risk group and apply
preventive treatment to medium- and high-risk
diabetic patients. The system was successful at
increasing preventive care, reducing unnecessary visits
for low-risk patients, and substantially reducing the
number of amputations of high-risk patients.57
Mental Health. Approximately 9.5% of the U.S.
population or 44.3 million Americans suffer from a
mental disorder in a given year.64 Computer
administered screening instruments at primary care
clinics were found to be useful for diagnosing mental
health disorders, increasing the quality of patient care,
and eliciting medical information from patients.65, 66, 67
In a separate study, a computer reminder system in an
outpatient mental health clinic was shown to be
superior to a paper system, improving adherence to
clinical guidelines by 25%. The system also improved
problem documentation by over 90%.68
Tuberculosis. The recent increase of tuberculosis in
the United States makes appropriate preventive
therapy a priority. A computer-based decision support
system was found to be more effective and efficient
than standard guideline tables for physicians in
applying TB preventive therapy. Physicians who
utilized the system applied the appropriate treatment
95% of the time, whereas physicians who used paper-
Hypertension. The prevalence of hypertension is high
in the United States and the associated cost is a burden
on the health care system. A computer-based medical
record was used by Harvard Community Health Plan
to significantly improve follow-up of newly discovered
elevation in diastolic blood pressure. Follow-up was
attempted or achieved in 84 % of patients in the
9
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
based resources applied the appropriate treatment only
56% of the time.69
Improved Drug Dosing. Warfarin and other
anticoagulant drugs are widely prescribed for a variety
of clinical disorders that have high medical and
administrative costs. Patients are found to be
inadequately coagulated 35-50% of the time.77 Several
studies have found that computer-assisted therapies
and decision support systems can improve outpatient
anticoagulant control in early and long-term treatment
which may lead to a reduction in adverse events.78, 79, 80,
HIV/AIDS. Advances in treatment are enabling
persons living with HIV/AIDS to live longer, healthier
lives. Getting current HIV treatment guidelines in the
hands of practitioners is a major challenge considering
the large amount of emerging information. Beth Israel
Deaconess Medical Center in Boston saw a positive
impact from computerized alerts and reminders on the
dissemination and adoption of clinical practice
guidelines in the area of HIV. Specifically, the median
response time with the electronic alert system for a set
of conditions was 11 days, as compared to 52 days
without the alert system.70
81, 82
Reduction in Medical Errors
Reduction in Adverse Drug Interactions. Adverse
drug interactions are a major source of morbidity and
mortality resulting in high costs in both inpatient and
outpatient settings. An estimated 770,000 people are
injured due to adverse drug events annually in the
U.S.83, 84, 85 and up to 70% of these incidents may be
avoidable.86, 87 A study at Brigham and Women's
hospital evaluated a physician order entry system that
resulted in a 55% reduction in medical errors and a
17% decrease in the preventable adverse drug event
rate with potential cost savings of at least $480,000
annually.88 Another study at the same institution
documented that more than 80% of medication errors
unrelated to missed dosage were eliminated by
computerized physician order entry.89
Improved Drug Prescribing and Administration
c. Improved Drug Pr
Improved Antibiotic Usage. Studies show that up to
50% of antibiotic use may be inappropriate in a variety
of clinical settings.71 A study at LDS Hospital in Salt
Lake City found that an antibiotic information system
that helped physicians select antibiotic regimens was
more likely to address the pathogens causing a
patient’s infection and offer the most cost effective
alternative. Additional use of this system in an
intensive care setting at the same site resulted in fewer
susceptibility mismatches and allergic reactions.
Adverse drug reactions were reduced by over 70%,
patients received excessive dosages for 2.9 fewer days,
and the overall cost of antibiotic therapy was reduced.
Reduction in Errors of Omission. Errors of omission,
such as failure to act on results or failure to carry out
indicated tests, are a common problem in medicine.
Computer generated reminders have been found to be
effective in reducing errors of omission in both
inpatient and outpatient settings.90, 91 A study at the
Regenstrief Institute found that physicians who
received a computer-generated reminder followed
practice guidelines 25% more often in an inpatient
setting.92
72, 73, 74
Increase of Appropriate Prescription Use and Refill
Compliance. Patient noncompliance with medication
use is a source of high medical costs associated with
poor disease management. A study among outpatients
demonstrated that only 22% of prescribed therapies
were being taken properly.75 At a university health
center a computer reminder system was found to be
significantly more effective in increasing medication
refill compliance in a controlled study.76
Improved Medical Data Capture and Display
A number of studies have found that computerized
information systems enable more comprehensive and
accurate documentation by physicians and nurses.93, 94
A more complete record is achieved with computer
prompts that ask for missing information and can
10
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
improve the clinical decision process.95, 96, 97 One study
found that charting errors occurred in 25% of
handwritten flow sheets. An automated patient data
management system eliminated these errors and
increased the number of progress notes documented.98
that utilized the order entry system than those that did
not. 103
Outpatient Utilization Reduction. Kaiser Permanente
has evidence that the clinical information systems in its
Colorado and Northwest regions helped reduce
outpatient visits in the first two years after
implementation. Total office visit rates decreased
approximately 7% (on an age-adjusted basis) in both
regions. The data indicate a reduction in the
proportion of members with high visit frequency after
the system implementation. While the visit rate has
dropped, key measures of quality remain stable or
improved.104
A study at Northwestern University documented
increased completeness of progress notes in their
electronic medical record compared to notes in paper
records of a control group. A blinded expert panel
scored completeness on a 0-2 scale—electronic
medical records scored 1.8 vs. 0.9 for paper record
users. The study also found that the clinical decisions
of providers using the electronic medical records were
deemed more appropriate compared with providers
who used paper medical records.99
Scheduling of Follow-up Appointment..
Computerized support for the scheduling of follow-up
appointments for anticoagulant treatment was found
to be effective in providing more efficient care by
having longer, more appropriate intervals between
appointments, without compromising quality of
care.105
Improved Access to Pertinent Literature and
Clinical Information
Some clinical information systems make it possible for
physicians to have access to knowledge bases and
literature sources. This access may be useful at the
diagnostic or treatment stage or in enhancing patient
understanding of a clinical problem.100 A study at the
University of Washington found that integrating the
electronic health record with the Internet also led to
efficiency gains.101
Reduction of Unnecessary Diagnostic Test.
Unnecessary diagnostic tests are a financial burden on
the health care industry and an imposition on patients.
A Regenstrief Institute study found that computers
can assist physicians in estimating the risk of disease,
thereby reducing the ordering of unnecessary
diagnostic tests in an outpatient setting. This was true
particularly for the two most popularly ordered tests,
electrolyte level tests and blood cell counts. 106 In a
separate study at the Regenstrief Institute, physicians
ordered 14%-17% fewer diagnostic tests when they
were given computerized information about the
charges for tests. The resulting reduction in charges to
the patient or insurer was almost $7 per patient visit.107
Patient inconvenience and risk were also reduced.
B. Improved Efficiency, Productivity and
Cost Effectiveness
More Appropriate Utilization of Services
Inpatient Utilization Reduction. Clinical information
technology has been used to improve quality of care
while reducing hospital utilization. A clinical
laboratory alerting system used at LDS Hospital to
increase the likelihood that patients in life-threatening
situations receive appropriate care decreased length of
stay by up to 6 days for some conditions.102 In a study
conducted at the Regenstrief Institute, a physician
order entry system in an inpatient setting was found to
lower patient charges and hospital costs mainly by
reducing length of stay by 10.5%, reducing test charges
by 12.5%, and reducing drug costs by 15.3%. The
total charges per admission were 12.7% less for teams
Better Use of Formulary and Generic Drugs
In 1998, pharmaceuticals were estimated at 8% of
overall health care costs, which translates to
approximately $90.6 billion.108 More recently, it has
been estimated that pharmaceuticals have risen to 11%
of overall health care costs.109 One strategy to reduce
pharmaceutical costs is to use computer prompts and
recommendations to encourage the use of generic and
11
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
formulary drugs. A study at Duke University found
that providing physicians with monthly-computerized
feedback of prescribing charges and patterns
encouraged physicians to substitute generic
alternatives for brand name prescriptions 30% more
often.110 Another study in Scotland saw a reduction in
prescription costs by more than 20% with the use of a
computerized prescribing program and personalized
formulary.111
medical center estimates savings between $300,000$500,000 per year.118 The Kaiser Permanente
Colorado and Northwest regions have documented
annual payroll cost savings of approximately $4 million
and $5.7 million respectively. Both Kaiser Permanente
regions enjoy nearly 100% chart availability with their
CIS. 119
Improved Workflow and Time Saving
A study at Duke University found that the use of a
computerized medical record system resulted in an
overall time saving of 13% for physicians. The
computerized medical record was also found to
improve physicians’ response to information regarding
diagnosis and treatment.112 In another study,
physicians who utilized the computer-based medical
record indicated that it contributed to time saving in
follow-up phone conversations and office visits.
Physicians found value in not needing to reenter
patient and laboratory data.113
Information systems have been historically successful
in the area of charge capture. Both Beth Israel
Deaconess Medical Center and Brigham and Women's
Hospital utilized an integrated hospital system and
reduced the time to collect unpaid bills from 65 days
to 39 days, and from 100 days to 59 days, respectively.
At the latter hospital, outstanding debts in the
outpatient clinics have been reduced by more than $6
million and revenues have increased by 45% over the
three years since implementation. The systems at
these hospitals enabled more than 90% of each
patient’s charges to be captured as a byproduct of
clinical computing.120
Savings Related to the Storage of Medical Records
Elimination of Transcription Notes
Electronic medical records can save a substantial
amount of space required for paper storage. It has
been estimated that an annual cost saving from storage
due to a computerized medical record ranges from
$4,000 at small centers to $100,000 at large centers. 114
The Memorial Sloan Kettering Cancer Center
estimates that they have realized space savings of 2,000
sq. feet with an electronic medical record, worth
$100,000 annually.115 Kaiser Permanente’s Colorado
and Northwest regions estimate annual savings of
approximately $400,000 and $500,000, respectively, in
avoided lease cost after implementation of an
electronic medical record.116
Medical transcription has grown to a $15 billion to $20
billion industry. Beth Israel Deaconess Medical Center
has eliminated the use of transcription notes in the
outpatient setting after the implementation of an
electronic patient record system.121
Improved Charge Capture and Revenue
C. Improved Service and Satisfaction
Improved Communication
Communication and the sharing of information
between providers are important for the delivery of
quality care and prompt service to the patient.122 A
study at Brigham and Women’s Hospital found that an
outpatient clinical referral system decreased the
amount of time needed to complete the referral
process compared to the manual process. The time
saving improved the quality of care, the
communication between physicians and created cost
savings.123 The Mayo Clinic’s clinical information
Savings Related to Reduction in Chart Pulls
The electronic medical record reduces the time
necessary for clerical staff to locate, pull and deliver
charts.117 Beth Israel Deaconess Medical Center uses a
CIS to reduce the paper chart pulls for processing
telephone messages. At $4 per chart pulled, the
12
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
system enables electronic submission of consult
requests in the hospital with immediate appearance of
the request on a provider’s work list. Anecdotes have
been collected at the Mayo Clinic of patients being
discharged a full day earlier because of rapid
notification.124
Conclusions about the Evidence
As evidenced above, the impact of clinical information
systems on health care outcomes, safety, and service is
growing. While only a limited number of the
individual studies contain compelling evidence of
benefit realization, when one looks at the studies
collectively there is mounting and persuasive evidence
of the positive impact of clinical information
technology. Additional research in this area is
imperative. Randomized controlled trials are
recommended as they provide the most valid
information about the efficacy of clinical information
systems.132 Further evidence of improvements in
productivity, service, and cost savings in non-clinical
areas will also help to build the case for CIS.
Improved Satisfaction
Physician and Health Care Worker Satisfaction
Since an estimated 20-30% of clinician time is spent
searching for or organizing information125, access to a
clinical information system is likely to improve
provider satisfaction. For example, physicians at the
Mayo Clinic have outpatient data available immediately
when patients are admitted to the hospital. Prior to
CIS implementation the charts might take several
hours to arrive following patient admission to the
hospital patient care floor.126
VI.
Roadblocks
Unlike other information-rich industries, health care,
the largest service industry in the U.S. economy, has
not fully benefited from the information revolution.
The typical health care organization spends
approximately two percent of its capital budget on
information technology, as compared with an average
of 10 percent in other industries.133 Why have health
care providers failed to migrate to clinical information
technologies?
Patient Satisfaction
Clinical information systems are likely to increase
patient satisfaction with the care they receive.
However, very few articles address the impact of these
systems on the patient experience or on patients' level
of satisfaction.127 A couple of studies have found
improved patient satisfaction due to an automated
nursing discharge summary128 and access to bedside
technology.129
A. Competing Priorities
Part of the answer lies in IT resource allocation
decisions. In the 1970s, 1980s and 1990s the IT
investments of health care organizations were focused
primarily on financial and administrative systems.134
Patient registration and accounting applications were
introduced to make workflow and billing processes
more efficient. Avoiding the anticipated Y2K crisis
occupied the health care IT agenda during 1998 and
1999. Most health care organizations froze new IT
investments and focused their resources on
remediation of projected Y2K problems in their
financial and administrative applications.135 Today, a
variety of IT solutions are competing with clinical
information technology for the scare capital funds of
health care organizations. These competing nonclinical technologies include practice management
applications, supply chain automation, and web-based
customer relations software.
In a survey of 60 hospitalized patients and their
families, the Mayo Clinic documented that 89% of
patients’ families had positive comments related to the
use of bedside computers in patient care; 87% of
patients felt that the computer increased the amount
of time nurses spent with patient care; and 63% felt
they had more involvement in their own plan of
care.130 The Palo Alto Medical Foundation uses an
online application that gives patients access to health
care services and their medical record. Patients who
use the Internet services feel that online
communication with their physician and staff increases
their satisfaction compared with telephone services
and promotes the feeling of shared decision making.131
13
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
CHART 1
Source: 12th ANNUAL HIMSS LEADERSHIP SURVEY Sponsored by Superior Consultant Company and Dell
Computer Corporation.
cost of technology acquisition is beyond their means.
Lack of adequate financial support for IT and
difficulty proving quantifiable benefits/return on
investment have been the top two IT implementation
barriers cited in 2000 and 2001 by senior executives
from health care provider and vendor organizations
from across the U.S. in the Healthcare Information
and Management Systems Society (HIMSS) Leadership
Survey (see Chart 1).137
B. Cost is a Significant Barrier
The high cost of the basic infrastructure of clinical
information technology is a significant hurdle for
many provider organizations, many of which are
suffering from multiple years of margin deterioration
from thinning Medicare or other reimbursement.
Getting a clinical application up and running requires a
minimum IT infrastructure (i.e., fiber-optic backbone
network, staff training and development, clinical
workstations136, or high-speed internet access, T1 line
etc.). For most hospitals, access to capital for new
medical technology is extremely scarce. Likewise,
small to mid-sized IPAs and independent physician
offices, where the majority of physicians practice, are
plagued with financial instability and cannot afford the
infrastructure cost.
C. The Elusive Business Case for Clinical IT
While quality and safety are important concerns,
information technology investment decisions are often
painstakingly evaluated by health care organizations
based on measures such as cost-per-doctor-permonth, cost-per-member-per-month, and short-term
return on investment. Tang and McDonald observed
that part of the difficulty in understanding the relative
costs and benefits of a computerized patient record is
Many prospective purchasers decide not to make the
investment in clinical systems after realizing that the
14
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
that have large IT investments they don’t want to
abandon.
the inability to accurately measure the actual costs of
using paper-based records.138 Return on investment
for clinical information systems is not easy to measure
nor is it necessarily the most appropriate indicator of
CIS success. CIS benefits that have a significant
impact on health care operations, such as provider
convenience, patient satisfaction and service efficiency
are not easily captured on the bottom line in terms of
an increase in revenue, a decrease in expense, or an
avoidance of expense. As a result many provider
organizations have not been able to justify a significant
resource investment in clinical information systems in
the absence of a strong business case or market forces
that dictate such an investment.
• Vendor selection is an arduous process of sifting
through a bewildering array of systems, RFPs, and
prospective vendors. In a survey conducted by the
Medical Records Institute in 2000, over one-third of
companies indicated that the inability to find a
vendor or technical solution that addresses their
organization’s needs was a major barrier.141
• Fragmented delivery system—Most care in the
U.S. is still highly fragmented, involving many
unrelated providers who struggle to communicate
with each other and provide continuity of care. This
system leads to dysfunctional relationships between
stakeholders and offers few incentives for
collaboration to overcome the multiplicity of
conflicting data standards that exist across the care
continuum.
Prospective CIS purchasers are becoming increasingly
aware that the true cost of a clinical information
system is a lot higher than the price quoted. Further, it
often takes much longer to implement a new system
than is promised.139 Savvy clinical IT investors also
understand that the realization of CIS benefits is very
dependent on how the business and practice of
medicine is managed—the information technology
only facilitates any gains. And capturing the gains
requires focused management attention.
• Provider resistance can be an obstacle to clinical IT
implementation, particularly if the system is counterintuitive to physicians’ practice methodologies and
preferences. Also, new processes can mean
additional responsibilities for physicians and the
need to change.
D. Other Barriers
• Lack of industry standards— Interoperability
refers to the ability of two or more systems to
interact with one another and exchange data
according to a prescribed method in order to achieve
predictable results. Interoperability significantly
impacts the accessibility of information across the
continuum of care in multiple outpatient and
inpatient facilities. Despite the substantial work
toward health informatics standards, no standard for
electronic health records allows true interoperability
among various providers.142
Beyond resource constraints and the soft business case
for CIS investment, there are a variety of other
challenges and barriers to the widespread
implementation of clinical information systems in the
U.S. health care delivery system:
• Data security and patient privacy—Security and
privacy of electronic medical records have been
under intense scrutiny in recent years by both the
U.S. government and the health care industry. A
breach of patient data confidentiality leading to a
crisis situation and media circus that threatens a
hospital or health plan's good name is a nightmare
that no health care organization wants to envision.140
• Risk averse health care organizations may be
waiting to let others be the clinical IT pioneers.
Numerous implementation efforts have been
undertaken by organizations on the basis of vendor
assurances, often at great expense, with an end result
that is significantly less than promised. In an era of
• Integrating legacy systems with new clinical
information systems is a challenge to organizations
15
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
clinical information systems have been successfully
introduced into other health care organizations and
interest is increasing in many more. In a recent survey
of health care providers, 71% indicated that that
clinical information systems were a priority for
them.144 However the vast majority of health care
practitioners and institutions in the U.S. are not well
positioned financially or organizationally to implement
the IT infrastructure necessary to deploy clinical
applications.
shrinking margins, health care organizations are
increasingly reluctant to “gamble” on a promised
solution vs. one that has a tried and tested track
record. Steady technical advances also complicate
this—today’s system may be outmoded tomorrow.
This is a particular concern with major IT projects
that require more than a year to implement.
• Lack of individuals trained in medical
informatics with experience as leaders – Most
large-scale clinical information system
implementation projects are still led by individuals
who lack both formal training in the field of medical
informatics and experience in the field of health care
itself.
A recent California HealthCare Foundation report
points out that only physicians who care for patients
under a pre-paid reimbursement arrangement have
financial incentives to use clinical information
technology, whereas other providers are not financially
rewarded for using information technology to achieve
better outcomes.145 As the financial risk for
population care management increases, the case for
clinical IT investment is strengthened. Populationlevel financial incentives such as capitation reward
health care providers that successfully manage
populations, rather than units of service. However,
physician group involvement in capitation has
decreased in recent years rather than increase.
• Time and cost required to choose, buy, and
implement or build a clinical information
system – The time required to fully implement a
robust, integrated clinical information system is on
the same order of magnitude as that of other major
construction projects within a medical center (e.g.,
building a replacement hospital).143 In addition to
explicit costs such as training, there are hidden costs
such as initial reduction in productivity.
Our nation’s health care delivery system is in danger of
becoming divided between the clinical information
technology “haves” and “have-nots”. Widespread
adoption of clinical IT in the near term is unlikely
unless policy action is taken to address the significant
challenges mentioned above.
• Benefit realization is not easily transferable—the
benefits gained in a successful application
implementation are not easily transferable to other
organizations because of differences in care delivery
models, leadership factors, and organizational
culture.
VII.
E. The Digital Divide
A small minority of care delivery systems has uniquely
positioned themselves to overcome many of the
barriers discussed above and has pioneered the use of
clinical information technology in health care. For
example, delivery systems such as LDS Hospital, the
Mayo Clinic, Beth Israel Deaconess Medical Center,
Brigham and Women’s Hospital, Kaiser Permanente,
and the Veterans Health Administration have
successful systems in place today that represent multimillion dollar investments that have evolved over
many years—decades in some cases. In addition,
Need for a National Health Care
Information Infrastructure
As noted above, lack of standards has been a barrier to
CIS implementation. Much has recently been written
and discussed about the need for a national health care
information infrastructure. This infrastructure has
been described as “a set of technologies, standards,
applications, systems, values, and laws that support all
facets of individual health, health care, and public
health.”146 It will be essential to facilitate
communication and care among providers from
different organizations. Four organizations have
focused recent attention on the need for the U.S.
16
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
health care system to move toward a national health
care information infrastructure and the need for
broader adoption of clinical information technology.
The National Quality Forum, a not-for-profit
organization created to develop and implement a
national strategy for health care quality measurement
and reporting, issued a resolution from its Board of
Directors in March 2001 that addresses the need for a
national health information infrastructure. The
National Quality Forum is planning an IT Summit on
this subject in March of 2002. Topics to be discussed
at the IT Summit include architecture and
infrastructure, funding and incentives, governance, and
information standardization.
The Institute of Medicine (IOM) in February 2001,
published Crossing the Quality Chasm, a report that
highlights six areas for focused improvement in U.S.
health care: safety, effective, patient-centered, timely,
efficient, and equitable. The report argues that
information technology must play a central role in the
redesign of the health care system and has enormous
potential to improve the quality of care with regard to
each of the six areas of focus listed above. The report
calls for the establishment of a national health
information infrastructure that will encourage publicand private-sector investments in information
technology. This builds on the 1991 IOM report, The
Computer-based Patient Record: An Essential Technology for
Health Care in which the IOM recommends the
promulgation of uniform national standards for data
and security to facilitate implementation of clinical IT.
The recent attention on the national health care
information infrastructure points to a growing
awareness of the need for national standards for
defining, collecting, communicating, storing, and
protecting health care data. CIS and other information
management tools are integral components of the
larger vision for a national health care information
infrastructure. This type of national strategy has a
significant bearing on the advancement of CIS in the
U.S. because it will enable provider organizations to
overcome many of the implementation barriers that
they currently encounter.
The National Committee on Vital and Health
Statistics (NCVHS), the public advisory body for the
Secretary of the U.S. Department of Health and
Human Services on national health information policy
articulated a vision of a National Health Information
Infrastructure (NHII) in a report submitted to the
Secretary in December 2001. Information for Health: A
Strategy for Building the National Health Information
Infrastructure builds on findings from public hearings
and consultation with various health care stakeholders
and outlines a vision and process for creating a NHII.
The authors “…determined that the most important
missing ingredient, which could accelerate and
coordinate progress on the NHII, is leadership,
specifically, Federal leadership.”147
VIII.
Policy Recommendations
Public policy that encourages and enables the health
care industry to realize the potential of CIS outlined in
this document is needed to promote broader use of
clinical information technology. The following are
recommendations to policymakers as they seek to
advance the clinical IT revolution and as they consider
the specific proposals for the development of a health
information infrastructure in the United States:
Recommendation 1: The federal government
should provide leadership to encourage
development of a standard clinical vocabulary,
standards for the exchange of clinical information,
and other standards for interoperability as they
emerge.
The President’s Information Technology Advisory
Committee’s February 2001 report, Transforming
Health Care Through Information Technology, delivers the
message that advances in information technology can
provide the foundation for important improvements
in health care delivery. The report specifies six
recommended actions for the federal government to
advance the technological capability of U.S. health
care.
The fragmented nature of the U.S. health care system
underscores the importance of standards to enable
interoperability between various provider entities.
17
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
the government can be a stimulus for the private
sector to adopt the same or similar standards.
Without standards it is not practical for many health
care organizations to adopt information technologies
because of the numerous obstacles to sharing and
maintaining digital clinical information. In addition,
prospective investors risk that what they buy today will
be obsolete tomorrow. Similar to the banking and
airline industries, health care needs to create a level of
standards that pertain to content, vocabulary and the
format of data.148 Despite substantial work toward
health information standards by a variety of private
sector organizations, there is still no agreement on
uniform standards.
Recommendation 2: State and federal privacy
policy should avoid establishing barriers to the
legitimate development and use of clinical
information technology.
Given the difficulty of implementing clinical
information systems and their growing list of benefits,
it is important that barriers to further implementation
not be established by public policy. Public interest and
attention regarding the confidentiality and privacy of
personal health information have increased as
information technology expands into health care.
Opinion polls have found that the American public is
skeptical of the data management practices of large
health organizations.149 Public distrust of these
practices should not be underestimated. Yet our
review of the evidence demonstrates that significant
benefit can be achieved with further implementation
of CIS. A delicate balance needs to be found between
the public’s right to privacy and a provider's ability to
coordinate quality medical care in a fragmented
delivery system and perform medical research for the
benefit of society. Reconciling consumers’ desire for
the most advanced medical care that technology allows
with privacy concerns remains a challenge for health
care providers and policymakers.
The limited progress of the private sector in standards
development is in part due to the fact that efforts have
been fragmented among a large number of
organizations. The leadership of the federal
government is needed to encourage the development
of standards in a coordinated and efficient manner.
As a purchaser and provider of care and a user of
clinical data, the federal government has a legitimate
interest in expediting the development of health care
data standards. The federal government needs to
become more actively involved in the standards
development process. Specifically the federal
government should:
• Appoint a senior information technology leader to
provide strategic leadership for the establishment of
national standards for health information
interoperability.
HIPAA privacy regulations have established a national
“floor” of privacy standards to protect the personal
health information of Americans. Furthermore, twothirds of states have enacted additional statutory
protections in response to public concerns about
privacy.150 State and federal policymakers should apply
the adage "first do no harm" to the developing field of
CIS as they consider additional legislation or regulation
to protect against improper disclosure of personal
health information. Privacy rules have the potential to
create unintended but significant barriers to the
delivery of health care services and medical research.
It is entirely appropriate to impose security standards
that assure that protected health information is not
accessed improperly. It is not appropriate to preclude
access to and use of essential medical information
needed to provide medically needed care.
• Articulate a strategy for the establishment of national
standards for health information interoperability.
• Convene an expert group (including key players in
health care delivery, medical informatics, consumers,
and researchers) to provide recommendations on a
national mechanism to establish standards for use in
health care settings and a process for the national
maintenance of these standards and other
mechanisms to facilitate interoperability.
• Additionally, the federal government should work to
achieve consensus on standards for health care
information across federal agencies. By doing so,
18
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
• Legislation, modeled after the Hill-Burton
Construction Act of 1946, to encourage the
development of a national health information
infrastructure should be explored. The Hill-Burton
Construction Act encouraged the modernization of
the nation’s decaying health care facility
infrastructure by providing funds for its expansion
and development. In return, facilities agreed to
provide free or reduced charge medical services to
persons unable to pay. Health care information
infrastructure legislation modeled after the HillBurton Act would assist organizations with the high
cost of capitalizing clinical IT. Linking funding to
Medicaid and Medicare participation would ensure
that these technologies are distributed equitably.
Policymakers should become better informed of the
benefits of clinical information technologies and their
growing impact on the quality and safety of health
care. Knowledge that certain legislative proposals
have the potential to restrict the legitimate and optimal
use of clinical IT will help policymaker make informed
decisions.
Recommendation 3: The cost of health
information technologies should be shared among
those who benefit from them. Public investment
is needed to encourage the adoption of these
important technologies.
The high capital and operating costs of information
technology remains a significant barrier for many
health care organizations. While the costs of CIS are
borne by health care providers, the benefits accrue to
patients and to society as a public good. This implies
that purchasers, including the federal government,
should be a more explicit part of the information
technology reimbursement equation.
• Appropriate reimbursement changes in the Medicare
program and by other payers could go a long way
toward promoting the development and use of
health information technologies. Reimbursement
legislation in fee-for-service Medicare could include
changes such as reimbursement for remote visits
that CIS can facilitate or bundled payments for
chronic conditions.151 Organizations that receive
Medicare capitation could receive a supplemental
amount to cover system development.
In 1991, the Institute of Medicine Committee on
Improving the Patient Record issued a landmark
report entitled The Computer-Based Patient Record: An
Essential Technology for Health Care, which advocated the
prompt development and implementation of
computer-based patient records. One of the policy
recommendations in the report suggested that the cost
of implementing and operating clinical IT should be
factored into reimbursement levels or payment
schedules for both public and private sector thirdparty payers. In the revised 1997 edition of the report,
the authors noted that no progress could be observed
toward clinical IT cost-sharing. Lack of progress in
this area can still be observed today.
Recommendation 4: Research and development
focused on implementation and effective use of
health information technologies should be
encouraged and supported.
Research is an ongoing theme in the evolution of
information technology in medicine. Much of the
research that has been done to date on health
information technologies has focused on clinical
outcomes and cost savings. This work should
continue. However, research also needs to be focused
on the critical success factors for CIS implementation
and benefit realization. Variation in CIS
implementation strategies directly impacts the tangible
benefits that are achieved and how quickly they are
realized. Leadership factors, organizational factors,
cultural factors, and lessons learned from past
implementations have been understudied.
Organizations with mature clinical information
systems should be encouraged to continue their
• Following are some of the funding possibilities that
have been discussed in various forums:
• The IOM recommends that Congress establish a $1
billion "Health Care Quality Innovation Fund" to
support the adoption of quality-improving
innovations in medical practice and technology. The
fund would be used in the next three to five years to
help subsidize promising projects and communicate
the need for these technologies in health care.
19
CLINICAL INFORMATION SYSTEMS: ACHIEVING THE VISION
support this work. Support is also needed to
disseminate the growing body of knowledge regarding
the benefit of these systems to further encourage their
adoption.
development work and document their
implementation experience.
Research to explore further innovation in technology
development (e.g., new software and operating
platforms) would also be beneficial. Along a similar
vein, research in the development and standardization
of health care applications and processes, including
standards for authentication and electronic signatures
that enable organizations to discontinue paper records
is needed.
IX.
Conclusion
Despite the modest uptake of clinical information
technology in the practice of medicine to date,
evidence is growing that clinical information systems
hold significant potential for positive benefit in the
delivery of health care. It is increasingly evident that
the electronic medical record and other emerging
clinical information management tools are the enabling
technologies that will lead to a new paradigm of
evidence-based, patient-centric health care. A review
of the literature and the experience of various
organizations support the conclusion that CIS can
improve provider performance, enhance clinical
outcomes, and decrease medical errors. It is likely that
many of these applications and the benefit they
produce will become the “community standard of
care” over the next decade, improving both patient
care and satisfaction. Thus, there is an imperative to
encourage broader implementation and use of clinical
information systems with sound public policy focused
on the identification of solutions to the barriers and
disincentives to implementation that are experienced
by health care organizations today.
Finally some of the questions that need to be
considered are:
• What are the lessons learned from the
implementation of information systems as they
pertain to people strategies, workflow patterns,
physician and patient involvement, etc?
• What should be the role and responsibility of the
patient in health care as information technologies are
incorporated into the care delivery system?
• What is the optimal set of system applications and
features that should be recommended to health care
organizations to maximize benefit?
• What are the recommended implementation
strategies for both large and small health care
organizations?
• Given the profusion of medical evidence that exists,
what are the best processes by which the validity of
this information can be assessed?
In addition to implementation questions and in light of
the many challenges to benefit documentation, more
randomized controlled trials are recommended, as they
provide the most valid information about the efficacy
of information systems.152 Studies that focus on
demonstrating improved health outcomes and cost
savings should continue to be emphasized as well.
Congress should provide the Agency for Healthcare
Research and Quality specific authority and funds to
20
Appendix
Meeting Participant List
The Benefits of Clinical Information Systems
A Meeting Sponsored by the Institute of Medicine and
the Kaiser Permanente Institute for Health Policy
Denver, Colorado
October 4-5, 2001
• E. Andrew Balas, MD, PhD, Director, Center for Health Care Quality, and Weil Distinguished Professor,
University of Missouri
• David W. Bates, MD, MSc, Chief, Division of General Medicine, Brigham and Women's Hospital
• Homer L. Chin, MD, Medical Director for Clinical Information Systems, Kaiser Permanente—Northwest
• Gary A. Christopherson, Chief Information Officer, Veterans Health Administration, U.S. Department of
Veterans Affairs
• Janet Corrigan (Meeting Co-Sponsor), Director, Board on Health Care Services, Institute of Medicine
• Molly Joel Coye, MD, MPH, MA, President and Chief Executive Officer, Health Technology Center
• Robert M. Crane (Meeting Co-Sponsor), Senior Vice President and Director, Kaiser Permanente Institute for
Health Policy, Kaiser Foundation Health Plan, Inc.
• Daniel Davis, MD, Chief, Department of Internal Medicine, Queen's Medical Center
• Don E. Detmer, MD, Gillings Professor of Health Management, Judge Institute of Management, University of
Cambridge
• Reed M. Gardner, PhD, Department of Medical Informatics, University of Utah
• William A. Gillespie, MD, Executive Vice President, Kaiser Foundation Health Plan, Inc.
• Elisabeth Hallman, MBA, RN, CAN, Director, Implementation, Cedars-Sinai Health System
• Andrew M. Lum, MD, Assistant Medical Director, Service Quality and Informatics, Kaiser Permanente—
Colorado
• David Mohr, MD, Chair, Mayo Integrated Clinical Systems Committee, Mayo Clinic
21
• Donald M. Nielsen, MD, Senior Vice President for Quality Leadership, American Hospital Association
• Eduardo Ortiz, MD, MPH, Senior Service Fellow, Agency for Healthcare Research and Quality
• Daniel Z. Sands, MD, MPH, Clinical Director of EPR and Communication, Beth Israel Deaconess Medical
Center, Center for Clinical Computing
• Dean F. Sittig, PhD, Senior Informatics Researcher, Kaiser Permanente
• Paul C. Tang, MD, Chief Medical Information Officer, Palo Alto Medical Foundation and Associate Clinical
Professor, University of California, San Francisco
• Peter Waegemann, Chief Executive Officer, Medical Records Institute
• Andrew M. Wiesenthal, MD, Associate Executive Director, The Permanente Federation
22
Notes
1
Collen MF, A history of medical informatics in the United States: 1950-1990, American Medical Informatics Association, 1995.
2
3
Waegemann CP, Healthcare Informatics. May 2001. 55-56.
Institute of Medicine, The computer-based patient record: an essential technology for health care, revised edition, 1997.
5
Balas EA, Boran SA, Managing clinical knowledge for healthcare improvement. Yearbook of Medical Informatics, NLM, Bethesda,
MD. 65-70; 2000.
6
Evidence-based medicine: A new approach to teaching the practice of medicine, Evidence-Based Medicine Working Group. JAMA
1992;268:2420.
7
ibid.
8
Elson RB, et al, An industrial process view of information delivery to support clinical decision making: implications for systems
design and process measures; Journal of the American Medical Informatics Association, 4:266-278, 1997.
9
Bowes L, Improving physician productivity: a case study; Department of Medical Informatics, Intermountain Health Care, Salt Lake
City, Utah, (presented at TEPR 2001 conference).
10
Tang PC and McDonald CJ, Computer-based patient-record systems, Medical Informatics: Computer Applications in Health Care
and Biomedicine, Second Edition, 2001.
11
ibid.
12
Shortliffe EH, Barnett GO, Medical data: their acquisition, storage, and use, Medical Informatics: Computer Applications in Health
Care and Biomedicine, Second Edition, 2001.
13
Bielorey L, The evolution of patient records, Annals of Allergy, Asthma, & Immunology, Volume 81, October, 1998.
14
Institute of Medicine, The computer-based patient record: an essential technology for health care, 1997, 180-181.
15
Chassin M, Is health care ready for six sigma quality?, The Milbank Quarterly, 1998 76(4)
16
McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectibility of man. N Engl J Med. 1976 Dec 9;
295(24): 1351-5.
17
Tolentino H, University of the Philippines Manila College of Medical Informatics Unit, 1997, (www.veranda.com.ph/hermant).
18
Weed, L.L., Weed, L. Opening the black box of clinical judgement, British Journal of Medicine, November 1999.
19
Eng TR. The eHealth Landscape: A Terrain Map of Emerging Information and Communication technologies in Health and Health
Care. Princeton, N.J: The Robert Wood Johnson Foundation, 2001.
20
Tang PC and McDonald CJ, Computer-based patient-record systems, Medical Informatics: Computer Applications in Health Care
and Biomedicine, Second Edition, 2001.
21
Institute of Medicine, The computer-based patient record: an essential technology for health care, revised edition, 1997
22
ibid.
23
Sittig DF, Personal health records on the internet: a snapshot of the pioneers at the end of the 20th century, International Journal of
Medical Informatics (in press).
24
Tang PC and McDonald CJ, Computer-based patient-record systems, Medical Informatics: Computer Applications in Health Care
and Biomedicine, Second Edition, 2001.
25
ibid.
26
ibid.
27
ibid.
28
ibid.
29
A health information network for Australia: report to health ministries by the national electronic health records taskforce, July 2000.
30
National Institution of Health, Evaluation of the Medical Information System at the NIH Clinical Center. Volume 1, Summary of
Findings and Recommendations, Publication No. PB82-190083, Washington, DC: Government Printing Office, 1982.
31
Rose JS, Kaiser Permanente Rocky Mountain Division clinical information system: comprehensive review, Kaiser Permanente.
1998.
32
This taxonomy expands upon categories presented in articles by Paul Clayton and David Mohr. However, assigning the benefits of
clinical information systems to a single category can be challenging because the groupings are not mutually exclusive and many
benefits cross several categories.
33
Mohr D, Benefits of and electronic clinical information system, Healthcare Information Management, Vol. 11, no. 4, Winter 1997.
34
Balas A, et al. The clinical value of computerized information services: A review of 98 randomized clinical trials. 1996. Arch Fam
Med. 5:271-278.
35
Hunt D, et al. Computer-based clinical decision support systems. JAMA. 1998. 280(15):1339-1346.
36
Shea S. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care
in the ambulatory setting. JAMIA. 1996; 3:399-409.
4
23
37
McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, McCabe GP, Reminders to physicians from an
introspective computer medical record: a two-year randomized trial, Annals of Internal Medicine, 1984, 100:130-138.
38
Weingarten MA, Bazel D, Shannon HS. Computerized protocol for preventive medicine: a controlled self-audit in family practice,
Family Practice 1989, 6(2):120-124.
39
Frame P, Zimmer J, Werth P, Hall WJ, Eberly S, Computer based vs. manual health maintenance tracking, 1994, Arch Fam Med.
3:581-88.
40
Centers for Disease Control, State and national vaccination coverage levels among children aged 19-35 months, MMWR, 1995,
44:613-23.
41
Lieu TA, Black SB, Ray P, Schwalbe JA, Lewis EM, Lavetter A, Morozumi PA, Shiefield HR, Computer-generated recall letters
for underimmunized children: how cost-effective? Pediatric Infectious Disease Journal, 1997, 16(1):28-33.
42
Szilagyi PG, Rodewald LE, Savageau J, Yoos L, Doane C. Improving influenza vaccination rates in children with asthma: a test of
a computerized reminder system and an analysis of factors predicting vaccination compliance, Pediatrics, 1992, 90(6):871-875.
43
Hall CB. Influenza: a shot or not? Pediatrics, 1987; 79:564-566.
44
CDC, National vital statistics report. 48(11)
45
Yarnall KSH, Rimer BK, Hynes D, Watson G, Lyna PR, Woods-Powell CT, Terrenoire J, Barber LT, Computerized prompts for
cancer screening in a community health center, J Am Board Fam Pract 1998, 11(2):96-104.
46
McPhee SJ, et al. Promoting cancer screening: A randomized, controlled trial of three interventions, 1989, Arch Intern Med.
149:1866-1872.
47
Landis SE, Hulkower SD, Pierson S, Enhancing adherence with mammography through patient letters and physician prompts: a
pilot study, NCMJ, 1992, 53(11):575-578.
48
McPhee S, Bird JA, Fordham D, Rodnick JE, Osborn EH. Promoting cancer prevention activities by primary care physicians:
results of a randomized, controlled trial, JAMA, 1991, 266(4):538-544.
49
Ornstein SM, Garr DR, Jenkins RG, Rust PF, Arnon A, Computer-generated physician and patient reminders: tools to improve
population adherence to selected preventive services, J Fam Pract, 1991, 32(1):82-90.
50
Fordham D, McPhee SJ, Bird JA, Rodnick JE, Detmer WM, The cancer prevention reminder system, M.D. Computing, 1990,
7(5):289-295.
51
McDowell I, Newell C, Rosser W, Computerized reminders to encourage cervical screening in family practice, Journal of Family
Practice 1989, 28(4):420-424.
52
Turner BJ, Day SC, Borenstein B, A controlled trial to improve delivery of preventive care: physician or patient reminders?,
Journal of General Internal Medicine. 1989, 4:403-409.
53
CDC, National Vital Statistics Reports, Vol 48(11), 1998.
54
Karson AS, Kuperman GJ, Horsky J, Fairchild DG, Fiskio JM, Bates DW, Patient-specific computerized outpatient reminders to
improve physician compliance with clinical guidelines, JGIM, 1999, 14(April supplement 2):126.
55
Kleschen MZ, Holbrook J, Rothbaum AK, Stringer RA, McInerney MJ, Helgerson SD, Improving the pneumococcal immunization
rate for patients with diabetes in a managed care population: a simple intervention with a rapid effect. Journal on Quality
Improvement, 2000, 26(9):538-546.
56
Lobach D, Hammond E, Computerized decision support based on a clinical practice guideline improves compliance with care
standards, Am Jrl Med, 1997, 102(1):89-98.
57
Khoury A, Landers P, Roth M, Rowe N, DaMert G, Dahar W, Nystrom H, Szczepanik R, Computer-supported identification and
intervention for diabetic patients at risk for amputation, M.D. Computing, 1998, 15(5):307-310.
58
Barnett GO, Winickoff R, Morgan M, Zielstroff R, A computer-based monitoring system for follow-up of elevated blood pressure,
Medical Care, 1983, 21(4): 400-409.
59
CDC, Vital and health statistics series 10, No.200, 1996.
60
CDC, Advance data 322, 1999.
61
Kable S, Henry R, Sanson-Fisher R, Ireland M, Corkrey R, Cockburn J, Childhood asthma: can computers aid detection in general
practice?, British Journal of General Practice, 2001, 51:112.
62
Curtin K, Hayes BD, Holland CL, Katz LA, Computer-generated intervention for asthma population care management, Effective
Clinical Practice 1998, 1(1):43-46.
63
Heparin protocol results were provided by Queen’s Medical Center, Honolulu in response to a survey of CIS benefits conducted in
October 2001 by the Kaiser Permanente Institute for Health Policy.
64
Regier DA, Narrow WE, Rae DS, et al., The de facto mental and addictive disorders service system: epidemiologic catchment area
prospective 1-year prevalence rates of disorders and services, Archives of General Psychiatry, 1993; 50(2):85-94.
65
Kobak KA, Taylor L, Dottl SL, Greist JH, Jefferson JW, Burroughs D, Mantle JM, Katzelnick DJ, Norton R, Henk HJ, Serlin RC,
A computer-administered telephone interview to identify mental disorders, JAMA, 1997, 278(11):905-910.
24
66
Spitzer RL, Kroenke K, Williams JB, and the Patient Health Questionnaire Primary Care Study Group, Validation and utility of a
self-report version of PRIME-MD: the PHQ primary care study, JAMA, 1999, 282(18):1737-1744.
67
Rollman BL, Hanusa BH, Gilbert G, Lowe HJ, Kapoor WN, Schulberg HC, The electronic medical record, Archives of Internal
Medicine, 2001, 161(2):189-197.
68
Cannon DS, Allen SN, A comparison of the effects of computer and manual reminders on compliance with a mental health clinical
practice guideline, J Amer Med Inform Assoc, 2000, 7(2):196-203.
69
Dayton CS, Ferguson JS, Hornick DB, Peterson MW, Evaluation of an internet-based decision-support system for applying the
ATS/CDC guidelines for tuberculosis preventive therapy, Medical Decision Making, 2000, 20:1-6.
70
Safran C, Rind DM, Davis RB, Ives D, Sands DZ, Currier J, Slack WV, Cotton DJ, Makadon HJ, Effects of a knowledge-based
electronic patient record on adherence to practice guidelines, M.D. Computing, 1996, 13(1):55-63.
71
Berman JR, Zaran FK, Rybak MJ, Pharmacy-based antimicrobial-monitoring service, Am J Hosp Pharm, 1992: 49:1701-6.
72
Evans RS, Pestotnik SL, Classen DC, Burke JP, Development of an automated antibiotic consultant, M.D. Computing, 1993,
10(1):17-22.
73
Pestotnik SL, Classen DC, Evans RS, Burke JP, Implementing antibiotic practice guidelines through computer-assisted decision
support: clinical and financial outcomes, Annals of Internal Medicine, 1996, 124(10):884-890.
74
Burke JP, Pestotnik SL, Antibiotic use and microbial resistance in intensive care units: Impact of computer-assisted decision
support, Journal of Chemotherapy, 1999, 11(6):530-535.
75
Boyd J, Covington T, Stanaszek W, Coussons T. Drug defaulting, part II. Analysis of non-compliance patterns, Am J Hosp Pharm,
1974, 31:485-91.
76
Simkins CV, Wenzloff NJ, Evaluation of a computerized reminder system in the enhancement of patient medication refill
compliance, Drug Intelligence and Clinical Pharmacy, 1986, 20(Oct):799-802.
77
Duxbury B. Therapeutic control of anticoagulant treatment, Br Med J, 1982, 284:702.
78
Poller L, Wright D, Rowland M, Prospective comparative study of computer programs used for management of warfarin, Journal
of Clinical Pathology, 1993, 46:299-303.
79
Mungall DR, Anbe D, Forrester PL, Luoma T, Genovese R, Mahan J, LeBlanc S, Penney JB, A prospective randomized
comparison of the accuracy of computer-assisted versus GUSTO nomogram-directed heparin therapy, Clinical Pharmacology and
Therapeutics, 1994, 55:591-596.
80
White RH, Mungall D, Outpatient management of warfarin therapy: comparison of computer-predicted dosage adjustment to skilled
professional care, Therapeutic Drug Monitoring, 1991, 13(1):46-50.
81
Fitzmaurice DA, Hobbs FD, Richard, Murray ET, Bradley CP, Holder R, Evaluation of computerized decision support for oral
anticoagulation management based in primary care, Br Jour Gen Pract, 1996, 46:533-535.
82
Vadher B, Patterson DLH, Leaning M. Evaluation of a decision support system for initiation and control of oral anticoagulation in a
randomized trial, BMJ, 1997, 314:1252.
83
Lazarou J, Pomeranz B, Corey P, Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective
studies, JAMA, 1998, 279: 1200-5.
84
Classen DC, Pestonik SL, Evans RS, et al., Adverse drug events in hospitalized patients, JAMA, 1997, 277(4):301-6.
85
Cullen DJ, Sweitzer BJ, Bates DW, et al., Preventable adverse drug events in hospitalized patients: a comparative study of intensive
care and general care units, Crit Care Med., 1997, 25(8):1289-97.
86
Leape LL, Error in medicine, JAMA, 1994, 272: 1851-1857.
87
Raschke RA, et al., A computer alert system to prevent injury from adverse drug events: development and evaluation in a
community teaching hospital, JAMA, 1998, 280:1317-20.
88
Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, Burdick E, Hickey M, Kleefield S, Shea B, Vander Vliet M,
Seger DL, Effect of computerized physician order entry and a team intervention on prevention of serious medication errors,
JAMA, 1998, 280(15):1311-1316.
89
Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, Boyle D, Leape L, The impact of computerized physician order
entry on medication error prevention, J Amer Med Informatics Assoc, 1999, 6(4):313-321.
90
McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, McCabe GP, Reminders to physicians from an
introspective computer medical record: a two-year randomized trial, Annals of Internal Medicin, 1984, 100:130-138.
91
Litzelman DK, Dittus RS, Miller ME, Tierney WM, Requiring physicians to respond to computerized reminders improved their
compliance with preventive care protocols, J Gen Intern Med, 1993, 8(June):311-317.
92
Overhage JM, Tierney WM, Zhou X, McDonald CJ, A randomized trial of "corollary orders" to prevent errors of omission, J Amer
Med Informatics Assoc, 1997, 4(5):364-375.
93
Apkon M, Singhaviranon P, Impact of an electronic information system on physician workflow and data collection in the intensive
care unit, Intensive Care Medicine, 2001, 27:122-130.
25
94
Nahm R, Poston I, Measurement of the effects of an integrated, point-of-care computer system on quality of nursing documentation
and patient satisfaction, Computers in Nursing, 2000, 18(5):220-229.
95
Sullivan F, General practitioner computing made a difference to patient care?, A systematic review of published reports, BMJ, 1995,
311:848-52.
96
Nahm R, Poston I, Measurement of the effects of an integrated, point-of-care computer system on quality of nursing documentation
and patient satisfaction, Computers in Nursing, 2000, 18(5):220-229.
97
Whiting-O'Keefe QE, Simborg DW, Epstein WV, Warger A, A computerized summary medical record system can provide more
information than the standard medical record, JAMA, 1985, 254(9):1185-1192.
98
Hammond J, Johnson H, Varas R, Ward C, A qualitative comparison of paper flowsheets vs. a computer-based clinical information
system. Chest, 1991, 99:155-157.
99
Tang PC, LaRosa MP, Gorden SM, Use of computer-based records, completeness of documentation, and appropriateness of
documented clinical decisions, J Amer Med Informatics Assoc, 1999, 6(3):245-251.
100
A Health Information Network for Australia: Report to Health Ministries by the National Electronic Health Records Taskforce,
July 2000
101
Tarczy-Hornoch, et al., Meeting clinician information needs by integrating access to the medical record and knowledge resources
via the Web, Proc AMIA Annu Fall Symp, 1997, 809-13.
102
Tate KE, Gardner RM, Weaver LK, A computerized laboratory alerting system, M.D. Computing, 1990, 7(5):296-301.
103
Tierney et al., Physician inpatient order writing on microcomputer workstations: effects on resource utilization, JAMA, 1993,
269.379-83.
104
Office visit utilization information provided by Kaiser Permanente in response to a survey of CIS benefits conducted in October
2001 by the Kaiser Permanente Institute for Health Policy.
105
Fihn SD, McDonell MB, Vermes D, Henikoff JG, Martin DC, Callahan CM, Kent DL, White RH, for the National Consortium of
Anticoagulation Clinics, A computerized intervention to improve timing of outpatient follow-up: a multicenter randomized trial in
patients treated with warfarin, J Gen Intern Med, 1994, 9(March):131-139.
106
Tierney, et al., Computer predictions of abnormal test results, JAMA, 1988, 259:1194.
107
Tierney WM, Miller ME, McDonald CJ, The effect on test ordering of informing physicians of the charges for outpatient
diagnostic tests, NEJM, 1990, 322:1499-504.
108
Levit K, et al. Health spending in 1998: signals of change, Health Affairs, 2000, 19(1):124-132.
109
Interview with Anthony Barreuta, Government Relations, Kaiser Permanente. September 21, 2001.
110
Karson AS, Kuperman GJ, Horsky J, Fairchild DG, Fiskio JM, Bates DW, Patient-specific computerized outpatient reminders to
improve physician compliance with clinical guidelines, JGIM, 1999, 14(April supplement 2):126.
111
Donald JB, Prescribing costs when computers are used to issue all prescriptions, British Medical Journal, 1989, 299:28-30.
112
Garrett Jr LE, Hammond WE, Stead WW, The effects of computerized medical records on provider efficiency and quality of care,
Methods of Information in Medicine, 1986, 25(3):151-157.
113
Safran C, Rury C, Rind DM, Taylor WC, A computer-based outpatient medical record for a teaching hospital. M.D. Computing,
1991, 8(5):291-299.
114
Renner K, Cost-justifying electronic medical records, Healthcare Financial Management, 1996, 50(10):63-64, 66, 68, 70.
115
Evans J and Hayashi A, Implementing on-line medical records, Document Management, Sept/Oct 1994, 12-17.
116
Medical record cost savings estimates were provided by Kaiser Permanente in response to a survey of CIS benefits conducted in
October 2001 by the Kaiser Permanente Institute for Health Policy.
117
Renner K, Cost-justifying electronic medical records, Healthcare Financial Management, 1996, 50(10):63-64, 66, 68, 70.
118
Chart pull cost savings estimates were provided by Beth Israel Deaconess Medical Center, Boston in response to a survey of CIS
benefits conducted in October 2001 by the Kaiser Permanente Institute for Health Policy.
119
Medical record cost savings estimates were provided by Kaiser Permanente in response to a survey of CIS benefits conducted in
October 2001 by the Kaiser Permanente Institute for Health Policy.
120
Bleich HL, Safran C, Slack WV, Departmental and laboratory computing in two hospitals, M.D. Computing, 1989, 6(3):149-155.
121
Information provided by Beth Deaconess Medical Center in response to a survey of CIS benefits conducted in October 2001 by the
Kaiser Permanente Institute for Health Policy.
122
Monalto W, Using referral letters to measure quality and performance in general practice, J Qual Clin Pract, 1995, 15(1):45-50.
123
Sittig DF, Gandhi TK, Franklin M, Turetsky M, Sussman AJ, Fairchild DG, Bates DW, Komaroff AL, Teich JM, A computerbased outpatient clinical referral system, International Journal of Medical Informatics, 1999, 55:149-158.
124
Information provided by the Mayo Clinic in response to a survey of CIS benefits conducted in October 2001 by the Kaiser
Permanente Institute for Health Policy.
125
Soule D, Handler S, Simplifying data entry in electronic health record systems, Conference proceedings-towards and electronic
health record Europe, 1999, Center for Advancement for Electronic Records.
26
126
Information provided by the Mayo Clinic in response to a survey of CIS benefits conducted in October 2001 by the Kaiser
Permanente Institute for Health Policy.
127
Nahm R, Poston I, Measurement of the effects of an integrated, point-of-care computer system on quality of nursing documentation
and patient satisfaction, Computers in Nursing, 2000, 18(5):220-229.
128
Siders AM, Perterson M, Increasing patient satisfaction and nursing productivity through implementation of an automated nursing
discharge summary, Proceedings of the Annual Symp Comp Appl Med Care, 1991, 136-140.
129
Happ MA, The effect of point of care technology on the quality of patient care, Proceedings of the 17th Annual Symp Comp Appl
Med Care, 1993: 183-187.
130
Survey results provided by the Mayo Clinic in response to a survey of CIS benefits conducted in October 2001 by the Kaiser
Permanente Institute for Health Policy.
131
Anecdotal information provided by the Palo Alto Medical Foundation in response to a survey of CIS benefits conducted in October
2001 by the Kaiser Permanente Institute for Health Policy.
132
Balas EA, et al., The columbia registry of controlled clinical computer trials, Proc Annu Symp Comput Applic Med Care, 1993:
16: 220-224.
133
Millenson M, Demanding medical excellence, 1997, Chicago: University of Chicago Press.
134
Marhula DC, Clinically oriented solutions in healthcare, Healthcare Informatics, February 2001.
135
ibid.
136
Sittig DF, Sengupta S, Payne T. The clinical information architect as a member of the healthcare team, www.asca.com/updates/16/24.htm.
137
12th Annual HIMSS Leadership Survey, Sponsored by Superior Consultant Company and Dell Computer Corporation, 2001.
138
Tang PC and McDonald CJ, Computer-based patient-record systems, Medical Informatics: Computer Applications in Health Care
and Biomedicine, Second Edition, 2001.
139
Watters D, Can clinical information systems do the job for a reasonable price? Professor of Surgery, Barwon Health, The Geelong
Hospital, Geelong, Victoria 3220, Australia (presented at TEPR 2001 conference).
140
Rudi Rodsjo S, Coping with insecurities, Healthcare Business, May 2001.
141
Second annual survey of electronic health record trend and usage, Medical Record Institute, 2000.
142
Waegemann CP, Standards inventory for health informatics, e-healthrecord.news Volume 2, Number 3, March-April 2001.
143
Stead WW, Sittig DF, Integrated advanced information management systems: strategies and architectures for fast track
development, in hospital information systems: design and development characteristics; Impact and future architecture, eds.
Prokosch HU, Dudeck J. Elsevier, Amsterdam, The Netherlands, 1995.
144
Morrissey J, High on tech, low on budget, Modern Healthcare, January 28, 2002.
145
LeGrow G, Metzger J, E-disease management, Prepared for California HealthCare Foundation by First Consulting Group,
November 2001.
146
The National Committee on Vial and Health Statistics, Workgroup on the National Health Information Infrastructure, Towards a
national health information infrastructure, Interim Report, June 2000.
147
National Committee on Vital and Health Statistics, Information for health: a strategy for building the national health information
infrastructure, November 15, 2001.
148
Institute of Medicine, The computer based patient record: An essential technology for health care, 2000, 173.
149
CPRI-HOST, CPRI toolkit: managing information security in health care, Vol 3.
150
Institute of Medicine, Crossing the quality chasm: A new health system for the 21st century, 2001.
151
Medical Technology Leadership Forum, Reimbursement for clinical information technologies, 2001.
152
Balas EA, et al., The columbia registry of controlled clinical computer trials, Proc Annu Symp Comput Applic Med Care, 1993,
16: 220-224.
27