Journal of the American Medical Informatics Association Volume 9 Number 3 May / Jun 2002
Implementation Brief
209
■
Desiderata for Personal
Electronic Communication
in Clinical Systems
INDRA NEIL SARKAR, BSC, JUSTIN STARREN, MD, PHD
Abstract
Electronic communication among clinicians and patients is becoming an essential
part of medical practice. Evaluation and selection of these electronic systems, called personal
clinical electronic communication (PCEC) systems, can be a difficult task in institutions that have
no prior experience with such systems. It is particularly difficult in the clinical context. To directly
address this point, the authors consulted a group of potential users affiliated with a nationally
recognized telemedicine project, to determine important characteristics of a hypothetical PCEC
system. They compiled a list of these characteristics and produced a desiderata, or list of desired
features, for PCEC systems. Two conventional e-mail implementations and three Web-based PCEC
systems were evaluated with respect to the features. The Web-based systems all scored higher than
conventional e-mail. It is the hope of the authors that this paper will initiate further discussions
about the features of PCEC systems and how to evaluate them.
■
J Am Med Inform Assoc. 2002;9:209–216.
Communication among care providers and between
care providers and their patients is an essential component of medicine.1 Systems that utilize information
flow across a modern electronic medium, such as the
Internet, have been developed to facilitate communication among various persons involved in patient
care, such as clinicians, administrators, and patients.
These systems are the latest trend in health care’s
quest to improve the transfer of clinical information.
A recent MEDLINE search on the term “e-mail” produced more than 200 references.
Such systems do not yet have a universally accepted
name. Labels range from the very narrow “Webbased doctor–patient communication services”2 to
Affiliation of the authors: Columbia University College of
Physicians and Surgeons, New York, New York.
This work was supported by cooperative agreement 95-C-90998
from the Health Care Financing Administration, medical informatics training grant LM 07079-09 from the National Library of
Medicine, funding from the New York State Office of Science,
Technology & Academic Research, and a grant from Lifescan.
Correspondence and reprints: Justin Starren, MD, PhD, Medical
Informatics, VC-5, 622 W. 168th St., New York, NY 10032; e-mail:
<[email protected]>.
Received for publication: 10/15/01; accepted for publication: 1/14/02.
the broader “electronic communication with
patients,”3 “physician practice–patient communication network,”4 and “electronic patient-centered
communication system”5 to the very nonspecific
“electronic messaging.”6 None of these terms conveys both the clinical content of the communication
and the frequent use of such systems for direct
provider–provider communication. To capture these
characteristics, we chose “personal clinical electronic
communication” (PCEC) as the best descriptor.
Medicine is historically quick to adapt new technologies to facilitate communication.7 There are many
anecdotal stories of health-care professionals using
whatever means they have available to communicate
medically relevant items. For example, in the 1800s,
clinicians tied yellow kerchiefs on patients’ doors to
communicate that the patient residing in that house
was infected with a communicable disease and was
in quarantine.7 In modern times, care providers often
use the telephone to communicate with their
patients, to consult or share information with colleagues, or to arrange many aspects of care.6,8–10
Many health care providers already use some form of
electronic communication, such as electronic mail or
instant messaging.6,11–14
210
SARKAR, STARREN, Personal Clinical Electronic Communication
An individual practice or institution attempting to
select a PCEC system has a bewildering array of
options, including conventional e-mail, messaging
components embedded in electronic medical record
systems, and specialized stand-alone systems.
Studies have explored guidelines for the use of electronic mail and other electronic communication with
patients3,11,15,16 and the various other benefits of technical programs and their effects on patient
care.8–10,12,14,17–19 Typically, these studies have
focused on a specific messaging technology,11,12,15
the clinical needs arising from a specific disease,1,8–10,14,17,18,20 or anecdotal experience with a specific system.3,6,19 Formal studies addressing the necessary features of a PCEC system that functions
across diseases and specialties are lacking.
The Informatics for Diabetes Education and Telemedicine (IDEATel) project is a large home-telemedicine project that uses informatics technology to bring
maintenance care into the homes of patients who have
diabetes mellitus.21,22 A PCEC system was essential to
IDEATel as a means for clinicians to communicate
with one another and with their patients. Early in the
selection process, it became clear that clinicians were
not in unanimous agreement on the features of an
ideal PCEC system. It also became clear that no current PCEC implementation contained all the features
desired by even one of the clinicians. To rationalize
the process, it was necessary to develop a prioritized
desiderata (a list of desired features). This paper
describes the process of creating the feature list, the
resulting desiderata, and the use of the features list to
evaluate several existing PCEC systems.
Methods
We used an electronic-mail version of the Delphi
technique.23 This was done in part because of our
panelists’ disparate geographic locations and the
convenience of electronic mail messaging.23 We contacted 16 people who were affiliated with the
IDEATel project; 10 (one nurse, one administrator,
and eight physicians) agreed to participate. These 10
people made up our expert panel. Four of the panelists had significant experience in medical informatics, in that they were either faculty in informatics
departments or were actively involved in informatics
projects. Neither author was a member of the panel.
The study was part of a system selection process. It
was begun prior to selection of the IDEATel PCEC
system and was completed after selection, but prior
to installation, of the system.
The elicitation process began with a request that each
panelist, independent of all the others, present a list
of features that he or she believed were important in
the selection of a PCEC system, based on their personal experience. To maximize the generalizability of
the resulting features, we asked the panelists to consider general clinical practice and to not limit themselves to the context of the IDEATel project. We also
asked the panelists to not worry about any of the
technical limitations that may currently exist for
selection of what they thought of as an ideal system.
For this and each subsequent round, the panelists
had two weeks to respond, and all responded on time
for all rounds. We intentionally did not “seed” the
list with features from existing products or prior published guidelines. The goal was to product a usercentric list that could be compared with lists derived
by different methods.
We combined all the responses into a single, unaccredited list, which we then returned to all the panelists. We asked the panelists to add new features—
and refine any of their own features—that they felt
needed to be included on the list. We recompiled the
results and again returned them to the panel.
During the third round, we asked the panelists to
rank the features. Using a ten-point Likert scale,24
ranging from 5 (most detrimental) to 5 (most
essential), they were asked to quantify each feature in
the set in terms of its benefits or detriments to a hypothetical, ideal PCEC system.
In the fourth round, we sent the panelists a list that
included, for each feature, a mean score and that
panelist’s individual score; we asked them to adjust
their individual scores if they wished, based on the
overall means. As is standard in the Delphi method,
direct discussion among panel members was discouraged, to mitigate the influence of forceful or
highly respected persons. After we updated the
mean scores, we again sent the list to each panelist,
with the mean and most recent individual score and
a request that he or she update the score, if desired.
No panelist changed his or her individual score in
this round, so the collection process ended.
We removed duplicate items from the final list. The
criteria for concluding that two items were duplicates
included not only that they appeared to describe
the same feature but also that the were scored the
same by the panel. Remaining features were grouped
into thematic categories. Within each category, the
items were ranked in descending order first by mean,
then by standard deviation. The categories were then
Journal of the American Medical Informatics Association Volume 9 Number 3 May / Jun 2002
211
collectively ranked also by mean and standard
deviation.
Results
Further inspection of the feature set lead to the development of three categories of features, based on
mean rank and inter-rater agreement. Inter-rank
agreement was ascertained by examining the standard deviations of each feature. The “critical” features shared a mean greater than or equal to 3.5 and
a minimum score greater than or equal to 0. The
equivocal features shared mean less than 2.25 and a
minimum score less than 0. Between these two
groups are the “desirable” features.
In the initial round of the Delphi method, our panelists submitted 29 criteria; they added 5 criteria in
the second round. We then removed two duplicate
entries, for a total of 32 criteria. These criteria were
sorted into 21 categories, which are shown in Table 1.
We made no attempt to enforce uniform syntactic
structure on the submitted features; i.e., some features were phrased in the positive (“Strong
Authentication) and some in the negative (“No
SPAM”). We considered the effect of such phrasing
on the scoring by looking at the relative scoring of
positive and negative features. In particular, we
looked for phrases in which a statement was negatively set and then the grouping it was placed in.
Because of the nearly bimodal distribution of scores
for one negatively phrased feature, “Recipient
Disclosure,” we suspected possible misunderstanding. We contacted our panel for written descriptions
of their answers. Eight panelists responded, and the
“Recipient Disclosure” scores were recalculated on
the basis of just these responses.
The features and associated scores were converted to
a numeric evaluation method for PCEC systems. The
evaluation method was based on two components—
a weighting value, based on the median score from
the panel, and an implementation value from 0 to 1.
Each feature was individually considered for a candidate PCEC system. If a feature was fully supported,
an implementation score of 1 was assigned. Those
that were partially supported were given an implementation score of 0.5. The score for each feature of
the candidate system was determined by the implementation score (0 to 1) multiplied by the weighting
score (0 to 5). Only critical and desirable features
were used for the score.
As a proof of concept, we applied the evaluation to
two potential implementations of conventional electronic mail and to three existing Web-based PCEC
systems—Healinx2 (Healinx Corporation, Alameda,
California), the PCEC supplied by the American
Medical Association to its members4 (Medem Inc.,
San Francisco, California), and the PCEC system
selected for the IDEATel project (Siemens Health
Services, Malvern, Pennsylvania).
A total of 318 scores were received for the 32 criteria,
across the full possible range (5 to 5). The overall
mean, median, and mode were 3.3 ± 0.7 (SD), 4, and 5,
respectively. Using the rank definitions and the interrater agreements, our final list consisted of 16 critical
features (mean, 4.3 ± 1.0), 13 desirable features (mean,
2.7 ± 1.8), and 3 equivocal features (mean, 1.5 ± 2.1).
The scoring for each feature is summarized graphically in Figure 1.
The numeric evaluation based on the feature list
revealed that conventional e-mail scored between 19
and 33 percent overall. The range of scores for conventional e-mail was dependent on the number of
features that were available for different implementations of e-mail. This scoring assumed partial credit
for features, like digital signature, that are not supported by all e-mail readers. Without these features,
the electronic mail scores dropped to 19 percent, for
both the critical and overall scores. In comparison, all
three Web-based PCEC systems scored markedly
better. Scores ranged between 43 and 62 percent for
all features and between 49 and 75 percent for critical
features.
Discussion
The most striking finding was the level of consensus
among panelists in different specialties from different institutions. Across all the features, the average
standard deviation of the group was 1.3 units on a 10point scale. This consensus was especially strong for
the 16 features in the “critical” group (1.1 to 1.6, 2 to
8.3, and 9.1), in which the average standard deviation
was 1.0. Nine features had median scores of 5.
Further evidence of consensus was the fact that no
feature submitted by an individual panelist was
ranked as undesirable (average score less than 0) by
the panel. In spite of this agreement among clinicians, the highest-rated PCEC system scored only
75 percent for these features. More important, conventional e-mail, which is probably the most widely
used PCEC tool today, scored only 40 percent for critical features in an optimal implementation and only
19 percent in a more typical implementation.
212
Table 1
SARKAR, STARREN, Personal Clinical Electronic Communication
■
Desiderata for Personal Clinical Electronic Communication
Item
Sub-item
1
Median
Mean ± SD
Description
5
5 ± 0.7
1.1
5
4.9 ± 0.3
Backups—Backups should be maintained such that adequate and effective
recoverywill be efficient and accurate.
1.2
5
4.8 ± 0.4
Audit Trail—A record should be kept of every time anyone writes, reads,
forwards, or responds to a message. This should automatic, regardless of what
record was record was accessed or by whom.
1.3
5
4.8 ± 0.4
Encryption—All messages should be stored and transported in an appropriate
manner such as to deter unwanted intrusion.
1.4
5
4.8 ± 0.4
Security Policy—A formal security policy should be established and implemented.
1.5
5
4.8 ± 0.6
Strong Authentication—Secure methods should be implemented such that
authorization is restricted to appropriate users. This will include utilization
of user password validation, and may also incorporate techniques such as
digital signatures, cryptography, secureID, etc.
1.6
5
4.0 ± 1.3
Digital Signature—Through the use of software tools, a digital “signature”
should be used to sign every message indicating who the message is from.
Every user will be assigned a unique encryption key, such as a large number
carried on a smart card. This key is then used to uniquely identify any message
that is sent by that person.
1.7
3.5
3.2 ± 1.9
Automatic Timeout—The user interface should be such that a timeout mechanism
is in place to prevent unwanted use due to negligence.
2
5
4.7 ± 0.5
Cross Coverage—Communications should be in place such that any lack of response
should be automatically forwarded to another caregiver so that the issue can be
dealt with
3
5
4.65 ± 0.5
E-mail Directing—The system should be able to direct messages to more than one
addressee, identified as “To:” and “CC:,” in the message header.
4
4.8
4.35 ± 0.8
Footprint/Thin Client—The communication system should not require any special
software for accessing messages. Messages will be dealt with through commonly
available applications such as a Web browser.
5
5
4.1 ± 1.2
Security/Reliability
No SPAM—Unsolicited e-mails (SPAM) should be kept to a minimum.
6
4.5
4.0 ± 1.6
Convenient—The user interface should be easy to use and not foreign to the users.
7
4
3.9 ± 1.0
Copy-and-paste Features—The system should allow pasting from the Windows clipboard
to facilitate the addition of pertinent information into a message.
8
3.5
3.7 ± 1.1
Receipt System
8.1
4.5
3.9 ± 1.4
Receipt Record—A record of whether the recipient has received an e-mail and
responded to it in a timely fashion—i.e., a temporal tracking system with a
reminder to the appropriate party (or parties)—will be important if the volume
of messages is high.
8.2
3.5
3.7 ± 0.8
Receipt Management System—There should be a centrally located system that
maintains a record of the status of each message, such as whether the message has
been “received,” “received and read,” or “read and responded.”
8.3
3.5
3.6 ± 1.2
Acknowledgment—Senders should know that receivers have not only received but
have read and responded to an e-mail. Acknowledgments need to be managed so
that at any given time the status of all messages sent, whether read or not, can be
seen.
3.5
2.9 ± 1.5
4
3.8 ± 1.0
9
9.1
Escalation
Triage/Automatic Escalation—Urgent items should be automatically routed to the
appropriate person so that they can be processed in an expedited fashion. In the
event of no response, the message should be sent to someone else, until the issue
contained in the message is dealt with.
continued
Journal of the American Medical Informatics Association Volume 9 Number 3 May / Jun 2002
Table 1
213
■
Desiderata for Personal Clinical Electronic Communication, continued
Item
Sub-item
Median
Mean ± SD
9.2
3
2.0 ± 1.9
10
3.5
3.4 ± 1.4
Attachments—The system should be able to handle encoded attachments for file
transfers between different entities.
11
4.5
3.4 ± 2.1
Recipient Disclosure/No BCC (blind copying)—The system should not allow blind copies;
all users to whom a message is sent to should be made known.
12
3.5
3.2 ± 1.6
Alerting Reminders—The system should be able to individualize alerts and should have
a reminder system on both the patient and the provider sides.
13
3
3.2 ± 1.0
Group Addressing—The ability to send messages to various groups (such as all the
patients of Dr. C) should be in place to create a more efficient atmosphere for
communication.
14
3
2.9 ± 1.5
Threading Multiple Messages of Same Case—This is the ability to have threaded messages,
so that particular issues can be dealt with in a linear fashion. That is, when somone
replies to a message, the reply is linked to the original message. This allows for ease in
the review of related messages.
15
3
2.6 ± 1.3
Structured Messaging
15.1
3
2.6 ± 1.3
Beyond Standard Text—This is the ability to go beyond standard e-mail format, such
as structured messaging.
15.2
3
2.6 ± 1.3
Check Lists—Check lists give users options to respond to a query rather than
write a narrative account.
16
2.5
2.3 ± 1.6
Chat/Instant Communication—A feature of instant communication, possibly with other
members of a health care team, in the care of a particular patient.
17
2
2.1 ± 1.2
Calendaring—This function would assist in the scheduling of events by use of the
messaging system.
Prioritization
18
Description
Due-date Escalation—As a particular date approaches, the level of priority increases
proportionally.
1.8
1.4 ± 2.7
18.1
3.5
2.2 ± 3.1
Levels of Importance—Each message should be clearly established at a level of
importance set by the user.
18.2
0
0.6 ± 2.2
Priority Preservation—The priority of a sent message should maintain its level of
given priority throughout a given thread
19
2
1.9 ± 2.0
Separation of Roles—The accounts of certain persons should be maintained separately
from accounts for their roles, to allow for the maintenance of better audit trails.
20
1
1.0 ± 1.7
Asynchronous Alerting—A mechanism should be in place that gives notification out of
band; e.g., a page is sent each time a message is received.
21
0.5
0.6 ± 2.2
VIP Messaging—Messages of high importance should be given individual passwords
to add another level of security.
NOTE: Features are ordered by mean score. When several features are grouped into a single item, the overall mean is used for ordering and
sub-items ordered within the item.
Although the number of implementations evaluated
was limited, every Web-based PCEC evaluated outscored conventional e-mail.
These features are not the first collection of recommendations for electronic messaging. The features
selected by the panel compare closely with those
included in previous guidelines on e-mail use,
including the importance of message archiving,
encrypted transport, and documentation of when
messages are actually read.11
Of the 16 criteria ranked as critical in our study, 13
were also relevant to the HIPAA requirements.25
These criteria (1.1 to 5, and 8.1 to 8.3) all deal with the
security and reliability of messaging between various
health care players, such as clinicians among themselves and with their patients. This was a reflection of
214
SARKAR, STARREN, Personal Clinical Electronic Communication
F i g u r e 1 Scores for features of
personal clinical electronic messaging systems. Features are ordered
vertically by mean score. The order
of features is not completely monotonic, because of thematic grouping
of features, as shown in Table 1.
Center diamond indicates median
score; gray bar, one quartile above
and below the median; thin whiskers, range of scores.
how the critical elements, on the whole, are essential
to a PCEC system.
Strikingly absent from the list of features are many
that are typically discussed in relation to Web-based
sites for patients. These include statements regarding
the limits of liability3 or limits on the types of messages that are permitted.16 Also absent are features
relevant to developers and implementers. For example, there is no mention of open application program
interfaces or the use of standards, even though both
are critical for the integration of PCEC into complex
clinical computing environments.22
Similarly, disclaimers and controls to limit legal liability are not mentioned.16 This is because these features reflect the views of individual clinical users
rather than the views of developers or institution
administrators. It also reflects the focus on messaging
among individuals, not on broad communication
(e.g., Web sites). As a result, these features complement, but are not a substitute for, good system and
user interface design. The requirements of local clinical practice, pre-existing electronic medical record
systems, HIPAA, and other regulations (such as federal Section 508 guidelines26) will influence the final
design of a PCEC implementation.
Although this list of features represents a start at
addressing criteria that should be used in the selection of a PCEC system, this study is not without its
limitations. The number of panelists included in the
study was small. With such a small sample, it is conceivable that areas of disagreement were not fully
expanded and areas of agreement were not properly
extracted from the data.
The panelists came from two separate institutions but
were all affiliated with a similar project, and most
had some experience with electronic medical record
Journal of the American Medical Informatics Association Volume 9 Number 3 May / Jun 2002
systems. However, because the IDEATel PCEC was
not functional at the time of this survey, this is not
likely to have induced significant bias. Also, this
study focused on clinician users. Further studies will
need to assess the preferences of other groups, such
as patients, vendors, and other non-clinician users.
Finally, in the numeric evaluation, the weighting values for individual features may not translate directly
to the aggregate value of classes of features. For
example, the large number of security features (6 of
16 critical features) may lead to overestimation of the
aggregate importance of security in PCEC systems.
The “critical” features may be the most important for
system developers. However, equally interesting are
the “desirable” and “equivocal” features, which
elicited more disagreement among the panel. The
desirable group (features 1.7, 9 to 17, and 18.1) is distinguished from the critical group by both lower
mean scores and higher average deviations. Compared with the critical features, these features share
the characteristic that their value to a clinician may be
dependent on the functionality of other computer
systems at the institutions.
For example, a calendaring system may be essential
at one institution but of no particular use at another,
where an efficient scheduling system is already in
place. Structured messaging, which provides templates for certain message types (e.g., medication
renewal and appointment request), could act as a
surrogate for a forms or order entry system if one did
not already exist. Threaded message viewing would
allow the messages pertaining to a specific patient to
act almost as an electronic medical record. Thus, the
importance of these features could be expected to
vary among individuals and among institutions.
Institutions selecting PCEC systems will want to pay
particular attention to those features that receive both
beneficial and detrimental ratings from the panel. This
group included all three equivocal features, as well as
five features from the desirable group, including automatic timeout, due-date escalation, chat, levels of
importance, priority preservation, separation of roles,
asynchronous alerting, and VIP messaging. Each of
these features was also rated 4 or 5 by at least one
panelist, indicating that some panelists felt these features to be highly desirable. One feature, levels of
importance, received scores ranging from 5 (must
have) to 5 (must never have). This level of disagreement did not decrease in subsequent Delphi rounds.
Overall, these features tended to involve additional
effort, interaction, or liability by clinicians in the way
215
they practice medicine. Both chat and asynchronous
alerting involve the use other communication means,
such as paging or pop-up windows, that may interrupt the clinician. Paging systems are often seen as the
bane of life by hospital clinicians, and many clinicians
may be hesitant to permit computers to page them
directly. Computerized paging may also affect clinical
care, when the sensitivity is set too high and an excessive number of alerts are sent to clinicians.27
Four features in this subset involved the creation of
special high-priority messages (due-date escalation,
levels of importance, priority preservation, and VIP
messaging). Levels of importance and priority preservation involve the ability of a sender to flag a message
as high priority and retention of the flagging by the
reply message. Most clinicians have encountered
coworkers who label everything “STAT.” Consequently, some may wish to limit that behavior in a
PCEC system.
With the VIP feature, messages pertaining to persons
designated VIPs are associated with additional security and auditing procedures. In practice, most hospitals already have special procedures for protecting
the security of both the paper and the electronic
records of VIP patients. These may include admitting
the patient under an alias or removing the name label
from the paper chart. This practical need conflicts
with the ethical imperative to treat all patients equally, regardless of wealth or status.
The case of receipt disclosure/no BCC (blind copying) is also of note. Originally, the panel generated a
nearly bimodal distribution of strongly positive and
negative scores. Subsequent queries to the panel
revealed that there was confusion about whether a
negative score meant that blind copying was bad or
that preventing blind copying was bad. After clarification, the scores still had a somewhat bimodal distribution, but they were now clustered about 0 and
5. Panelists who gave high scores indicated that full
disclosure was imperative if patients were to trust the
system. Panelists who gave low scores indicated a
conflict between the need for disclosure and the fear
that full disclosure might also disclose patient names
and thereby violate patient privacy.11 This is an
example of a feature that is very different when
viewed from clinician and patient perspectives.
It is clear from these desiderata that no single PCEC
system will satisfy all the wants and needs of all clinicians or institutions. However, they also show that
a high level of consensus about the critical features
that should be included in future PCEC systems.
216
References
SARKAR, STARREN, Personal Clinical Electronic Communication
■
1. Shapiro S. Screening: assessment of current studies. Cancer.
1994;74(1:suppl):231–8.
2. Healinx at a glance. Healinx Corporation Web site. 2002. Available at: http://www.healinx.com/Corporate/v/Overview/
index.asp. Accessed Mar 28, 2002.
3. Prady SL, Norris D, Lester JE, Hoch DB. Expanding the guidelines for electronic communication with patients: application
to a specific tool. J Am Med Inform Assoc. 2001;8(4):344–8.
4. Medem Corporation. Medem Messaging Web site. Available
at: http://www.yourpracticeonline.com/ypolbuilder/splash_
faq.jsp. Accessed Mar 29, 2002.
5. Sands DZ. Electronic patient-centered communication resource center. Available at: http://www.e-pcc.org. Accessed
Mar 29, 2002.
6. Moorman PW, Branger PJ, van der Kam WJ, van der LJ.
Electronic messaging between primary and secondary care: a
four-year case report. J Am Med Inform Assoc. 2001;8(4):372–8.
7. Darkins AW, Cary MA. Telemedicine and Teleheath: Principles,
Policies, Performance, and Pitfalls. New York: Springer, 2000.
8. Leggett-Frazier N, Swanson MS, Vincent PA, Pokorny ME,
Engelke MK. Telephone communications between diabetes
clients and nurse educators. Diabetes Educ. 1997;23(3):287–93.
9. Ahring KK, Ahring JP, Joyce C, Farid NR. Telephone modem
access improves diabetes control in those with insulin- requiring diabetes. Diabetes Care. 1992;15(8):971–5.
10. Meneghini LF, Albisser AM. An electronic case manager for
diabetes control. Diabetes Care. 1998;21(4):591–6.
11. Kane B, Sands DZ. Guidelines for the clinical use of electronic
mail with patients. J Am Med Inform Assoc. 1998;5(1):104–11.
12. Costello R, Shaw A, Cheetham R, Moots RJ. The use of electronic mail in biomedical communication. J Am Med Inform
Assoc. 2000;7(1):103–5.
13. Strode SW, Gustke S, Allen A. Technical and clinical progress
in telemedicine. JAMA. 1999;281(12):1066–8.
14. Kinsella A. Home telehealthcare, disease management and
diabetes: a community effort for success. Remington Report.
1999;7(3):16–9.
15. Murphy G. Patient-centered e-mail: developing the right policies. J AHIMA. 2000;71(3):47–54.
16. eRisk Working Group for Healthcare. eRisk for Providers:
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
Understanding and Mitigating Provider Risk Associated with
Online Patient Interaction. Medem Web site. Available at:
http://www.medem.com/level2/downloads/eRisk_for_Provid
ers_Mar_01.pdf. Accessed Oct 4, 2001.
Marrero DG, Vandagriff JL, Kronz K, et al. Using telecommunication technology to manage children with diabetes: the
Computer-Linked Outpatient Clinic (CLOC) Study. Diabetes.
1995;21(4):313–9.
Gomez EJ, del PF, Hernando ME. Telemedicine for diabetes
care: the DIABTel approach towards diabetes telecare. Med
Inform. 1996;21(4):283–95.
Balas EA, Jaffrey F, Kuperman GJ, et al. Electronic communication with patients: evaluation of distance medicine technology. JAMA. 1997;278(2):152–9.
Hunt DL, Haynes RB, Hayward RS, Pim MA, Horsman J.
Automated direct-from-patient information collection for evidence-based diabetes care. Proc AMIA Annu Fall Symp.
1997:81–5.
Shea S, Starren J, Weinstock RS, et al. Columbia University’s
Informatics for Diabetes Education and Telemedicine
(IDEATel) project: rationale and design. J Am Med Inform
Assoc. 2002;9(1):49–62.
Starren J, Hripcsak G, Sengupta S, et al. Columbia University’s
Informatics for Diabetes Education and Telemedicine
(IDEATel) project: technical implementation. J Am Med
Inform Assoc. 2002;9(1):25–36.
Snyder-Halpern R, Thompson CB, Schaffer J. Comparison of
mailed vs. Internet applications of the Delphi technique in
clinical informatics research. Proc AMIA Annu Fall Symp.
2000:809–13.
Likert RA. Technique for the measurement of attitudes. Arch
Psychol. 1932;140:1–55.
The Health Insurance Portability and Accountability Act of
1996. Health Care Financing Administration Web site.
Available at: http://www.hcfa.gov/hipaa/hipaahm.htm.
Accessed Apr 24, 2001.
Rehabilitation Act, Section 508: Electronic and information
technology accessibility standards. Available at: http://www.
access-board.gov/508.htm. Accessed Oct 4, 2001.
Shultz ED, Bauman A, Hayward M, Holzman R. Improved
care of patients with diabetes through telecommunications.
Ann NY Acad Sci. 1992;670:141–5.