International Journal for Quality in Health Care 2004; Volume 16, Number 2: pp. 175–180
10.1093/intqhc/mzh027
Quality assessment program in primary
care clinics: a tool for quality improvement
DROR MANDEL, HOWARD AMITAL, EYAL ZIMLICHMAN, ROBERT WARTENFELD, LILACH BENYAMINI,
TZIPPORA SHOCHAT, FRANCIS B. MIMOUNI AND YITSHAK KREISS
Medical Corps, Israel Defense Forces, Israel
Abstract
Objectives. (i) To describe mechanisms of quality assessment (QA) in IDF primary care clinics; (ii) to compare quality of care
in different types of primary care clinics; and (iii) to test the hypothesis that implementation of the QA program results in
improved quality of care.
Research design. A prospective, single-blinded, uncontrolled, non-randomized study.
Measures. Teams of two physicians carry out the QA process once or twice a year according to clinic size. Five areas were evaluated: (i) physician–patient interaction; (ii) medical chart evaluation; (iii) high-risk patients management; (iv) medical care provided by specialists; and (v) medical staff guidance. Clinics were classiWed in two groups: single-physician clinics (battalion
troop clinics) and multi-physician clinics (home-front base clinics). General Linear Models were used for analysis. A P-value
<0.05 was considered signiWcant.
Results. In 2000 and 2001, 99 primary clinics and 162 primary care physicians were assessed. Seventy-four (45%) physicians
were evaluated twice. Single-physician clinics scored higher than multi-physician clinics on most QA parameters. Physicians
had signiWcantly better QA results at the second encounter, regardless of the type of clinic.
Conclusions. A primary care medicine QA system is feasible in the IDF. It allows for standardized, reliable, and comprehensive assessment of primary care across the military clinics. We postulate that the increase in QA assessment scores from one
examination to the next one indicates an improvement in quality due to the QA program.
Keywords: military medicine, primary care, quality assessment, quality control
Quality assessment (QA) in primary care is a process of
planned activities whose ultimate goal is to achieve a continuous improvement of medical care through the evaluation of
structure, process, and outcome measures [1–4].
The practice of health care in the Israeli Defense Forces
(IDF) is similar in many aspects to that of civilian health systems.
It is essentially based upon primary care clinics, and also upon
secondary specialist centers, and civilian hospitals and facilities.
Yet the military milieu is different from other clinical setups in
various aspects [5–7]: (i) pre-recruiting screening examinations
select draftees that are healthier than the general population;
(ii) there is an abrupt change in lifestyle of young adults
recruited in the army; and (iii) military constraints may conXict
with proper medical decision making. Over the years, the
medical corps has created a primary care system that is highly
accessible and available, leading to an average of seven to
eight visits per soldier per year (according to unpublished
internal IDF data). In this health system, primary care physicians
(PCPs) include board-certiWed physicians in general medicine
(general practitioners) that serve either on active duty or as
civilians employed by the army or as reservists. The differences between these groups might be fundamental with
respect to knowledge, skills, motivation, and the sense of
identiWcation with the medical corps. The characteristics of the
military medical establishment make it mandatory to develop
a comprehensive system able to identify variations in quality
of care and to study the impact of corrective measures.
The purposes of this study were: to describe the mechanisms of QA in the primary care clinics of the IDF; to
describe the results of QA studies conducted from the implementation of the program, on 1 January 2000, to 31 December
2001 in primary care clinics; to compare the quality of care
provided by the different types of primary care clinics and
physicians; and to analyze the preliminary impact of the QA
Address reprint requests to Dror Mandel, MD, Israel Defense Forces, Medical Corps, 3 Ha’Emek Street, Ramat Hasheron
47203, Israel. E-mail: [email protected]
International Journal for Quality in Health Care vol. 16 no. 2
© International Society for Quality in Health Care and Oxford University Press 2004; all rights reserved
175
Downloaded from http://intqhc.oxfordjournals.org/ at Pennsylvania State University on March 3, 2014
Background. Assessment of quality of health care is a major ongoing project of the Israeli Defense Forces (IDF) medical
corps.
D. Mandel et al.
program upon quality of primary care. We hypothesized that
the implementation of the QA program within the IDF
results in improved quality of care and that evaluation of a
speciWc PCP within the framework of the QA program results
in an improved score of the PCP during a second evaluation.
Methods
Definitions
Physician assessors
In the IDF, several teams of two experienced physicians carry
out the QA system. In each team there is at least one boardcertiWed family physician. The teams use a detailed, prospectively established QA protocol, where the emphasis is placed
on medical records. The assessment uses both speciWc obligatory
markers of adequate medical evaluation and treatment (an
explicit quality assessment method), and a direct inspection of
the primary physician (an implicit quality assessment method).
IdentiWcation and management of population at risk, further
medical training and guidance, and medical administration
with a direct effect on the quality of care are also evaluated.
Each item is subcategorized and is given a speciWc score. The
score reXects the relative importance of the item in question,
as determined (by consensus) by a committee composed of
the Surgeon General and the IDF scientiWc committee. The
QA forms that the assessors Wll in are shown as Supplementary
data (available at IJQHC Online), and include both the variables studied and their scoring assessment (maximum score).
Inspection
A board-certiWed family physician inspects the patient–physician
interaction (directly, in a live performance) and examines the
176
Medical record audit
The medical record is the soldier’s medical Wle, which consists of
past medical history data, previous referrals to military unit clinics,
laboratory results, and tests. In addition, the patient chart may be
used as a legal tool. The medical record is tested using three
criteria. (i) ‘Medical data’, which provide global, yet important,
medical information that must be included in the medical chart.
These data include medical rating, information about drug
allergy, an active medical problem list, and past hospitalizations
record and vaccine status. (ii) ‘Chart arrangement’ is evaluated by
the percent of medical charts found on request, chronological
continuity of the chart, presence or absence of physician signature
after each visit, and order and clarity of the charts. (iii) ‘Analysis
of the medical chart’ by the assessors—examination of the written
data inside 10 different records, according to problem-oriented
medical writing principles (the S.O.A.P. acronym: subjective,
objective, assessment, therapeutic and evaluative plan). In addition, Wve frequent (based on the army’s yearly statistical report;
internal unpublished data) medical conditions (respiratory tract
infection, urinary tract infection, abdominal pain, headache, and
low back pain) and one chronic condition (asthma) are evaluated.
At least three different physician–patient interactions for each
condition are analyzed. In order to address this part of the QA
program prospectively, teams of board-certiWed specialists analyzed each of these conditions, and deWned the necessary actions
that must be taken for their evaluation.
High-risk patient surveillance
Patients at high risk in the army are those soldiers whose
medical problems are either dynamic and may lead to serious
complications, or subjects with chronic illnesses. In this Weld,
two QA criteria are examined: (i) the initial surveillance of followup examinations of those patients at risk and the existence
of appropriate medical records for such examinations; and
(ii) the implementation of periodic follow-up examinations
and recommendations.
Complementary medical services
Evaluation of the medical services that relate to the physician–
patient interaction is effected according to the following criteria:
(i) appropriate drug supply; and (ii) secondary health care
characteristics. The latter is the interaction between the primary
physician and the secondary physician, which is examined in
Downloaded from http://intqhc.oxfordjournals.org/ at Pennsylvania State University on March 3, 2014
For the purpose of this analysis, the various PCP units evaluated
were classiWed as follows: (i) ‘single-physician medical clinics’
were those primary care clinics located at Weld units, serving
deployed troops, in mobile clinics, either during deployment or
during training. In this setting, primary care is provided by a single general practitioner, with no level of support from other
health care providers except for military medics. These military
medics provide only elementary medical care and are responsible
for administrative functions as well. All these unit clinics have a
restricted pharmacy on site (capable of providing basic medications). Physiotherapy and mental health services are not available
on site. (ii) ‘Multi-physician medical clinics’ were those primary
care clinics located in home-front bases, which employed
between two and 10 PCPs and were supported by medics and
nurses, which increases the number of available clinical staff. In
addition, full laboratory, pharmacy, and radiology services are
usually available on site or near by. Most centers also have physiotherapy and mental health services, while others include various
specialists’ services in the same building. Some centers also have
in-patient facilities. In addition, most of these primary care clinics
function in the evenings and at weekends for urgent visits.
following characteristics: (i) medical history taking: is the past
medical history properly studied? Are the correct questions
about the present illness asked? (ii) Physical examination: is
a thorough physical examination performed? (iii) Discussion:
the assessor discusses the case for differential diagnosis, further
evaluation option and suggested treatment. (iv) Follow-up:
evaluation of the follow-up that is offered to the patient.
(v) Communication: the communication between the physician and the patient during the interaction and the psychosocial
approach are evaluated. Finally, (vi) a general impression of
the assessor on the interaction is given.
Quality assessment in primary care clinics
terms of: (i) the proportion of referrals from the total visits in
the primary clinic; (ii) the quality of the referral letter; (iii) the
justiWcation of the referral and the attitude of the primary care
physician in regard to the specialist’s recommendations;
(iv) the availability of the medical services (determined by the
average waiting time in the clinic, the waiting list management
system, and the priority order); (v) the efWcacy of the administration procedures; and (vi) the evaluation of the in-patient
rooms in the units.
Guidance
Weighting of the results
Each item was ranked and subranked according to its importance to the QA committee. Individual scores for each item were
determined by consensus, and may be seen in the Supplementary data. The quality assessment of the clinic was arbitrarily
scored according to an internal weighting system, where each
component was weighted as a percentage of the total score, as
follows: the medical record, 35%; inspection of the PCP, 20%;
high-risk patient management, 15%; the medical services, 25%;
and guidance, 5%. In addition, a separate mark was given to the
physician, which included a score for the inspection and a score
for the markers. Each physician was given a Wnal mark and his
or her achievements were calculated, also on a percentile of all
military-physicians. The score for the inspection is expressed as
such, as a percentile of military physicians, but also as a percentile related to the speciWc assessor team.
Every clinic in the IDF is evaluated once or twice a year,
depending upon patient load. High volume clinics, where the
total physician–patient interactions exceed 1000 per month, are
evaluated twice a year. In order to avoid bias, the assessors’ teams
were not kept aware of the results of the previous evaluations.
The results are stored in a computerized data bank and analyzed
using special software speciWcally designed for this project. After
analysis of the results, two feedback reports are sent to the physicians and to the clinic managers, separately. The PCP receives
the feedback from the inspection and from the tracers that were
analyzed. The primary care clinic’s manager receives the whole
quality assessment feedback. In addition, a yearly executive
report is presented to the surgeon general command.
Statistical analysis
General Linear Models (SAS Institute Inc., Cary, NC) were used
for analysis. Student’s t-tests were used to determine differences
in scores by clinics. In addition, paired t-test analysis was used to
evaluate the differences between Wrst and second assessment of
physicians. Stepwise logistic regression was performed when
needed. A P-value of <0.05 was considered signiWcant.
During the period 1 January 2000 to 31 December 2001,
99 primary care clinics were evaluated. Fifty-Wve clinics (56%)
were single-physician medical clinics (unit troop clinics) and
44 (44%) were multi-physician medical clinics (home-front
clinics). One hundred and sixty-four PCPs were evaluated
during this period: 52 (32%) worked in single-physician medical
clinics and 112 (68%) worked in multi-physician medical clinics.
Table 1 depicts score differences by type of clinic (singlephysician versus multi-physician clinics), evaluated using the
implicit method (inspection). Scores ranged from 0 (lowest
quality assessment) to 5 (highest quality assessment). In general, PCPs working in single-physician medical clinics scored
signiWcantly higher than PCPs working in multi-physician
medical clinics in every single assessment criterion examined:
medical history taking, physical examination, discussion, program, communication, assessor’s impression, and mean score
( Table 1).
Table 2 depicts the score differences by type of clinic (singlephysician versus multi-physician clinics), evaluated using the
explicit method (markers of adequate medical evaluation and
treatment). Scores ranged from 0 (lowest quality assessment)
to 100 ( highest quality assessment). The data shown in the
table are the data obtained at the Wrst assessment only. Multiphysician medical clinics scored higher than single-physician
medical clinics in terms of high-risk patient surveillance, but
not in terms of medical record assessment, complementary
medical services, or guidance (Table 2).
Table 3 depicts score differences between the Wrst and the
second assessment, evaluated using the implicit method
(inspection). Seventy-four PCPs (46%) were evaluated twice.
In univariate analysis using paired t-tests, there was a signiWcant improvement in mean scores of the PCPs in four out of
the six Welds studied. The improvement was signiWcant in
Table 1. Score differences by type of clinic (implicit method:
inspection)
Single-physician
medical clinics
Multi-physician
medical clinics
4.04 ± 0.009
3.69 ± 0.011
4.01 ± 0.009
4.17 ± 0.007
4.24 ± 0.008
4.17 ± 0.007
4.05 ± 0.007
3.52 ± 0.008
3.09 ± 0.009
3.47 ± 0.009
3.54 ± 0.008
3.75 ± 0.007
3.66 ± 0.007
3.50 ± 0.007
..........................................................................................................
History taking
Physical examination
Discussion
Program
Communication
Impression
Mean score
SEM, standard error of the mean.
Data are expressed for the Wrst quality assessment as mean ± SEM.
Scores range from 0 (low assessment) to 5 (high assessment).
Single-physician medical clinics scored better than multi-physician
medical clinics for all parameters (P < 0.001).
177
Downloaded from http://intqhc.oxfordjournals.org/ at Pennsylvania State University on March 3, 2014
An important aspect of preventive medicine is patient education and continuous medical education of the medical staff.
Thus, the following characteristics were recorded: lectures to
the soldiers, knowledge of the medical staff and the medical
orderly, reserve army medical guidance, and the existence of
updated medical references.
Results
D. Mandel et al.
Table 2 Score differences by type of clinic (explicit method:
markers of adequate medical evaluation and treatment)
Table 4 Score differences between Wrst and second assessment
(explicit method: markers of adequate medical evaluation and
treatment)
Single-physician Multi-physician P-value
medical clinics medical clinics
..........................................................................................................
Medical record
assessment
High-risk patient
surveillance
Complementary
medical services
Guidance
79.22 ± 1.2
76.75 ± 1.01
NS
56.93 ± 2.24
71.67 ± 2.68
<0.008
86.72 ± 1.23
83.77 ± 1.01
NS
71.34 ± 2.3
72.9 ± 3.4
NS
Table 3 Score differences between Wrst and second assessment
(implicit method: inspection)
First
assessment
Second
assessment
P-value
3.39 ± 0.09
3.11 ± 0.11
3.61 ± 0.09
3.24 ± 0.1
NS
NS
3.29 ± 0.09
3.33 ± 0.09
3.67 ± 0.08
3.48 ± 0.09
3.65 ± 0.08
3.67 ± 0.08
3.99 ± 0.08
3.78 ± 0.08
<0.003
<0.005
<0.004
<0.008
.........................................................................................................
History taking
Physical
examination
Discussion
Program
Communication
Impression
NS, non-signiWcant; SEM, standard error of the mean.
Data are expressed as mean ± SEM. Scores range from 0 (low
assessment) to 5 (high assessment).
terms of discussion, program, communication, and impression.
Although the scores were also higher at the second assessment in terms of history taking or physical examination, the
difference between the two assessments was not statistically
signiWcant (Table 3). We used backward stepwise multiple
regression analysis to study the effect of potential confounders (type of clinic, year of study, and assessor team) on the
‘improvement’ between the two assessments. It allowed us to
conWrm that a statistically signiWcant improvement had
indeed occurred in terms of discussion, program, communication, and impression, but not in terms of history taking or
physical examination.
Table 4 depicts score differences between the Wrst and the
second assessment, evaluated using the explicit method.
There were no statistically signiWcant differences between the
two assessments in terms of any of the parameters studied,
i.e. medical record assessment, high-risk patient surveillance,
complementary medical services, or guidance.
178
Second
assessment
P-value
77.19 ± 2.39
75.66 ± 2.18
NS
53.26 ± 7.06
64.20 ± 5.67
NS
84.54 ± 3.20
83.59 ± 2.11
NS
81.54 ± 6.97
83.81 ± 4.49
NS
..........................................................................................................
Medical record
assessment
High-risk patient
surveillance
Complementary
medical services
Guidance
NS, non-signiWcant; SEM, standard error of the mean.
Data are expressed as mean ± SEM. Scores range from 0 (low
assessment) to 100 ( high assessment).
Discussion
Our Wrst aim was to describe the mechanisms of QA in the
primary care clinics of the IDF. We showed that such a
QA program was feasible in a military setting, geographically distributed over a whole country, despite operational
constraints.
The results of this study supported our hypothesis that the
quality of primary care provided in single-physician medical
clinics, such as is found in troop clinics, was higher than that
provided in multiple-physician medical clinics, such as in
home-front base clinics. This was true in terms of most of the
parameters studied, except for high-risk patient evaluation,
which scored better in multi-physician than single-physician
medical clinics. We speculate that the reasons for this Wnding
are multiple. It might be that the IDF primary care physicians
are medical ofWcers who graduate from a medical school, an
ofWcer academy, and a military medical academy. They are
then sent to unit troops clinics for periods of 12–18 months.
In contrast, PCPs serving in home-front bases, where several
PCPs work in the same clinic, are more likely to be civilian
physicians hired by the IDF or military reserve physicians.
Thus, the degree of motivation and identiWcation with the
organization goals and values may be different between these
groups, and thus may render quality of medical care uneven.
Another potential reason is that the setting of a Weld unit is
similar to that of a small community, in which the PCP is
familiar with the population and in terms of military medicine, with the tasks of the unit as well; thus, it helps the PCP
to perceive his/her role as a ‘family’ practitioner and assures
continuity of care by the same physician. Moreover, since
a unit troop clinic serves a smaller population than that of
a home-front clinic, the patient load, a parameter that may
affect quality, is respectively lower, and thus the measured
quality of primary care is higher. A similar Wnding was made
by Campbell and colleagues [9], who stratiWed a random sample
of 60 general practices in six areas of England and identiWed
Downloaded from http://intqhc.oxfordjournals.org/ at Pennsylvania State University on March 3, 2014
NS, non-signiWcant; SEM, standard error of the mean.
Data are expressed for the Wrst quality assessment as mean ± SEM.
Scores range from 0 (low assessment) to 100 (high assessment).
First
assessment
Quality assessment in primary care clinics
while the major aim of the program was to assess the process
of rendering care rather than outcomes. The program examined speciWc aspects of patient-centered care, as it may be
noted that all variables studied were done so for the beneWt
of the patient, and did not address whether it would be to the
beneWt of the IDF. Timeliness was studied in terms of physician availability, availability of specialists and waiting time at
the clinic. In terms of efWciency of care, aspects such as appropriate use of specialists’ services and in-patient services
were examined. And Wnally, a major variable examined in this
report, i.e. variability of quality of care by geographic
location, was a measure of the equitable nature of care in
the IDF.
In conclusion, this study conWrms that a wide variation in
the quality of care exists in IDF general practice. The study
also allowed us to conWrm that evaluation of primary care
clinics and primary care physicians is feasible, and leads to an
improvement in the quality of primary care in the military
setting. We believe that our program should help us to meet a
major challenge raised by the CQCA, i.e. the incorporation of
performance and outcome measurements for improvement
and accountability.
References
1. Donabedian, A. Explorations in Quality Assessment and Monitoring.
Vol. 1. The DeWnition of Quality and Approaches to its Assessment.
Ann Arbor, MI: Health Administration Press, 1980.
2. Blumental D. Quality of health care—what is it? New Engl J Med
1996; 335: 891–894.
3. Brook RH, McGlynn EA, Shekelle PG. DeWning and measuring
quality of care: a perspective from US researchers. Int J Qual
Health Care 2000; 12: 281–295.
4. Jeffer EK. Quality assurance and quality improvement: the
1990s and beyond. J Healthcare Qual 1992; 14: 36–40.
5. Gilutz H, Shamis A, Ben-Amitay D et al. Primary quality control
in Israel Air Force clinics. Harefuah 1994; 126: 570–573.
6. Mankuta D, Vinker S, Itzhak B et al. A quality management
project in Israeli navy primary care clinics. Am J Med Qual 1996;
14: 211–215.
7. Vinker S, Mankuta D, Ischak B et al. Medical record audit in
primary care clinics of the Israel Navy. Harefuah 1996; 131: 477–
479.
8. Epstein A. Performance reports on quality: prototypes, problems and prospects. N Engl J Med 1995; 333: 57–61.
9. Campbell SM, Hann M, Hacker J et al. Identifying predictors of
high quality care in English general practice: observational study.
Br Med J 2001; 323: 784–787.
10. Morey JC, Simon R, Jay GD et al. Error reduction and performance improvement in the emergency department through formal
teamwork training: evaluation results of the MedTeams project.
Health Serv Res 2002; 37: 1553–1581.
11. Wallis M, Tyson S. Improving nursing management of patients
in a hematology/oncology day unit: an action research project.
Cancer Nurs 2003; 26: 75–83.
179
Downloaded from http://intqhc.oxfordjournals.org/ at Pennsylvania State University on March 3, 2014
factors associated with high quality care. They found a signiWcant association between size of practice and quality of care.
Smaller practices scored better than larger ones for access to
care, but for diabetes care larger practices had higher scores
than smaller ones. The authors concluded that no single type
of practice has a monopoly on high quality care; different
types of practice may have different strengths. In our study,
an analogy to Campbell’s statement was that only a single
parameter (high-risk patient surveillance) scored higher in
larger practices than in smaller ones.
The QA program that we described was effective in that it
improved primary care per physician examined in nearly all
aspects of the assessment. We believe that the improvement
was true in nature, and did not reXect only the experience of
the PCP in ‘passing’ the inspection. Indeed, while the part of
the assessment under direct inspection may be inXuenced by
the previous experience of the physician assessed, the part of
the assessment that relates to the quality of the chart evaluates
continuous care rendered while the physician is not under
investigation. Similar positive impact of QA programs, leading to improvement of care as judged from improved outcomes or processes, has been demonstrated in various
settings, such as the emergency department [10], the hematology/oncology day unit [11], among pneumonia patients in
very small hospitals [12], or in hemodialysis units [13].
There are several possible limitations to our study. Firstly,
we conducted it in a military setting. Although the nature of
medicine is at times similar in civil clinics and in military clinics,
some characteristics of military medicine are unique [5–7],
thus the interaction between the PCP and the patient may be
different. A study comparing this QA program between military and civil clinics has not yet been conducted. Such a study
may be useful in determining this system’s capabilities and
may offer a standardized tool for the assessment of quality of
care in primary care clinics. Secondly, we evaluated PCPs over
a limited period of 2 years, and the long-term impact of our
QA program cannot be evaluated at this point in time.
Furthermore, the Wnancial cost/beneWt aspects of this QA
program have not been calculated. In a country such as Israel,
such a QA program is feasible for several reasons: (i) the
physician-per-capita ratio is one of the highest in the world
[14]; and (ii) there is mandatory conscription of the whole
population, with continued military periods for physicians
until 45–51 years of age. In contrast, such a program may be
prohibitive in countries with a lower physician density, or
whose army physicians are only hired.
When placed into the general context of the recently published recommendations of the Committee on Quality of
Care in America (CQCA) (of the Institute of Medicine for
quality of medical care improvement [15,16]), our program
addressed many, but not all of the issues raised there. The
CQCA deWned six major aims for the 21st century health care
system, and recommended that it be ‘safe, effective, patientcentered, timely, efWcient, and equitable’. In terms of safety,
our QA program veriWed that appropriate records be kept,
and appropriate steps be taken for several speciWc tracers.
The program was not primarily designed to assess effectiveness, in that mortality or speciWc morbidities were not measured,
D. Mandel et al.
12. Chu LA, Bratzler DW, Lewis RJ et al. Improving the quality of
care for patients with pneumonia in very small hospital. Arch
Intern Med 2003; 163: 326–332.
13. Sehgal AR. Impact of quality improvement efforts on race and sex
disparities in hemodialysis. J Am Med Assoc 2003; 289: 1033–1034.
14. WHO. WHO Estimates of Health Personnel. Physicians, Nurses,
Midwives, Dentists and Pharmacists. Http://www.who.int/whosis/
health/health_personnel.cfm Accessed 18 April 2002.
15. Institute of Medicine/Committee on Quality of Care in America.
Improving the 21st century health care system. In Crossing the
Quality Chasm: a New Health System for the 21st Century. Washington, DC: National Academy Press, 2001, pp. 39–60.
16. Institute of Medicine/Committee on Quality of Care in America.
Building organizational supports for change. In Crossing the Quality Chasm: a New Health System for the 21st Century. Washington,
DC: National Academy Press, 2001, pp. 11–44.
Accepted for publication 24 November 2003
Downloaded from http://intqhc.oxfordjournals.org/ at Pennsylvania State University on March 3, 2014
180