Afghanistan Health Sector Balanced Scorecard
National and Provincial Results
Round 1 (2004)
JUNE 2, 2005
Ministry of Public Health
General Directorate of Policy and Planning
Monitoring & Evaluation Department
WITH
Johns Hopkins University
Bloomberg School of Public Health
Indian Institute of Health
Management Research
Afghanistan
Health Sector Balanced Scorecard
National and Provincial Results
Round 1
(2004)
June 2, 2005
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results
Round 1 (2004)
Table of Contents
The Balanced Scorecard ....................................................................................1
Introducing the Balanced Scorecard...........................................................1
How to Read the Balanced Scorecard........................................................2
Findings of the 2004 National Balanced Scorecard....................................3
Findings of the 2004 Provincial Balanced Scorecard .................................4
Limitations of the Balanced Scorecard .......................................................6
National Balanced Scorecard of the Health Sector (Table 1) ..........................7
Provincial Balanced Scorecard of the Health Sector (Table 2) .......................8
Methods Annex .................................................................................................10
Identification of Domains & Indicators ......................................................10
Data Collection .........................................................................................11
Detailed Description of Domains and Indicators .......................................14
i
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
The Balanced Scorecard
Introducing the Balanced Scorecard
•
•
•
•
•
•
•
The purpose of the Afghanistan Health Sector Balanced Scorecard is to
summarize the performance of the provinces of Afghanistan in the delivery of
the Basic Package of Health Services (BPHS).
The BSC is used by the Ministry of Public Health (MOPH) to clarify its vision
and strategies, and is a measurement tool to help it to manage change. It can
be used for organizing activities and learning from experience.
The MOPH and all stakeholders of the health sector can use the BSC to see
how provinces and the country as a whole are providing the BPHS.
The Balanced Scorecard (BSC) provides a framework to efficiently look at
multiple areas (called domains) of the health sector at a given point of time
(2004).
This year’s BSC can be considered a baseline level of delivery, and is used to
set the benchmarks to guide actions and compare results in the future.
The BSC has six domains made up from 28 indicators and 2 composite
indicators. The domains and indicators were determined through a
participatory process that involved the MOPH, NGOs and development
partners, and was facilitated by the 3rd party evaluation team from Johns
Hopkins Bloomberg School of Public Health (JHBSPH) and the Indian
Institute of Health Management Research (IIHMR).
The six domains of the BSC are:
1.
2.
3.
4.
5.
6.
•
•
Patients and Community
Staff
Capacity for Service Provision
Service Provision
Financial Systems
Overall Vision
The data for the BSC were taken from the baseline National Health Facilities
Performance Assessment (NHFPA).
o Data for the NHFPA were collected from June-October 2004, using a
stratified random sampling of all health facilities providing the BPHS
from all provinces of Afghanistan.
o The final sample included assessments of over 600 health facilities,
nearly 6000 patient observations and an equal number of patient
interviews, interviews with over 1600 health workers, interviews of
13,000 households in the communities surrounding the clinics, and 74
focus groups in communities held around the country.
o Results from the household interviews and community focus groups
are reported separately, and complement the findings of the BSC.
For each indicator of the BSC, upper and lower benchmarks were set to
indicate levels that are achievable in Afghanistan. The upper benchmark is
1
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
set at a level that is currently being achieved by at least six provinces and the
lower benchmark by 27 of the 33 provinces.1
How to Read the Balanced Scorecard
•
•
•
•
•
The BSC tables are color-coded:
o Green: green colored cells indicate that the result is above the upper
benchmark (top six).
o Red: red colored cells indicate that it is in the bottom quintile (bottom
six provinces).
o Yellow: all results in-between the top and bottom quintile are shown in
yellow cells.
The first 26 of the indicators are measured as percentages.
o In some cases, the number shows what percent of the results in a
province met a certain standard (e.g. Indicator 1 is the percent of
patients that rated their satisfaction with health services as “highly
satisfactory” on a 4-point scale; Indicator 5 shows the percent of staff
salary payments that are up to date).
o In some cases, the number shows what percent of items in an index
have been achieved (e.g. Indicator 6 shows the percent of selected
BPHS equipment that is present and working).
Indicator 27 and 28 are Concentration Indices, which are measures of
equality. The results can range from -1 to 1. Each attribute (outpatient visits
for Indicator 27 and satisfaction with services for Indicator 28) is analyzed
according to how it is distributed among the population that is grouped
according to their wealth.
o A result of -1 means that only the most poor people in the province
received the attribute (e.g. all outpatient visits or satisfaction).
o A result of +1 means that only the wealthiest people had the attribute.
o A “0” means that the attribute is equally distributed among the rich and
poor.
Detailed definitions for each of the indicators are described in the methods
section of this report.
The last two indicators in the BSC are composite indicators.
o “The percent of upper benchmarks achieved” demonstrates how well
provinces have done in meeting the upper benchmarks across the first
28 indicators.
o “The percent of lower benchmarks achieved” measures how well
provinces have done in meeting the lower level benchmarks across the
first 28 indicators.
1
This means that the upper level of performance is set at the level achieved by the top 20th percentile
(also called quintile) of the provinces, whereas the lower benchmark was set at the level achieved by
the bottom quintile. The one exception to this pattern is the upper benchmark for the Equipment
Index. It was set at 90%, indicating that it is expected that at least 90% of a package of equipment
needed to deliver the BPHS is functional at the facilities in the province. The top quintile mark was
actually 69%.
2
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
•
The National BSC is shown in Table 1.
o The first column on the left is the serial number of the variable.
o The next column to its right is the name of the indicator listed according
to the six domains.
o This is followed by a column that shows the unit of measurement
(usually a %).
o The “N” column shows the number of individuals or facilities that was
used to calculate the results (the number in the denominator).
o The colored columns to the right indicate the results accumulated
across the provinces.
o The median value (in yellow) represents the middle value of the
provinces, with 16 provinces having results above the median, and 16
provinces having values below the median.
o Minimum (red) and maximum (green) levels are the lowest and highest
provincial results respectively.
o The bottom (red) and top (green) quintile show the level of the bottom
and top 20th percentile of provinces, which usually marks the result for
the sixth province from each extreme, and are used for the lower and
upper benchmarks (except indicator 6, see footnote 1).
Findings of the 2004 National Balanced Scorecard
One of the most important findings of the National Scorecard (Table 1) is that there
is a wide variation for each indicator. This is shown by the differences in results
between the minimum and maximum levels, or between the bottom and top quintiles.
This suggests that provinces perform quite differently from each other in these
indicators of health sector performance. For the country as a whole, the results
concerning the Overall Vision (Domain F) show encouraging results. Overall, women
and the poor received more BPHS services in 2004, and the poor were more likely to
be satisfied with the services. However, the other domains also highlight low median
levels in 2004 where greater improvements are needed.
In examining Patients & Community Perspectives (Domain A), the provincial median
for meeting patient perceptions of high quality was quite low (Indicator 2, where the
cut-off score for high quality was set at the top quintile of all patients ratings). Ratings
for high overall satisfaction (Indicator 1) were higher. Relatively few communities
had active shura-e-sehie (median 32%), though one province appeared to be very
active (84% of communities in Kapisa had active shura).
The results for Staff Perspectives (Domain B) point out some causes for concern and
action. Less than half of salary payments are up to date (Indicator 5 – paid in the
last month), and only about 15% of provinces have overall health worker satisfaction
scores (Indicator 4) that are considered highly satisfactory by health workers.
Domain C describes the Capacity for Service Provision. The median level of
facilities having a tuberculosis registry (Indicator 17) is particularly low (12%), even
though this is necessary if tuberculosis treatment is to be carried out according to
directly observed treatment (DOTS). Functional laboratories (Indicator 9), meeting
3
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
minimum staff placement guidelines (Indicator 10) and staff training (Indicator 12)
are also relatively low, despite their importance in delivering the BPHS. However,
drug availability (Indicator 7 -- median 72%), use of Health Management Information
Systems (Indicator 13 -- median 73%), and use of patient records (Indicator 16 -median 67%) are considerably higher, though still suggest that there is much room
for improvement.
Domain D covers the technical assessment of providing health services. These
results are generally quite low. Basic Health Centres that see a targeted number of
outpatients (Indicator 20) was particularly low (median 10%), as was providing
delivery care according to the BPHS guidelines (Indicator 23) was also quite low
(median 14%). The indicator on health workers that use good outpatient care
practices (Indicator 18) was also relatively low (54% median).
The indicators in Financial Systems (Domain E) and Overall Vision (Domain F)
scored higher. Most facilities have user fee guidelines (Indicator 24) and systems for
exempting poor patients from paying fees (Indicator 25). In the 2004 BSC, more
females were seen as outpatients than males (56% of total was the median score).
The median Concentration Indices were also less than 0, meaning that the poor
were getting proportionately more outpatient visits and more satisfaction with care
than those who are less poor.
The composite scores on provinces meeting the upper benchmarks (Indicator 29)
show that most provinces have not met many of these benchmarks (median 18%).
The top six provinces achieved the upper benchmarks for at least 29% of the
indicators, with the highest level found in Zabul (43%).2 This means that each of the
provinces has considerable opportunity for improvement in meeting benchmarks that
other provinces in Afghanistan are achieving. Provinces were more consistent in
achieving the lower benchmarks. The finding that nearly all the provinces are
meeting the lower benchmarks (Indicator 30 – median 82%) means that it is only a
few provinces that are consistently near the bottom in levels of performance. This
suggests that in those provinces,3 more effort may be placed on at least meeting the
minimum levels across these indicators.
Findings of the 2004 Provincial Balanced Scorecard
The individual Provincial BSC results are shown in Table 2. The first (left) column in
this table gives the name of the indicator followed on its right by the unit of
measurement. This is followed by the three benchmark levels (the national median
and the lower and upper benchmark). Each province is then listed in the columns
from left to right in alphabetical order, with its actual value provided in the cell for
each indicator. As before, each cell is colored according to whether it is above the
upper benchmark (green), below the lower benchmark (red), or in-between (yellow).
In general, it will be important for managers to find ways to improve health services
in each of the domains of the BSC. Since the benchmarks have now been set,
managers in each province may put priority on trying to improve in those areas that
2
3
Zabul had only 4 facilities that could be assessed, so its results should be interpreted with caution.
These are identified from Table 2 as Ghor, Helmand, Khost, Nimroz, Paktika, and Urozghan.
4
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
are currently in the “red zone” (bottom quintile) for their province. A second strategy
is to try to find ways to reach the green zone (upper quintile) in areas that are
currently in the “yellow zone” (average).
If we take Badakshan as an example for how to interpret the Provincial BSC, we see
that the level of performance for all the indicators in Patients & Community (Domain
A) and Staff (Domain B) are in the middle level (all are colored yellow). This means
Badakshan’s performance in these areas is average compared to the other
provinces. However, in Domain C (Capacity for Service Provision), there are two
indicators where its performance is in the “red zone”. Thus, in the areas of providing
essential drugs and using patient records, it will be important to find an explanation
for the low results, and try to improve performance. On the positive side, Badakshan
is doing relatively well on training staff and having its infrastructure in place, two
indicators where its results are in the “green zone” (top quintile). This suggests that
managers in Badakshan (and elsewhere in Afghanistan) may want to see how they
have managed these issues, so that lessons can be learned for elsewhere.
Most service provision indicators (Domain D) in Badakshan are in the yellow zone,
suggesting that improvements can still be made if it wishes to reach the levels
achieved by the top provinces. The most striking area for improvement is on the
Patient-Provider Care Index, where its result is in the “red zone”. This is a
particularly important indicator, which is derived from actually observing health
workers take care of patients. In Badakshan’s case, only 38% of the steps a
provider should take in dealing with a patient had been done. Since Badakshan has
also done relatively well in providing training to its staff, managers there may want to
examine how well the training was done, and how well they have been able to
supervise and support its health workers in the field. The latter is particularly
important given the geographic constraints in the province.
In Domain E (Financial Systems), it is again apparent that health facilities in
Badakshan had low levels in one indicator – fee exemptions for the poor. It will be
important for health managers in Badakshan to ensure that its policies concerning
fee exemptions for the poor are put into practice.
In the indicators for achieving the overall vision (Domain F), Badakshan is among the
average group of provinces. Just under half of all outpatients were women (49%), so
improvements could still be made in ensuring that women are able to use health
services. The Concentration Indices are both negative numbers, which mean that
poorer people in Badakshan are slightly more likely to use outpatient health services
than less poor people (Indicator 27), and that the poor are also more likely to be
satisfied with those services than those who are better off (Indicator 28). Finally, it
can be seen that Badakshan has done relatively well in at least meeting the lower
benchmarks (86% of them). Although its ability to reach the upper benchmarks is
similar to most other provinces, it achieved those upper levels in only 3 of 28
indicators (11%), and could strive to do better.
5
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Limitations of the Balanced Scorecard
The Balanced Scorecard gives a powerful indication of how provinces and the
country are doing in delivering the basic package of health services, but it also has
limitations. When interpreting the Provincial BSC, is important to not consider that a
result is entirely due to the ability of the providers of health services in that province.
People’s health situations are different, as are other conditions (e.g. security, roads,
and poverty). So provinces are expected to be at different levels in providing the
BPHS because of these factors. This suggests that in each province, managers
should first try to find out why they are getting the results that they get. After trying to
get an explanation for their results, they should then try to learn from themselves and
other provinces that are doing well, using the benchmarks as a guide.
In addition to being careful to first examine why a province may perform the way it
does, it is also important not to rely on only one indicator in the BSC to rate a
Province’s performance. This is because one indicator is too small a part of the
overall picture to use reliably.
The BSC is only measured at health facilities that were actually providing services,
and thus does not take into account places where BPHS is not being provided. This
is an important consideration, especially in provinces where there are populations
with poor access to health facilities. Some provinces have particularly few facilities
providing the BPHS (notably Zabul, Uruzghan, Samangan, and Nimroz). The
measurements in the BSC for these provinces are also not as reliable as the other
provinces, and should be interpreted with caution. There were also a few provinces
that did not have reliable data for each of the indicators. These results have been
left blank, and where possible, data is being collected again to fill in these gaps.
The Methods Annex that follows the BSC tables describes how the BSC was set up,
as well as how the data were collected and analyzed. The detailed definition of each
indicator is also provided for reference.
6
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Table 1: Afghanistan National Health Sector Balanced Scorecard
Measure
N
National
Median
Minimum
Bottom
Quintile
Top
Quintile
Maximum
40.2%
79.9%
96.2%
A. Patients & Community
1
Overall Patient Satisfaction (Score = 4/4)
%
5443
62.7%
0.0%
2
Patient Perception of Quality Index (Score > 34/36)
%
5397
23.5%
0.8%
5.8%
38.3%
65.7%
3
Written Shura-e-sehie activities in community
%
598
32.4%
0.0%
10.7%
58.8%
84.4%
B. Staff
4
Health Worker Satisfaction Index (Score > 62/76)
%
1309
14.7%
0.0%
3.8%
32.6%
62.6%
5
Salary payments current
%
1517
42.1%
15.8%
31.2%
57.6%
66.9%
C. Capacity for Service Provision
6
Equipment Functionality Index
%
570
62.5%
31.2%
56.2%
68.8%
83.3%
7
Drug Availability Index
%
580
72.1%
10.4%
51.5%
82.2%
99.3%
8
Family Planning Availability Index
%
555
61.2%
0.0%
43.5%
76.0%
91.0%
9
Laboratory Functionality Index (Hospitals & CHCs)
%
284
28.8%
0.0%
12.7%
45.9%
58.4%
10
Staffing Index -- Meeting minimum staff guidelines
%
606
36.8%
0.0%
0.0%
55.1%
69.7%
11
Provider Knowledge Score
%
1502
51.3%
21.7%
44.7%
57.5%
72.0%
12
Staff received training in last year
%
1,539
37.3%
0.0%
24.8%
55.0%
78.0%
13
HMIS Use Index
%
566
73.3%
39.1%
60.7%
89.3%
96.5%
14
Clinical Guidelines Index
%
460
39.2%
12.9%
26.9%
51.3%
69.2%
15
Infrastructure Index
%
569
51.9%
21.4%
43.8%
60.9%
92.6%
16
Patient Record Index
%
2624
66.6%
39.5%
53.0%
94.9%
100%
17
Facilities having TB register
%
605
11.8%
0.0%
6.5%
20.9%
47.1%
Patient-Provider Care Index
%
2060
54%
37.7%
40.9%
65.2%
91.5%
19
Proper sharps disposal
%
600
46.4%
2.6%
28.0%
86.7%
100%
20
Average new outpatient visit per month (BHC > 768 visits)
%
202
10.0%
0.0%
0.0%
57.1%
100%
21
Time spent with patient (> 9 minutes)
%
2908
22.2%
0.0%
10.0%
57.1%
100%
22
BPHS facilities providing antenatal care
%
606
57.7%
7.9%
21.4%
76.0%
100%
Delivery care according to BPHS
%
606
13.9%
0.0%
3.5%
34.3%
56.6%
Facilities with user fee guidelines
%
424
93.4%
19.5%
71.7%
100%
100%
Facilities with exemptions for poor patients
%
417
74.8%
22.5%
51.6%
100.0%
100%
%
CI (-1 to
1)
CI (-1 to
1)
527
55.7%
39.1%
48.6%
61.2%
67.8%
5,194
-0.038
0.119
0.018
-0.104
-0.185
5,160
-0.002
0.046
0.019
-0.018
-0.048
D. Service Provision
18
23
E. Financial Systems
24
25
F. Overall Vision
26
Females as % of new outpatients
27
Outpatient visit concentration index
28
Patient satisfaction concentration index
Composite Scores
29
Percent of Upper Benchmarks Achieved
%
33
18%
0%
7%
29%
43%
30
Percent of Lower Benchmarks Achieved
%
33
82%
54%
71%
93%
100%
KEY
Score Below Bottom Quintile
Score Between Bottom & Top Quintiles
Score Above Top Quintile
RED
YELLOW
GREEN
7
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Provincial Results
Measure
National
Median
Lower
Benchmark
Upper
Benchmark
Badakhshan
Badghis
Baghlan
Balkh
NGO
Performance Based Partnership Agreement
A. Patients & Community
1 Overall Patient Satisfaction (Score = 4/4)
%
2 Patient Perception of Quality Index (Score > 34/36)
%
3 Written Shura-e-sehie activities in community
%
Bamyan
Daykundi
NGO
Farah
Faryab
Ghazni
Ghor
NGO
Helmand
Herat
Jawzan
Kabul
NGO
62.7
23.5
32.4
40.2
5.8
10.7
79.9
38.3
58.8
73.6
25.1
40.9
60.3
30.8
0.0
81.1
36.7
36.5
64.3
32.9
10.7
96.2
43.5
23.8
68.7
18.9
17.4
40.5
0.8
27.9
73.4
38.3
40.9
23.3
1.9
18.5
49.8
10.8
8.6
40.2
14.4
24.1
33.2
7.5
32.4
79.9
49.9
46.0
14.7
42.1
3.8
31.2
32.6
57.6
18.9
35.8
3.1
53.7
23.9
31.1
36.0
31.9
11.5
49.2
8.7
66.9
2.5
15.8
46.0
57.8
11.9
25.7
13.7
50.6
28.8
49.2
2.6
42.1
34.3
23.7
62.5
72.1
61.2
28.8
36.8
51.3
37.3
73.3
39.2
51.9
66.6
11.8
56.2
51.5
43.5
12.7
0.0
44.7
24.8
60.7
26.9
43.8
53.0
6.5
90*
82.2
76.0
45.9
55.1
57.5
55.0
89.3
51.3
60.9
94.9
20.9
63.5
51.4
55.7
32.9
37.5
45.6
69.4
66.1
28.4
60.9
49.7
33.2
62.7
51.5
55.6
20.2
35.0
41.3
39.7
70.3
26.4
48.7
100.0
24.8
56.2
72.3
74.2
19.2
37.2
52.7
37.3
51.3
35.2
43.8
80.1
11.2
64.5
56.6
65.0
6.9
57.9
53.9
49.9
81.7
23.0
61.6
95.0
12.3
76.2
81.5
81.4
58.4
65.3
63.0
33.8
67.1
40.3
53.3
66.6
0.0
56.8
10.4
0.0
0.0
56.6
46.6
36.5
96.5
43.7
59.0
100.0
17.7
57.0
38.7
72.6
11.2
13.0
54.4
44.6
89.3
37.8
37.2
81.4
6.5
64.3
75.8
53.5
31.4
0.0
52.4
24.3
86.1
39.2
66.2
99.2
11.9
57.3
74.9
52.8
13.9
0.0
38.9
29.9
86.9
27.7
51.4
65.5
13.7
39.4
84.2
73.4
27.1
23.9
46.6
22.6
39.1
12.9
21.4
76.1
13.0
59.3
70.3
60.0
21.1
50.0
53.5
32.4
48.9
45.3
45.7
83.1
21.0
60.2
62.8
58.5
3.8
44.0
57.8
55.0
67.8
31.1
48.7
58.4
9.7
71.0
79.7
68.7
36.4
60.5
56.9
46.1
76.9
47.6
51.9
61.5
7.1
54.0
46.4
10.0
22.2
57.7
13.9
40.9
28.0
0.0
10.0
21.4
3.5
65.2
86.7
57.1
57.1
76.0
34.3
38.4
60.6
14.3
22.2
25.9
22.9
56.7
42.8
10.0
55.6
56.9
34.3
49.6
67.3
0.0
0.0
39.4
3.5
76.3
74.5
33.3
20.0
64.2
29.1
63.0
81.0
16.7
30.0
95.3
45.4
40.7
32.2
0.0
75.0
64.6
56.6
57.7
34.8
14.3
12.5
62.4
13.0
64.7
100.0
28.6
0.0
61.2
27.2
59.3
81.5
0.0
16.7
22.4
5.7
41.9
9.5
NA
22.2
20.6
19.4
65.2
42.7
57.1
20.0
80.2
0.0
65.9
28.6
20.0
12.5
74.0
13.0
53.7
54.2
10.0
57.1
74.6
28.3
93.4
74.8
71.7
51.6
100
100
98.8
22.5
93.4
29.8
94.7
64.8
33.1
84.6
90.5
90.5
100
53.7
94.1
92.3
91.9
100
19.5
83.1
67.3
66.9
100
100
50.0
100
100
29.22
55.7
-0.038
-0.002
48.6
0.018
0.019
61.2
-0.104
-0.018
49.3
-0.080
-0.017
49.7
0.020
-0.021
55.0
-0.034
-0.013
57.5
0.018
0.001
55.8
-0.031
-0.005
51.5
-0.038
0.018
60.9
-0.145
-0.010
61.4
-0.092
0.020
55.7
0.119
-0.003
61.9
-0.090
0.046
53.0
-0.003
-0.006
59.2
-0.010
-0.027
61.4
-0.110
0.019
18
82
7
71
29
93
11
86
14
75
4
75
18
82
29
96
25
82
7
75
29
86
0
71
4
67
25
86
18
86
29
89
20
189
183
46
16
604
24
218
216
67
10
500
27
245
241
88
5
681
24
224
229
72
8
471
11
101
99
37
11
371
21
215
211
69
14
441
20
206
210
22
6
404
17
167
150
30
5
356
24
197
181
57
0
578
25
241
244
88
20
479
21
196
196
47
1
396
25
227
224
65
0
487
B. Staff
4 Health Worker Satisfaction Index (Score > 62/76)
%
5 Salary payments current
%
C. Capacity for Service Provision
6 Equipment Functionality Index
%
7 Drug Availability Index
%
8 Family Planning Availability Index
%
9 Laboratory Functionality Index (Hospitals & CHCs)
%
10 Staffing Index -- Meeting minimum staff guidelines
%
11 Provider Knowledge Score
%
12 Staff received training in last year
%
13 HMIS Use Index
%
14 Clinical Guidelines Index
%
15 Infrastructure Index
%
16 Patient Record Index
%
17
Facilities having TB register
%
D. Service Provision
18 Patient-Provider Care Index
%
19 Proper sharps disposal
%
20 Average new outpatient visit per month (BHC > 768 visits)
%
21 Time spent with patient (> 9 minutes)
%
22 BPHS facilities providing antenatal care
%
23 Delivery care according to BPHS
%
E. Financial Systems
24 Facilities with user fee guidelines
%
25 Facilities with exemptions for poor patients
%
F. Overall Vision
26 Females as % of new outpatients
%
27 Outpatient visit concentration index
CI (-1 to 1)
28 Patient satisfaction concentration index
CI (-1 to 1)
Composite Scores
29 Upper Benchmarks Achieved
%
30 Lower Benchmarks Achieved
%
KEY
Score Below Bottom Quintile
Score Between Bottom & Top Quintiles
Score Above Top Quintile
N/A = Not Available
Number of Facilities
Number of Patient Observations
Number of Exit Interviews
Number of Health Workers
Number of CHWs
Number of Households
RED
YELLOW
GREEN
618
5,852
5,730
1,569
167
12,904
* Benchmark set at 90%, though top quartile is 68%
24
214
204
74
10
482
8
0
0
0
0
0
0
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Provincial Results
Kandahar
Performance Based Partnership Agreement
A. Patients & Community
1 Overall Patient Satisfaction (Score = 4/4)
2 Patient Perception of Quality Index (Score > 34/36)
3 Written Shura-e-sehie activities in community
Kapisa
Khost
Konar
Kunduz
Laghman
Logar
Nangarhar
SM
Nimroz
Nooristan
Paktika
Paktiya
NGO
Panjsheer
Parwan
Samangan
Saripul
SM
SM
NGO
NGO
Takhar
Urozghan
Wardak
Zabul
NGO
77.5
42.5
28.1
87.8
24.3
84.4
18.5
1.9
68.5
23.2
3.7
61.0
54.4
10.4
32.4
55.9
26.8
58.0
90.0
30.5
58.8
61.8
17.8
55.2
74.3
5.8
0.0
91.2
23.5
58.8
41.1
27.5
49.3
60.2
25.1
41.0
NA
11.6
9.9
77.7
28.8
25.5
66.9
5.8
26.7
50.7
4.1
18.8
78.3
20.5
70.9
0.0
65.4
67.0
63.6
10.7
57.3
52.2
65.7
8.4
24.6
44.0
14.7
31.2
0.0
33.5
14.7
66.2
14.7
34.0
27.7
41.7
62.6
32.7
18.3
38.6
32.6
46.7
5.3
34.9
19.2
63.2
5.6
57.6
12.5
37.2
32.5
45.4
40.0
30.2
3.8
36.9
6.8
60.6
14.4
53.9
17.1
47.3
0.0
51.5
60.0
82.2
56.1
48.9
28.0
45.3
36.6
64.6
46.6
53.5
53.0
20.9
45.9
31.8
83.2
12.7
54.6
72.0
35.2
91.0
26.1
62.7
56.7
8.7
63.1
67.3
27.1
29.3
0.0
49.1
31.9
94.8
35.6
59.1
69.5
1.4
61.7
72.1
87.6
27.8
36.8
55.0
59.6
62.8
51.3
48.9
44.9
7.9
69.8
99.3
76.0
28.8
39.2
57.5
58.3
88.1
69.2
57.9
84.0
34.9
67.4
53.8
68.4
45.9
4.0
48.1
35.9
72.1
57.3
53.4
71.0
16.7
61.7
75.1
61.2
28.7
55.1
42.8
36.1
68.3
44.1
59.0
53.6
11.8
74.2
63.6
47.6
37.6
69.7
56.7
59.5
73.3
63.6
57.6
52.3
47.1
65.6
52.9
64.2
52.5
48.7
50.7
16.4
56.7
27.2
31.2
69.7
12.9
58.2
90.1
44.5
18.3
0.0
55.5
9.4
91.2
44.2
50.5
56.1
16.2
31.2
49.6
75.0
24.0
7.3
61.5
0.0
75.2
26.9
45.4
90.8
7.3
62.5
77.6
66.5
33.7
0.0
46.2
28.2
90.1
41.7
31.8
70.5
8.9
62.5
74.5
35.7
29.7
0.0
66.4
24.8
87.0
28.4
55.6
97.3
0.0
51.6
68.7
43.5
36.4
43.7
47.3
44.1
60.7
37.4
48.9
49.7
10.1
68.2
62.2
91.0
15.9
8.4
47.0
51.9
68.3
38.6
49.0
55.8
0.0
52.5
54.5
34.8
31.4
31.1
44.7
45.3
79.7
26.1
50.5
94.9
15.7
68.8
73.2
86.0
39.5
22.3
53.5
50.4
58.3
42.1
29.4
55.7
9.7
83.3
86.6
75.0
0.0
0.0
21.7
78.0
86.7
66.7
83.3
56.7
0.0
65.5
87.4
60.6
48.6
44.7
51.3
52.4
78.4
44.3
54.8
65.4
13.4
64.1
79.3
36.2
46.2
51.8
42.8
44.9
62.0
53.4
92.6
39.5
7.7
38.8
38.2
16.7
12.5
57.7
5.8
54.8
82.3
0.0
55.6
28.8
17.3
37.7
28.0
0.0
80.0
21.4
5.7
40.1
46.4
0.0
40.0
7.9
8.4
52.3
100.0
0.0
0.0
86.6
0.0
46.7
77.1
60.0
42.9
50.4
12.7
55.2
43.4
30.8
11.1
44.9
13.9
42.8
72.2
100.0
33.3
54.5
6.9
65.6
25.8
0.0
66.7
87.1
40.4
64.0
42.3
NA
80.0
9.8
0.0
53.1
94.0
0.0
0.0
15.2
7.4
40.9
2.6
25.0
33.3
58.5
35.3
91.5
57.0
66.7
20.0
64.2
23.0
67.4
3.1
5.6
42.9
43.1
17.3
50.1
17.8
0.0
100.0
17.8
34.3
59.5
37.5
0.0
40.0
69.4
17.3
54.0
86.7
9.1
40.0
76.0
16.6
56.5
100.0
NA
0.0
66.7
0.0
51.6
34.5
30.8
10.0
42.7
12.5
43.5
100.0
100.0
0.0
100.0
0.0
86.9
66.3
98.0
74.8
81.5
36.3
92.3
72.3
100
59.3
84.1
85.3
85.2
61.8
90.3
78.0
100
100
100
91.2
74.1
61.9
93.3
92.8
N/A
N/A
100
51.6
71.7
60.6
100
85.6
95.0
62.9
100
100
43.9
32.1
100
100
60.1
-0.104
0.022
52.4
-0.028
-0.005
46.3
-0.072
0.021
48.6
-0.087
-0.033
58.2
0.020
0.016
61.8
N/A
N/A
61.2
-0.076
0.011
59.8
N/A
N/A
42.8
0.002
0.032
39.1
-0.167
0.000
56.2
0.095
0.004
52.3
-0.066
-0.048
51.9
-0.119
N/A
53.2
-0.011
0.009
45.9
0.018
-0.002
61.1
0.010
-0.018
57.6
-0.076
-0.004
47.7
0.088
0.000
52.5
-0.045
-0.014
67.8
-0.185
-0.030
18
89
21
79
11
54
21
75
32
86
15
100
18
96
23
100
29
68
22
85
11
71
14
86
21
79
7
79
14
75
11
75
18
93
33
67
7
93
43
75
23
216
194
60
1
326
21
186
176
32
0
560
14
140
139
34
5
278
14
130
127
33
0
279
16
153
155
46
0
316
23
230
225
49
6
0
24
232
237
61
0
478
24
243
241
57
3
0
8
80
78
23
2
306
13
122
121
27
6
257
16
190
189
18
0
380
16
171
162
53
2
306
14
138
127
34
3
465
28
251
257
64
8
735
7
69
70
27
0
304
15
151
140
34
15
464
25
203
207
56
3
494
4
40
36
12
0
30
26
227
222
72
5
661
4
40
39
15
2
15
B. Staff
4 Health Worker Satisfaction Index (Score > 62/76)
5 Salary payments current
C. Capacity for Service Provision
6 Equipment Functionality Index
7 Drug Availability Index
8 Family Planning Availability Index
9 Laboratory Functionality Index (Hospitals & CHCs)
10 Staffing Index -- Meeting minimum staff guidelines
11 Provider Knowledge Score
12 Staff received training in last year
13 HMIS Use Index
14 Clinical Guidelines Index
15 Infrastructure Index
16 Patient Record Index
17
Facilities having TB register
D. Service Provision
18 Patient-Provider Care Index
19 Proper sharps disposal
20 Average new outpatient visit per month (BHC > 768 visits)
21 Time spent with patient (> 9 minutes)
22 BPHS facilities providing antenatal care
23 Delivery care according to BPHS
E. Financial Systems
24 Facilities with user fee guidelines
25 Facilities with exemptions for poor patients
F. Overall Vision
26 Females as % of new outpatients
27 Outpatient visit concentration index
28 Patient satisfaction concentration index
Composite Scores
29 Upper Benchmarks Achieved
30 Lower Benchmarks Achieved
KEY
Score Below Bottom Quintile
Score Between Bottom & Top Quintiles
Score Above Top Quintile
N/A = Not Available
Number of Facilities
Number of Patient Observations
Number of Exit Interviews
Number of Health Workers
Number of CHWs
Number of Households
9
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Methods Annex
Identification of Domains & Indicators of the BSC
The Balanced Scorecard was designed through a series of workshops and discussions
with the MOPH, NGOs, and other development partners. The team from Johns Hopkins
University and the Indian Institute of Health Management Research served as
facilitators and technical advisors. The original two workshops involved participants
working as individuals, small groups, and in plenary to arrive at a consensus on:
1) The purpose of the BSC in Afghanistan’s health sector
2) The meaning and selection of the domains to be used in the BSC
3) An initial shortlist of indicators to be used in the BSC based on a long list of
over 340 potential indicators derived from the National Health Services
Performance Assessment (NHSPA)
Each of the proposed indicators was then analyzed for: 1) reliability; 2) completeness
(i.e. missing values); 3) outlying values; and 4) sufficiency of variation. Indicators
comprising a domain were then assessed as a group to ensure that there was a good
balance of inputs, processes, outputs, and outcomes. Based on this analysis, a revised
list of indicators was proposed for each domain, and a second round of workshops held
to agree on a reduced set of indicators and proposed target levels for the domains.
The proposed BSC with results was then reviewed by the Monitoring and Evaluation
Advisory Board of the MOPH in a series of meetings. Several clarifications and
adjustments were reviewed, including a decision to use only the facility-based indicators
for the annual BSC, and a subset of the BSC to be used for those provinces that have
Performance Based Partnership Agreements (PPAs).4 A final BSC was approved by
the Advisory Board in late April 2005.
4
The PPA provinces are: Badghis, Balkh, Farah, Helmand, Kapisa, Nimroz, Panjsheer, Parwan,
Samangan, Saripul, and Wardak.
10
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Data Collection
The National Health Facilities Performance Assessment (NHFPA) was conducted from
June-October, 2004 across the whole country. It was the first nation-wide survey of its
kind in Afghanistan, and will serve as a baseline for future assessments. In addition to
the items in the BSC, a community assessment was also performed using a sample of
12,904 households from the catchment areas of the health facilities, and 74 focus group
discussions in a sample of communities. The results from the household and
community assessments are reported separately, and provide further explanations
about the results found in the BSC.
Instruments
The health facility assessments included seven instruments, each of which were pretested, translated and back-translated from English to Dari and Pashto. The facility
instruments included:
1.
2.
3.
4.
5.
6.
7.
Observation of patient consultation, under 5 years old
Observation of patient consultation, 5 years or older
Exit interview, guardian of new patient under 5 years old
Exit interview, new patient 5 years or older
Health worker interview
Community health worker interview
Health facility assessment
Study Teams
Study teams consisted of staff from the MOPH from the central and provincial
administrations, locally recruited data collectors, and members of JHBSPH and IIHMR.
Teams were trained and tested to be data collectors, supervisors, monitors and master
trainers, or regional coordinators. Others were trained to edit forms and enter data.
Data analysis was conducted by JHBSPH and IIHMR staff. In total, over 350 people
were involved in the collection of data.
Supervision instruments were used to ensure that data were collected in a complete
and consistent manner. 10% of facilities and one household per village were re-sampled
and given a repeat survey assessment by a different team member.
11
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Sampling
Sample Size of the National Health Facilities Performance Assessment 2004
Number of Facilities providing BPHS
Number of Basic Health Centers (BHCs)
Number of Comprehensive Health Centers
Number of BPHS Hospitals
Number of Patient Observations
Number of Patient Exit Interviews
Number of Health Worker Interviews
Number of Community Health Worker (CHW) Interviews
Number of Households
606
314
241
51
5,852
5,730
1,569
167
12,904
Sampling Facilities
A sampling frame of facilities was created for each province by first compiling the
MOPH list of facilities from Kabul, and updating them with the Provincial Health
Directors and key informants from NGOs and the MOPH in each Province to remove
non-existent facilities, add new ones, and find their correct names and locations. A rule
was set that if more than 1/3 of the facilities were deemed unsafe, the province would
not be surveyed until sufficient numbers were available. Facilities deemed unsafe for
travel were removed from the sampling frame.
The following steps were taken to sample the health facilities in each of 33 provinces
(there were no BPHS health facilities in the newly created province of Daykundi):
1. Facilities to be surveyed will include:
• 3 DHs
• 7 CHCs
• 15 BHCs
2. If fewer than above in any province, all the facilities will be covered
3. Two additional back up sites will be selected to replace facilities that could not be
surveyed due to security reasons
Sampling Patients
The first step in sampling for patients is to estimate the expected new patient load for
the day. The estimate is made by looking at the patient register to gain an informed
approximation of average daily volume of new patients and by asking health center staff
how many new patients they expect to see that day. At each facility, the team should
conduct five observations of consultations involving new patients under five years old
and five observations of consultations involving patients five years of age or older.
Therefore, if looking at the register and querying health center staff indicates that fewer
than ten new patients from each age stratum (under five years vs. five years or older)
12
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
are likely to receive care in the facility that day, the team should observe the first five
new patient consultations in each age stratum. If between 10 and 15 patients from each
age stratum are expected, the team should observe every other patient consultation in
each age stratum until five of each type have been conducted. If fewer than five new
patients from each age stratum come to the clinic on the day of the assessment, then
the surveyors stayed an extra day to collect the needed number of patients. The
following table shows the sampling pattern that should be followed for different
estimates of the daily patient load.
Determining sampling pattern on basis of expected number of new patients
Expected # of new patients
Sampling pattern
in age stratum (under five
years vs. five years or older)
<10
Select each eligible patient until 5 observations of
consultations involving patients in that age stratum
have been completed
11-15
Sample every second eligible patient until 5
observations of consultations involving patients in that
age stratum have been completed
16-20
Sample every third eligible patient until 5 observations
of consultations involving patients in that age stratum
have been completed
>20
Sample every fourth eligible patient until 5
observations of consultations involving patients in that
age stratum have been completed
Sampling for patient exit interviewers corresponds exactly to the pattern of sampling for
observations. An exit interview must be conducted with every patient for whom an
observation was conducted, as long as they give their consent to be interviewed.
Sampling Health Workers
For community health worker interviews, a list of all CHWs in the catchment area will be
developed for each facility. Two will be randomly selected for each facility, and asked to
come to the facility to be interviewed.
For other health workers, either all workers were interviewed, or A random sample was
taken according to the following targets:
• BHCs: 2 nurses
• CHCs: 1 doctor; 2 nurses
• District Hospitals (delivery area): 3 doctors; 2 nurses
If there were fewer health workers at a health facility, then all the health workers were
interviewed.
13
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Detailed Description of Domains and Indicators
This section defines the domains and indicators used in the BSC, including details of
how they were measured and analyzed. In nearly all cases, the results were weighted
according to a standardized distribution of facility types based on the number of each of
the three types of health facilities providing the BPHS found in the national sample.
Since the sampling of facilities was stratified by the type of facility, and since many of
the results vary according to the level of facility, this weighting compares each province
on the same basis.
Domain A: Patients and Community
Description. This domain captures how the health sector appears from the perspective
of clients and community members. It includes patient and community satisfaction,
perceptions of the level of quality of available services, and levels of community
involvement in the health system.
Aims:
• Improve community and patient perceptions of the availability and quality of health
services
• Improve community and patient satisfaction with health services
• Ensure that communities are involved in important decisions on provision of health
services, both at the health facility and community levels
Indicators
1.
2.
3.
Overall Patient Satisfaction (Score = 4.4)
Patient perceptions of Quality Index (Score > 34/36)
Written record shura-e-sehie activities in community
1. Patient Satisfaction
Description
This indicator assesses patients’ overall level of satisfaction with their visit to the health
facility. Patients were asked to point to the number of naan that best represents their
level of agreement or disagreement with the statement, “Your overall visit was
satisfactory.”
Technical Details
• The observations from Panjsher were removed from the analysis, since the standard
deviation of the responses from the province was zero.
• The weighted scores were converted to Z scores and the % above 1 Z score is
reported. The cut off score (i.e., 1 SD above the mean) for this indicator is 4.
14
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
2. Patient Perceptions of Quality Index
Description
This indicator measures patient perceptions of quality of care. The index includes nine
items, each of which was measured on the naan scale (rating from 1 to 4 naans, with 1
being lowest quality and 4 being highest quality):
1. Convenience of travel to health facility
2. Cleanliness
3. Courtesy and respect of staff
4. Trust in skills and abilities of health workers
5. Explanation of illness
6. Explanation of treatment
7. Ease of getting prescribed medications
8. Reasonableness of cost
9. Adequacy of privacy
Technical Details
• Scores on each of these 9 items were combined to create a simple index with scores
ranging from 9-36.
• The scores for each province were weighted according to a standardized distribution
of facility types.
• The weighted scores were converted to Z scores and the % above 1 Z score is
reported. The cut off score (i.e., 1 SD above the mean) for this index is 34.
3. Written shura-e-sehie activities in the community
Description
This index measures the percentage of facilities with a written record of shura-e-sehie
activities conducted in the community.
15
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Domain B: Staff
Description. This domain addresses the perspectives of staff within the health system,
including clinical staff, administrative staff, and community health workers (CHWs).
Relevant aspects include: staff satisfaction and perceptions of the work environment.
Aims:
• Improve staff satisfaction with their working environment
• Ensure provision of appropriate staff incentives, including salaries, working
conditions, required training, and relationships with colleagues
• Provide regular and timely support from management, such as supervision and
on the job training
Indicators
4.
5.
Health Worker Satisfaction Index (Score > 62/76)
Salary payments current
4. Health worker Satisfaction Index
Description
This indicator measures the satisfaction of the health worker under the following 19
items. Each of them was measured using the 4-point naan scale.
Working relationship with facility staff
Working relationship with provincial MoH staff
How well the MoH or NGO facility is managed
Relationship with local leadership
Availability of medicines
Availability of equipment
Physical condition of health facility
Availability to provide high quality care
Respect and standing in the community
Training opportunities to upgrade skills
Ability to meet needs of the community
Salary
Employment benefits
Security
Living accommodation for your children
Access to education for your children
A boss who recognizes your good work
Opportunities for promotion
Overall job satisfaction
16
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Technical Details:
• Scores on each of these 19 items were combined to create a simple index with scores
ranging from 19-76.
• The scores for each province were weighted according to a standardized distribution
of health worker type rather than type of health facility.
• The weighted scores were converted to Z scores and the % above 1 Z score is
reported. The cut off score (i.e., 1 SD above the mean) for this indicator is 62 out of a
possible 76.
5. Salary Payment Current
Description
This indicator provides information about the health workers who have been getting
their salary on regular basis at date of the interview. Payment was considered up-todate if they had been paid within the last month.
17
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Domain C: Capacity for Service Provision
Description. This domain refers to the capacity of a facility to provide a high quality of
basic package of health services. It looks at various input or structural level indicators
of a facility’s readiness to provide services, including the presence of: trained staff; staff
clinical knowledge; drug stocks; appropriate physical infrastructure; equipment and
supplies; health management information system reporting and record keeping system;
and key administrative processes.
Aims:
• Ensure adequate numbers and categories of staff with appropriate levels of skills
• Ensure adequate physical infrastructure, including buildings, drugs, equipment,
communication systems
• Ensure effective and efficient management and administrative systems, including
budgets, job descriptions, work plans, and organization communication systems
Indicators:
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
Equipment Functionality Index
Drug Availability Index
Family Planning Availability Index
Laboratory Functionality Index (Hospitals & CHCs)
Staffing Index – Meeting minimum staffing guidelines
Provider Knowledge Score
Staff received training in last year
Health Management Information Systems (HMIS) Use Index
Clinical Guidelines Index
Infrastructure Index
Patient Record Index
Facilities having tuberculosis register
6. Equipment functionality index
Description
This indicator measures the presence of functional equipment in health facilities. The
index contains the following 15 items:
• Children’s scale
• Height measure
• Adult scale
• Blood pressure cuff
• Thermometer
• Stethoscope
• Otoscope
• Sterilizer
• Section/aspiration device
18
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
•
•
•
•
•
•
Vision chart
Minor surgical set
Fetoscope
Speculum
Delivery kit
Vaccine refrigerator
For each of the above items that are present and functional, a facility received 1 point.
If the item is either not present or present but not functional, the facility received 0
points. The range of scores possible for each facility is 0-15.
7. Drug availability index
Description
This indicator measures the availability of current drugs in health facilities. The index
contains the following five items:
• Tetracycline ophthalmic ointment
• Paracetamol tabs
• Amoxicillin
• ORS packets
• Iron tablets
For each of the above items that were in stock throughout the last month, a facility
received 1 point. The range of scores possible for each facility is 0-5.
8. Family planning availability index
Description
This indicator measures the availability of family planning supplies in health facilities.
The index contains the following four items:
• Condoms
• OCP
• DMPA
• IUDs
For each of the above items that were in stock throughout the last month, a facility
received 1 point. The range of scores possible for each facility is 0-4.
19
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
9. Laboratory functionality index
Description
This indicator measures the functionality of the facility’s laboratory. The index contains
the following 11 items:
• Complete blood counts
• Malaria smears
• Rapid diagnostic test for malaria
• TB smears
• Gram stains
• Blood type and cross match
• Urine dipstick tests
• HIV testing
• Hepatitis testing
• Syphilis testing
• Pregnancy testing
For each of the above tests that a facility was able to perform on the day of the survey,
the facility received 1 point. The range of scores possible for each facility is 0-11.
Technical details
• In this analysis, only CHCs and District Hospitals were included as per guidelines
of the BPHS, even though many BHCs have functioning laboratories
10. Staffing index – meeting minimum staff requirements
Description
This indicator measures whether facilities meet BPHS requirements for total number of
clinical staff per facility type.
Technical details
•
•
Clinical staff is defined as a doctor, nurse or midwife.
The required number of clinical staff per facility type required by the BPHS is as
follows:
o BHC: 2 clinical staff required
o CHC: 6 clinical staff required
o Hospitals: 21 clinical staff required
20
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
11. Provider knowledge score
Description
This indicator measures the level of knowledge possessed by doctors, doctors’
assistants, nurses, midwives, auxiliary midwives and vaccinators.
Technical details
•
The mean % of questions answered correctly for health workers in each province
is reported, after adjusting for type of health worker.
The questions dealt with the following topics:
Doctors & Nurses
Signs indicating need for referral
Home care to provide a 6 month old
When should a child be brought back
Treatment of a child with a cough
Case of two year old boy with coughing
Correct treatment for child with cough
Midwives & Auxiliary midwives
Danger signs in pregnancy
Actions during delivery
Obstetrical complications
Danger signs during delivery
Definition of a maternal death
Vaccinators
When to give the following vaccines:
BCG
OPV1
OPV2
OPV3
DPT1
DPT2
DPT3
Measles
Child with a fever
To whom is TT given
21
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Community Health Workers
CHW who know proper hand washing
CHW who know health drinking water
CHW who know pregnancy danger signs
CHW who know neonatal danger signs
CHW who identify iodine deficiency problems
CHW who know age of starting breast weaning
CHW who identify vaccine preventable diseases
CHW who know 2 methods of contraception
CHW who know signs of TB
CHW who know danger signs of malaria
12. Staff received training in last year
Description
This indicator measures the % of staff who have attended post-service training within
the last year.
Technical detail
•
Respondents who reported attending within the last year any kind of post-service
training were included in the numerator for this variable.
13. HMIS Use Index
Description
This indicator measures the availability and upkeep of records and usage of the Health
Management Information System by under the following 5 items.
Monthly integrated activity report
Monthly aggregated activity report
Facility status report
Notifiable diseases report
Vaccination activity report
Technical Details:
• Scores on each of these 5 items were combined to create a simple index with scores
ranging from 0-5.
• The scores were converted into percentages.
22
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
14. Clinical Guidelines Index
Description
This indicator measures the availability of Clinical Guidelines at various facilities under
the following 9 items.
Integrated Management of Childhood Illness (IMCI)
Growth monitoring
Tuberculosis diagnosis and treatment
Malaria
ORT corner
Patient education materials
HMIS guidelines are present
Immunization schedule is present
Family planning guidelines are present
Technical Details:
• Scores on each of these 9 items were combined to create a simple index with scores
ranging from 0-9.
• The scores were converted into percentages.
15. Infrastructure Index
Description
This indicator measures the available infrastructure at the facility level under the
following 6 items.
Facilities having reception/ registration room
Facilities having waiting room
Facilities having room for emergencies, injections, treatments or
dressings
Facilities having minor surgery theater
Facilities having separate pharmacy in the facility
Facilities having accommodation for persons on-call
Technical Details:
• Scores on each of these 6 items were combined to create a simple index with scores
ranging from 0-6.
• The scores were converted into percentages.
23
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
16. Patient Record Index
Description
This indicator measures the maintenance of patient record at the facilities by the health
worker under the following 6 items.
Health worker completed individual record for under-five patients
Patient tally sheet is marked for under-five patients
Patient record is made in the register for under-five patients
Health worker completed individual record for five-and-over patients
Patient tally sheet is marked for five-and-over patients
Patient record is made in the register for five-and-over patients
Technical Details:
• Scores on each of these 6 items were combined to create a simple index with scores
ranging from 0-6.
• The scores were converted into percentages.
17. Facilities having TB register
Description
This indicator measures the availability and upkeep of tuberculosis registers at the
facility.
Technical Details:
• Only one indicator has been considered for this index
• The score has been given as percentage of facilities maintaining TB register.
• The original intention was to assess the treatment completion rate of tb patients (a 6
month treatment course), but only one facility in the country maintained a register with
12 months of patient records.
24
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Domain D: Service Provision
Description. This domain is used to demonstrate actual delivery of health services, in
contrast to the previous domain which examined the readiness to provide services.
This domain has two main categories: clinical quality of care (from a technical or
professional perspective) and health service utilization. Clinical quality of care is
measured largely by looking at processes and outcomes, whereas health service
coverage/utilization indicators are comprised of outputs.
Aims:
• To increase the coverage and utilization of the BPHS
• To improve the clinical quality of BPHS delivery
Indicators
18.
19.
20.
21.
22.
23.
Patient-Provider Care Index
Proper sharps disposal
Average new outpatient visit per month (BHC > 768 visits)
Time spent with patients (>9.1 minutes)
Facilities providing antenatal care
Delivery care according to BPHS
18. Patient-Provider Care Index
Description
This indicator measures the level and quality of patient provider interface under the
following 18 items.
Consultations in which age was asked by the health worker
Consultations in which health worker greets the patients
Consultations in which nature of complaint asked by the HW
Consultations in which duration of illness asked by HW
Consultations where other complaints were asked by the HW
Consultations in which health worker asks about previous treatment sought for same
condition
Adult consultations where blood pressure was taken
Consultations in which health worker examines patients body closely or actual contact
Consultations in which privacy was observed
Consultations in which patient is informed of the name of the disease
Consultations in which the causes and course of the disease are explained to the patient
Consultations in which precautions or home nursing care to be taken explained to patient
Cases receiving drugs where the name of the medicines is told to patient
Cases receiving drugs where how to take medication is explained to the patient
Cases receiving drugs where potential adverse reactions and what to do about them are
explained to patient
Cases where the signs and symptoms that should prompt patient to return to the clinic are
explained
Cases when patient is informed about the follow-up date or referral to other facility, if
follow-up or referral required
Cases where the health worker asked if the patient had any queries
25
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Technical Details:
• Scores on each of these 18 items were combined to create a simple index with scores
ranging from 0-18.
• The scores were converted into percentages.
• Where no observation was recorded, the item was removed from both the numerator
and denominator
19. Proper Sharps Disposal
Description
This indicator measures the management of sharps at the facility level.
Management of sharps
Technical Details:
• Only one indicator has been considered to assess sharps disposal at each facility.
20. Average new outpatient visit per month (BHC)
Description
This indicator assesses the new outpatient visit per month at BHCs.
Technical Details:
• Only outpatient visits at BHCs were considered for calculating the scores
• Scores have been given on the basis of 1 SD from the overall national median (768
visits per month).
21. Time spent with patient
Description
This indicator assesses the average time spent by the health worker on each patient.
Technical Details:
• Scores have been given on the basis of 1 SD from the national median (9.1 minutes
per patient).
26
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
22. BPHS facilities providing antenatal care
Description
This indicator measures the facilities providing antenatal care.
Technical Details:
• Only one indicator has been considered to assess whether antenatal care was
currently being provided at the facility.
23. Delivery care according to BPHS
Description
This indicator measures whether the facility is providing delivery care according to the
guidelines of the BPHS.
Technical Details:
• It comprises of only one indicator but with different standards for each type of facility
• BHCs are expected to provide normal deliveries
• CHCs are to provide normal deliveries, and have ability to cross-match and type blood
for transfusion and have a partograph to monitor labor
• District hospitals are additionally supposed to be able to provide caesarian section
27
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Domain E: Financial Systems
Description
This domain refers to the effectiveness and transparency of financial systems at various
levels of the health system.
Aims:
• To ensure availability of required funds in districts and clinics in a timely manner
• To promote active involvement of relevant stakeholders in a transparent financial
management process (including the budgets and grants process)
• To ensure a unified, affordable and transparent cost-sharing system
Indicators
24.
25.
Facilities with user fee guidelines
Facilities with exemptions for poor patients
24. Facilities with user fee guidelines
Description
This indicator assesses whether facilities that currently charge user fees maintain
written guidelines. Written guidelines are important to ensure that fees are administered
in a transparent and consistent manner.
Technical Details
• The sample sizes are particularly small for this variable for some provinces; 9
provinces have less than 10 facilities charging user fees that have responses for this
question, including: Kapisa (9); Parwan (9); Samangan (4); Jawzjan (2); Herat (5);
Farah (9); Nimroz (1); Zabul (2); and Uruzgan (4).
25. Facilities with exemptions for poor patients
Description
This indicator looks at whether facilities that charge user fees (70.2% of assessed
facilities nationally) have any exemptions for poor patients. When facilities charge fees,
it is important that they maintain systems for ensuring that the fees do not pose financial
barriers to patients who need to access health services. Therefore, exemptions for poor
patients is an important indicator of whether facilities attempt to ensure equitable access
for all patients, regardless of their ability to pay.
Technical Details
• The numbers are quite small for some provinces; 9 provinces have fewer than 10
facilities charging user fees that have responses for this question, including: Kapisa
(9); Parwan (9); Samangan (3); Jawzjan (2); Herat (4); Farah (9); Nimroz (1); Zabul
(2); Uruzgan (4).
28
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
Domain F: Overall Vision
26.
27.
28.
Females as % of new outpatients
Outpatient Visit Concentration Index
Patient Satisfaction Concentration Index
26. Females as % of new outpatients
Description
This indicator measures the percent of patients attending the assessed facility who are
female. This is one measure of how well health services are reaching women, which is
a key priority area of the Ministry of Public Health.
27. Outpatient visit concentration index
Description
This indicator is a measure of equity (by wealth status) of utilization of the assessed
BPHS facilities. Concentration indices range from -1 to +1, with zero indicating perfect
equality. Negative numbers between -1 and 0 indicate inequality in favor of the poor,
while positive numbers between 0 and +1 indicate inequality in favor of wealthier
groups. The outpatient visit concentration index measures how equitable utilization of
assessed BPHS facilities is; zero indicates equal utilization by all wealth groups;
negative numbers indicate higher rates of utilization by poorer groups, while positive
numbers indicate higher rates of utilization by wealthier groups. For the purposes of
this indicator, a negative number is considered more favorable.
Technical details
The data for this indicator comes from the combined under-five and over-five exit
interviews. Wealth quintiles were assigned to each exit interview by running principal
component analysis on each case, and using the cut-offs determined by the household
survey to assign quintiles. For each province, the percent of exit interviews in each
wealth quintile, weighted by facility type, was entered into the Wagstaff CI calculator,
along with the true sample size in each wealth quintile, to determine the concentration
index for each province.
The following household-level variables were included in the principal component
analysis used to create the wealth quintiles:
•
•
•
•
•
Main source of income
Main source of lighting
Main source of drinking/cooking water
Main source of fuel
Main type of toilet
29
Afghanistan Health Sector Balanced Scorecard
National and Provincial Results, Round 1 (2004)
•
Ownership of various assets, including:
o Sewing machine
o Clock or watch
o Gold jewelry
o Pressure cooker
o Radio
o Television
o Bicycle
o Motorbike
o Electricity generator
o Car
o Tractor
28. Patient satisfaction concentration index
Description
This indicator is a measure of equity (by wealth status) of patient satisfaction with the
assessed BPHS facilities. Concentration indices range from -1 to +1, with zero
indicating perfect equality. Negative numbers between -1 and 0 indicate inequality in
favor of the poor, while positive numbers between 0 and +1 indicate inequality in favor
of wealthier groups. The patient satisfaction concentration index measures how
equitable satisfaction with assessed BPHS facilities is; zero indicates the same level of
satisfaction with the health facility by all wealth groups; negative numbers indicate
higher satisfaction rates among poorer groups, while positive numbers indicate higher
satisfaction rates among wealthier groups. For the purposes of this indicator, a
negative number is considered more favorable.
Technical details
The data for this indicator comes from the combined under-five and over-five exit
interviews. Wealth quintiles were assigned to each exit interview by running principal
component analysis on each case, and using the cut-offs determined by the household
survey to assign quintiles. For each province, the average satisfaction by wealth
quintile, weighted by facility type, was entered into the Wagstaff CI calculator, along with
the true sample size in each wealth quintile, to determine the concentration index for
each province.
30