Community Learning Systems:
Increasing Accountability for Results
from Development Assistance
Joy P. Mukaire
Lawrence L. Jackson
Annual Conference 2009
Christian Connections for International Health
The Shift in Accountability
 Change to a transitional phase should be
reflected in the shift from the traditional role of
NGOs in health of service provision
 From little accountability to anyone other than
the individual NGO and their donors
 To a change demanding that NGOs implement
health care on behalf of government, working
alongside MoH GOSS to support national
frameworks, supporting MOH in health
systems strengthening at all levels.
Struggles of M&E Systems
 Harmonization of M&E approaches


Having one National Health Strategic
Framework

Having one National M&E framework

Having one National Coordinating body
Strengthening and expanding M&E Capacity at
all levels

Promoting decentralization of M&E

Improving human capacity for M&E

Allocating adequate resources to M&E
Struggles of M&E Systems
 Using national data to “prescribe solutions” for
local level problems/needs

Treating every community/district the same
even when priorities differ in nature

Not knowing the magnitude of the problem
being addressed

Mismatch between national priority and local
needs
 Promoting utilization of program data

To decide on priorities

Make tactical changes to improve programs

To scale up proven project models
Management for Results
There is increasing pressure to manage programs with focus
on achieving results that are visible.
Monitoring and Evaluation has two purposes:


Aid program managers to improve their programs by
using data

Sub-national level (Implementation Teams)

National level (Policy and Sr. Management)
Support a reporting function

National stakeholders

Internationally [Donors, International Organizations
(RBM)]
Macro vs Local M&E Outcome Data
NATIONAL LEVEL
Program
District 1
Program
District 2
Local managers need M&E data that
show what is happening in the areas
where they work
Efforts to Strengthen M&E
 Rapid Assessment using Lot Quality Assurance
Sampling
 Sampling method used in industry since the 1920s
 Adapted to public health settings in mid-1980s
 Local managers at a sub-district level using small
samples determine whether a performance target has
been reached
 Yes or No Judgment
What is LQAS?
An analysis method that can:
 be used locally – in


“supervision areas”
to identify priority areas: those not reaching an established
performance benchmark for an indicator
data for local management decision making and for sharing
information across supervision areas
 measure coverage at an aggregate level
(e.g., program catchment area or district or nation)
 be used locally – in

“supervision areas”
suitable for Reporting Purposes
 LQAS uses small samples (less than 20)
A
Assume a program area
that has 7 supervision areas
or municipalities
B
Each one is supervised by
one person
C
D
Each one has between
25-35 promoters/wards
/communities to supervise
E
F
G
A
Good
B
C
D
E
F
G
Below Average
or Established
Benchmark
Maintain the program at the
current level
Identify Supervisors and Health
Workers that can help other Health
Workers improve their performance
Good
Identify the reasons for
program problems
Develop targeted
solutions
Below Average
or Established
Benchmark
LQAS Table: Decision Rules for Sample Sizes of 12-30 and Coverage Benchmarks or Average Coverage of 10% to 95%
Coverage Benchmarks or Average Coverage
Sample Sizes 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95%
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
1
1
1
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
2
2
2
2
2
2
2
3
3
3
3
3
3
4
4
4
4
4
4
2
3
3
3
3
3
3
4
4
4
4
4
4
5
5
5
5
5
5
3
3
4
4
4
4
5
5
5
5
5
6
6
6
6
7
7
7
7
4
4
4
5
5
5
6
6
6
6
7
7
7
8
8
8
8
9
9
5
5
5
6
6
6
7
7
7
8
8
8
9
9
9
10
10
10
11
5
6
6
6
7
7
8
8
8
9
9
10
10
10
11
11
12
12
12
6
6
7
7
8
8
9
9
9
10
10
11
11
12
12
13
13
13
14
7
7
8
8
9
9
10
10
11
11
12
12
13
13
14
14
15
15
16
7
8
8
9
9
10
11
11
12
12
13
13
14
14
15
15
16
17
17
For all coverage benchmarks (except where noted) LQAS is at least 92% sensitive and specific
N/A = Not Applicable -- Indicates that LQAS should not be used since coverage is too low for LQAS to detect.
Alpha and Beta Errors are > 10%
Alpha and Beta Errors are > 15%
8
8
9
10
10
11
11
12
13
13
14
14
15
16
16
17
18
18
19
8
9
10
10
11
12
12
13
14
14
15
16
16
17
18
18
19
20
20
9
10
11
11
12
13
13
14
15
16
16
17
18
18
19
20
21
21
22
10
11
11
12
13
14
14
15
16
17
18
18
19
20
21
21
22
23
24
11
11
12
13
14
15
16
16
17
18
19
20
21
21
22
23
24
25
26
Delivery Attended by a Clinically Trained
Provider
Benchmark = 75%, Decision Rule=12;
Mean = 65%, Decision Rule=10
* = Below Benchmark; # = Below Mean
District
These are
the
problem
supervision
areas. The
Decision
Rule is 12
women
delivering
with a
Trained
Provider
Area
Yes
1
11
2
No
*
Total
8
19
13
6
19
3
19
0
19
4
8
*#
11
19
5
8
*#
11
19
6
15
4
19
Total
74
40
114
District
Nation or
Region
District
District
SA= Supervision Area
S.A. = 19
S.A. = 19
S.A. = 19
S.A. = 19
S.A. = 19
S.A. = 19
S.A. = 19
S.A. = 19
S.A. = 19
S.A. = 19
S.A. = 19
S.A. = 19
LQAS Application in CHAS
Applied in two counties of Leer and Koch in Unity State
to establish baseline for a CHAS implemented Access
to Basic Service “Quick Impact Project”




Estimate immunization coverage among children aged
12 – 59 months;
Assess knowledge and practice of correct feeding
practices among pregnant women and children aged 0
– 23 months;
Estimate coverage of ITNs among children under 5 and
pregnant mothers;
Estimate prevalence of ARI, fever, and diarrhea among
children under 5;
LQAS Application in CHAS
 Assess knowledge and practice of mothers and
caretakers of children under 5 of critical danger
signs of and correct response to ARI, fever and
diarrhea;
 Utilization of ANC, delivery and PNC services
among pregnant women; and
 Knowledge and use of family planning methods
among women in the reproductive age bracket.
 Use of sanitary means of excreta disposal
 Use of improved water source of drinking water
LQAS Application in CHAS
The sample:
A total of 5 parallel samples corresponding to the 5 project
target groups, namely,

Heads of household

Mothers of children 0 – 11 months

Mothers of children 12 – 23 months

Mothers of children 0 – 59 months

Women aged 15 – 49
From each randomly selected village

133 households in Koch x 5 interviews =

152 households in Leer x 5 interview =
LQAS – A Catalyst for Change
A team of 14 people were trained in Unity State,
six at HQ and 8 at county level
Mobilizes managers and their organizations to
appraise their programs
Empowers them by creating a learning
environment
Supports leadership at national, district and
sub-district levels
Advantages of Using LQAS
 Can be used at a local level with modest amounts of
supervision
 Produces information that can be rapidly interpreted by
local managers
 Identifies where the successes and challenges are
located
 Paper/pencil analyses rather than requiring computer
analyses
 Data can be used for national reporting, and
 Sampling theory is rigorous
Lessons Learned
 LQAS findings revealed substantial variations in
estimates of counties
 The magnitude of “Sanitation” problem became more
visible to implementers than before something the
project was not even planning to address.
 The substantial variation imply that local managers are
better placed to make tactical changes in their
Programme
 Those responsible for providing services are better
placed to analyze the challenges in their areas and
decide on the strategic change
The Christian Perspective
 This is one solutions to increased STEWARDSHIP of
faith-based organizations
 This tool is just but a means to acceptable standards of
ACCOUNTABILITY where resources are limited and
competing priorities are many
 This tool builds a culture of result-oriented management
and the culture of accountability, a contribution Christian
health services can make to Strengthening Health
Information Systems and Monitoring and Evaluation
The Power of Measuring Results
 If you don’t measure results you cant tell success from
failure
 If you don’t see success, you cannot reward it
 If you cannot reward success, you are probably
awarding failure
 If you cannot recognize failure, you cannot correct it
 If you cannot demonstrate results, you cannot win
public support
Source: Adapted from Osborne & Gaebler 1992
Getting Ready
Household Interviews
Household Interviews