Does Community Monitoring Improve Public Services?
Based on evidence from Uganda and India the answer is “Yes” and “No”
T
wo evaluations of ambitious efforts to improve
public services show strikingly different impacts.
Both projects attempted to mobilize communities to achieve this. One project on health in Uganda
succeeded, while another on education in India had
no impact on public schools even as a private initiative
outside the public school system was able to improve
the reading skills of students.
The push toward community monitoring of public
services in development projects is based on the notion
that local oversight raises the social accountability of
public service providers. The diverging results from the
two evaluations suggest that local monitoring does not
guarantee better services.
In Uganda community monitoring led to
improved quality of health delivery
In 25 randomly selected communities local NGOs organized meetings of residents and health service providers about the quality of care in public clinics. Quality
of care was measured previously through user surveys
about the quality of service received at the clinics and
was compiled in “citizen report cards.” In another 25
randomly selected “control” communities, no such discussion took place.
In the communities where the meetings were held,
absenteeism by providers decreased and the quality of
service (measured by wait time and quality of care) improved. Ultimately, immunization rates rose and child
mortality rates dropped in these communities, suggesting that mobilizing the community to monitor providers more actively can improve services. In Uganda,
this oversight could be done through the beneficiarycontrol institutions called Health Users Management
Committees (HUMCs).1
In India community monitoring did not lead to
improved quality in public education
A leading education-focused local NGO mobilized
communities about learning in schools in 195 randomly
selected villages in the state of Uttar Pradesh. It facilitated information sharing about the quality of schools and
how to improve them through agencies like the Village
Education Committee (VEC). The VECs operate much
like the HUMCs for health services in Uganda.
September 2008
Village volunteers prepared “report cards” on the
reading ability of children; these were shared with
teachers, local government representatives, and residents in village-wide meetings. An evaluation found
no difference in community participation, teacher effort, or learning outcomes in public schools between
the villages where the meetings took place and 85 randomly selected “control” villages where no meetings
were held.
However, in 65 villages (among the 195) where a
local NGO held additional classes to improve reading
skills outside the public school system, reading scores
increased. Children who could only decipher alphabet
letters at baseline attended the reading classes and were
more than twice as likely to read words and stories
when surveyed one year later (see fig.). A large number of local youths volunteered for training in the use
of the new reading tool, and they held more than 400
reading classes across 55 villages that involved almost
7,500 children.2,3
A large number of factors may affect the
performance of community monitoring
The success of this private initiative outside the public
school system compared with the failure to improve the
public schools through the village meetings suggests that
collective public action does not guarantee better public
services. VECs in India and HUMCs in Uganda were
Reading improvement among children who could only
decipher alphabets at baseline
Control
Ablity to read a story
Ablity to read word or paragraph
Report card intervention 1
Report card intervention 2
Report card intervention 3
Private reading class
0
10
20
30
40
50
Students tested for reading (percent)
econ.worldbank.org/research
60
largely inactive at baseline, and both had only indirect
influence, mainly in the form of complaints from parents and users about non-performing teachers or health
workers. Action against them would have to be taken by
a senior bureaucrat.
So what accounts for the different outcomes in the
two countries?
Variation in NGO activism in the interventions. In Uganda, the local NGOs seemed more active in pressuring
the health care providers to improve performance than
the education NGO in India. The facilitators in Uganda directly negotiated with the dispensary staff before
involving the villagers, and the villagers who became
involved were hand-picked by them. Because the community served by each health clinic is almost ten times
larger than a village in India, the activist role for external facilitators may have been more scalable. In India
the focus was on community-led engagement.
Differences between health and education services.
Poor health services are more directly observed by users than poor teaching which can remain invisible to
parents. As a result, the users of health clinics could
be more easily spurred to demand better services they
did not know they were entitled to. In contrast, young
pupils need advocates. Monitoring the quality of teaching requires parental visits to the classroom to observe
teacher effort and performance, and one parent’s complaint may go unheeded unless it sparks complaints
from more parents.
Differences in country political economy. Teachers are often organized, politically powerful, and therefore resistant to social or bureaucratic reproach. Larger political
obstacles can constrain local collective action. Mobiliz-
Human Development and Public Services Research
ing citizens in the Indian political economy context may
be more successful by providing information for comparisons and benchmarking purposes across jurisdictions, thereby putting public pressure on politicians to
improve the quality of services.4
The contrasting results in Uganda and India underscore the importance of continuing to experiment with
information interventions in different socio-political
contexts.
Stuti Khemani, Senior Economist
[email protected]
Notes
Martina Bjorkman and Jakob Svensson. Forthcoming
“Power to the People: Evidence from a Randomized Field
Experiment of Community-Based Monitoring in Uganda.” Quarterly Journal of Economics.
2
Abhijit V. Banerjee, Rukmini Banerji, Esther Duflo, Rachel
Glennerster, and Stuti Khemani. 2008. “Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India.” World Bank Policy Research
Working Paper 4584, April.
3
Abhijit V. Banerjee, Rukmini Banerji, Esther Duflo, Rachel
Glennerster, and Stuti Khemani. 2006. “Can Information
Campaigns Spark Local Participation and Improve Outcomes? A Study of Primary Education in Uttar Pradesh,
India.” World Bank Policy Research Working Paper 3967,
July.
4
Stuti Khemani. 2007. “Can Information Campaigns Overcome Political Obstacles to Serving the Poor?” In The
Politics of Service Delivery in Democracies: Better Access
for the Poor, ed. Shantayanan Devarajan and Ingrid Widlund. Expert Group on Development Issues, Ministry for
Foreign Affairs, Sweden. Report available at http://www.
egdi.gov.se/pdf/51587.pdf.
1
econ.worldbank.org/programs/hd_and_public_services