Sustainable Handpump
Projects in Africa
Draft Guidelines for
Field Evaluation of
Handpump Projects
Produced under a Knowledge and Research (KAR)
project funded by the Department for International
Development (DFID), UK. Contract No. R7817
S. Parry-Jones, R. Reed and B. H. Skinner
Water, Engineering and Development Centre
Loughborough University
2001
Water, Engineering and Development Centre
Loughborough University
Leicestershire
LE11 3TU UK
© WEDC, Loughborough University, September 2001
Any part of this publication may be copied, reproduced or
adapted to meet local needs, without permission from the
authors or publisher, provided the parts reproduced are
distributed free, or at cost and not for commercial ends,
and the sources is fully acknowledged as given below.
Please send copies of any materials in which text has
been used to WEDC Publications at the address given
above.
A copy of this publication is available online via:
http://www.lboro.ac.uk/departments/cv/wedc/projects/shp
/index.htm
Parry-Jones, S., Reed, R. A., and Skinner, B. H.
WEDC, Loughborough University. UK.
This document is an output from a project funded by the
UK Department for International Development (DFID)
for the benefit of low-income countries.
iii
The views expressed are not necessarily those of DFID.
Contents
Contents
1
1.
Introduction
1
2.
What is the purpose of the evaluation?
3
3.
Who should be involved in the evaluation?
4
4.
What techniques are available?
6
5.
What data and information should be gathered?
7
6.
How should the fieldwork be carried out?
11
7. What should be done with the data and information
collected?
13
References
15
Appendix 1 Tools
16
A1.1....... Sustainability snapshot
A1.2....... Checklists
A1.3....... Village handpump data summary sheet
A1.4....... Participatory tools
17
22
34
36
1. Introduction
This draft report comprises a set of tools and suggested methodology for carrying out field evaluation of existing handpump projects in Africa. It is based on
the information gathered in the literature review and the main author’s personal
experience in collecting field data and evaluating rural water supply and sanitation projects.
This guideline is set out logically to follow the evaluation process (see Box 1)
from inception to completion, and is structured around the following set of
questions:
1.
What is the purpose of the evaluation?
2.
Who should be involved in the evaluation?
3.
What type of data and information need to be gathered?
4.
What techniques are available?
5.
How should the fieldwork be carried out?
6.
What should be done with the data and information collected?
This report is one of the interim outputs from the DFID Funded Knowledge
and Research Project (KAR) carried out by WEDC under Contract R7817. A
printable version of this report and further details about this research project
and other outputs from it can be found on web page:
http:/www.lboro.ac.uk/wed/projects/shp/index.htm
Comments on the content of this report are welcomed, and should be sent by email to [email protected] or sent by post to WEDC. It is hoped that the
suggested methodologies will also be the subject of a discussion on the HTN email discussion list. This can be joined at http://jiscmail.ac.uk/lists/htn.html.
An archive of past discussions on a number of handpump topics is also found
at this web site.
1
Box 1: Checklist of the evaluation process
1 .Planning
Deciding
Why are you doing it?
For whom?
What is to be evaluated?
By whom?
How will it be managed?
When will it be done?
What are the resource implications?
How will the results be used?
Designing
Stating what is to be evaluated
Choosing and designing methods
Choosing measurements and indicators
Choosing methods for recording, disseminating & storing findings
2. Action
Collecting information
What? (facts, quantitative and qualitative data)
How? (questioning, observing, reading)
Analysing
Interpreting information
Assessing results
Judging
Deciding on the value of the results
Presenting
Validating analysis and judgements
Format of report/presentation
3. Using results
Implementing
Agreeing recommendations
Making changes
Feedback
Reviewing needs and design of the evaluation
Storing
Storing results so they are accessible
(Source: Save the Children, 1995)
2
2. What is the purpose of the evaluation?
The purpose of an evaluation is to assess whether the project objectives have
been achieved and what the impact of the project has been. In the context of
sustainability it is also important to also try and look into the future to determine which factors may affect the long-term use of the handpumps.
For the purposes of this work a definition of a sustainable handpump project
has been drawn from the various definitions and criteria identified in the literature review (Parry-Jones et al., 2001). A handpump project can be defined as
sustainable if the pumps:
 are being used efficiently, effectively and equitably by users
 can be managed and financed by users with limited external support
 will continue to deliver benefits for a long period after project inputs cease
The purpose of this evaluation is therefore to examine the factors relating to
use, management and financing of the handpump and to assess the likelihood
of the continued flow of benefits. The timeframe is a difficult issue since it
may vary depending on context and perspective. It probably needs to be discussed with project staff and other stakeholders.
Box 2: Key questions
Key questions to be asked in this handpump sustainability evaluation:
 Are the handpumps presently sustainable? Why?
 Will the handpumps be sustainable in the future? Why?
 What information should be shared with different stakeholders?
 What lessons can be learned and how can they be disseminated?
3
3. Who should be involved in the
evaluation?
Anyone that either is involved in service delivery, maintenance or use of the
handpumps should be involved in the evaluation since all these people will
have valuable information or views relating to the sustainability of the pumps.
People who have a particularly important role, for example a village committee
chairperson, or a project officer, are known as key informants. The range of
stakeholders that should participate in the evaluation process is shown in Table
1 below.
Table 1: Stakeholders and key informants to be involved in evaluation
Stakeholders
Key informants
Role in evaluation
National and regional
government staff
Technical staff
Providing information
Local government staff
Technical and political
representatives
Providing information
and data collection/facilitation
Project team
Technical and software
staff
Providing information
and data collection/facilitation
NGOs
Staff from waterrelated projects
Providing information
Private sector artisans
Well diggers, handpump mechanics
Providing information
Water vendors
Vendors/transporters of
water from handpumps
Providing information
Local businesses
Handpump manufacturers, stockists of
pumps/spares
Providing information
Village water committee
Chairperson, treasurer,
caretaker
Providing information
and data collection/participation
Water users
Particularly women,
disadvantaged groups
Providing information
and data collection/participation
Institutions (schools,
clinics)
Head teachers, community health workers
Providing information
and data collection/participation
Donor (if present)
Technical staff
Providing information
As shown in Table 1, the different stakeholders will be involved in the evaluation in different ways. The project staff and government partners should be ac-
4
tively involved in the evaluation process and facilitate data collection in the villages. The primary stakeholders (i.e. communities) should participate fully to
allow a two-way flow of information. The people on the external (WEDC)
evaluation team should both be familiar with data collection in rural areas, and
one person should have experience of using participatory tools such as PHAST.
5
4. What techniques are available?
There are a number of different techniques for collecting data and information.
Broadly these are:
 Secondary sources (reviewing project documentation)
 Direct observation
 Questionnaire and household surveys
 Semi-structured interviews
 Focus group discussions
 Group meetings or brainstorming
 Participatory approaches
In order to get the maximum depth and objectively of data, it is always recommended to mix data collection techniques. For this handpump evaluation process a variety of tools and techniques should be used to gain objective
quantitative and qualitative data. The formal questionnaire survey is not considered to be necessary or appropriate since it requires relatively high resource
inputs to achieve statistically robust data.
Section 5 details the type of information and data to be gathered, and links the
tools to be used which are presented in Appendix 1.
6
5. What data and information should be
gathered?
In this project, we are focusing on existing handpump projects to try and discover which factors promote sustainability. Although there are many factors
that are interlinked, or contribute, to sustainability it is important not to get
sidetracked when carrying out fieldwork. For example, whilst it may be interesting to discuss farming practices or religious beliefs in the villages visited,
this information will not necessarily contribute to understanding handpump
sustainability. It is easy to be drawn away from the main issues and consequently miss important information. The use of this draft guideline should ensure that only relevant data are collected.
The key areas of influence relating to sustainability as identified in Table 1 in
the literature review (Parry-Jones et al., 2001) are:
 Policy environment
 Institutional arrangements
 Technology
 Natural environment
 Community and social aspects
 Financing and cost recovery
 The project process
 Key linkages (training, IEC, supply chains)
Data need to be collected on all these areas, but the difficulty is to identify exactly what should be measured or quantified and how? Under each area, there
are issues that can be measured quantitatively (e.g. How much money is available for maintenance? How many people use the pump? How many times has
the pump broken down?). Measurement of others issues is more qualitative or
subjective, and the findings will depend on whom you speak to (e.g. sense of
ownership, user satisfaction, willingness to maintain the pump).
Table 2 below gives an overview of the types of qualitative and quantitative information that need to be collected for each sustainability factor. It also indicates which tools should be applied; these tools are detailed in Appendix 1.
It is difficult to score or rank projects on sustainability issues, but this would
allow different projects to be compared objectively. The WaterAid ‘Sustainability Snapshot’ provides a crude scoring system so a range of handpump projects could be compared for sustainability (see Appendix 1).
7
Table 2: Tools for quantitative and qualitative data collection on sustainability factors
Sustainability
factor
Quantitative data
Qualitative data
Tools
Issues
Tools
Issues
Policy environment
 Policy documents and legislation

Checklist A
 Political will

Checklist A
Institutional
arrangements
 Responsible bodies in place


Venn diagram
Sustainability snapshot
Checklists B & C
 Functions performed satisfactorily

 Roles and responsibilities
clear
 Cooperation

 Staff commitment/motivation

Sustainability snapshot
Checklists B & C
Handpump data sheet
Transect walk
Handpump data sheet
Sustainability snapshot
Checklists B, C & D
 Quality of construction

 Willingness to maintain
Handpump data sheet
 Sustainability snapshot
Sustainability snapshot
Checklists B & C
 User perception - taste

Transect walk
Community map
Venn diagram
Handpump data sheet
Sustainability snapshot
Checklist C
 Sense of ownership

 Exclusion or conflict

Handpump data sheet
Sustainability snapshot
 Poverty in village

 Trust and transparency
Sustainability snapshot
 Community map
Sustainability snapshot
Checklists B & C
 Participatory approaches

 Coordination/communication

 Resources
Technology
 Handpump performance

 Availability of spares

 Design criteria – e.g. number
of users

Natural environment
 Water quality - corrosion

 Aquifer properties

Community and
social aspects
 Use of other sources

 Water usage patterns

 Community roles e.g. in
preventive maintenance





Financing and cost
recovery
 Capital contributions

 Maintenance money available

The project process
 Clear objectives/benefits

 Maintenance in project document

 Timing/phasing
8
 User satisfaction
 Demand responsive
Sustainability snapshot
Handpump data sheet
Sustainability snapshot
Sustainability snapshot
 Checklists B & C
Key linkages
 Training activities

 IEC (information, education,
communication) activities

Sustainability snapshot
Checklists B, C & D
 Private sector stimulus
9
Note that the literature (Parry-Jones et al., 2001) identified some key areas of
weakness that often undermine the chances of sustainability unless they are addressed:
 On-going use of alternative (non-potable) sources
 Lack of user involvement in choice of technology
 Failure of community to undertake preventive maintenance
 Poor systems for cost recovery for maintenance
 Inadequate training and ongoing support for community
 Excessive numbers of users on handpumps
 Inadequate attention to water quality leading to taste and corrosion problems
Below is a summary of the tools referred to in Table 2, which are all detailed in
Appendix 1.
 Sustainability snapshot: comprises 14 questions to be used with communities to provide an overview of village-level sustainability of handpumps
 Checklist A: To guide discussions and meetings with national and regional
level stakeholders
 Checklist B: To guide discussions and meetings with local government and
other local level partners
 Checklist C: To guide discussions and meetings with village-level organisations and to supplement information from the sustainability snapshot
 Checklist D: To guide interviews with private sector partners (suppliers,
manufacturers, local traders and artisans)
 Transect walk: informal walk through village to assist with observation to
be made with Checklist C.
 Community map: participatory mapping exercise with a community that
can also lead to discussion on issues of primary importance to villagers
 Venn diagram: participatory tool to map the inter-relation of different
formal and informal institutions and power structures in the village
 Village handpump data sheet: data sheet summarising data for each
handpump visited in a village, and some village-specific information on
handpump operation and maintenance
10
6. How should the fieldwork be carried
out?
The actual fieldwork programme will depend on the time available, distance to
project sites, logistical problems and so on. Table 3 below provide a suggested
schedule of activities, showing which tools should be matched with different
stakeholders and activities. It is important to note that the national/regional
level consultations should be carried out prior to fieldwork (day X) so that
evaluators are familiar with the context.
It is suggested that the sustainability snapshot should not be attempted until the
evaluators (and the project staff and local government) have had a chance to
see how things are really going in the field. The tool can be introduced in the
initial meetings prior to fieldwork, but the answers should not be decided upon
until after the fieldwork (visiting handpumps and villages) has been completed.
The final synthesis and analysis of data will be carried out by the evaluation
team with the use of the ‘Project Sustainability Summary Sheet’ (see Section
7).
Table 3: Suggested project evaluation schedule and phased use of
tools
Day
Location/activities
Key Stakeholders
X
Regional/national
administrative
centre for project
- meetings
National staff
(government/project/donor
)
Checklist A
1
Project office and
local government
offices – briefing
and interviews to
gather data.
Project officers and
local government
staff
Checklist B
Field visits to
villages – handpump inspections and
participatory
discussions
Community leaders, typical users,
CBOs, artisans,
pump mechanics,
traders
Sustainability Snapshot
2, 3,
4
Tools
(Introduce the Sustainability
Snapshot tool)
Checklist C
Transect walk
Community map
Venn diagram
Handpump data sheet – one
per village
2,3,4
5
Visits or meetings
with private
sector
Suppliers, manufacturers, traders,
artisans
Analy-
Evaluation team
Checklist D
11
sis/synthesis of
data collected
6
12
Feedback and
concluding
discussions
Evaluation team,
local government
staff, community
representatives
7. What should be done with the data and
information collected?
Synthesis of information and data collected is often the most difficult part of an
evaluation. The handling of qualitative data is particularly difficult since it is
largely subjective and may be difficult to support with hard facts and figures.
The set of tools in this guideline will result in a range of quantitative and qualitative data. The sustainability snapshot and the village handpump data sheet
will both provide easily interpretable data on whether the handpumps are currently sustainable. These two tools will also allow for some comparing and
contrasting between projects.
In order to get a better understanding of why the projects are sustainable or not,
and to predict future sustainability, the data from the participatory tools and the
interviews and meetings with various stakeholders provides a deeper insight.
The evaluators will have to use their judgment in interpreting this data and
drawing conclusions.
It will help if all sustainability issues are considered under the eight key sustainability factors identified in Section 5. Particular attention should also be
given to the common areas of weakness related to sustainability that are also
detailed in that Section of this document. Table 4 below may help to summarize and analyse the field data, although it can only capture some of the key issues.
13
Table 4: Project sustainability summary sheet
Sustainability
factor
Sustainability
snapshot scores
(maximum sustainability = 3)*
Key areas of sustainability strengths
Community level
(1) Policy environment
N/a
(2) Institutional
arrangements
1.25/1.00
(3) Technology
2.25/2.25
(4) Community
and social
aspects
1.25/2.75/1.25/1.
00
(5) Financing
and cost recovery
1.25/1.00
(6) Natural
environment
N/a
Local support structures
Key areas of sustainability weakness
Community level
Local support structures
N/a
2.25/2.5
(7) The project
process
2.00
(8) Key linkages
2.75
Summary of sustainability prospects
for handpumps installed under this
project:
* Scores shown in this column are examples for each issue investigated for that sustainability factor (See Appendix 1.1)
14
References
Parry-Jones S., Reed R. and Skinner B.H. (2001) Sustainable Handpump Projects in
Africa: A literature review, WEDC, Loughborough University, UK
(available at http:/www.lboro.ac.uk/wed/projects/shp/index.htm)
Save the Children (1995) Toolkits. A practical guide to assessment, monitoring, review
and evaluation. Development Manual 5. Save the Children 1995, reprinted 1998
Sugden S. (2001) ‘Assessing sustainability – the sustainability snapshot’. Paper presented at forthcoming 27th WEDC conference, Lusaka, Zambia, 2001
(available at http://www.lboro.ac.uk/departments/cv/wedc/conferences/index.htm)
15
Appendix 1 Tools
This section contains a complete set of tools, which can be used to evaluate the
sustainability of handpump projects. Most of them have been developed by the
author and have already been used in the field on project evaluations. The Sustainability Snapshot tool has been adapted and expanded from the original
work carried out by WaterAid (Sugden, 2001). However, all these tools are
presently in draft format and it is intended that they should be amended and
improved according to local circumstances and as experience is gained during
the evaluation process.
The tools contained in this section are:
A1.1
A1.2
A1.3
A1.4
16
Sustainability snapshot
Checklists
Village Handpump data summary sheet
Participatory tools
A1.1 Sustainability snapshot
The ‘sustainability snapshot’ tool was developed by WaterAid to assist project
officers and partners in assessing the sustainability of water sources within
their projects. The idea is that both the process and the outcome should prompt
practitioners to discuss the challenges and problems surrounding handpump
operation and maintenance. It can lead to identification of systemic weaknesses and potential action for change.
WaterAid are still in the process of developing and testing this tool, but are
keen for other practitioners and researchers to take on the concept. The basic
tool has been expanded and developed by the main author during the preparation of these draft guidelines for field evaluation. Since it has not yet been
field tested, it will certainly need to evolve and be improved as fieldwork progresses.
The 14 sustainability issues are arranged around seven of the eight key sustainability factors that were identified by the literature review (Parry-Jones et al.,
2001) associated with this project. These are:
1. Policy environment
2. Institutional arrangements
3. Technology
4. Community and social aspects
5. Financing and cost recovery
6. Natural environment
7. Project process
8. Linkages (training, IEC (information, education, communication), private
sector)
The issue of policy environment is not considered to be directly relevant when
assessing sustainability at village level and no questions have been included on
policy.
For each issue identified, three statements have been developed which represent a continuum from least sustainable (score of 1) to most sustainable (score
of 3). The facilitators using the tool need to arrive at a consensus for each
question as to where the village is on the scale. So for example, the issue relating to maintenance funds (Question 5a) has three statements:
1. No funds available for maintenance when needed
2. Some funds available but not sufficient for most expensive jobs
3. Fund available and sufficient to cover most expensive jobs
17
The evaluation or facilitation team should discuss this issue with the community and decide which of the three statements most closely relates to the current
status in the village. This could be done through a focus group discussion, or
meeting with the village committee or even through a voting process. Once
agreement has been reached on the issues, a corresponding 1, 2 or 3 is then inserted into Sustainability grid. This is repeated for all 14 issues in the sustainability snapshot. A completed grid is shown below. The average scores can
provide a crude means of comparison between villages and also serve to highlight which key areas are weakest in relation to sustainability. In the example
below, it can be quickly seen that the weak factors are institutional arrangements, community and social issues and financing and cost recovery.
This rapid assessment should lead to further discussion around the issues that
have been identified as a potential threat to handpump sustainability.
Sustainability snapshot grid (filled out for four villages in a particular project)
Project name:
Village name:
Date:
Handpump & and reference
number:
18
XX rural water supply project
W
X
Y
Z
23/06/
01
23/06/
01
23/06/
01
23/06/
01
Afridev
Afridev
India
Mk3
Tara
Ave
score
Question
Issue
2(a)
Management systems
1
3
2
1
1.25
2(b)
Major repairs
1
1
1
1
1.00
3(a)
Technical skills
2
2
3
2
2.25
3(b)
Equipment and
spares
2
1
3
3
2.25
4(a)
Use
1
2
1
1
1.25
4(b)
Access/exclusion
2
3
3
3
2.75
4(c)
Preventive maintenance
1
1
2
1
1.25
4(d)
User satisfaction
2
2
1
1
1.00
5(a)
Maintenance funds
1
1
2
1
1.25
5(b)
Capital contribution
1
1
1
1
1.00
6(a)
Quality
2
1
3
3
2.25
6(b)
Source reliability
3
3
3
1
2.50
7(a)
Participation
2
2
2
2
2.00
8(a)
Training
2
3
2
2
2.25
1.64
1.89
2.07
1.64
Average score:
19
Sustainability snapshot questions
Factor
Issue
1
.
Policy environment
None at village level
None
2
.
Institutional arrangements
(a) Management systems
1. No village organization has responsibility for water point
2. Village has organisation but is not managing point satisfactorily
3. Village organization actively managing system to everyone’s satisfaction
1. Community would not know what to do in event of major breakdown
2. No clear procedure, responsibility unclear in case of major breakdown
3. Confident that pump would be quickly repaired in case of major
breakdown
1. Technical skills not available to community for maintenance when
needed
2. Some technical skills available for maintenance, but not all
3. Technical skills for all maintenance processes available
1. Maintenance equipment and spare parts not available
2. Some availability but not for all repairs
3. Available for all repairs
1. Handpump source never used for drinking water
2. Handpump source sometimes/normally used for drinking water
3. Handpump source always used for drinking water
1. Some people never get access to the pump even when they want to
use it
2. Some people sometimes do not get access to the pump
3. All the people who want to use the pump gain access all the time
1. No preventive maintenance being carried out on pump
2. Some preventive maintenance being carried out, but not regularly
3. Regular programme of preventive maintenance carried out
1. Don’t like the handpump and would prefer other water sources
2. Like the handpump but are concerned about sustainability
3. Happy with the pump and believe they will be able to sustain it
(b) Major breakdowns
3
.
Technology
(a) Technical skills
(b) Equipment and spares
4
.
Community and social
aspects
(a) Use
(b) Access/exclusion
(b) Preventive maintenance
(c) User satisfaction
20
Statements
Factor
5
Financing/cost recov. ery
Issue
(a) Maintenance funds
(b) Capital contribution
6
.
Natural environment
(a) Quality
(b) Source reliability
7
.
Project process
(a) Participation
8
.
Linkages
(a) Training
Statements
1. No funds available for maintenance when needed
2. Some funds available but not sufficient for most expensive jobs
3. Funds available and sufficient to cover most expensive jobs
1. Community did not make any financial or in-kind contribution
towards pump
2. Community made significant in-kind contribution (set by project)
3. Community made financial contribution (set by project)
1. None of the people who use the pump perceive it to be good for
drinking
2. Some of the people who use the pump perceive it to be good for
drinking
3. Everyone who uses the pump perceives it to be good for drinking
1. The pump yield is poor – people have to use other sources all the
time
2. Sometimes (dry season) the pump yield is inadequate to meet needs
3. The pump always meets everyone’s needs
1. The pump was “given”, community not offered choice if they wanted to participate
2. Community was asked if they wanted to participate
3. The community initiated the project themselves
1. No-one in village received any structured training from project or
government staff
2. Some people trained but cannot remember or apply what was
learned
3. Useful training was provided which still benefits trainees now
21
A1.2 Checklists
This is a set of checklists that should be used when holding interviews, meetings or discussions with different stakeholders. It is important that notes are
taken on the issues discussed, and that all of the key areas on the checklists are
covered (even if no useful or relevant information is forthcoming!).
The checklists in this section are:
Checklist A: National and/or regional stakeholders
Checklist B: Local government and project partner stakeholders
Checklist C: Community organizations and primary stakeholders
Checklist D: Private sector stakeholders
22
Checklist A: National and/or Regional Stakeholders
Demographic or baseline data
 Data available for project area at Regional/National level?
 Plans to improve data on rural water supply?
Policy
 National Water Policy in place? What stage is it at?
 Does it cover standardisation, local manufacture, gender, cost
recovery?
 Is the policy being implemented?
What are the main con-
straints?
Institutional arrangements
 Organogram of key stakeholders at national/regional level?
 Roles and responsibilities of stakeholders clearly defined?
 Communication and coordination between levels?
 Relationship with local level structures?
 Role of private sector with respect to handpumps?
 Role of NGOs in rural water supply projects?
Funding and flow of funds
 How is funding arranged for rural water supply sector?
 How do funds flow to local level?
 How are recurrent costs paid for?
 What are the cost sharing arrangements for handpump projects?
Resources
 Are there adequate resources at national/regional level to fulfil roles?
23
 What areas are most lacking?
Capacity building
 Have staff at national/regional level received capacity building?
 Do they provide capacity building to local government or other
partners?
24
Checklist B: Local Government and project partner stakeholders
Policy
 Aware of national policies?
Are they relevant to handpump
projects?
 Are staff trying to implement the policy?
What are the con-
straints?
Institutional arrangements
 Organogram for rural water supply?
 Responsibility for mobilisation, installation and maintenance of
handpumps?
 Are roles and responsibilities clearly defined?
 What constraints are there to fulfilling roles?
 What is relationship like with national/regional level and communities?
 What is the role of the community in handpump maintenance?
 Are NGOs and/or private sector involved in any handpumprelated activities?
Handpump technology issues
 What percentage of handpumps are functioning at present?
 What types of handpump are installed? Are they standardized?
 Is there a standard design for a handpump installation?
 Where are handpumps obtained? Where are they manufactured?
 Where are spares available?
Who buys them? Is there adequate
supply?
 What are the commonest causes of handpump failure?
 Is there is system in place for carrying out major repairs?
 What is the design criterion for number of users per pump?
Community and social issues
25
 What mobilisation work is done with communities for handpump projects?
 Have staff been trained in participatory approaches?
 What type of sources do people use if they do not have access to
handpumps?
 What is the role of women in handpump projects?
 How is the community organized to operate and maintain the
pump?
 Have they been given toolkits for maintenance?
 Are they generally carrying out preventive maintenance?
 Who owns the pump? And the borehole or well?
Financing and cost recovery
 Are there clear cost sharing mechanisms in place?
 What do communities contribute towards cost of handpump and
installation?
 Do communities know how much it costs to maintain a handpump?
 Are they regularly collecting money for routine maintenance?
 Can communities afford the full cost of maintenance?
 Who pays for the cost of major repairs (e.g. dropped pipes, new
rising main)?
Checklist B: (continued)
Local Government and project partner stakeholders
Natural environment
 What is the most significant aquifer in the region?
 Typical depth of borehole or well?
 Who does the exploration and siting for boreholes?
With what
equipment?
 What is the success rate of drilling?
 Is there a problem with boreholes drying up during the dry season?
26
 Is chemical composition of groundwater tested during drilling?
Subsequently?
 Is bacteriological quality of water tested or monitored?
Project process
 Are water supply activities based on data about coverage or scarcity?
 Is there a mechanism for communities to apply for a source?
 Are technologies other than handpumps offered to communities?
 Is a Memorandum of Understanding signed?
 What is the planning and implementation process for handpump
installation?
 Who is responsible for quality control during construction?
 Is there a formal handover of the handpump?
 What is done to monitor performance of handpumps once installed?
Key linkages (training, IEC, supply chains)
 What training is provided to communities?
 How is training phased or linked with implementation?
 How are participants for training selected?
 Is implementation of water supply linked to hygiene education?
 How do communities communicate with local government and
vice versa?
 Is there a supply chain for spare parts? Could it be improved?
27
Checklist C: Community organisations and primary stakeholders
Institutional arrangements
 Is there a formal organization responsible for managing the
pump?
 Who is on this committee or organization (gender)?
 Are roles and responsibilities of organization members clearly
defined?
 What constraints are there to fulfilling roles?
 Does everyone trust the organization?
 How much contact does the village have with the local water
supply department?
 What is the role of the community in handpump maintenance?
 Are there private people in village or locality who can work with
the pump?
Handpump technology issues
 How many handpumps are in the village?
 How many are functioning at present?
 Are breakdowns frequent?
 What are the common problems with the pumps?
 Do they have access to spare parts locally?
 Can they afford to buy them?
 What do they do when the pump breaks down?
Community and social issues
 Who owns the pump? And the borehole or well?
 Are all handpumps used equally or are some more popular?
Why?
 What other sources are there in or around the village? Are these
used?
 What is the borehole water used for?
28
 What is the role of women in relation to the handpumps?
 How is the community organized to operate and maintain the
pump?
 Have they established any rules with regard to the pump?
 Have they got a toolkit for maintenance?
 Are they doing preventive maintenance? (Specify what and
when)
 Are they satisfied with the handpump? Why?
 Who owns the pump?
Financing and cost recovery
 Did they contribute towards cost of handpump and installation?
 Do communities know how much it costs to maintain a handpump?
 Are they regularly collecting money for routine maintenance?
How?
 How much money do they have at present?
 How much have they spent on maintenance in the past year?
 Who would pay a major repair (e.g. dropped pipes, new rising
main)?
 Do they consider they can afford to maintain the pump?
29
Checklist C: (continued)
Community organisations and primary stakeholders
Natural environment
 How is the taste of the water? Is it acceptable for drinking?
 Is it acceptable for washing clothes (no discolouration)?
 Does the quality vary at different times of the year?
 Is the quantity of water adequate for everyone or is it rationed?
 Does the quantity available vary at different times of the year?
 Is the pump used all year round? If not, why not?
Project process
 When did the community first get involved in the provision of a
handpump?
 Were they offered any alternative technologies (e.g. bucket and
windlass)?
 Were they clear about what their responsibilities were throughout the project?
 Did they sign a Memorandum of Understanding?
 To what extent did they participate during installation?
 Are they happy with the quality of the work done?
 Was there a formal handover of the handpump?
 Does the community monitor performance of the pump?
 Do they report back to local government on performance?
Key linkages (training, IEC, supply chains)
 What training did they receive in relation to the pump?
 When was this training received (before or after installation)?
 How were participants for training selected?
 Are they confident with the skills they gained from training?
 Have they had any training on hygiene education?
 How do communities communicate with local government and
vice versa?
30
 Are there regular visits from extension staff?
 Are there local businesses that would be interested in supplying
parts?
31
Checklist D: Private sector stakeholders
Private pump mechanics
 Where and when did they receive training?
 How were they selected to be trained?
 What did the training comprise?
 How long have they been working as a pump mechanic?
 How many pumps do they work on now?
 Who pays them for work? How much?
 Where do they get spare parts from?
 Can they carry out all repairs? What do they do if it is beyond
their capacity?
 Do they have other employment?
Pump or spare part suppliers
 Which models of pump do they stock (or hold parts for)?
 How long have they been stocking pumps/parts?
 How did they initially go into business (e.g. with support from
project)?
 Is the business now making a profit without external support or
subsidy?
 Do they supply the full range of parts?
 Where do they purchase these items?
 Who are normally their customers for pumps/parts?
 How could business be improved?
Water vendors
 How long have they been vending water?
 How much do they charge for water?
32
 What type of people are their typical customers (trade, private,
farmers)?
 Do they pay towards pump maintenance?
 What would they do, or do they do, when the pump breaks
down?
33
A1.3 Village handpump data summary sheet
Date of visit and location/name of
village:
Information on handpump maintenance
Number of Households/Population
Number of handpumps in village
Active organization for handpump
management
Number of men in committee
Number of women in committee
How/when is money collected for O&M?
Do they currently have funds set aside
for O&M?
How much money is available at present?
Who, if anyone, does preventive maintenance?
Who fixes broken pumps?
Where are spare parts available from?
Who buys spare parts when needed?
Who owns the pump(s)?
Details of each handpump in village
Type of handpump (model, make, country of origin)
Type of source (drilled borehole, handdug well)
Date of source completion (and by
whom)
Depth to cylinder (m)
Date of installation (and by whom)
Riser pipe material (uPVC, steel, GI)
Currently functioning
When did it last break down/fail?
What was the problem?
How long was it broken?
Who fixed it?
Who paid and how much did it cost?
How many times has it broken:(1) in the
34
Nr 1
Nr 2
Nr 3
Nr 4
last year?
was installed?
(2) Since it
Approximate yield (litres/second)
Does this vary with seasons?
How many users/households use this
pump?
A1.3 Village handpump data sheet (continued)
Details of each handpump in village
(continued)
Nr 1
Nr 2
Nr 3
Nr 4
Is rationing or restriction in place?
Is the pump lockable?
How is the taste?
What is the water used for?
If it functions but is not used, what is
the problem?
Condition of apron
Drainage adequate
General quality of construction/installation
General observations and comments on
pump quality, operation, usage, problems etc
35
A1.4 Participatory tools
The fieldwork should be supplemented with some basis participatory tools to
ensure that valid and useful data are obtained from community members. It is
suggested that the following tools are used in conjunction with the sustainability snapshot, checklist C and the ‘village handpump data summary:
 Transect walk: This is simply an informal walk through the village to
identify key features, and to visit and inspect all the handpumps and other
sources that the villagers are using. The evaluation team should be accompanied by local residents so that questions can be asked and issues clarified
as they arise. Much of the data for the handpump summary sheet will be
obtained during the transect walk.
 Village Map: A group of residents is asked to map out their village on a
large scale, using pen and paper or local material as available (it is often
done on the ground with stones, sticks, leaves etc). It can act as a good icebreaker when starting to work with the village, but it also provides important information on the relative location of different sources, zonal uses
of village, location of chief’s house, institutions etc.
 Venn diagram: This is an excellent visual tool for mapping the interrelations of different formal and informal institutions and power structures
in the community. It will help to clarify where the responsibility lies for
management of handpump and other sources. Using pen and paper, different circles are drawn to represent institutions and individuals with decisionmaking powers. Where the circles are separate there is no contact between
them and where they touch, information passes between them. If the circles
are drawn overlapping this indicates that there is some cooperation in decision-making. The degree of overlap represents the amount of interaction of
cooperation between groups. So, for example, if a village water committee
could not take any decision without first getting approval or agreement from
a village development committee, then the water committee circle would be
drawn inside the village development committee circle.
 Focus discussion group: A small group (say, maximum 15) of representatives from the village is asked to attend a closed meeting to discuss certain
issues. In the fieldwork associated with handpump evaluation it would be
useful to get a representative cross-section of handpump users and managers
to attend a focus group discussion to carry out the sustainability snapshot
exercise. The facilitation of this type of group is important since it needs to
be well-managed and controlled without directing or manipulating the responses from the group.
36