United Nation Development Programme in Libya
Outcome evaluation country programme
Greater awareness, capacity and means to
preserve the environment at global, national
and local level
Evaluation report
January 2011
United Nation Development Programme in
Libya
Outcome evaluation country
programme
Greater awareness,
capacity and means to
preserve the environment at
global, national and local
level
Inception report
Team members
Fathia Abdulgawad
Giovanni Morsiani
DISCLAIMER:
The content of this report is the sole responsibility of the
consultant and can in no way be taken to reflect the views of the
UNDP and Government of Libya
January 2011
Document no.
1
Version
2
Date of issue
13-01-11
Page 0 of 30
Acknowledgment
The mission is particularly indebted to all those who extended help and cooperation in
completing its assignment. Thanks are due to Environment General Authority, General
Water Authority and Secretariat of Agriculture officials, project coordinators and UNDP staff.
Page 1 of 30
List of abbreviations and acronyms
ADB
Asian Development Bank
DIAS
Desktop Information Access System
EGA
Environment General Authority
EU
European Union
GEF
General Environment Fund
GWA
General Water Authority
LRIMS
Land Resource Information Management System
MDGs
Millennium Development Goals
M&E
Monitoring and Evaluation
NAP
National Action Programme
NCSA
National Capacity self Assessment
NGO
Non Governmental Organization
SoA
Secretariat of Agriculture
UNCBD
United Nation Convention on Biological Diversity
UNCCD
United Nations Convention to Combat Desertification
UNDP
United Nation Development Programme
UNFCCC
United Nations Framework Convention on Climate Change
WB
World Bank
WIAS
Web Information Access System
Page 2 of 30
TABLE OF CONTENTS
SECTION 1:
BACKGROUND, PURPOSE AND METHODOLOGY ..........................................................5
1
PROGRAMME BACKGROUND AND STRUCTURE .............................................................................5
2
EVALUATION PURPOSE AND METHODOLOGY ................................................................................6
2.1
2.2
2.3
2.4
EVALUATION PURPOSE ............................................................................................................................... 6
EVALUATION SCOPE AND OBJECTIVES ............................................................................................................ 7
EVALUATION QUESTIONS ............................................................................................................................ 7
EVALUATION STEPS .................................................................................................................................... 8
SECTION 2:
PROGRAMME LOG FRAME ...........................................................................................9
SECTION 3:
PROJECTS EVALUATION .............................................................................................13
3
PROJECT: MAPPING OF NATURAL RESOURCES .............................................................................13
3.1
3.2
3.3
4
PROJECT: UNDP- GEF: ENABLING ACTIVITIES PROJECT .................................................................15
4.1
4.2
4.3
5
PROJECT DESCRIPTION.............................................................................................................................. 19
EVALUATION FINDINGS ............................................................................................................................. 19
RECOMMENDATIONS AT PROJECT LEVEL ....................................................................................................... 20
PROJECT: ENHANCING NATIONAL PARTNERSHIP FOR PROTECTED AREAS MANAGEMENT ..........20
8.1
8.2
8.3
9
PROJECT DESCRIPTION.............................................................................................................................. 17
EVALUATION FINDINGS ............................................................................................................................. 18
RECOMMENDATIONS AT PROJECT LEVEL ....................................................................................................... 18
PROJECT: EQUIPPING THE LABORATORY OF EGA .........................................................................19
7.1
7.2
7.3
8
PROJECT DESCRIPTION.............................................................................................................................. 16
EVALUATION FINDINGS ............................................................................................................................. 16
RECOMMENDATIONS AT PROJECT LEVEL ....................................................................................................... 17
PROJECT: STRENGTHENING EGA CAPACITY FOR SOUND ENVIRONMENT MANAGEMENT ............17
6.1
6.2
6.3
7
PROJECT DESCRIPTION.............................................................................................................................. 15
EVALUATION FINDINGS ............................................................................................................................. 15
RECOMMENDATIONS AT PROJECT LEVEL ....................................................................................................... 16
PROJECT: ESTABLISHING ENVIRONMENT GEOGRAPHIC INFORMATION SYSTEM .........................16
5.1
5.2
5.3
6
PROJECT DESCRIPTION.............................................................................................................................. 13
EVALUATION FINDINGS ............................................................................................................................. 13
RECOMMENDATIONS AT PROJECT LEVEL ....................................................................................................... 14
PROJECT DESCRIPTION.............................................................................................................................. 20
EVALUATION FINDINGS ............................................................................................................................. 20
RECOMMENDATIONS AT PROJECT LEVEL ....................................................................................................... 21
PROJECT: NATIONAL FRAMEWORK AND ACTION PLAN FOR SOLID AND HAZARDOUS WASTE .....21
9.1
9.2
9.3
PROJECT DESCRIPTION.............................................................................................................................. 21
EVALUATION FINDINGS ............................................................................................................................. 22
RECOMMENDATIONS AT PROJECT LEVEL ....................................................................................................... 22
Page 3 of 30
10
PROJECT: STRENGTHENING GWA IN WATER RESOURCES PLANNING AND MANAGEMENT ..........23
10.1
10.2
10.3
PROJECT DESCRIPTION.............................................................................................................................. 23
EVALUATION FINDINGS ............................................................................................................................. 23
RECOMMENDATIONS AT PROJECT LEVEL ....................................................................................................... 23
SECTION 4:
PROGRAMME EVALUATION .......................................................................................25
11
RELEVANCE ..................................................................................................................................25
12
OUTPUT EVALUATION ..................................................................................................................25
13
OUTCOME EVALUATION ..............................................................................................................25
14
FACTORS INFLUENCING RESULTS .................................................................................................26
15
RECOMMENDATIONS AT PROGRAMME LEVEL .............................................................................26
ANNEXES ...............................................................................................................................................28
ANNEX 1:
ANNEX 2:
LIST OF MEETINGS .......................................................................................................................... 29
LIST OF DOCUMENTS REVIEWED ........................................................................................................ 30
Page 4 of 30
SECTION 1:
1
BACKGROUND, PURPOSE AND METHODOLOGY
Programme background and structure
Libya is a North African country located along the southern coast of the Mediterranean
Basin. Its total land area is about 1.76 million km2, most of which (95.2%) is desert, while the
rest is either rangeland (4%), or agricultural land (0.4%), and less than 0.3% is a scattered
forested area. The annual average rainfall is estimated at 300-400mm depending on climatic
and topographic features. Four phyto-geographical regions are present in Libya and these
include a narrow coastal strip, semi-coastal hills, sub-Saharan areas and the Sahara desert
belt. Libya’s environmental challenges include limited water resources, land degradation,
fragmented mechanisms for environmental management and monitoring, inadequate solid
and hazardous waste management, and oil spills.
Libya’s main concern has been to attain a sustainable water management strategy during its
agricultural and economic development. The falling water tables in Libya’s most productive
agricultural area caused by over-irrigation pose severe long-term ecological threats. The
government began to recognize this in the beginning of the 1980s and took measures to
discourage citrus orchards and vegetable plantations, both of which require large amounts
of water. The government signed and ratified the United Nations Framework Convention on
Climate Change (UNFCCC) on 14 June 1999. Libya also signed the Convention on Biological
Diversity in 1992, and in 2001 it ratified the convention, and subsequently established a
National Committee on Biodiversity. The Libyan Government ratified the United Nations
Convention to Combat Desertification (CCD) on 22 July 1996, and assigned the People
Committee of the Agriculture General Authority (AGA) as the Focal Point. A first national
report (in English) to the CCD was submitted in 1999 and a second national report was
prepared and submitted (in Arabic) in 2002.
A National Action Programme for Combating Desertification (NAP) has yet to be developed
and for this the National Committee to Combat Desertification has been established under
the General Secretariat of Agriculture.
The project partners are:
- The General Water Authority (GWA)
- The Environment General Authority (EGA)
- Secretariat for Agriculture
The evaluation concerns the Outcome 6 of the Country programme: “Greater awareness,
capacities and means to preserve the environment at global, national and local levels” and it
is driven by following projects:


00013422 – Mapping of Natural Resources
00050071 - UNDP-GEF Enabling Activities Project - National Capacity Self Assessment
PIMS 2714
Page 5 of 30






00054203 - Establishing Environmental Geographic Information System (EGIS)
00054207 – Strengthening EGA capacity for sound environmental management
00054208 – Equipping the laboratories of EGA
00054209 – Enhancing National Partnership in support of Protected Areas
Management
00055477 - National Framework for Solid Waste Management and Action Plan for
Hazardous Waste Disposal in Libya
00069233 – Strengthening GWA technical capacity in water resources planning and
management
The environment component of the Country Programme represents about 40% of the total
UNDP annual development budget. All projects are fully financed by the Government of
Libya but one: Enabling activities project which is financed by GEF. With regard to project
management (preparation of ToR or/and technical specifications, procurement advertising,
short listing, interviews, contracting and execution monitoring) the following arrangements
might be in place:

Agency execution: UNDP controls funds, the concerned UN agency is responsible for
project management;

Direct execution: UNDP controls funds and is responsible for project management

National execution: UNDP controls fund and the concerned national authority
manages the project.
In case of environment component all project but Mapping of natural resources, are
nationally executed. Mapping of natural resources is subject to agency execution (FAO).
During the implementation of the projects several changes have taken place in the national
authorities’ senior management. This implies that project coordinators and senior staff have
changed. This might undermine the evaluation process since some of information might not
be any more available.
2
Evaluation purpose and methodology
2.1
Evaluation purpose
This evaluation exercise is commissioned in line with the Evaluation Plan of the UNDP
Country Programme (2006-2010) in the Libyan Arab Jamahiriya. The evaluation will place a
major emphasis on assessing the impact and overall contribution of this outcome to the CPD
and to the Environment Plan 2009-2011, and will also produce recommendations for
eventual adjustments and focusing on:



Evaluating the impact of UNDP interventions and their contribution to building
national capacity for addressing environmental challenges;
Assessing the appropriateness and relevance of UNDP strategies to address the
identified problems and hence contribute to the achievement of the outcome;
Assessing if the outcome has been achieved or will be achieved given the activities
supported by UNDP;
Page 6 of 30


2.2
Assessing and providing feedback on the validity/relevance of the outcome for UNDP
supported interventions, and the extent to which the set results and outcomes have
or can be achieved;
Identifying gaps/weaknesses in the current programme design and providing
recommendations as to its improvement with particular reference to the stated
outcome
Evaluation scope and objectives
This evaluation is carried out in the last year of development interventions under the current
CPD. The evaluators should give greater importance to assessing the efficiency and, as far as
possible, the effectiveness of UNDP’s contribution to national priorities and CPD Outcome 6.
Evaluators shall take into account and rank the following items:
 Status and degree of change in the outcomes, and factors influencing the outcomes
 Level of incurred changes: enabling environment, organizational and/or individual
levels
 UNDP strategic positioning on achieving the outcomes
 Relevance of the outcomes and outputs
 Partnership strategy
 Sustainability: whether there is ownership and capacity to maintain and manage
development in the outcomes
2.3
Evaluation questions
Specifically, the outcome evaluation should address, but not be limited to, the following
questions and issues:
1. Outcome analysis











Are the outcomes and associated projects relevant, appropriate and strategic to
national priorities and the UNDP mandate?
Were the actions to achieve the outputs and outcomes effective and efficient?
Were there multi-level interventions conducted (environment, organization,
individual)? How many?
Are the outputs and outcomes leading to benefits beyond the life of the existing
projects?
Which findings may have relevance for eventual adjustments and/or future
programming?
What is the current status and prospects for achieving the outcome with the indicated
inputs and within the indicated timeframe?
What are the main factors (positive and negative) within and beyond UNDP’s
interventions that affected or are affecting the achievement of the outcome? How
have these factors limited or facilitated progress towards the outcome?
Were UNDP’s proposed contributions to the achievement of the outcome appropriate,
sufficient, effective and sustainable?
How did UNDP support gender equality and women’s participation in the
achievement of the outcome?
Page 7 of 30
2. Output analysis




What are the key outputs that have been produced by UNDP to contribute to the
outcome?
Are the UNDP outputs relevant to the outcome?
Are the monitoring and evaluation indicators appropriate to link these outputs to the
outcome, or is there a need to improve these indicators?
Has sufficient progress been made with regard to UNDP outputs?
3. Resources, partnerships, and management analysis
 Was UNDP’s resource mobilization strategy in this field appropriate and effective in
achieving this outcome?
 Was UNDP’s partnership strategy in this field appropriate and effective in achieving
this outcome?
 Are UNDP’s management structures and working methods appropriate and effective
in achieving this outcome?
 Overall, assess the scope, relevance, efficiency and sustainability of UNDP’s resources
mobilization, partnership and management arrangements in achieving this outcome.
2.4
Evaluation steps
The study was structured in four phases:
i.
Desk study
It included the collection and the analysis of all relevant programmes documentation and
projects outputs (policy papers, strategy, action plans, etc.
ii.
Revision of projects log-frame
The project log-frame (results, indicators and outputs) will be reviewed with the aim of
harmonizing and standardizing the approach and the intervention logic; see next
section: evaluation matrix.
iii.
Interviews
Project directors, coordinators, department heads, steering committees members will be
interviewed to assess project spending, outputs delivered, results achieved in
accordance with the evaluation matrix with particular reference to the identified set of
indicators.
iv.
Analysis and report writing
The fourth phase includes further interviews, an in-depth analysis of all documentation
and the writing of the draft report including recommendations focusing on measures to
deal with bottlenecks and improve programme effectiveness and potential for projects
follow-up.
Page 8 of 30
SECTION 2:
PROGRAMME LOG FRAME
The projects log-frames (hierarchy of activities, outputs, objectives, outcomes), as presented
in the project documents, are rather confused and puzzling: sometimes what is stated as
objective is an output (i.e. Project: National framework for solid waste management –
Objective: Municipal solid waste management plan); other times an indicator of outcome is
actually an indicator of impact (i.e. Project: Strengthening GWA technical capacity –
Indicator of outcome: National access to safe water and sanitation improved).
In order to harmonize and standardizing the intervention logic and the approach the
evaluation team has re-design the programme and projects log-frames. The revised logframe is presented in the table below.
Page 9 of 30
Table1: Programme log-frame
Programme outcome
(as stated in the Country Programme)
Greater awareness, capacity and means to preserve the environment at global, national, and local levels
Project title
Government
partner
Implementation
period / Budget
Specific project outcome
Indicators of oucome
Mapping of natural resources
(agency execution)
Secretariat of
Agriculture
2001-2010
US$ 3,200,000
Strengthening GAAAMW capacity
to manage land resources
 Gefara pilot plan approved and
implemented
 Plan
 Plan approval process
 Full system definition
 Core database approved and
implemented
 Core database
 Metadata standards and
process
 Maintenance tools
 LLRIMS for physical planning
approved and implemented





DIAS
WIAS
Agro-climatological database
End users application
Users training
 Land cover/evaluation model
approved and implemented




Land cover maps
Crops evaluation models
Evaluation for selected areas
Training
 Improved planning for
agricultural and rural
development approved and
adopted
 General soil legend
 Land evaluation and land use
potential system
 Digital soil maps
 Agro-ecological zoning and
potential
Main outputs
Page 10 of 30
UNDP – GEF Enabling activities project
(national execution)
EGA
2006-2011
US$ 200,000
Upgrading stakeholders
institutional capacity
Self-assessment capacity needs
report approved
Action plan approved and
implemented
 NCSA report
 Action plan
 Monitoring and Evaluation
plan
 Procurement
 System design and
implementation strategy
 Pilot programme (production
of priority mapping)
 Training
 Training needs assessment
 Training and awareness
 Training system including
M&E of results
Establishing Environmental Geographic
Information System
(national execution)
EGA
2006-2011
US$ 1,300,000
Environmental risks mapped and
prevented
GIS operational and environment
protection action undertaken
Strengthening EGA capacity for sound
environmental management
(national execution)
EGA
2006-2011
US$ 1,300,000
Capacity for global and local
environment conservation
enhanced
Strategy and action plan
developed and implemented
Equipping the laboratories of EGA
(national execution)
EGA
2006-2011
US$ 1,000,000
Strengthening environmental
analysis capacity
Increased number and enlarged
scope of analysis
 Procurement
 training
Enhancing national partnership in support of
protected areas management
(national execution)
EGA
2006-2011
US$ 1,000,000
Protected areas management
improved
Protected areas assessed and
established




National coordination enhanced
(MOU signed)
Legislation reviews approved and
enforced
 Workshop
 prototype MOU
 Law review based on
consultative process
Culture of awareness introduced
 Awareness campaign
Protected area management
model developed and
implemented
 Pilot project
 Business plan
Training
GIS procurement
GIS Implementation
Assess biodiversity and
propose protected areas to be
established
Page 11 of 30
National framework for solid waste
management and action plan for hazardous
waste
Strengthening General Water Authority in
water resources planning and management
EGA
GWA
2006-2011
US$ 1,700,000
2009-2010
US$ 200,000
Solid and hazardous waste
management improved
Availability and quality of water
improved
National plan for solid waste
management developed and
implemented




National action plan for
hazardous waste disposals
developed and implemented
Plan developed and implemented
 Inventory
 National action plan
 Training
Water quality analysis capacity
enhanced
 Equipment assessment and
procurement
Review of national legislation
Current system assessment
Awareness campaign
National framework for solid
waste management
Page 12 of 30
SECTION 3:
3
3.1
PROJECTS EVALUATION
Project: Mapping of natural resources
Project description
The objective of the project is to construct a pilot model for the Gefara Plain and to establish
user needs, system functionality requirements, and test the prototype components as a
precursor to the national implementation of the Land Resource Information Management
System (LRIMS) in order to support the renewal of natural resources. After developing and
testing the pilot model in the Gefara Plain, the model will be replicated in other identified
areas. This is an ongoing project implemented by FAO with the Agricultural Research Centre
at the Secretariat for Agriculture. The estimated budget is US$ 3,294,148. The project
started in 2001 and, according to plan, was supposed to end in 2010.
3.2
Evaluation findings
Relevance
The project might be relevant in relation to national strategic objective since it mostly aims
at increasing agricultural production since land suitability aims primarily at identifying which
crops are more suitable to the prevailing agro-climatic and meteorological condition, the
impact on environment conservation/protection is limited to the potential conservation of
natural soil fertility.
Output evaluation
With regard the development of LRMIS all forecasted activities were implemented even if
this required several extensions of the project in particular for data entry (executed by an
Egyptian contracted company). The project started in 2001 and it was supposed to end in
2004 but it is currently on-going.
The LLRIMS includes the: (i) core database software, (ii) maintenance tools for database
administrator, (iii) Desktop Information Access System (DIAS), (iv) Web Information Access
System (WIAS), (v) collection and entry of agro-climatic data, (vi) soil maps (1:50,000) and
(vii) the end users application.
All these outputs were produced but with severe limitations that make the entire system of
limited use since the data are incomplete and out of date. Climate change, water and soil
exploitation quickly modify soil and water chemical composition and meteorology, therefore
there is a need for updating information in real time. The core database contains
information on soil chemical structure, quality of ground water and meteorological data. Soil
and water data were provided on hardware support (paper) by the General Water Authority
(GWA), the meteorological data were provided by the Meteorological Authority. The data
on soils are available only for the costal area and were collected in 1980; with regard to
Page 13 of 30
water (ground water and wells) the data are missing from many areas in particular northwest; the most recent meteorological data are from 2003. With regard to maps they should
have been more detailed (1:25,000). It is worth noticing that the GWA has implemented, in
parallel, the same exercise, it entered soil and water data (previously on paper support) in its
own database. This was a potential duplication of efforts.
The crop evaluation models (9) have been developed and the pilot project in Gefara area, to
test the models and to define land suitability (the intermediate objective of the project), is
being implemented. The implementation of the models requires the execution of specific
surveys to collect socio-economic data.
Currently not all surveys have been completed and only three models out of nine have been
tested. Since the project will end in few months and the financial resources are available
only for the organization of the final workshop, it is rather questionable that the testing
exercise will be completed. It is also questionable that the Agricultural Research Institute
has the technical capacity to autonomously complete the exercise since limited training was
delivered. Since the pilot project includes an area of about 65,000 ha out of a total
agricultural area of about 600,000 ha it is evident that the most remains to be done.
Outcome evaluation
The project has limited impact on environment, this means that even if it was able to
produce complete and sustainable outputs (and it was not the case), it would not have
necessarily contributed to the achievement of the programme outcome. To get sustainable
results very substantial investments would be needed and at least further ten years of
implementation. Furthermore the database should be kept constantly updated. However the
availability of land suitability maps does not imply that the farmer will be willing to change
their cropping patterns accordingly. This would require the implementation of very costly
incentive policies similar to those implemented in Europe by the European Union. According
to the opinion of the evaluation team the project is based on old and dirigistic approach
which is typical of FAO: believing that farmer behaviour is more sensitive to technical
recommendations rather than price incentives.
Project management
A CTA was appointed for the first year but, according to interviews, his performances were
so poor that the position was deleted. The evaluation team was reported that also the
performances of short-term expert were below standards.
3.3
Recommendations at project level
Any possible follow-up should be implemented by FAO since the agency is opening an office
in Libya. However, since obtaining any valuable outputs would imply very relevant
investments and the outcome would be, at best, uncertain, the extension is not
recommended.
Page 14 of 30
4
Project: UNDP- GEF: Enabling activities project
4.1
Project description
The objective of the National Capacity Self Assessment (NCSA) project is to provide national
stakeholders in Libya with the opportunity to articulate a thorough, participatory self
assessment and an analysis of national capacity building needs and priorities, as well as of
constraints against the achievement of global environmental objectives as set forth in the
Rio Conventions and related international instruments.
The NCSA facilitates a cross-sectoral consultative process informed by capacity building
assessments and the sequencing and prioritizing of capacity needs, in order to identify
challenges that prevent the country from fully meeting its obligations under the relevant
environmental conventions. Specific outputs to be accomplished through the NCSA process
include the identification of priority areas of action within and across the GEF thematic areas
of biodiversity, climate change, and land degradation. This will act as a catalyst for targeted
and co-ordinated actions as well as requests for future funding to protect the global
environment within the broader perspective of sustainable development. The sustainable
institutional system that will be created within the framework of the initial NCSA will ensure
that capacity needs assessment continues in a comprehensive and systematic manner. The
project is being implemented by EGA through a National Execution Modality with a total
budget of US$200,000 GEF – (actual $148,000) -$50,000 Libyan Gov-(actual $25,000) -The
project started in February 2006 and, according to plan, was supposed to end in 2010.
4.2
Evaluation findings
Relevance
Given the need for defining responsibility and function with regard to the implementation of
the three conventions1 and the lack of strategic plan at EGA the project is relevant.
Output evaluation
Nobody at EGA officials the team had the opportunity to meet (including the responsible for
donor coordination) was aware about the project or its outputs. The consultant could not
meet with the primary stakeholders since they were unknown; either the EGA Director, who
was personally involved in this project or the GEF focal point for Libya, and that was not
available. Since the design of the action plan and its implementation would have required
the participation of most EGA departments, this lack of even basic knowledge tells already
something about its potential outcome. However the team managed to get a copy of the
final report at UNDP office. The report make an assessment of institutional capacity building
1United
Nation Convention on Biological Diversity (UNCBD), United Nations Convention to Combat Desertification (UNCCD), United Nations
Framework Convention on Climate Change (UNFCCC)
Page 15 of 30
needs and ends proposing a number of projects aimed at effectively implementing the three
conventions that might be implemented under the EGA coordination.
Outcome evaluation
The project did not contribute yet to programme outcome since it was not endorsed. The
evaluation team suspects that the limited involvement of national authorities and EGA
departments might have negatively affected ownership and skills transfer.
4.3
Recommendations at project level
The documents should be reassessed, presented in a workshop looking for Government
endorsement. Whenever endorsed it might be used for planning further UNDP environment
project and as strategic basis for the preparation of EGA strategic plan (see
recommendations related to the project “Strengthening EGA capacity for sound
environment management”.
5
Project: Establishing environment geographic information system
5.1
Project description
The object of the project is to support the Environment General Authority (EGA) in the
planning, design and implementation of a Geographic Information System (EGIS) to enable
EGA to in carry out its mandated responsibilities of monitoring, assessing and responding to
various environmental conditions and emergencies that occur in Libya. It will also help to
build institutional capacity through strengthening the capacities of EGA’s staff to effectively
run the system in a sustainable manner. The EGIS will be developed in such a manner that
the technology will be logically and functionally integrated into the organization’s
operational processes.
The project outputs can be summarized as follow:





Outlining general requirements and development needs
Detailing the blueprint of the system
Developing a Strategic Plan
Developing a Pilot Programme
Staff Training Programme
The project is being implemented by EGA using the National Execution Modality with a total
budget of US$1,303,000. The project was signed in 2006 and is planned to close in 2011.
5.2
Evaluation findings
Relevance
The project is relevant since GIS and related database are instrumental to any assessment of
environmental risks and to plan protection action. All EGA departments need this service
even if, at this stage, only Protected Area Department has the capacity to make use of it.
Page 16 of 30
Output evaluation
The procurement has been accomplished, no doubt that the equipments are in line with
required quality standards. The issue is that, according to the GIS Department Director
nobody in the GIS department is able not only to use them but even to understand their
potential. This is due to lack of training and in particular to the lack of on-the job training
once the system was operational.
Outcome evaluation
Under these conditions the project did not contribute to the achievement of programme
outcome.
Project management
The project management was not particularly effective and this is one of the main causes of
poor results. Two long-term experts were fielded. The first one was a GIS expert,
unfortunately he spent one year without being in the condition to provide assistance since
the system was not in place yet. The second one, who spent two years at EGA, was an IT
expert and therefore, according to the Director, inappropriate to the scope. Wrong timing
and consultant inappropriateness were the main reasons for lack of training and technical
assistance.
5.3
Recommendations at project level
A follow up project focusing on capacity building should be urgently implemented to make
use of the equipment and of the human resources deployed in the office (ten staff), and to
contribute to the forecasted outcome.
6
6.1
Project: Strengthening EGA capacity for sound environment management
Project description
This project is designed to support the Environment General Authority (EGA) with its evergrowing environmental activities through developing a consultative process that includes
training, awareness raising, information dissemination, and the formulation of policies,
strategies and programmes addressing various environmental issues.
The project aims to enhance institutional and individual staff effectiveness in applying
innovative management techniques to help carry out environmental protection tasks.
Additionally, the project will contribute to improving EGA efficiency and enhancing its
competence in its environmental management and coordination role. The project is being
implemented by EGA through a National Execution Modality with a total budget of
US$1,303,000. The project started in 2006 and will end in 2011.
Page 17 of 30
6.2
Evaluation findings
Relevance
The project is relevant since EGA needs a lot of institutional capacity building to perform its
mandate. However, since the implementation focused on individual capacity building
(training) rather than institutional building (strategic planning, identification of functions,
organizational set-up) its relevance was strongly hampered.
Output evaluation
The outputs were quite poor. During the first phase a number of randomly selected training
and study tours were implemented. After having realized the weakness and ineffectiveness
of the approach a structured assessment was implemented. The team was not impressed by
the quality of the capacity building needs report (prepared by experts fielded by the MSIRBAS team) since the analytical part is quite poor in terms of functional analysis; however it
ends with a number of recommendations that, even if do not represent a structured
strategic plan, might help in the formulation of a sectoral strategy.
However, despite the report, project activities continued focusing on training. The team was
not able to assess the effectiveness of training delivered since the project coordinator was
not available. However its impact, given the approach, is really questionable.
Outcome evaluation
The project contribution to programme outcomes, if any, was limited.
Project management
The project was not attached to the EGA training department and the project coordinator is
from a different office. As a consequence the project did not contribute setting up a
permanent human resources development system.
6.3
Recommendations at project level
EGA still needs institutional as well as human resources capacity building. The
recommendation is to implement a new project aimed at setting up a proper strategic
planning system (which includes functional analysis, organizational set-up, department work
plans, staff assessment and other) as well as a performance management system from which
training needs can be detected. In this framework EGA should better assign implementation
responsibilities to the Training and to the Planning departments.
Page 18 of 30
7
7.1
Project: Equipping the laboratory of EGA
Project description
This project aims to equip the laboratories of the Environment General Authority (EGA) in
order to qualify them to carry out a variety of monitoring functions that will include
performing inspections for pollutants. Furthermore, these laboratories, once fully equipped,
will oversee the country's imported goods to ensure their adherence to Libyan
environmental specifications and standards. The laboratories will also support environment
research initiatives. The project is being implemented by EGA using a National Execution
Modality with an original total budget ofUS$1,000,000, which was later increased to
US$5,486,788.62. The project started in 2006 and it is expected to end in 2011.
7.2
Evaluation findings
Relevance
The project is relevant since well equipped and efficient laboratories able to perform a wide
spectrum of analysis are essential to detect any kind of environmental risks due to industrial
and other man-made pollution of water, soil and air.
Output evaluation
The procurement has been accomplished; the laboratories are very impressive, fully
equipped with modern technological apparatus in line with required quality standards. Some
concerns are related to quality of civil works (whose execution, however, was not under the
mandate of the project). The laboratories are able to perform any type of analysis
(compulsory those for companies - oil companies in particular) and, potentially, are
financially self-sustainable since the analysis are priced.
The issue is that no one of the laboratory staff is able to run the very sophisticated
equipments for analysis and, even worst, the management does not have the capacity to
manage such complex laboratories that require a very structured work organization and
timely procurement of consumable. This is due to lack of focus on capacity building
worsened by the not always inappropriate identification of trainees. Most likely the project
should have been planned in two phases: the first one to procure and install the equipment,
the second one to train, formally and on the job, the staff. However there is a strong need
for new staff recruitment since part of existing staff is not more trainable.
Outcome evaluation
The project has achieved one of its output - equipping the EGA laboratories – but since the
capacity of using them is not yet developed, at this stage, the project has been not able to
contribute to the programme outcome. A second phase, focusing on capacity development,
might modify this conclusion. Furthermore the legislation and the concerned institutional
arrangements for the enforcement of measures aimed at preventing pollution should be
carefully scrutinized to assess their effectiveness.
Page 19 of 30
7.3
Recommendations at project level
A follow up project focusing on capacity building should be urgently implemented to make
use of the equipment and of the human resources currently working at the laboratory. A
long-term laboratory manager should be deployed for at least 9 months, short-terms
chemists for each type of analysis should provide formal and on the job training to the staff.
Given the fact that the chemical reagents have an expiration date and that any inappropriate
use of equipment might lead to damages the follow-up is particularly urgent. In this
framework EGA should guarantee the recruitment of new suitable and qualified staff.
8
Project: Enhancing national partnership for protected areas management
8.1
Project description
The project aims at proposing an umbrella intervention at the national level to complement
national initiatives towards establishing EGA as a leader in environment protection,
legislation and sensitization, as well as an agency with a cross cutting mandate to mobilize
national efforts towards environmental protection.
The project outputs could be summarized as follows:




Strengthening the institutional capacity of EGA in PA management
Enhancing national coordination on protected area issues
Introducing a culture of awareness on Protected Areas.
Developing a model for protected area management institutes and establishing the
basis for a protected areas management model that could be easily replicated.
The project will also support EGA in building partnerships on environment protection with
government institutions and the private sector. The project is being implemented by EGA
using the National Execution Modality with a total budget of US$850, 000. The project
started in 2006 and will end in 2011.
8.2
Evaluation findings
Relevance
The project is relevant since there is a need for establish well managed protected areas
(national parks) to protect Libyan bio-diversity. This might also contribute to further
development of the tourism sector by providing new attractive tourist sites.
Output evaluation
All activities are on-going: assessment of biodiversity to propose protected areas, awareness
on protected areas (stakeholders’ workshop and awareness campaign addressed to general
public – TW interviews, brochures, seminars in the schools and universities), and
Page 20 of 30
implementation of a pilot project aimed at producing a protected area business plan. The
latter a quite critical issue since, in accordance with the current legislation, EGA does not
have authority on protected areas and on their management since these responsibilities falls
under the authority of the Secretariat for Agriculture. The management of the first selected
protected area was not responsive, therefore two new areas were selected and the drafts of
the business plans were produced. With regard to the revision of the legal framework (one
of the project outputs) with the aim of empowering EGA, finally the Secretariat of
Agriculture accepted to establish a joint commission responsible for the revision of the law.
Finally EGA rightly deleted one project component concerning the setting up of a GIS system
within the department that would have been a duplication of the one installed in the
framework of the project “Establishing environment geographic information system”.
Outcome evaluation
The project outputs, even if the activities are just started and require a long-term
commitment, are contributing to the programme outcome.
Project management
The project was very well managed by the project coordinator that, in this case, is the head
of the concerned department, his commitment and his appropriate background have
contribute to the achievement of positive results. There was also a very careful and
appropriate management of consultancy services, consultants were recruited for the
preparation of stakeholders’ workshops and for the assessment of business plans and law
amendments prepared by the department.
8.3
Recommendations at project level
The project should be refinanced to extend activities, furthermore funds should be provided
for the implementation of the pilot business plans.
9
9.1
Project: National framework and action plan for solid and hazardous waste
Project description
The project aims to support the development of appropriate, affordable and sustainable
waste management practices for Libya and at strengthening national capacity. It is also
aimed at building ownership to meet the country’s obligations under the Basel and
Stockholm conventions, in addition to preparing a hazardous waste national action plan.
Under the overall objective of developing and implementing appropriate and sustainable
waste management practices throughout Libya, the project covers the following activities:





Development of a National Solid Waste Management Policy
Institutional Strengthening
Public Education & Awareness Raising
Improving Investment in Waste Management Infrastructure & Promotion of Cost
Recovery Mechanisms
Page 21 of 30

Encouraging Private Sector participation in Service Delivery
The project is being implemented by EGA using the National Execution Modality and with a
total budget of US$1.7 million. The project started in 2006 and, according to plan, was
supposed to end in 2010.
9.2
Evaluation findings
Relevance
The project is very relevant since solid waste is not properly handled (simply discharged in
dump sites) and this has a very negative impact on the environment. Even worst is the
situation regarding hazardous waste (from hospital, industrial chemicals, expired fertilizers)
that are abandoned without any protection.
Output evaluation
Two UN volunteers were recruited with the support of UNDP office in Bonn. At this stage the
two national strategies and action plans are ready (solid and hazardous waste). They need to
be translated into Arabic (on-going) and presented in a workshop to the government for
endorsement and follow up. A Solid Waste Committee was established for steering the
project. The project coordination was assigned to an EGA official that was from a different
department. The decision was rational since he is the only staff with specific waste
management experience; however this has limited the potential positive project impact on
the Waste Management Department. The participation of Department staff was limited to
the execution of surveys to detect waste management shortcomings. The departments need
the formulation of a strategic plan defining functions, organizational set-up and staffing
needs. The two national strategies and action plans look quite well structured. The analysis
of the current situation is very detailed, the recommendations related to the single cases of
waste mismanagement seem to be rational, the action plans, despite not priced, contain a
lot of valuable suggestions.
Outcome evaluation
There is no contribution yet to the achievement of the programme outputs, however, if the
planned dissemination phase and the concerned workshop will be properly managed, the
outputs may even contribute to the formulation of UNDP/Government country strategy in
the sector.
9.3
Recommendations at project level
The formulation of a strategic plan for the department should be implemented in the
framework of the recommended follow up of the project “Strengthening EGA capacity for
sound environment management”. With regard the national strategies it is recommended to
carefully review the recommendations identifying potential programme and projects to be
jointly implemented by UNDP, other agencies, IFI (WB) and Government. A single workshop
for the two strategies should be organized taking the form of national or even regional
event, donors, UN agencies, concerned central government bodies’, municipalities’ and
Page 22 of 30
private companies’ representatives should be invited. It is also recommended inviting
representatives from regional governments waste management institutions and
representatives from regional UNDP offices.
10 Project: Strengthening GWA in water resources planning and management
10.1 Project description
This project is designed to support the General Water Authority (GWA) through developing a
consultative process that includes training, upgrading of the laboratories and providing the
GWA’s departments with technical support. The project is geared to enhance institutional
and staff effectiveness and ability to carry out sustainable water resource management tasks
as well as the other functions of GWA.
The project is being implemented by the GWA using a National Execution Modality with a
total budget of US$202,647. The project started in 2009 and, according to plan, was
supposed to end in 2010.
10.2 Evaluation findings
Output evaluation
The genesis of the project is quite peculiar. There was a previous project implemented by
UNOPS which ended with an unspent balance of US$ 200,000. Therefore it was agreed upon
to use the remaining fund to support GWA need on ad-hoc basis. A project receptacle was
created to employ the funds for short-term capacity needs, instead of formalizing the
adoptions of different implementation modalities which, however, are not necessarily wrong
or ineffective.
However resources were slowly mobilized. From 2006 only two consultancies were required.
The first to assess chemical laboratory, which was regularly implemented, producing the
expected results, the second was on assessment of material suitability for drilled wells. In
this case UNDP was unable to provide the suitable expertise. It is true that the type of
expertise required was very specialized, but UNDP, using its international network of UN
agencies , should have better managed the request. However, most likely, the GWA itself has
better know-how for the identification of the required expertise.
Outcome evaluation
Not applicable.
10.3 Recommendations at project level
The project director confirmed the need for this type of ad-hoc approach in particular for
evaluating GWA activities in relation to international standards and his will to replenish the
fund whenever needed.
Page 23 of 30
Page 24 of 30
SECTION 4:
PROGRAMME EVALUATION
11 Relevance
In general all projects were relevant in relation to national policy, UNDP country programme
objectives and to the specific issues they were meant to address. The only project that was
not relevant was “Mapping of natural resources” that did not concern environment but
agricultural production. The project “Strengthening GWA in water resources planning and
management” is a case apart. It was not a structured project but the issue of efficient use of
water resources is extremely relevant to Libya and the establishment of a facility aimed at
providing ad-hoc assistance might be an effective way to contribute addressing the problem
since it is a flexible and potentially effective.
12 Output evaluation
Most of projects are not completed yet. However in terms of outputs delivery they might be
classified in four categories:

Overambitious projects: this is the case of mapping “Mapping of natural resources”
that has produced incomplete, unusable and unsustainable outputs

Project with focus on procurements of goods rather than capacity building: this
category includes “Establishing environment geographic information system” and
“Equipping the laboratory of EGA”; this was due respectively to poor design or poor
use of available technical assistance resources

Projects whose potential interesting outputs were disregarded by the Government
and possibly by UNDP (since no follow up actions have been undertaken): this
category includes “UNDP- GEF: Enabling activities project” and “Strengthening EGA
capacity for sound environment management”. The risks is that the outputs of the
“National framework and action plan for solid and hazardous waste” will have the
same destiny

Relatively successful projects which are producing the expected outputs and in
prospective might positively affect the programme outcome: this category includes
“Enhancing national partnership for protected areas management”
13 Outcome evaluation
At this stage the impact of outputs on outcomes is very limited. This is due to: (i) incomplete
and poor outputs, (ii) limited focus on capacity and institutional building, (iii) promising but
disregarded outputs. With regard to gender issues the team has not detected any particular
targeted programme strategy or project actions.
Page 25 of 30
14 Factors influencing results
In general the projects results were quite poor. This was due to a number of factors that are
discussed in this paragraph. First of all there is a need for underlining the prevailing
environmental conditions. It is undoubted that some partners have limited experience in the
“development process” including policy design and implementation. This makes
development planning and development partnership a challenge. This is further complicated
by the fast turn-over of senior government officials that affects programmes and projects
policy and approach stability.
In this little conducive environment the UNDP should be able to pursue its development
mandate proving its comparative advantage in term of policy and strategy development and
coordination. This would require a high degree of determination, perseverance, and
motivation, but in particular outstanding professionalism and capacity.
This is not what the evaluation is showing. Project design were often quite poor in terms of
strategic approach and even in term of appropriate use of logical framework approach; the
latter complicates an effective project monitoring. Project coordination, as proved by the
duplication detected in the projects evaluation process is not a common feature.
Monitoring, or at least monitoring feedback, including the identification of measures aimed
at reorienting project activities is virtually lacking. Prospective outputs might be ignored due
to lack of understanding of their potential.
15 Recommendations at programme level
As we stated above the comparative advantage of UNDP should be its policy and strategy
development capacity. This would require the recruitment of outstanding consultants for
programme and project formulation. In this framework templates and log-frames should be
enhanced and standardized; appropriate indicators for outputs, outcome and impact should
be identified to allow effective monitoring and evaluation. It is also recommended to
implement mid-term programme evaluation to allow for projects assessment and
reorientation whenever needed. Evaluation and project monitoring should be implemented
as joint exercise which implies the participation of government designated officials.
With regard to the environment programme the unproductive and unsustainable projects
should be discontinued, the focus should be on institutional building (EGA strategic plan)
capacity building (GIS and laboratories), extension of effective projects (protected areas)
and, in particular on the valorisation of prospective but, at this stage, neglected outputs.
National or, even better, regional workshops/events to present the national strategies and
the action plan aimed at the implementation of the three conventions should be organized
as the forums to debating and identifying objectives and actions to be pursued and
implemented in the framework of next country programme. To this scope it is important to
further develop the partnership with the government, other UN agency and donors
(including IFIs). In this framework it is also recommended inviting representatives from
other governments of the region and representatives from regional UNDP offices since
Page 26 of 30
environment protection strategy formulation would substantially benefit from regional
approach.
In this framework UNDP should act as a catalyser for UN agencies with specific expertise on
environmental issues (such as UNEP), for other donors agencies currently involved in the
sector or that might be involved in the future (such as World Bank in relation to the follow
up of solid and hazardous waste management), government agencies that are responsible
for the implementation of the three conventions or for any other environment concerned
issues. Furthermore, given the regional dimension of environment issues (i.e. convention
implementations, water shortage) UNDP should promote, in partnership with EGA and
whenever appropriate with other government agencies, regional policy and programmes to
address these issues at regional level. Regional dimension implies a partnership not only
between regional governments but also with the regional UNDP and other donors’ offices.
The team believe that the recommended workshops might be the starting point to promote
the debate on national and regional partnership, policies and programmes with the aim of
establishing more formal forums to coordinate national and regional initiatives. In this
framework UNDP should adopt a partnership approach identifying opportunities and
measures to promote: (i) national and regional UN agencies partnership, (ii) donors
partnership, (ii) national government agencies partnerships and (iv) regional governments
partnership. The first recommended action is to revitalize its national and regional networks;
this implies calling preparatory meetings at national (government’s agencies, donors’ and
UN agencies representatives) and regional levels (government’s concerned agencies UNDP
regional offices).
UNDP should also reflect on the benefit that might derive from better focusing its country
programme by selecting few areas. This would allow achieving the needed critical mass to
put in motion transformational changes and would avoid the dispersion of resources
(financial and human) in micro-interventions that unlikely produce visible results.
Environment (including water) might be the focal country programme area.
Page 27 of 30
Annexes
Page 28 of 30
Annex 1: List of meetings
Government Officials and Project technical assistance

Elmabrok Affan, Project Coordinator of Mapping of Natural Resources, Secretariat of Agriculture

Eltaher Ali salem, General Secretary, General Water Authority

Omer Salem, Director of Planning General Water Authority

Esam Aborass, Project Coordinator, Protected area management

Nuri El-Meseghimasoge Director of Technical Cooperation Department, EGA

Abdulah El Tarabulsipolsi, Project Coordinator, Environmental geographic information system, EGA

Abdul Baset El mere, Project Coordinator, Equipping the Laboratories of the EGA

Representative of technical assistance, Equipping the Laboratories of the EGA


Ramesh Kaji Tuladhar, Municipal Solid Waste Management Specialist
Abdoulaye allassane, UNV Technical Assistance Team, Solid and hazardous waste management
national strategy, EGA
UNDP

Costanza Farina, Resident Representative

Ramanathan Balakhrishnan, Deputy Resident Representative

Amal El-Mograbi, Programme Analyst, M& E Focal Point

Abdelmenam Mohammed, Programme Analyst, Environment Focal Point

Sarah Edongali, former Programme Officer Environment
Page 29 of 30
Annex 2: List of documents reviewed

UNDP, Libya, project document “Strengthening GWA Technical Capacity in Water Resources Planning
and Management”. December 2009

UNDP, Libya, project document” Enhancing National Partnership in support of Protected Areas
Management in Libya”. November 2006

UNDP, Libya, project document” National Framework for Solid Waste Management and action plan for
Hazardous Waste Disposal in Libya”. February 2009

UNDP, Libya, project document” Equipping the Laboratories of the Environment General Authority”.
December 2006

UNDP, Libya, project document” Mapping of National Resources for agriculture Use and planning”.
September 2000

UNDP, Libya, project document” Establishment of an Environmental Geographic Information system.
September 2000

UNDP, Libya, project document” Strengthening EGA Capacity for Sound Environment Management.
December 2006

UNDP, Libya, project document” GEF Enabling Activities Project”. May 2006

UNDP, Libya, Evaluation of UNDP in the net contributor countries of the Arab region, conducted by the
evaluation office. May 2008

UNDP, Libya, Draft country programme document for the Libya Arab Jamahiriya (2011-2014). March
2010

UNDP, Libya, Country programme outline for Libya (2006-2009). April 2005

UNDP, Libya, Draft country programme document for the Libyan Arab Jamahiriya (2006-2009). July
2005

UNDP, Libya, Results Oriented Annual Report. January 2011

UNDP, Libya, “National Strategy and Action plan for Hazardous Waste management in Libya”. June
2010

UNDP, Libya, “Solid waste management project, Tripoli Libya Draft Report On Municipal Solid Waste
Management in Tripoli. July 2010

UNDP, Libya, Organizational Chart and Laboratory Management System. October 2010

UNDP, Libya, Assessment of Existing Capacities and Capacity Building Needs of the Environment
General Authority, final report. February 2009

UNDP, Libya, Final Report for synergies among the three convention
Page 30 of 30