Terms of Reference – Final Evaluation for MRE TERMS OF REFRENCE FOR CONSULTANT Project “Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities” MARCH 3, 2016 0 Terms of Reference – Final Evaluation for MRE Contents 1. BACKGROUND ............................................................................................................................................. 2 2. OBJECTIVES OF THE EVALUATION ............................................................................................................... 2 3. EVALUATION CRITERIA AND QUESTIONS .................................................................................................... 5 4. METHODOLOGY AND SCOPE ....................................................................................................................... 3 4.1 Design ........................................................................................................................................................ 3 4.2 Sampling and sample size .......................................................................................................................... 3 4.3 Tools .......................................................................................................................................................... 4 4.4 Data collection ........................................................................................................................................... 8 4.5 Data Analysis ............................................................................................................................................. 8 4.6 Stakeholder Engagement .......................................................................................................................... 8 4.7 Scope of the evaluation ............................................................................................................................. 8 5. REPORT WRITING ........................................................................................................................................ 8 6. ETHICAL GUIDANCE AND CONSIDERATION ................................................................................................. 9 7. EVALUATION TEAM ..................................................................................................................................... 9 8. TIMELINES .................................................................................................................................................. 10 9. REQUIREMENTS FOR CONSULTANT .......................................................................................................... 10 9.1 Consultant Deliverables........................................................................................................................... 10 9.2 Qualifications and Experience Required of External Consultant............................................................. 10 9.3 Period of Performance ............................................................................................................................ 11 9.4 Compensation.......................................................................................................................................... 11 9.5 Application requirements ........................................................................................................................ 11 ANNEX 1: The Project Result Framework .......................................................................................................... 13 1 Terms of Reference – Final Evaluation for MRE TERMS OF REFERENCE – FINAL EVALUATION “Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project” 1. BACKGROUND Catholic Relief Services Vietnam (CRS) is a US non-governmental organization that was established in 1943 and that has been headquartering in Baltimore, America. CRS started programs in Vietnam in 1994 with an office in Hanoi. The project, “Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project” (Grant # SPMWRA12GR1096) is implemented in Quang Tri (QT), Quang Binh (QB), and Quang Nam (QN). The project has two components, including Mine Risk Education (MRE) for primary students and assistance to landmine survivors (LMS). In the MRE component, CRS in collaboration with its partners, provincial Departments of Education and Training in QB, QT, and QN, developed an MRE integration guideline (MREIG) to help teachers integrate MRE content into compulsory curricula in schools. The MREIG is now being used at all 475 primary schools in these three provinces. Additionally, in the second component, CRS and its partners, provincial Departments of Labor, Invalids, and Social Affairs in QB, QT, and QN, provide support of education, agriculture livelihood, and health, to 300 LMS. LMS’ identified needs are being met through the project’s resource or existing available service providers in their locality. The Results Framework is included on Annex 1. Local project partners—including the provincial Departments of Education and Training (DOETs) and stakeholders such as Departments of Labor Invalid and Social Affairs (DOLISA) at the provincial and district levels— play a crucial role in all stages of project implementation. Moreover, CRS works with Quang Tri Teachers Training College to replicate MRE to other primary-education students at Teachers Training Colleges in Hue, Quang Nam, Quang Ngai, Nghe An, Ha Tinh, and Quang Binh. The project final evaluation, planned from 1st March to 13 May 2016, will assess its relevance, effectiveness, efficiency, impact, and sustainability; identify lessons learned and potential promising practices; and make recommendations for future similar project development. 2. OBJECTIVES OF THE EVALUATION The final evaluation has five objectives as follows: 1. To assess and document the extent to which CRS/VN has achieved the Project’s strategic objectives and contributed to the project goal including intended and unintended, positive and negative outcomes resulting from the project (the project impact). 2. To determine major successes and innovations accomplished during the project’s implementation as well as challenges and lessons learnt 3. To collect feedback from the project participants on the relevance, effectiveness and efficiency of project’s strategies/approaches, including the participants/beneficiary selection process. 4. To analyze the level of lasting change and sustainability as well as replicability of the project’s interventions; 5. To make recommendations for CRS/VN and its key implementing partners in terms of linking services to landmine survivors, implementing MRE as well as informing future programing. 2 Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project 3. METHODOLOGY AND SCOPE 4.1 Design The methodology for the final evaluation will include 1) a desk review of project document; 2) quantitative survey using self-administered questionnaires; 3) qualitative tools in targeted communities; 4) Participatory Reflection Events Desk review: Below is a list of documents that will be made available to the evaluation team. Project Result framework; Project Proframe MEAL Plan Update IPTT Baseline tools Baseline survey report Project proposals and project agreements with implementing partners; Partners’ reports CRS quarterly reports National and sectorial policies (national program 504, PWDs law, and policies of MOLISA for PWDs) Award Provisions and Standard Terms and Conditions of US Department of States Quantitative Assessment: A self-administered questionnaire will be delivered to interviewees who are students and teachers of selected schools to collect feedback. Qualitative method: Focus group discussions (FGDs) and key informant interviews (KIIs) will be the principal qualitative methods used to collect in-depth information related to the evaluation’s objectives and evaluation questions with targeted groups according to Table 1. Quantitative and qualitative data collection will be conducted from 17 March to 31 March 2016, respectively, including training and field-testing. Participatory Reflection Events After data collection, there will be two reflection events. 1) Participatory internal learning meeting among CRS staff. One day will be allocated for staff to summarize and analyze the information collected during the FGDs, KIIs, and questionnaires. 2) Based on the information collected, CRS will share preliminary findings with partners to collect their opinion in the reflection workshop in April. This participatory reflection session will allow partners to reflect on issues of project relevance, effectiveness, sustainability, successes, challenges and recommendations. This reflection event will result in lessons learned, potential promising practices, and recommendations for future programming. 4.2 Sampling and sample size Qualitative assessment: The qualitative assessment using purposeful sampling method will be conducted in three provinces where MRE-VA projects are being implemented. The number of FGDs and KIIs with different target groups are as follows: Target group Number of FGDs (at least 6 participants per Number of KIIs 3 Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project DOETs representative DOLISA representative Community volunteers Village head Landmine survivors Students Teachers CRS program staff TOTAL group) 0 0 3 3 6 6 0 0 18 3 3 6 6 9 6 6 1 40 Quantitative assessment: Sampling method The sampling method and sample size of quantitative survey of the final evaluation will be the same as those in the baseline. A stratified two-stage cluster sampling method was applied. According to CRS guidance on monitoring and evaluation, , clustered sample without stratification will be applied to select teachers and clustered sample with stratification (boy vs girls) will be used to select students 1 Therefore, sample size for quantitative assessment will be as follows: Old schools New schools Teachers Students 260 260 Boys 169 169 Girls 169 169 Total 338 338 At the first stage, probability proportional to size sampling method was applied to randomly select 20 schools (10 among old schools and 10 among new schools) from target population of 44 old schools 2and 397 new schools 3 in three project provinces. At the second stage, each school will randomly select 34 students (17 girls, 17 boys) to participate in the final evaluation. 4.3 Tools Final evaluation tools including FGD guide, in-depth interview guide and self-administered structured questionnaire will be developed. The baseline tools will be modified slightly for the objectives of the evaluation. Tools will be developed in English, translated into Vietnamese and cross-checked for accuracy in terminology translation. 1 Guidance on monitoring and evaluation, page 57 2 Old schools are schools which implemented MRE project from Nov 2012 to February 2014. 3 New schools are schools which have implemented MRE project since Oct 2014. 4 Terms of Reference – Final Evaluation for MRE 4. EVALUATION CRITERIA AND QUESTIONS In order to achieve evaluation objectives, the following key evaluation questions have been developed and grouped by the 5 evaluation criteria of OECD4.. The evaluation team will develop data collection tools with more specific questions based on these evaluation questions. Table 1 - Key Evaluation Questions and Informants Relevance Were the project interventions in line with priorities and policies of the country, provinces and donor? Qualitative, Desk Review Did the project staff have the appropriate capacity to implement the project? Why/ why not? Qualitative KII, FGD KII and internal meeting KII and internal meeting x x x x x x x x x x x Did the project meet following strategic objectives and to what extent? Why/ why not? 1. Target community reduce the LM/UXO risks '• # of UXO/LM accidents that happen in target communities during the project period • % of students in target schools practicing safe behaviors 4 Desk review Quantitative DOLISA’s and DOET’s reports Structured questionnair es OECD, “DAC criteria for evaluating development assistance” at http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm 5 CRS Commune Head/vice head Landmine survivors x Students Qualitative Teachers Did the project design (project's objectives, activities, timing) meet the priorities and needs of the community (victims, teachers, students)? Were there differences between men and women; adults and children? If so, how? If not, why not? Tool Community volunteers How/ method DOLISA What information you need to know DOET Who to talk to /respondent x Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project Quantitative Structured questionnair es • % of DOETs that have officially approved the MREIG for mandatory use in all primary schools across the target provinces Desk review Progress report 2. LM/UXO survivors have increased benefits/social services through the volunteers’ networks and DOLISA • # (300) of UXO/LM survivors that receive assistance through the project and other available services. Desk review Progress report/IPTT table x • # of link between survivors' needs and available existing service providers are implemented by VNs and DOLISAs (desk review/progress report/IPTT table) Desk review Progress report/IPTT table x Did the project achieve its planned outputs on the planned timeline? Why/ why not? Reflection events and desk review IPTT table x Has the working in partnership (meet deadline, time allocation for the project, staffing) influence to the effectiveness and quality of the project? Why/ Why not? Qualitative and refection events KII guide x x Has the project been effective in building partner capacity? If so, how has partner capacity been built? If not, why not? If not, how can this be improved for next time? Qualitative KII and FGD guide x x What were the major factors influencing the achievement or nonachievement of the objectives? Qualitative KII guide x x x KII guide x x x KII guide x x x • % of target schools with >=80% of students practicing safe behaviors Effectiveness Are the project's staffing and management structure efficient? Why/ why not? Efficiency Did the M&E system provide the right information at the right time to allow for timely project management and decision-making? Why/ why not? What is the cost per project participant by project component? Is this reasonable given project impact? Why/ why not? Were objectives achieved on time? Were objectives achieved on budget? Refection events Qualitative and refection events Refection events and desk review Refection events and x x x x x x Relevant project documents x Relevant project X 6 Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project What are significant changes in beneficiaries’ life after participating in the project? What are evidence for better quality of life of beneficiaries? What is the most valuable impact to beneficiaries? Why? Impact What major successes, accomplishments, and innovations were achieved during the project for each component (MRE and VA)? What key factors led to these successes? Which impacts were most valued (and valuable) to beneficiaries in particular, and society in general? What are the unintended outcomes (both negative and positive) of the project? How many people benefit from the project? What were the program’s biggest challenges or failures? Were these adequately addressed during implementation? What are the most relevant lessons learnt from the program implementation? Which activities/results can the DOLISAs/DOETs/communes/schools continue after the project funding ends? Why/ why not? If not, What support do they need and what and how do they need to do to sustain it? Sustainability desk review documents Qualitative and reflection events KII guide x x x x x x x x Qualitative KII and FGD guide x x x x X x x x x x x x x x x x Qualitative Desk review Qualitative Qualitative Have sustainability plans been developed and implemented? Are there any indications that services/ activities/models developed during program implementation will be still used, functioning or scaled out once the project ends? Qualitative What are the threats to the sustainability of project benefits? Why? Qualitative What are the successful aspects of the project can be replicated? How? Qualitative KII and FGD guide IPTT table KII and FGD guide KII and FGD guide KII and FGD guide KII and FGD guide KII and FGD guide x x x x x x x x x x x x x x x x x x x x x x x x x x 7 Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project 4.4 Data collection It is expected that data collection will be conducted by consultants with support from CRS team in the mid of March 2016. A team of data collectors will be trained on all data collection tools. 4.5 Data Analysis Analysis of qualitative data and quantitative data will be carried out by the national consultant with input from the Evaluation team in Vietnam. 4.6 Stakeholder Engagement Students X X X X X X X X X X X X X X X X X X X X X X X X X X CRS Teachers Landmine survivors Commune Head/vice head DOLISA Progress toward end of project targets for key indicators at SO, IR and output levels Assessment of project relevance, efficiency, impact, sustainability and potential for scale-up or replication Unexpected outcomes (positive or negative) of the project Lessons learnt and recommendations for future programing. DOET Interest/Information Need Community volunteers Given the objectives of “Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities” project, the final evaluation must include opportunities for consultation and participation of provincial DOET and DOLISA representatives in 3 provinces: Quang Binh, Quang Nam, and Quang Tri. The evaluation methodology and tools as well as data collection plan will be shared with relevant partners for their input prior to the start of data collection. DOLISA and DOET will join in data collection with CRS. The preliminary evaluation findings will be reflected at the reflection workshop. The final evaluation report will be shared with all relevant stakeholders. 4.7 Scope of the evaluation The evaluation will be conducted in Quang Nam, Quang Tri, and Quang Binh from 29 February to 13 May 2016 5. REPORT WRITING The national consultant is responsible for writing the full evaluation report (in English and the report will be translated into Vietnamese to send to partners). The CRS Evaluation team, including the Sub-regional HoP, will provide review and feedback on the first draft evaluation report and on the revised draft. The total final evaluation report should not be more than 20 pages, excluding annexes. 8 Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project The following outline is suggested for the full evaluation report: Executive Summary (1.5 pages) Background (0.5 pages) Methodology and limitations(1 page) Findings with Analysis and discussions (14 pages, including tables and graphs) Lessons Learned and promising practices (1 page, including summary of lessons learned and potential promising practices) Conclusion and Recommendations (2 page) Annexes (ToR, tools, etc.) 6. ETHICAL GUIDANCE AND CONSIDERATION The Consultants must ensure that the baseline study adheres to ethical guidelines as outlined in the American Evaluation Association’s (AEA) Guiding Principles for Evaluators. A summary of these guidelines is provided below: 1. Verbal Informed Consent: All participants are expected to provide verbal informed consent following standard and pre-agreed consent protocols. For children respondents (under 18 years) in the qualitative assessment, written parental consent is required for each child participating. 2. Systematic Inquiry: Evaluators conduct systematic, data-based inquiries. 3. Competence: Evaluators provide competent performance to stakeholders. 4. Integrity/Honesty: Evaluators display honesty and integrity in their own behavior, and attempt to ensure the honesty and integrity of the entire evaluation process. 5. Respect for People: Evaluators respect the security, dignity and self-worth of respondents, program participants, clients, and other evaluation stakeholders. It is expected that the evaluator will obtain the informed consent of participants to ensure that they can decide in a conscious, deliberate way whether they want to participate. 6. Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation. 7. A link to a more detailed description of AEA’s Guiding Principles for Evaluators can be found at: http://comm.eval.org/eval/Go.aspx?c=ViewDocument&DocumentKey=ba879c95-f810-4c6bbf50-524da31144c1 7. EVALUATION TEAM The national consultant will be hired to participate in designing tools, conduct qualitative data collection in the field, co-facilitate two participatory reflection events with CRS staff, quantitative data entry, qualitative data transcription, and overall data analysis and develop the report. CRS Project team (including 1 PM and 2 POs) will support evaluation planning, tool design, quantitative and qualitative data collection, and report review. The PM will be responsible for overseeing and coordinating the overall evaluation activities and process, including data analysis. POs will liaise with partners to support the data collection in provinces. CRS MEAL team will participating in evaluation planning, provide technical support in tool design, quantitative and qualitative data collection, and report review. 9 Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project Support from CRS Sub-regional HOP will also be there to ensure the quality of the evaluation. 8. TIMELINES The following is a proposed action plan and tentative timeline for the evaluation: Action Point TOR for final evaluation Finalization and approval of TOR Hiring an external consultant and local contract signed Data collection plan with partners Design Tools Tentative date 29/2/2016 4/3/2016 16/3/2016 17/3/2016 18/3/2016 Field-test tools 21/3/2016 Finalization of Tools 22/3/2016 Data collection 31/3/2016 Data transcription and quantitative data entry Data analysis CRS internal team reflection 7/4/2016 15/4/2016 19/4/2016 Reflection workshop with partners Draft report written Comments on the draft report 25/4/2016 3/5/2016 10/5/2016 Finalization of evaluation report Evaluation report will be shared with relevant stakeholders 12/5/2016 13/5/2016 Responsible (involved) CRS Team CRS team and HoP CRS CRS Team CRS Team (Consultant) CRS Team CRS Team (Consultant) Consultant (Data collectors and CRS Team) Consultant Consultant Consultant (CRS team) CRS team Consultant MEAL, project team, HoP Consultant CRS Team 9. REQUIREMENTS FOR CONSULTANT 9.1 Consultant Deliverables - A written evaluation plan detailing the full design and methodology of the evaluation, which will be subject to CRS/VN approval; Data collection tools and work plan, also subject to approval by CRS/VN and partners; Clean quantitative data set; qualitative KII/FGD notes and transcripts Quantitative and qualitative data matrix Presentation of preliminary evaluation findings for reflection workshop A draft evaluation report, for review by CRS and partners; A final evaluation report in English and Vietnamese, taking into account suggestions and changes recommended during the review/validation process; 9.2 Qualifications and Experience Required of External Consultant - Master degree in social sciences or a related field; 10 Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project - - Demonstrable expertise in designing and conducting qualitative and quantitative program evaluation, including data analysis, impact assessment, to provide strategic recommendations for continued support/development of programming/strategies; Strong knowledge and 5-year experience in relevant field (education, landmine) in Vietnam; Excellent and demonstrable skill of Participatory monitoring and evaluation methodologies and approaches; Excellent analytical, communication and report writing skills; Excellent written skill, in English and in Vietnamese; Ability to work under pressure, as an individual and in a team; Ability to solve problems. 9.3 Period of Performance As shown in the table above, the period performance for the final evaluation is expected to be from 1 March 2016 to 13 May 2016. The fee for consultant will be negotiated and applied following CRS Vietnam regulations The contract is expected to start by 17 March to 12 May 2016: Tentative 36 days Action Point Design Tools Finalization of Tools Data collection Data transcription and quantitative data entry Data analysis CRS internal team reflection Reflection workshop with partners Draft report written Finalization of evaluation report Tentative date 18/3/2016 22/3/2016 31/3/2016 7/4/2016 15/4/2016 19/4/2016 25/4/2016 3/5/2016 12/5/2016 No. of days 1 1 9 7 6 1 1 7 3 9.4 Compensation: CRS will pay the consultant a fixed daily rate. The rate will depend on the selected consultant’s level of experience 9.5 Application requirements Institutions and Individuals interested in this job are invited to submit your applications via email to Catholic Relief Services, Ms. Trần Thị Thu Hằng, at: [email protected] . Applications should include: i) Expression of Interest; ii) A concise technical proposal; iii) A tentative work plan with indicative budget, iv) CVs demonstrating relevant capacity and experience. Deadline for submission: by 10 March 2016 For further information, please contact: Ms. Ta Thi Hai Yen - Program Manager 11 Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project Tel: 04. 37738300, ext: 137 Email: [email protected] CRS is an equal-opportunity employer and does not discriminate on the basis of ethnicity, religion, sex, national origin, disability, or HIV/AIDs. 12 Terms of Reference – Final Evaluation for Consultant Responding to the social/economic effects of Landmine/UXO in Vietnam: UXO/Mine risk education and comprehensive community support for landmine incident survivors in high risk communities project ANNEX 1: The Project Result Framework Program Goal: The negative humanitarian, social and economic effects of explosive remnants of war (ERWs) is reduced in QB, QN, and QT provinces. SO 2: LM/UXO survivors have increased benefits/social services through the Volunteers’ Network and DOLISAs. SO 1: Target communities reduce the LM/UXO risks. IR 1.1 : Trained teachers implement the MREIG in 514 (470+44 already piloted schools) schools in 20 districts in QB, QN and QT provinces. rights and needs Output 1.1.1: Teachers in 470 new schools in QB, QT, and QN have complete knowledge and skills on the revised MRE integration guideline IR 1.2: Trained students practice safe behaviors in schools and at home in 20 districts in QB, QN and QT provinces. priority needs Output 1.1.2: All teachers in 44 old/existing/ schools have refresher knowledge and skills on the Revised and approved MRE guideline Output 1.2.1: All primary students in QB, QT, QN provinces have complete knowledge on MRE IR 1.3: Future teachers in TTCs in 7 high risk provinces have improved understanding on how to use the MRE integration guideline in their future job. Output 1.3.1: Future teachers in TTCs in 7 high risk provinces are trained on how to use the MRE integration guidelines. Output 2.1: 11 new volunteer networks are established and 9 old volunteer networks are maintained IR 2.1: Volunteer networks in 20 communes (11 new communes and 9 currentlytargeted communes) in 9 districts of QT, QB and QN provinces link LM/UXO survivors to services and Output 2.2: Volunteer Networks in new areas have knowledge and skills to link LM/UXO survivors to services and support Output 2.3: Volunteer Networks in old areas have refresher knowledge and skills to link LM/UXO survivors to services 13
© Copyright 2026 Paperzz