Consultancy - Evaluator

Duration: 3 months

The Position:

UNFPA Evaluation Office (EO) is seeking a social media professional with proven social media and digital skills to manage EO’s social media channels and create content that reflects EO’s communication priorities. This position reports directly to the Communication and Knowledge Management Specialist at UNFPA Evaluation Office.

How you can make a difference:

UNFPA is the lead UN agency for delivering a world where every pregnancy is wanted, every childbirth is safe and every young person’s potential is fulfilled. UNFPA strategic plan 2018-2021 focuses on three transformative results: to end preventable maternal deaths; end unmet need for family planning; and end gender-based violence and harmful practices. The evaluation function supports UNFPA to achieve its mandate by demonstrating accountability to stakeholders on its performance. Evaluation also supports evidence-based decision-making and learning by enhancing the existing knowledge base on how best to advance sexual and reproductive health and rights, and on how UNFPA can effectively support the achievement of the Sustainable Development Goals.

In a world where fundamental human rights are at risk, we need principled and ethical staff, who embody these international norms and standards, and who will defend them courageously and with full conviction.

UNFPA is seeking candidates that transform, inspire and deliver high impact and sustained results; we need staff who are transparent, exceptional in how they manage the resources entrusted to them and who commit to deliver excellence in programme results.

 

Job Purpose:

The humanitarian situation in Yemen continues to escalate, as fighting continues to erupt in multiple frontlines around the country and new infighting emerging among hitherto allied groups. According to humanitarian actors project that in 2020, 24 million people are going to be in need, of which 15.6 million will be targeted. Among the estimated 3.9 million women of reproductive age are some 624,000 vulnerable women who will be pregnant or delivering during the course of the year with some 93,600 likely to develop complications. Government financing of health services has all but ceased and access to lifesaving RH services has become difficult with only 37 percent of health facilities providing maternal and newborn health care services. Women in most remote areas lack access to services due to lack of providers and transportation means. One in five people suffer mental disorders due the ongoing crisis. More than 300,000 residents have been displaced between January and mid-April 2019 alone. Most of these reside in collective sites. While the RRM provides immediate assistance, the post-RRM assistance is mostly delayed.

UNFPA with the support from ECHO is implementing the project “Providing Integrated Lifesaving Reproductive Health and Mental Health Specialized Services to Under-served Communities and Rapid Response Mechanism to newly displaced persons affected by the Humanitarian Crisis in Yemen”. The Action supports improved access to the Minimum Initial Service Package (MISP) for RH, mobile RH teams, specialized psychological centers, and immediate assistance to newly displaced persons. The proposed action comprise three main elements, namely; 1) support 52 health facilities in selected districts to provide basic and comprehensive EmONC services, including integrated GBV services; 2) community-based reproductive health through 50 midwives and two mental health specialized centers; and 3) responding to those newly displaced with immediate safe assistance throughout the country, with measures to accelerate the sequencing of the assistance to more sustainable forms, including enrollment in 110 districts with highest displacement.

The results and objectives of the project will be evaluated through the MTE, according to the UNFPA ECHO project logical framework, are as follows:

Long-term objective: Improved access to integrated basic lifesaving reproductive, maternal and newborn health, and nutrition services including GBV prevention, mental health specialized services, and Rapid Response Mechanism to newly displaced persons affected by the Humanitarian Crisis in Yemen

 

Specific objective: Support targeted facilities to provide basic and comprehensive EmONC quality services. Community based RH services through the deployment of mobile teams and/or community midwives. Support to community based specialized psychological health centers. Life-saving assistance made available to newly displaced persons, and accelerating the access of the most vulnerable to sustainable forms of humanitarian assistance.

 

Specific objective detailed description:

The action support timely response to newly displaced population by ensuring RRM delivers urgent lifesaving kits within 72 hours of displacement. This action contributes to the coordination, pre-positioning and distribution costs for all the three kits, and procurement of the dignity kit component of the RRM package. The other components comprise of hygiene kits and IRR being provided by UNICEF and WFP respectively. It also supports the enrollment of the newly displaced persons, which reduces the risk of exclusion, and provide quicker and better quality data to food and MPCA actors who can provide more sustainable assistance. The action strengthens health facilities to provide lifesaving RH services integrated with GBV services by providing the required health and social workers to augment the under staffing that has resulted from lack of salary payment and beef up existing staff to cope with increased caseload due to displacement. UNFPA works with NGO partners to support payment of incentives to retention of staff required to provide EmONC services. The NGOs role is to ensure quality assurance by deploying the right service delivery team and quality of care. Medical supplies including the emergency RH Kits that contain basic equipment and supplies for management of lifesaving RH services are provided. Health facilities are also supported to provide GBV services. The action supports community level RH services through Community Midwives or mobile teams to complement facility based services that have currently been overstretched. The midwives received orientation training, midwifery kits and supplies and equipment to conduct ANC, normal home deliveries, identify danger signs and timely refer complicated cases to referral level health facilities as defined in the Yemen Minimum Service Package. The action supports two specialized community-based psychological centers providing psychological and psychiatric assistance to those in need. This intervention is complemented by WHO and protection actors. UNFPA has identified the locations where the midwives and mobile team’s models will be deployed/implemented. Currently, mobile teams are in place in Ta'iz, Al Jawf and Ma'rib. While midwives are in place in most of the governorates through home-based clinics and community midwifery. In the course of March, and in consultation with ECHO field, the locations supported by this action have been defined through this MR, the action will support an additional 36 health facilities to reach a total of 52 facilities (16 CEMONC and 36 BEMONC). It ensures the service provision is done safely, in line with the COVID-19 preventive and infection control measures. This is in line with the available evidence that the RH service uptake is at almost the pre-COVID-19 levels in the facilities which continued to run. This is partially attributed to the presence of infection control measures in place, sustained demand from the communities and possibly the most vulnerable women and girls accessing these services. The action will also support the provision of the RRM distribution and coordination costs for an additional month to ensure the continuation of the enrollment of those newly displaced, assisting them with the RRM kits, and alerting other actors, mainly the DRC/IOM cash consortium and WFP/GFD partners to provide sequenced assistance.

 

Objectives and Scope

 

The purpose of this MTE is to provide an independent assessment of the project to improve project implementation, and making necessary course corrections. The evaluation will assess relevance, effectiveness, and sustainability of the project.

 

More specifically, the objectives of the MTE will be to assess:

• Progress made toward the achievements against the logic of the partnership and performance after 28 months of implementation (from July 2018 to date).
•  Relevance of the project strategies and design in the evolving context of changing socio-economic developments and realities in Yemen such as inclusion of accountability to affected population and alignment of partners’ strategies on RRM and capacity sequenced responses and common registry Effectiveness of the project in achieving the specific results of the project.
• Recommend adjustments, if any, to project strategies and directions for the remainder of the project.
• Potential sustainability of the project

 

Criteria and Key Questions

 

The MTE will look at the following criteria.

 

Relevance will look at the planning, design, implementation and coherence of the project to UNFPA Yemen concerns. The MTE will make suggestions on improvement of the project design to achieve the results focusing on targeting and evidence based prioritization of targeted facilities, beneficiaries and stakeholders

 

Effectiveness will examine factors contributing to achievement of the results in time, quality of service and beneficiaries’ views. The MTE should also examine UNFPA’s coordination role and look at the level of ownership among the stakeholders in the project. Any lessons learnt at this stage of the project should be identified by the MTE.

 

Efficiency will examine the extent to which the Project is making best use of available human, technical, technological, financial and knowledge inputs to achieve its desired results.

 

Sustainability will look at the aspects of the project which is contributing to the continuation of the results of the project after its completion. Suggestions on further activities to improve transferability of the project.

Methods, Process and Timeframe

 

The evaluation will be a transparent and participatory process involving relevant UNFPA Yemen stakeholders and partners at the regional, and country levels. The evaluation methodology will employ mixed methods for data collection. The logic model based on ECHO project will be used to assess whether the project is on the right track and whether the proposed interventions have the potential to achieve proposed outcomes in the suggested time-frame.

 

The evaluation will have two levels of analysis and validation of information:

 

Level 1 will start with a desk review of information sources on UNFPA Yemen ECHO project work available through project document, detail AWP, progress reports and annual project reports, monitoring reports prepared by UNFPA staff and relevant government reports on the implementation of the project; technical products developed during the project implementation, meeting minutes of the important meetings, capacity assessment report conducted in the beginning of the project implementation.

 

Level 2 will involve in-depth analysis of the project both by qualitative and quantitative data collection. Level 2 will involve field visits and will employ a number of evaluation methods ranging from document review, interviews, focus group discussions, surveys, observation depending on the final methodology. A qualitative comparative case study analysis can be applied too.

The following data sources will be utilized and data will be triangulated to ensure validity and reliability:

• Review of key documents: Strategic Plan; ECHO Project Annual Reports; baseline study; monitoring/reporting information, including donor reports; previous evaluations; guidance notes, etc.;
• Interviews and focus groups with a purposive sample from UNFPA ECHO Project staff, staff from partners, other UN agencies, any civil society partners, beneficiaries and local-national authorities.

 

This evaluation is taking place at the end of 28 months’ implementation of project activities. The MTE will look at the progress made towards the results and suggest any changes to management to improve the project. The entire process of the MTE will take place over a period of six months, in which collection of field data should be completed over a period of one month from the start of data collection. The project is implemented in partnership with IPs all over Yemen. The MTE is expected to cover all stakeholders engaged in this project.

 

The MTE will include the following steps:

 

Step 1: Inception meeting, desk review of key programme documents and key stakeholder interviews to understand the scope of the MTE. Review project theory (desk review and meetings) and Stakeholder analysis.

The evaluator will attend an inception meeting where s/he will be oriented on project objectives and key progresses made. At this stage of the MTE, the evaluator will have the chance to meet with UNFPA staff and staff from IPs working on this project to be oriented with the project and define scope as well as design of the MTE. At this stage, UNFPA will provide the evaluator with key project documents for review. The documents could include agreement with donor, letter of agreements between UNFPA and IPs, capacity assessment report, monitoring reports by UNFPA staff, post activity reports by IPs. Documents should provide a sense of the intent of the project as well as what is actually occurring.

 

 

Step 2: Submission of Inception Report and finalization of methodology and evaluation design based on UNFPA feedback.

The inception report should include final evaluation questions, identified stakeholders for interviews and discussions, and present the methodology of the midterm evaluation. This will be finalized in agreement with UNFPA.

 

Step 3: Data collection (mixed-methods).

Interviews and focus group discussions, or questionnaire interview should focus on what stakeholders know and perceive to be true about the project. Data from project staff, documentation, and stakeholder interviews and group discussions will be used to determine plausibility of the project model. That is, data are analyzed to determine the extent to which the project is properly implemented, sufficiently developed, and activities appropriate, to reasonably predict that desired outcomes will be met.

 

Step 4: Data analysis.

Collected data should be analyzed. Analysis framework should be clearly explained in the report.

 

Step 5: Share preliminary findings and presentation draft report with UNFPA.

The evaluator will share preliminary findings and recommendations with UNFPA at the end of the field visit and interviews with stakeholders. The evaluator will draft the report and present the initial report to a group with representatives from UNFPA and key stakeholders. Feedback will be taken into consideration and incorporated into the final report.

 

Step 6: Draw conclusions and make recommendations (analysis and report writing).

The evaluator makes conclusions and recommendations. Conclusions and recommendations are drawn from the data. The evaluator is encouraged to guard against validity threats, such as personal bias.

 

Step 7: Draft report.

The evaluator finalizes a draft report. The report structure should follow the UNEG’s evaluation report guidance. UNFPA will review the report as part of quality assurance and will share it with the reference group for their feedback.

 

Step 8: Finalization of the MTE report.

The evaluator will present the final draft MTE report to the stakeholders in a validation workshop. Recommendations of the MTE report will also be presented and stakeholders will prioritize recommendations to draft action plans for the Management Response. UNFPA will be responsible for finalizing the Management Response and follow up of the Actions Points.

 

Step 9: Dissemination of MTE report, and plan specific steps for utilization of MTE data.

Based on the stakeholder analysis, UNFPA will develop a dissemination plan from the beginning of the evaluation process. It will follow the principles of gender and human rights, particularly participation and inclusion of stakeholders.

 

Key Deliverables

 

Inception report: The inception report should detail the evaluators’ understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures. The inception report should also include a proposed schedule of tasks, activities and deliverables. This inception report should also identify the sites visits and it should elaborate on the selection criteria for those sites selected.

 

Preliminary findings: The evaluator shall share initial findings and recommendations with the UNFPA Yemen programme team prior to the stakeholders’ consultation.

 

Draft reports: Report structure should follow UNEG evaluation report guidance.

 

Facilitate and presentation of draft MTE report: A presentation will be done on the draft report and the draft report will be shared with the UNFPA Yemen Country Office for comments. In addition to validation of data, the evaluator should draft practical recommendations in consultation with stakeholders.

 

Final MTE report: The final report will not exceed 40 pages (not including annex) in hard and soft copy to be submitted to UNFPA (please follow UNFPA’s evaluation report guidance).

 

Qualifications and Experience

The following are the required skills and competencies expected from the selected applicant for conducting this MTE. In the case of an evaluation team, it is expected that the team leader should fulfill all required skills and competencies.

• Education:
  
  • Advanced degree in relevant discipline (e.g. public health, development and social studies, sociology, political science, etc.)
  • Advanced degree in evaluation is an asset.

• Professional experience:
  
  • At least 10 years’ experience in programme evaluation in a humanitarian and/or development contexts and proven accomplishment in undertaking evaluations, including leading evaluations of multi-stakeholder programmes for multilateral organizations.
  • Experience in evaluating reproductive/mental health, and Rapid Response Mechanism related programmes/project.
  • Experience in working in Yemen will be an advantage.

• Knowledge and skills:
  
  • Knowledge of evaluation.
  •  Knowledge in results-based programming.
  • Proven expertise in evaluating programmes focusing on reproductive/mental health, and Rapid Response Mechanism.
  • Extensive knowledge of qualitative and quantitative evaluation methods.
  • Excellent written and spoken English and presentational capacities.
  • Knowledge of Arabic language would be an asset.
  • Knowledge of the UN system would be an asset.

Required Competencies 
Values:
•    Exemplifying integrity, 
•    Demonstrating commitment to UNFPA and the UN system, 
•    Embracing cultural diversity, 
•    Embracing change    

Core Competencies: 
•    Achieving results,
•    Being accountable,
•    Developing and applying professional expertise/business acumen,
•    Thinking analytically and strategically,
•    Working in teams/managing ourselves and our relationships,
•    Communicating for impact 

         Functional Competencies

• Job knowledge/technical expertise
• Conceptual innovation in the provision of technical expertise
• Promoting organizational learning and knowledge sharing (IERD/DHR)
• Organizational awareness
• Promoting accountability and results-based management

Evaluation Ethics

Evaluations in the UN will be conducted in accordance with the principles outlined in both UNEG Norms and Standards for Evaluation in the UN System and by the UNEG ‘Ethical Guidelines for Evaluation’. These documents will be attached to the contract. Evaluators are required to read the Norms and Standards and the guidelines and ensure a strict adherence to it, including establishing protocols to safeguard confidentiality of information obtained during the evaluation.

 

How to apply

Interested applicants must submit the following document/information (in PDF format) to demonstrate their qualifications.

 

Technical component:

• Letter of interest explaining why they are the most suitable for the work.
• Technical proposal.
• Two writing samples/reports in English.
• Signed Curriculum vitae with contact details of 3 clients for whom you have rendered preferably the similar service.

Financial proposal (with your signature):

• The financial proposal shall specify a total lump sum amount in US Dollar including consultancy fees and all associated costs
• Please note that the cost of preparing a proposal and of negotiating a contract, including any related travel, is not reimbursable as a direct cost of the assignment.
• If quoted in other currency, prices shall be converted to US Dollar at UN Exchange Rate at the submission deadline.

Interested and qualified candidates should submit a cover letter along with their CV and P11 form by email to Yem.vac@unfpa.org with the subject line "Application: ECHO Mid-term Evaluation" by 6 April 2021.