International Consultant: Team Leader of the BERANI 2 Evaluability Assessment

Purpose of consultancy:

To conduct the evaluability assessment of the BERANI 2 UN Joint programme to strengthen the BERANI II programme by aligning its design with intended goals.

 

Background:

In 2018, a program called BERANI, meaning "courageous" in Indonesian, emerged to tackle critical issues surrounding Sexual and Reproductive Health (SRH) for women, girls, and youth. Backed by a powerful alliance (the Indonesian government, UNFPA, UNICEF, and Global Affairs Canada), BERANI addressed harmful norms and social barriers for five years, transforming countless lives. While BERANI achieved significant success, challenges remained. Maternal mortality persisted, access to quality adolescent reproductive health services needed improvement, and gender-based violence (GBV) remained a concern. Harmful practices like child marriage and female genital mutilation/cutting (FGM/C) also demanded continued attention.

UNFPA, UNICEF, and UN Women Indonesia are jointly conducting an Evaluability Assessment (EA) of the BERANI Phase II program (2023-2027) as stipulated in its project document. BERANI, which stands for Better Reproductive Health and Rights for All in Indonesia, aims to directly benefit 13,000 individuals, including 7,000 women and young girls.

BERANI II Steps Up: Building on this strong foundation, BERANI II expands its reach to vulnerable populations in 26 districts across 14 Indonesian provinces. Two districts will serve as pilot locations, showcasing the entire program model. Aligned with national efforts for gender equality, BERANI II reflects a collective commitment to empowering women.

The Ultimate Goal:  Improved SRH & RR of all women and girls in Indonesia. The pathway to success are as follows:

• Enhanced SRH & RR Services: BERANI II focuses on improving the quality of adolescent reproductive health education and Sexual and Reproductive Health and Reproductive Rights (SRH & RR) services by training teachers, healthcare providers, and midwives.
• Empowered Individuals and Communities: The program empowers individuals and communities to prevent GBV and harmful practices through life skills education, financial skills training for women, and workshops promoting healthy relationships and parenting. Additionally, BERANI II trains community leaders, religious figures, and young people to advocate for change. Public awareness campaigns further amplify these efforts.
• Stronger Policies and Implementation: BERANI II works with national and sub-national governments to develop evidence-based policies that strengthen SRH & RR and address GBV and harmful practices. This involves technical assistance and support for advocacy by civil society organizations.
• Sustainability: Building a Lasting Impact. BERANI II prioritizes long-term impact through collaboration with government bodies, civil society organizations, and academic institutions. The project fosters knowledge exchange and encourages replication of successful models across Indonesia. Additionally, BERANI II works to ensure promising interventions are incorporated into national budgets for long-term sustainability. Evidence generated by the program will be used to advocate for continued government support.

To effectively evaluate the BERANI Phase II program's impact and ensure it meets its intended objectives, it is crucial to conduct a thorough and systematic assessment. This evaluability assessment aims to determine the extent to which progress towards the program's objectives can be readily and reliably measured, monitored, and evaluated. It will focus on ensuring universal access to SRH & RR, particularly for vulnerable women and girls, and achieving transformative results in ending preventable maternal deaths, addressing unmet needs for family planning, and eliminating GBV and harmful practices in Indonesia. By conducting a thorough assessment, we aim to identify areas for improvement, address potential challenges, optimize the program's implementation strategy, and evaluate the effectiveness of data collection and utilization within the program.

 

Scope of work:

Objectives and Scope of the evaluability assessment (EA)

Purposes: 

Sharpening BERANI II: A Focused Evaluability Assessment. The Evaluability Assessment (EA) aims to strengthen the BERANI II programme by aligning its design with intended goals. Through a thorough review, it is expected to identify evaluability of project indicators, areas for improvement, address potential challenges, and refine the programme's indicators and implementation strategy.  It will also assess the effectiveness of data collection and utilisation.

Key Objectives:

• Strengthening the Core: assessing program design and results framework, ensuring clarity and measurability.
• Alignment & Improvement: Verifying alignment with National development plan and SDGS goals, identify areas for improvement, and propose implementation strategy refinements.
• Results & Measurement: Evaluating the results chain's logic and effectiveness of chosen indicators and measurement tools.
• M&E System Strength: Assessing established monitoring and evaluation systems for their ability to track progress and measure program impact and suggesting suitable approaches and methodologies for the final evaluation of the joint programme. 
• Data-Driven Decisions: Evaluating data collection (data from PUNOs and relevant stakeholders), analysis, sharing, and utilization to ensure it informs decision-making and program improvement.

Benefits: By achieving these objectives, the EA will:

• Position BERANI II for Success: A strong foundation sets the stage for impactful execution.
• Maximize Program Impact: Effective data use leads to continuous improvement and program results.

In relation to copyright, all materials or documents resulting from this consultation services will remain the property of the UNFPA. The consultant is obliged to provide soft copies of the deliverables, including all annexes so that all results can be independently verified for their accuracy before the report is finalised and released. 

The primary audience and users of the evaluability assessment report include decision-makers and program managers in UNFPA, UNICEF, and UN Women Indonesia, the Government of Canada, Ministry of National Development Planning/National Development Planning Agency (BAPPENAS), other counterparts in the Government of Indonesia, the United Nations Resident Coordinator Office (UNRCO) in Indonesia, and civil society organizations. This evaluability assessment, managed jointly by UNFPA, UNICEF, and UN Women, will be an external, independent exercise conducted by an independent team of evaluators. It will adhere to UNFPA guidance on program evaluations, ethical norms, and the standards of The United Nations Evaluation Group (UNEG).

Approach and Methodology

Approach

The EA approach will be transparent, inclusive, and participatory by involving the related BERANI implementing partners and stakeholders, as well as gender and human rights responsive to promote (i) ownership, (ii) impartation of evaluative thinking in planning and programming and (iii) accountability for results in a relevant final product. The EA will utilize mixed methods and draw on quantitative and qualitative data. The methods will include document review and key informant interviews (KIIs) and FGDs with PUNOS, United Nations Resident Coordinators, government partners at the national and subnational level, and any other relevant partners, right holders, and duty bearers. 

The EA will follow the guidance on the integration of gender equality, human rights principles and disability inclusion in the evaluation focus and process as established in the UNEG Handbook, Integrating Human Rights and Gender Equality in Evaluation - Towards UNEG Guidance. It will follow UNEG Norms and Standards for Evaluation in the UN system and abide by UNEG Ethical Guidelines and Code of Conduct3 and any other relevant ethical codes. The EA will utilize a theory-based approach taking into consideration the programme planning document, the programme theory of change and results framework. 

Methodology

A consultant’s team will design the EA methods and tools to answer the BERANI’s project questions and indicators; and come up with assessment results backed by clear evidence. The consultant will propose a provisional methodological design that will include an analytical framework; a strategy for collecting and analyzing data; a series of specifically designed tools; and a detailed work plan.  The main elements of the method will be further developed during the inception phase in line with the agreed EA questions (incl. assumptions to be assessed, indicators, data collection tools, and analysis approach) and analytical framework and shall include, but not be limited to, the following:

Documentary review and secondary data: A preliminary list of relevant documentation (together with electronic copies) including key documents related to BERANI II activities and existing literature on the theme will be prepared. A full set of available documents will be shared with the consultant team during the inception phase. In addition, evaluators will be responsible for identifying and researching further information (both qualitative and quantitative) at global, regional and country levels. The available documentation will be reviewed and analysed during the inception phase to determine the need for additional information and finalisation of the detailed evaluability assessment methodology. 

Interviews with key informants: will be conducted by the consultant team with PUNOS and relevant stakeholders. Key staff from programme will be interviewed during the inception phase. During the data collection phase, interviews will be conducted with PUNOS and national stakeholders and implementing partners. Interview protocols will be developed during the inception phase.

Focus Group Discussions: will be conducted by the consultant. The specific protocols for focus group discussions will be developed during the inception phase.

 

Duration and working schedule

The duration of the consultancy is 40 working days, commencing in August 2024 and to be completed in October 2024.

The assessment will be conducted in five phases:

 I – Preparatory Phase

The BERANI coordinator/ PUNOS M&E focal point will lead the preparatory work in consultation with technical teams and evaluation reference groups. This phase includes the: drafting of terms of reference for the evaluability assessment; initial document collection and review; selection and recruitment of the independent evaluability assessment team; and the constitution of a reference group for the evaluability assessment.

 II - Inception Phase

The consultant will conduct the design of the EA in consultation with BERANI coordinator/ PUNOS M&E focal point. This phase includes:

• The inception phase will involve a briefing from the BERANI coordinator/ PUNOS M&E focal point. 
• Discussions with selected PUNOS programme staff. 
• A desk review of all relevant documents. 
• A review of the intervention logic and the BERANI II Programme. 
• The development of the detailed list of evaluability assessment questions, the identification of the assumptions to be assessed and the respective indicators, sources of information, and methods and tools for the data collection
• The development of a data collection and analysis strategy.
• A concrete work plan for the field and reporting phases.
• During this phase, the consultants will produce an inception report (of around 20-30 pages) that presents the initial outline and plan of the assessment. A detailed work plan that outlines the expected work outputs and planned timing for each element should be presented to the country office at the end of the inception phase. It will further refine the assessment’s objectives, scope, and key questions to be answered by the evaluability exercise.
• The inception report will follow the structure set out in Annexes.

III - Data Collection and Analysis Phase

This phase of the EA will involve an extensive desk review, interview, and analysis. The EA consultant team will carefully review the sources and reliability of the information, determine what gaps there may be in the information required, and suggest methods needed to fill the gaps. The consultant will review the design, technical and management aspects of the programme, the M&E and the reporting systems in place and assess the constraints, opportunities, contextual and substantive issues in operationalising the programme and the monitoring and evaluation work.

During this phase, the consultant will produce a draft evaluability assessment report that includes a detailed analysis of the documentation and processes reviewed, as well as evidence-based conclusions and recommendations. The report should be between 20-30 pages maximum (excluding the Annexes).

 IV - Validation and Reporting Phase

In this phase, the debriefing meetings will be held to validate major findings of the draft evaluability assessment report. The report draft will be developed based on meeting recommendations after consolidating the comments received during the meeting. The consultant team then submitted the draft for approval and final clearance. 

The exercise will culminate with the completion and approval of the evaluability assessment report which includes an evaluability assessment matrix, findings, and recommendations.

 V – Dissemination and Follow-up Phase

 The management of the evaluability assessment will follow a participatory approach in close collaboration with PUNOS programme colleagues and development partners concerned to engage them in key moments of the evaluability assessment process. A final evaluability assessment report based on comments and feedback received on the draft report during the validation phase (60-75 pages excluding annexes).

 

Place where services are to be delivered:

The consultant must have extensive experience in leading evaluability assessments and/or evaluations of a similar scope and type, complexity, and character, as well as technical expertise in the areas required. 

The consultancy is home-based with a regular visit to UNFPA Indonesia and site visits for the purpose of data collection after the discussion with the team. She/he is responsible for carrying out all the tasks described in the ToR. All deliverables are to be produced as per the accepted quality standards as per UNEG guidelines:

 

Delivery dates and how work: 

The list below outlines the breakdown of the expected deliverables of the evaluability assessment:

Inception phase: a draft and final inception report (20-30 pages excluding annex): 10 Working days

• Scope and methods
• Finalized list of questions 
• Workplan, timeline, and schedule to conduct interviews/ discussions. 
• Report template for the final evaluability assessment report

Data collection and analysis phase: 15 working days

• Desk review of relevant documents
• Interviews with staff and key stakeholders, data analysis and development of a draft report with the preliminary findings and recommendations
• Data collection phase debrief.

Validation meeting and final report: 15 Working days

• A PowerPoint presentation (15-20 slides): A summary of key findings and conclusions prepared towards the end of the evaluability assessment and submitted before the final validation meeting with the country office
• A draft evaluability assessment report (maximum 60-75 pages including the executive summary and excluding annexes) to be discussed and validated by the country office team
• A final evaluability assessment report based on comments and feedback received on the draft report during the validation phase.

Monitoring and progress control, including reporting requirements, periodicity format and deadline:

The consultant is expected to report the progress to the BERANI coordinator, UNFPA M&E Programme Analyst, and PUNOS focal point. 

 

Supervisory arrangements: 

The Evaluation Manager will be the UNFPA M&E programme analyst in close coordination with UNFPA Assistant Representative and  Reference Group, under the overall guidance and responsibility of the UNFPA Indonesia Representative. The evaluation manager will be responsible for the day-to-day oversight and management of the evaluation including management of the evaluation budget, ensuring the quality and independence of the Evaluation and its alignment with UNEG Norms and Standards and Ethical Guidelines.

An evaluation reference group (RG) will be created to provide guidance/technical inputs to the evaluation and quality assure all evaluation deliverables (from a technical point of view) which include the inception report, draft report, and final report. The Reference Group composed of PUNOS and selected government stakeholders will be the main decision-making body for the evaluability assessment and have overall responsibility for the management of the evaluation process including hiring and managing the team of external consultants. The RG is responsible for ensuring the quality and independence of the assessment and guaranteeing its alignment with UNEG Norms Standards and Ethical Guidelines. 

Key roles and responsibilities of the RG include: 

• To review, provide substantive comments and approve the inception report, including the work plan, analytical framework, methodology, and selection of countries for in-depth case studies;
• To review and provide substantive feedback on interim deliverables and draft/final evaluability assessment reports;
• To quality assure the entire evaluability assessment process;
• To identify and ensure the participation of relevant stakeholders in coordination with the RG throughout the evaluability assessment process;

To contribute to learning, knowledge sharing, the dissemination of the evaluability assessment findings, and follow-up on the joint management response.

 

Expected travel:

The consultant may need to carry out a number of field missions as necessary, as per the agreed work plan in the design/inception report. The travel is for attending and facilitating consultative meetings and workshops for in-depth interviews and reviews.

 

Required expertise, qualifications and competencies, including language requirements:

The team leader is an International/national consultant who must have extensive experience in leading evaluability assessments and/or evaluations of a similar size, complexity, and character, as well as technical expertise in areas related to evaluation and results-based management. A general understanding of BERANI II areas of programming i.e. SRH, youth, GBV and harmful practices will be a key advantage. 

His/her primary responsibilities will be: 

• Guide and manage the team throughout the evaluability assessment phases;
• Develop and set out the methodological approach;
• Reviewing and consolidating the team members’ inputs to the evaluability assessment  deliverables;
• Liaise with UNFPA, ERG, and represent the evaluation team in meetings with stakeholders;
• Delivering the inception report, and EA report in line with the requested quality standards;
• Ensure adherence to UNEG guidelines and ethical standards throughout the assessment process.

Minimum qualifications required:

• At least 10 years of experience conducting or managing evaluations, with a preference for those with experience in evaluability assessments or programmatic evaluations in areas such as maternal health, family planning, adolescents and youth, gender-based violence, and population and development.
• Experience working with the United Nations, particularly on sexual reproductive health, GBV, harmful practices, and children and women-related programmes.
• Master’s degree or equivalent in Development Studies, Public health, Sociology, Economics, Social Studies, International Relations, or other related fields.
• Awareness of ethical risks in programming around sensitive issues, both in programme delivery and in all aspects of M&E.
• Proven skills in evaluation methodology, and research analysis, including quantitative and qualitative data collection and analysis techniques.
• Demonstrate expertise/experience in developing results frameworks, tools or guide for monitoring and evaluation.
• Familiarity with results-based management orientation and practices.
• Excellent report writing skills, analytical skills, and computer skills.
• Excellent command of written and spoken English
• Experience leading and managing teams.

Language

Excellent command of written and spoken English

 

Inputs / services to be provided by UNFPA or implementing partner (e.g support services, office space, equipment), if applicable:

The consultant will be assisted by the UNFPA M&E Analyst and BERANI coordinator to facilitate communication and coordination with PUNOS and the programme partners for evaluation-related activities, such as data collection or dissemination. 

The draft concept note of the evaluability assessment and other supporting documents for further reference can be found here

 

How to Apply

To apply to this position, click “Apply Now” below. Upload following documents in the “attachment” section of the application: 

1. Cover letter or Letter of motivation
2. Curriculum Vitae / Resume Highlighting Relevant Experience

Due to the volume of applications received, only shortlisted candidates will be contacted for an interview

 

UNFPA Work Environment:

UNFPA provides a work environment that reflects the values of gender equality, diversity, integrity and healthy work-life balance. We are committed to ensuring gender parity in the organization and therefore encourage women to apply. Individuals from the LGBTQIA+ community, minority ethnic groups, indigenous populations, persons with disabilities, and other underrepresented groups are highly encouraged to apply. UNFPA promotes equal opportunities in terms of appointment, training, compensation and selection for all regardless of personal characteristics and dimensions of diversity. Diversity, Equity and Inclusion is at the heart of UNFPA's workforce - click here to learn more.

 

Disclaimer:

Selection and appointment may be subject to background and reference checks, medical clearance, visa issuance and other administrative requirements. 

UNFPA does not charge any application, processing, training, interviewing, testing or other fee in connection with the application or recruitment process and does not concern itself with information on applicants' bank accounts. 

Applicants for positions in the international Professional and higher categories, who hold permanent resident status in a country other than their country of nationality, may be required to renounce such status upon their appointment.