About Plan International
Plan International is an independent development and humanitarian non-profit organisation that advances children’s rights and equality for girls. We support children’s rights from birth until they reach adulthood, and enable children to prepare for – and respond to – crises and adversity. We believe in the power and potential of every child, but know this is often suppressed by poverty, violence, exclusion and discrimination.
Working together with children, young people, supporters and partners, we strive for a just world, tackling the root causes of the challenges girls and vulnerable children face. We have been building powerful partnerships for children for over 85 years and are now active in more than 70 countries, driving changes in practice and policy at local, national and global levels using our reach, experience and knowledge.
Project Background
The World Bank (WB) and Global Financing Facility (GFF) supported Accelerating Nutrition Results in Nigeria (ANRiN) Project aims to increase utilization of quality, cost-effective nutrition services for pregnant and lactating women, adolescent girls, and children under five years of age in 12 states of Nigeria. Plan International was contracted to implement the ANRiN Project in 12 Local Government Areas (LGAs) of Niger State which seeks to achieve the following outcomes:
Outcome 1: Utilisation, quality and coverage of BPNS – Increased utilisation of BPNS for children and pregnant and lactating females in targeted LGAs of Niger State that is capable of meeting the established and agreed targets for quality and coverage.
Outcome 2: Behaviour change – Mothers/caregivers of children 0-59 months of age in targeted LGAs of Niger State practice optimal behaviours related to maternal, infant, and young child feeding.
Outcome 3: Accountability and safety – Mothers/caregivers and pregnant females in targeted LGAs of Niger State are supported to access an accountable BPNS and feel safe to do so.
Outcome 4: Capacity building – Local actors in targeted areas of Niger State have a strengthened capacity to manage and supervise the delivery of BPNS.
Specifically, through a mix of counselling and service delivery, in an equitable and sustainable manner, Plan International is working to increase access to and utilization of the BPNS that consists of the following 7 services:
- knowledge of mothers/caregivers of children 0-23 months of age on improved behaviours related to maternal, infant, and young child feeding, notably early, exclusive, and continued breastfeeding, appropriate complementary feeding, and early stimulation,
- thirty sachets of micronutrient powders per month among children 6-23 months to improve the quality of complementary feeding,
- at least 90 iron-folic acid (IFA) tablets by pregnant women by specifically addressing barriers that inhibit women from taking a full course of IFA tablets during pregnancy through counselling during ante-natal care sessions,
- at least three doses of intermittent preventive treatment for malaria during ante-natal care by pregnant women,
- at least four sachets of zinc/oral rehydration solution (ORS) per year for treatment of diarrhoea among children 6-59 months of age,
- semi-annual vitamin A supplementation among children 6-59 months of age, and,
- semi-annual deworming among children 12-59 months of age.
- The project is fully results-based and leverages a performance-based contract (commercial contract) with Plan International for community-based delivery of nutrition services. The project is assessed against annualised quantitative targets for the doses (contacts) of BPNS services delivered (Annex 3). This performance is independently verified every half year by an verification agency contracted and managed by the World Bank. Plan International employs its internal implementation approaches including project management, monitoring, and accountability processes to achieve the specified targets. The service delivery data for this project is captured through a customized mobile application developed by the WB.
Purpose of the Evaluation
Plan International is seeking to procure the services of an evaluation team to conduct a mixed-methods, gender-sensitive endline evaluation for the ANRiN project which is in its final year of implementation. The purpose of the evaluation is to assess the success of project’s objectives and explore reasons why change occurred for the project participants and other stakeholders.
The evaluation will be forward looking and will identify lessons learned and good practices, and provide recommendations both for accountability towards the funder, the World Bank, and for informing future intervention models and delivery approaches of Plan International and its partners. The main focus of the evaluation is to demonstrate evidence of outcome level change and the associated barriers and enablers for various stakeholders. The consultant will be responsible for collecting and analysing data/information to address specific evaluation questions and sub-questions.
Evaluation Questions:
The final evaluation will largely focus on the Development Assistant Committee (DAC) evaluation criteria of effectiveness, efficiency, and sustainability and will specifically address the following evaluation questions. These questions help define the scope and focus of the evaluation process; however, the successful bidder will be expected to work with the Project Management Team to review and revise these questions as appropriate at the outset of the evaluation.
- Impact and Quality
- To what extent has the project been effective in achieving its intended outcomes/what are the key outcomes/most significant changes for the different stakeholder groups of the project?
- What were the major drivers and barriers influencing the achievement or non-achievement of intended outcomes? Has there been any differential results across stakeholder groups and if so, why?
- What worked and didn’t work about Plan International’s approaches to achieving the intended outcomes (training, supervision, logistics, security, monitoring etc.)
- To what extent has the project been effective in improving the knowledge and practices of pregnant and lactating women on appropriate maternal and infant feeding? To what extent has the project been successful in shifting knowledge and practice of husbands, partners, fathers and community members regarding the health of pregnant and lactating women?
- What value has the project added to the lives of Community Health Volunteers (CHVs) and Mother to Mother facilitators (MTMs) with regards to their socio-economic status, technical skills, agency and self-confidence and beyond?
- Equity of Service delivery
- Was the selection of target communities and populations done equitably and inclusively (consider age, gender, ethnicity, location etc.)? Did the commercial angle of the project influence these decisions?
- What do we know about the acceptance and uptake of BPNS services/interventions? Has there been any differential results across project participant groups and if so, why?
- Were feedback mechanisms for service users utilized effectively and were complaints and feedback acted upon?
- Did the project contribute to transforming unequal gender-based power relations? How and to what extent? What were the barriers to change?
- Operational Efficiency and Effectiveness
- To what extent has the strategies, interventions, and implementation approaches of the project been effective in making nutrition services more accessible to the target population?
- What has been the contribution of the different entities of Plan International- Niger team, Abuja Country Office, and UK National Office- and its partners in delivering this project? What could be changed in future programmes?
- With consideration to the commercial component of the project, how efficient were the project management processes (risk management, resourcing, procurement, monitoring etc.) at different stages of the project life-cycle?
- How well did the project adapt to changing contexts over the course of the project (e.g. security situation, MIS – World Bank back End challenges) and utilise lessons learned from implementation of the interventions? What was the impact of these adaptations on different stakeholders?
- Sustainability
- To what extent were programmatic activities designed and implemented in a participatory and empowering manner with the key stakeholders?
- Did the programme enhance local ownership and capacity at the systemic level?
Scope of Work
The evaluation questions/sub questions will be refined and agreed upon by Plan and the Consultant as part of the inception phase of the endline evaluation. It is anticipated that the evaluation will have three elements or components:
1. Quantitative data collection and analysis
2. Qualitative data collection and analysis
3. Consolidation of lessons learned
The consultant is expected to combine the findings from the quantitative and qualitative data components into a final evaluation report. The report should also include relevant recommendations. The lesson learnt and good practices report could be an appendix to the endline evaluation report or a separate document – to be decided at inception. The report should also provide clear and concise recommendations and key lessons concerning the approaches used to achieve intended and unintended results on the project.
The study will be largely conducted among the participants reached by the project. Overall, the consultant will be responsible for detailing the methodology for carrying out the quantitative and qualitative data collection, refining/adopting the KAP survey tools for use in the endline evaluation, developing data collection tools and translating them if required, analysis plan, training and supervising data collectors, managing the data collection process, analysing, reporting and presenting data, detailing a plan for dissemination of study findings, and detailing aspects of quality assurance/ethical considerations (Safeguarding Children and Youth People Policy, informed consent).
Ethics and Safeguarding
Plan International is committed to ensuring that the rights of those participating in data collection or analysis are respected and protected. All applicants should include details in their proposal on how they will ensure ethical considerations are fully integrated into every stage of the evaluation process.
Before work can start, the Evaluation Team will be required to understand, comply with and sign Plan International’s Global Policy on Safeguarding Children and Young People (available on request). Within the proposal, the Evaluation Team will need to clearly outline their approach to managing and reporting suspected or actual cases of abuse.
In addition, the Evaluation Team will need to demonstrate the following:
- How they have considered the protection of children, vulnerable groups (including girls and people living with disabilities) and adults with specific vulnerabilities
- A consideration of how to ensure the Prevention of Sexual Exploitation, Abuse and Harassment (PSEAH) within their approach to the assignment
- How appropriate, safe, non-discriminatory participation of all stakeholders will be ensured through the different evaluation stages, including recruitment and training of research staff, data collection, data analysis and report writing
- How confidentiality and anonymity of participants will be guaranteed
- Experience of putting safeguarding approaches and procedures in place, as well as how they ensure compliance from their staff (including enumerators), at all times
In line with Plan International’s values and organisational ambition, all evaluations should seek to prioritise a focus on child rights, gender and inclusion and try to understand the extent to which the project or programme applied gender and inclusion sensitive approaches and explicitly aimed for results that improve the rights of children and young people and gender equality. As a minimum, this requires:
- the disaggregation of all data points from the study by age and sex,
- the development of an ethics protocol that safeguards children, youth and vulnerable adults
- robust sampling design ensuring representation of diversity groups
Other Ethics that the Consultant should adhere to include the following:
- All terms/conditions stipulated in the Terms of Reference (TOR).
- Show good conduct while undertaking assignment, this includes not making any commitments to communities and any other persons, on behalf of Plan International Nigeria.
- Timeframe and conditions outlined in the TOR
Users of the Evaluation
Plan International is the primary user of this end-line evaluation as well as the government, World Bank and Plan International’ program partners. It is expected that the evaluation report will contribute to provide an in-depth insight into the project achievement and serve as a guide for stakeholders and partners in the implementation of future projects.
Proposed Methodology
The consultant is expected to propose an appropriate evaluation design to assess the evaluation questions cited above which will be further refined in consultation with the ANRIN team. Evaluators should propose methodologies that can adequately demonstrate the project’s interventions’ contributions to outcomes for project participants and stakeholders, and that acknowledges the ground-realities of data collection.
The endline evaluation will largely adopt a non-experimental design and the use of exploratory qualitative methodologies like Outcome Harvesting and Most Significant Change are highly encouraged. There won’t be a need to compare baseline and endline findings because KAP survey conducted at the beginning of the project focus only on four indicators. The evaluation will involve the utilization of mixed data collection methods – primary and secondary quantitative and qualitative data collection and analysis. As outlined in the scope of work, the consultant will lead and manage all the components of the evaluation in consultation with Plan and relevant stakeholders. The study will include the use of methods that are sensitive to the inclusion of all target vulnerable groups and capable of generating disaggregated data as specified by Plan International standards.
Quantitative Data Collection and Analysis
To answer the evaluation questions adequately and appropriately, it is envisaged that both primary and secondary quantitative data will be required. Annex 7 outlines a standard list of indicators for the Plan International’s Area of Global Distinctiveness (AOGD) in Early Childhood Development (ECD) which may be considered for designing the quantitative data collection tools.
Desk review and analysis of project data: The project’s Database has data on the people directly reached by specific project interventions and some sociodemographic details. Also available are the KAP Survey report, annual operations and results reports and other relevant documents. It is anticipated that consultant would conduct a review and analysis of the data included/reported in these data sources as part of the evaluation process.
Qualitative Data Collection and Analysis
The Consultant will be expected to use appropriate qualitative approaches to validate and interrogate findings from analysis of the quantitative data, as well as evaluate lessons learned and curate best practices amongst project stakeholders. Qualitative data is needed to respond to some of the evaluation questions as well as to complement, and help ‘unpack’ the findings of the quantitative surveys and monitoring data. The Consultant should propose a methodology for sampling and sample size determination that selects key informants, interviewees and FGD participants such that theoretical saturation is reached. The approach may be either random or purposive (e.g., extreme case, convenience).
For both the quantitative and qualitative data collection, the consultant shall propose a sampling strategy that meets the needs of the assignment, including how they will identify respondents. Consultants are requested to include in their proposal the approach they will take for administering surveys – face to face interviews, tablet based, computer assisted, group facilitated, or alternatives will be considered. However, the consultant is expected to conduct face to face interview with research respondents across all locations.
Data Quality Assurance
The bidder is required to submit a brief quality assurance plan that sets out the systems and processes for quality assuring the evaluation and research process and deliverables of the project from start to finish. This plan should include the proposed approaches to:
- Robust evaluation matrix
- Training of enumerators and researchers conducting the research, including in research ethics
- Piloting of all research activities
- Logistical and management planning
- Field work protocols and data verification including back-checking and quality control by supervisors (if required)
- Data cleaning and editing before any analysis
- Analysis and validation of results
Risk Management
It is important that the successful bidder has taken all reasonable measures to mitigate any potential risks to research participants and the delivery of the required outputs for this evaluation. Therefore, the Evaluation Team should submit a brief risk management plan covering.
- The assumptions underpinning the successful completion of the proposals submitted, the anticipated challenges, estimates of the level of risk for each risk identified and proposed contingency plans
- Specific safeguarding risks (for both children and adults) and mitigating strategies, including reference to the child protection policy and procedures that will be in place
- Health and safety issues that may require significant duty of care precautions
- Reporting and Contracting Arrangements
The Evaluation Team will be expected to identify a Project Manager for communication and reporting purposes. At the inception meeting, they will be expected to submit a full contact list of all those involved in the evaluation.
The Evaluation Team will be working directly with the ANRIN MEL Manager with support as required from the ANRIN project team.
The ANRIN project will provide:
- The project proposal, logical framework, coverage data, and other relevant documents
- Overview of the project, list of communities, CHVs per district
- Contact list of key people in-country and in the field
- Support in setting up introductory meetings with relevant stakeholders and communities
- Logistical support during the fieldwork phase.
The Evaluation Team will be responsible for the timely delivery of all outputs and deliverables as outlined above, as well as for the prompt reporting and presentation of raw data, draft and final reports (inclusive of responding to feedback from Plan International and the donor on the draft reports). More specifically, the Evaluation Team will:
- Submit a full list of contacts for all those involved in the evaluation during the inception meeting
- Report to the Evaluation Steering Committee and attend meetings as agreed with the ANRIN team
- Contact the Plan International focal points on a daily basis during the fieldwork stage of the evaluation
- Submit (by email) to the Plan International ANRIN focal point on fortnightly progress reports against the workplan during the evaluation period, summarizing activities / tasks completed to date (% achieved), time spent, etc.
- Recruit and train research assistants such as enumerators for the assessment
- Make own logistical arrangements to reach the selected communities and organise interviews and ensure full logistical support for the entire exercise across all wards
- Supervise and take full responsibility for the behaviour and performance of data collectors, including data collection checking in the field
- Perform child protection and safeguarding background checks on all their staff involved in evaluation activities, including contractors
- Ensure data collection reports are completed by field researchers and that these are compiled into an overall data collection report (the essence of which should be in the evaluation reports along with any mitigation, etc. where issues arise);
- Design or modify tools in consultation with the ANRIN Project MEL staff
- Run analysis of the findings and produce reports which sufficiently explore and explain the results
- Produce any other relevant dissemination materials and share findings with the Project Team
Expected Qualification
The consultant will possess the following skills and experiences:
- Advanced Degree in a relevant discipline (Statistics, Evaluation Science, Social Sciences, Public Health or other related Degree),
- Over five years of work experience related to nutrition and Maternal, Infant, Young Child Feeding (MIYCF) programmes (experience in Niger State is desirable)
- Strong analytical and research skills, with skills in quantitative and qualitative data collection and analysis, drawing findings from multiple sources and handling potential contradictions between data sets, including ensuring a greater understanding of quantitative data results through the triangulation of qualitative data
- Expert skills in qualitative data collection and analysis using methodologies like Outcome Harvesting and Most Significant Change are highly desired.
- Experience in undertaking External Evaluation in the field of nutrition, health, community distribution, behaviour change or other related disciplines in LMICs
- Proven leadership and coordination skills with the ability to quality assure an enumerator team
- Fluency in English, Hausa, Nupe and other local languages required.
Disclosure and Ownership of Information
It is understood and agreed that the Consultant/Consulting firm shall, during and after the effective period of the contract, treat as confidential and not divulge unless authorized in writing by Plan International Nigeria, any information obtained in the course of the performance of the Contract. Information will be made available for the consultants on a need-to-know basis.
All ownership and copyright for final data collected during the endline evaluation are held by Plan International Nigeria. Obligations regarding non-disclosure of information and intellectual property rights will be specified in the contract for this consulting mandate.
Further information on this Terms of Reference can be found via this link : https://tinyurl.com/3skuku9y
How to apply
Submissions should be sent to [email protected] on or before 31st July, 2023 5:00pm referencing “End-line Evaluation ANRiN in the subject line.
A proposal detailing the technical and financial aspects for the Endline evaluation (with consultant’s understanding of the assignment, a clear reporting format, consultant(s)’ up to date CVs, detailed study cost) should be submitted.
All proposals must include the following details:
- A proposed evaluation matrix
- Proposed methodology and justification
- How the consultant will ensure quality at all steps of the process
- How analysis of each type of data will be completed
- Implementation plan
- Specifically, considerations for child and young people safeguarding and gender throughout the study, demonstrating what approaches will be used to ensure that child protection, safeguarding and gender considerations will be applied throughout the design and the data collection phases of the study
- Demonstration of how the consultant (and their team) meet the qualification and experience required, as detailed above
- Demonstration of how the consultant would addressing the project indicators.
- At least two samples of previous, similar work done (soft copy of reports, certificate of completion or engagement letter)
The technical proposal will be evaluated based on the Quality Cost-Based Selection (QCBS) criteria:
- The individual general reliability (in terms of previous work done etc.) as well as experience and capacity in carrying out the assignment (45%).
- The approach in responding to the TOR and detailed work plan (30%).
- The qualifications and competencies of the proposed personnel for the assignment (15%).
- The proposal budget (10%)
Proposals obtaining more than 70% of the technical points will be considered technically suitable and qualify for a financial assessment.
