Afghanistan: Terms of Reference for the Ready for Anything Project End of Project Evaluation
Terms of Reference for the Ready for Anything Project End of Project Evaluation
1) Summary of the Evaluation Terms of Reference
This is a terms of reference for an end of project evaluation being undertaken by the Ready for Anything (RFA) project in Afghanistan, Malawi, Myanmar and Nepal which started in Aug 2013 and will end in July 2016. These terms of reference outline the key purpose and objectives, expected methodology and deliverable of the evaluation. The Evaluation is expected to highlight the project results, impact and the processes which led to these, key lessons learnt, and the sustainability of the project benefits.
2) Background: Ready for Anything End of Project Evaluation
ActionAid is a global movement of people working together to further human rights for all and defeat poverty. We work in 45 countries with over 15 million people across Asia, Africa and the Americas. A Human Rights Based Approach (HRBA) underpins all of our work, in both development and humanitarian contexts.
The concept of resilience is gaining increasing importance for policy and programs in development work around the world. Communities and practitioners are facing growing challenges due to climate change, conflict and disasters. Cognizant of this, ActionAid has been working with communities and donors to enhance community resilience. RFA is one of the resilience projects being implemented by ActionAid and its partners.
RFA is a three years project (Aug, 13 to July 16) funded by DFID with a total budget of £1,361,149. The project has been implemented in four countries (Afghanistan, Malawi, Myanmar and Nepal) with elements of the project implemented by ActionAid’s International Secretariat (IS). Table 1 and 2 below indicate the number of people/communities the project aimed to reach per country and project result chains with its indicators.
Table 1: Number of communities and people to reach
Name of country
No of Local Rights Program
No of communities
People to reach
No of households
Afghanistan
2
50
37,452
5,944
Malawi
4
28
40,800
9,714
Myanmar
1
30
18,072
3,845
Nepal
3
57
18,612
4,230
Sum
10
165
114,936
23,733
Table 2: Project results and target
Result chains and indicators
Target
Impact: Communities are resilient and able to protect their lives and livelihoods in the face of disasters and climate change
Impact indicators
· Number of national government’s policies, practices and/or budgets that better reflect the resilience priorities of poor and vulnerable communities
4 (one per country)
· % of HHs with improved income and/or food security
60% out of 23,733 HH ( 14,239 hh)
Outcome: 114,936 beneficiaries in 165 communities have the capacity and appropriate resources to reduce risks and recover from the impacts of disasters and climate change
Outcome indicators
· Number of communities with new DRR mechanisms in place [as a result of the project]
150
· % of HHs that report they are better prepared in case of a disaster
75% out of 23,733 (17,801)
· % of women who are engaged in decision-making on community risk management
60% (1,486 women)
· Number of examples of government response to communities’ recommendations on DRR and Climate Change
11
Output one: Women in poor and vulnerable communities are able to identify risks and implement risk reduction plans
Output one indicators:
· Number of women in disaster affected communities more aware of the impacts of disasters and climate change and ways to reduce risks and vulnerabilities
2,540
· Number of women participating to develop and implement community based risk reduction plans
2,140
· Number of communities in which risk reduction plans are developed and implemented
165
Output two: Communities adopt food security and sustainable livelihoods strategies that enhance their resilience
Output two indicators:
· Number of communities (represented by community DRR committee) which receive training on the impact of disasters and climate change on food security and sustainable livelihoods
165
· Number of communities which implement food security and sustainable livelihoods strategies
165
· Number of replicable good practice or alternatives in relation to food security and sustainable livelihoods documented and shared
20 (5 per country)
Output three: Communities, especially women, participate actively to contribute to government behavior and policy influencing processes
Output three indicators:
· Number of women ‘s in partnership with civil society organizations, review and analyze local, provincial and national government budgets, policies and practices related to resilience (e.g. DRR, CCA, Livelihoods)
790
· Number of communities linked to networks, forums and movements on disaster risk reduction (DRR) and climate change adaptation (CCA) to exchange knowledge for policy influence
165
· Number of engagements where recommendations representing the views of CSOs and communities are developed or presented
22
As part of the project M & E framework and learning, ActionAid International in collaboration with AAUK, and project implementing countries will undertake a final evaluation of the project. The TOR as defined in this document defines how this evaluation will be undertaken.
3) Objectives and Purpose of the Evaluation
As resilience is an emerging issue within and outside of the organization, the main purpose of the evaluation is to generate learning about effective strategies for building the resilience of vulnerable communities, and particularly women, to reduce their risks and improve their recovery from the impacts of disasters and climate change
Objectives for the final evaluation are:
To assess to what extent the project has achieved its intended outcome/impact;
To review to what extent the project has been implemented in line with ActionAid’s HRBA principles (particularly with regards to women’s rights and accountability to communities); and
Identify and document lessons learned and include recommendations for improvements for ActionAid’s ongoing work on resilience and women’s leadership.
Assessment of the project’s outcome will be in line with OECD-DAC Criteria, and include sub-questions on:
Relevance – have we done the right thing?
Effectiveness – have we done the right thing well?
Efficiency (and value for money) – have we got the most (and best) results for our inputs?
Impact – what changes (positive or negative, intended or unintended) to beneficiaries’ lives has the project contributed to?
Sustainability – will what we have done remain after the project ends?
4) Key evaluation questions
In fulfilling the aims and objectives of this ToR, the evaluation is also expected to address and answer the following questions:
What have been the most effective strategies for raising awareness of disaster risks and climate changes in communities, and why have they worked (or not) in different contexts?
How has the ActionAid program worked to support women’s leadership in communities and has this translated into any improvements in women’s lives or community resilience?
How has ActionAid’s conceptual understanding of resilience been used in different contexts, and to what end?
What has enabled, or been a barrier, to communities’ engagement in different policy processes at different levels?
To what extent have the knowledge, attitudes and practice of different stakeholders changed (as evidenced by project KAP surveys) and what has brought about any changes?
To what extent has the project been implemented in line with ActionAid’s Climate Resilient Sustainable Agriculture (CRSA) approach, and what evidence is there as to the effectiveness of this approach?
5) Use and users of the evaluation
The top three priority primary intended users of this evaluation are ActionAid program and policy staff and senior managers. It is expected that the evaluation report will be used by program staff to deepen their understanding of effective strategies for improving community resilience; by ActionAid policy staff to support evidence-based policy and advocacy activities; and by senior managers within ActionAid International to make informed decisions about strategic priorities. In addition, the evaluation report will be shared with DFID to evidence how DFID funding has been spent. The report will also be shared with ActionAid partners and with other external stakeholders, government and other NGOs etc., as appropriate.
6) Scope and Proposed Approach of the Evaluation
The evaluation will cover all implementing countries of Ready for Anything, but to varying extents. In Afghanistan, security challenges may limit the potential of the evaluation. To carry out the evaluation, the following approach is proposed. This approach is intended to ensure consistency (both in methodology and quality) across all countries, while reducing costs associated with international travel somewhat. However, ActionAid welcomes alternative suggestions that respond to the objectives and evaluation questions of this ToR and can be achieved within the available budget.
A lead consultant will be recruited to design the overall approach and methodology for the evaluation. In two countries they will directly carry out the evaluation (Malawi and either Nepal or Myanmar depending on the prevailing conditions). In the other two countries (including Afghanistan) they will remotely oversee the evaluation by an in-country consultant, meaning they will provide the local consultant(s) with the tools and guidance for data collection, and use the local consultants’ findings within their global report.
The overall approach to the evaluation will be a mixture of qualitative and quantitative from primary and secondary sources in all countries. Quantitative data will be collected through a household survey, repeating the process used at baseline to provide data to measure changes over time for a small number of key outcome level indicators. Quantitative data will be complimented by and expanded upon with in-depth qualitative research, most likely comprising of key informant interviews and focus group discussions in selected communities.
The consultant is expected to use a variety of methods to collect and analyze data. We welcome suggestions for methodological approaches that address the objectives and evaluation questions set out in this ToR.
The inception phase will include introductory briefings with ActionAid International and the four countries, and a literature review of existing project and contextual documents. During the inception phase, the lead consultant will also be expected to develop a detailed methodology, data collection tools and work plan, including but not limited to:
Reviewing existing data collection tools to identify key indicators (within the project log frame or not) for which comparable data is available (from baseline, mid-term and monitoring tools) and that can be used to demonstrate changes over time;
Finalize and agree overall methodological design and approach for the evaluation, including details of sampling strategy, size and analytical framework;
Finalize management and coordination arrangements for the evaluation in all countries/internationally, including sub-contracting/management and training of in-country consultants as necessary;
Adapt/develop specific data collection tools or processes (i.e. focus group discussion guides. KII etc.);
Develop clear and simple ethical protocol and guidelines for the evaluation.
7) Evaluation Standards
ActionAid expects all evaluations to be carried out in line with AA Evaluation Standards. These standards will be shared on request, but in practical terms include the need for the evaluation and the evaluator to:
Feeding back: the evaluators should as a minimum commit to feeding key findings back to the communities involved in the evaluation before leaving ‘the field’. Options for validation of results and feeding back final evaluation findings must be considered.
Transparency and ethical standards: the evaluators should explain clearly to communities involved in the evaluation what the purpose of the exercise is, and how people’s information will be used. The evaluation should follow ActionAid’s ethical standards for research and data collection, and should include a risk assessment covering security risks to communities. As a minimum the evaluation should ‘do no harm’.
Community voices: it is essential that the evaluation reflects the voices of women and men involved in the project. People should be consulted as part of the evaluation, and their voices should be included in the evaluation report as direct quotes and case studies. In line with ActionAid’s mandate the evaluation should prioritize people living in poverty and exclusion, especially women.
Women’s rights: Women’s rights must be respected in all evaluations. All evaluations should seek to explore how women have been affected by an intervention and the effect of the intervention on gender relations. It is essential that women’s voices are heard clearly in the evaluation. The timing and location of evaluation activities and the composition of the evaluation team should be designed to maximize women’s ability to participate in the evaluation.
Transparency about methodology: The evaluation should include a detailed and transparent discussion of the methodology used and key decisions taken in designing and implementing the evaluation. This should include information about the sampling (approach to sampling, numbers of people/communities covered, to what extent it was representative), what tools were used and why, methodological limitations and gaps in the evaluation.
Disaggregated data: All data, qualitative and quantitative, collected through the review must be disaggregated by sex and age as a minimum.
8) Coordination, Roles and Responsibilities for Implementing the Evaluation
ActionAid International is looking for an evaluation consultant to lead the evaluation of the multi-county project.
This lead evaluator will develop the methodology and tools for the project evaluation in all countries, in consultation with countries. The lead evaluator will be directly involved in the management and execution of the evaluation in two project countries.
Exact management details will depend on the geographical location of the lead consultant, whether or not they have research partners or contacts in relevant countries they propose using or if this will need to be managed separately with the support of ActionAid. Terms of reference of the national consultant shall be discussed and agreed upon by the lead consultant and ActionAid country programs. ActionAid country programs would be responsible for hiring the national consultant, payment of the national consultant, facilitate visa for international consultant & internal logistics, and the national consultant would conduct the evaluation using the methodology and tools discussed and agreed with the lead consultant. The national consultant will be expected to respond to any queries, clarifications and details required by the lead consultant before and during the evaluation.
The lead evaluator will be responsible for data quality from all countries, and for developing methodology, communicating with and briefing the national consultants, consolidating all data collected, analysing it, and producing a final evaluation report.
The lead consultant will report to AA International Secretariat (AAIS) and will be supported by the international project manager from the ActionAid International Secretariat. AAI IS will lead coordination of the evaluation across the four countries, and will provide all necessary project information; introductions to relevant stakeholders at the international level; and any general logistical support and advice.
9) Timetable*
The Ready for Anything grant finishes in July 2016 and the evaluation also needs to be completed by that time. Working backwards from that date, beginning to April to end of June would be for in-country data collection, analysis and report writing. March would be required for the preparation phase, recruiting national consultant(s), developing tools etc.
* Dates are subject to change pending a possible six month no cost extension.
10) Budget
The total budget available for this evaluation is £22,000, to cover all activities needed to fulfill these ToR. This includes lead consultant costs and all in-country costs (including national level consultants if necessary) in four countries – please refer to section 6 above.
11) Outputs / Deliverables of the Evaluation
In line with the purpose and methodology of the evaluation, the international consultant will deliver
Inception report with detailed methodology, tools and work plan
Field debriefings (as a note or a presentation remotely)
Draft evaluation report
Final evaluation report that incorporates evidence, learning, case studies, feedback received from ActionAid and meets agreed quality standards. The report should be precise, must answer each evaluation objective and question and should at least contain the following (this can be discussed within the inception phase):
Cover page (title of the evaluation report, date, name of consultants)
Contents table
Executive summary of no more than 2 pages outlining the key purpose of the evaluation, main points of analysis, key findings, conclusions and recommendations
Introduction outlining the background to the project and the evaluation
Purpose and objectives of the evaluation
Analytical framework of the evaluation
Methodology/approach, indicators used, ethical issues and limitations of the evaluation
Summary results against project log-frame indicators
Major findings (data analysis and response to evaluation questions)
Lessons learned and recommendations
Annexes: details of data collection tools, schedule of field visits and meetings; list of people interviewed; bibliography of key documents consulted; case studies; TOR for the evaluation
Draft summary report (summarising key findings and recommendations, up to 4 pages) that can be used to disseminate findings, written in English but in clear and plain language and style suitable for project communities
The raw data (all transcripts, quantitative data) must be handed over to ActionAid together with the evaluation report.
Management Response: The report will be followed by a management response from ActionAid, outlining areas that we agree with and will take forward in future responses; responding to areas highlighted as requiring improvement; outlining any findings that we disagree with which have not been resolved through providing comments on the draft report, and indicating how learning will be taken on board in this and future responses.
12) Evaluation Team Qualifications and how to Apply
Selection Criteria
ActionAid is seeking proposals with the following skills and experiences:
Expertise on human rights based approach and women’s rights and of evaluating similar projects
Understanding of rural livelihoods, disaster risk reduction, climate change adaptation, and resilience
Previous experience working with communities using participatory approaches
Substantial experience conducting qualitative research and data analysis
Ability to conduct statistical analysis of monitoring and survey data
Understanding of and commitment to ethical issues in research/evaluations
Experience in managing and coordinating evaluation/research exercises, including with or through country-based partners, delivering agreed outputs on time and on budget
Ability to write high quality, clear, concise reports in English
A financial proposal that offers good value for money and that maximizes potential efficiencies to deliver the outputs within budget
Selected consultant(s) will be expected to sign and abide by ActionAid values and key policies (including Anti-Sexual Harassment Policy, Child Protection Policy etc.).
How to apply
We invite interested individuals to submit the following application documents:
- Copy of CV of the consultant(s) who will undertake the research and evaluation (maximum 3 sides of A4 each);
Proposal (maximum 8 sides of A4) detailing a) how the Consultant(s) meets the selection criteria and b) their understanding of the TOR and methodology. Please ensure your proposal explicitly answers the following questions:
What evaluation approach and methods do you suggest to answer the evaluation questions given in the ToR?
How will your proposed approach involve women participating in the project in the project countries?
What do you foresee to be the main ethical issues and what will be your approach to addressing them?
What risks do you foresee in relation to this evaluation consultancy and how will you mitigate them?
A proposed activities schedule/work plan with time frame;
- Financial proposal detailing consultant(s) itemized fees, data collection and administrative costs
- One recent example of similar evaluation report written by the applicant (if joint authored to include a description of the role of the named consultant in the report);
- Contact details of two independent referees
Please send your applications to: yilma.muluken@actionaid.org
The deadline for applying is 10th February 2016. We are aiming to select the consultant by the end of February and start the contract in early March.
13) Terms of payment
The payment will be done according to the following time frame/arrangement:
Instalments
Amount
Time line
1st instalment
30% of the total amount
After Inception meeting and sharing inception report from the evaluator with detailed methodology, tools and work plan
2nd instalment
50% of the total amount
After presenting field data summery and sending draft report
3rd instalment
20% of the total amount
After satisfactory completion of final report according to agreed quality criteria & summary external report
0 comments:
Post a Comment