Afghanistan: Female Consultant (End of Project Evaluation ‘Women’s Economic Empowerment Project’)

1. Background information
   1.1. Donor and Fund Manager
   The Department for International Development (DFID) provides significant funding to civil society organisations (CSOs) annually in line with its overall strategy to alleviate poverty and promote peace, stability and good governance. The Programme Partnership Arrangements (PPA) and Global Poverty Action Fund (GPAF) are two of DFID’s principal funding mechanisms and have provided £480 million to approximately 230 CSOs between 2011 and 2013. The current political climate and results -based agenda demand a rigorous assessment of the effectiveness of funds disbursed to ensure that they are managed to provide value for money.
   Triple Line Consulting is the Fund Manager for the GPAF and is responsible for assessing performance of Afghanaid as the commissioning organization at the project level. Coffey International Development is the Evaluation Manager and is responsible for assessing the performance of the funding mechanisms as a whole. For more information on the fund level valuation, please see the 2012 Evaluation Strategy.
   1.2. Implementing organisation
   Afghanaid is a British international NGO that has been working in Afghanistan since 1983. Headquartered in Kabul and with 10 other offices across three provinces, Afghanaid’s work helps community-based recovery and development in some of the most remote areas of the country. A small marketing and fundraising office is based in London. More than 270 national and 7 international personnel are employed in Afghanistan.
   Afghanaid works alongside poor Afghans to enhance their opportunities and capabilities to achieve sustainable and equitable economic and social development. Afghanaid does this through a range of programme interventions that support grass-roots development of basic services, such as water and sanitation, access roads, education facilities and other community infrastructure; improved livelihoods, through natural resource management, agricultural and small enterprise development, and improved access to markets; as well as humanitarian assistance. Gender mainstreaming, local governance and community development are integrated into all of Afghanaid’s work. The results achieved have earned Afghanaid the respect and recognition of stakeholders as an effective and collaborative charity.
   1.3. Project description
   The official name of the project is “Improving and diversifying access to economic development opportunities for 13,947 poor rural women in 8 districts of Afghanistan” although among the communities and different stakeholders it is more known as “Women’s Economic Empowerment Project” (WEE project). WEE project is a three year project funded mainly by DFID and three other British Trusts and Foundations (The Finnis Scott Foundation, The JA Clark Charitable Trust; Aspect Capital) with Afghanaid contribution. The project has been implemented in 8 districts of Badakhshan province, namely Argo, Baharak, Darayem, Faizabad, Jurm, Keshim, Shuhada and Yaftal.
   The project’s goal is to contribute towards reduced household poverty and hunger and improved social status of women in Badakhshan province. It is expected that women in the target areas of Badakhshan province will benefit from improved economic status, resilience and empowerment. The project has started in May 2013 and will be ended in April 2016.
   The project is expected to achieve the following results:


2. Targeted beneficiaries (all women) have strengthened agricultural production inputs and skills, productivity processes and diversification of agro based livelihood


3. Target beneficiaries (all women) have improved knowledge and skills regarding livestock management and related livelihood activities


4. Non agro enterprise development and access to credit, microfinance institutions and financial services for its establishment


5. Project team members have improved capacity in M&E, PCM, etc. and beneficiaries improved in group dynamics.


6. 
   

   Policy dialogues and advocacy efforts contribute to more open discussion on women’s empowerment at the district, provincial and national levels

   
   


7. 
   

   The Consultancy Assignment
   2.1. Purpose of the independent final evaluation
   As per agreement with DFID and in line with Afghanaid’s own policy to promote accountability for performance and learning for future programming and wider sharing, an external end of project evaluation has been planned in order to assess the project performance and progress against the specific objectives, results and outcomes, to document lessons learnt and best practices and to inform future programming. This will enable DFID and the commissioning organization (Afghanaid) to know whether the good practice in development programming was practiced and guided the project cycle management. For this purpose Afghanaid intends to commission an individual consultant specialized in women’s economic and social empowerment through diversification of their livelihood opportunities to conduct this evaluation.
   The independent final evaluation reports that are submitted by grantees will be used to inform the Fund Manager’s understanding of the grantee’s performance at the project level and will also be used to inform the Evaluation Manager’s assessment of performance at the GPAF fund level. The independent final evaluation report needs to be a substantial document that (a) answers all the elements of the Terms of Reference (ToR); (b) provides findings and conclusions that are based on robust and transparent evidence; and (c) where necessary supplements the grantee’s own data with independent research.

   
   

2.2. Key objectives of the evaluation
The evaluation has two explicit objectives that are explained below:

1. To independently verify (and supplement where necessary), commissioning organization’s record of achievement as reported through its Annual Reports and defined in the project logframe;


2. To assess the extent to which the project was good value for money, which includes considering:
   a) How well the project met its objectives;
   b) How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
   c) What has happened because of DFID funding that wouldn’t have otherwise happened; and
   d) How well the project aligns with DFID’s goals of supporting the delivery of the MDGs.
   2.3. Verification of commissioning organization’s reporting
   The first task of the final evaluation is to verify commissioning organization’s achievement. The record of achievement will be presented in past Annual Reports and progress against the project logframe. This exercise could include verifying information that was collected by the commissioning organization for reporting purposes and possibly supplementing this data will additional information collected through primary and secondary research. Verifying the results from the project logframe will begin to capture what the project has achieved. However, there will be other activities and results that occur outside of the logframe that may require examination in order to respond to the different evaluation questions. Verifying reporting will also necessarily include a review of the data and systems that were used to populate results.
   2.4. Assessment of value for money
   The final evaluation should assess the extent to which the delivery and results of the project are good value for money. Value for money can be defined in different ways, but at minimum the evaluation report should include an assessment against:
   a) How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
   b) What has happened because of DFID funding that wouldn’t have otherwise happened.
   2.5. Evaluation questions
   To ensure comparability across the final evaluation reports, the evaluator(s) should respond to the questions below. The attention given to each evaluation question may vary depending on the objectives of the project and the availability of data, so the independent evaluator(s) should use his/her discretion in the level of effort used to respond to these questions. All evaluators are encouraged to structure their research questions according to the Organisation for Economic Co-operation and Development – Development Assistance Committee (OECD-DAC) criteria of relevance, effectiveness, efficiency, sustainability and impact.
   Relevance


3. To what extent did the commissioning organization support achievement towards the MDGs at the provincial level, specifically off-track MDGs?


4. To what extent did the project target and reach the poor and marginalised?


5. To what extent did the project mainstream gender equality in the design and delivery of activities (and or other relevant excluded groups)?


6. How well did the project respond to the needs of target beneficiaries, including how these needs evolved over time?
   Effectiveness


7. Did the project make sufficient progress towards its expected outcomes and to what extent (e.g. fully achieved, partially achieved, not achieved)?


8. To what extent are the results that are reported a fair and accurate record of achievement?


9. To what extent has the project delivered results that are value for money? To include but not limited to:
   a. How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
   b. What has happened because of DFID funding that wouldn’t have otherwise happened;


10. To what extent has the project used learning to improve delivery?


11. What are the key drivers and barriers affecting the delivery of results for the project?
    Efficiency


12. To what extent did the commissioning organization deliver results on time and on budget against agreed plans?


13. To what extent did the project understand cost drivers and manage these in relation to performance requirements?


14. What were the constraints that hindered the project implementation (e.g. political, religious and security) and were these managed effectively?


15. What mitigation measures were adopted to overcome the challenges faced by the project and what were the success rates of these measures?
    Sustainability


16. To what extent has the project leveraged additional resources (financial and in-kind) from other sources? What effect has this had on the scale, delivery or sustainability of activities?


17. To what extent is there evidence that the benefits delivered by the project will be sustained after the project ends?
    Impact


18. How many people are receiving support from the project that otherwise would not have received support?


19. To what extent and how has the project affected people in ways that were not originally intended?


20. Evaluation methods
    The consultant(s) commissioned to conduct the final evaluation and the commissioning organization are jointly responsible for choosing the methods that are the most appropriate for demonstrating impact. Evaluation methods should be rigorous yet at all times proportionate and appropriate to the context of the project intervention. Where possible, the evaluator(s) are encouraged to triangulate data sources so that findings are as robust as possible.
    3.1. Different approaches to assessing impact
    Although it is not strictly mandatory, the evaluator(s) are encouraged to apply a mixed-methods approach for assessing impact. This would combine qualitative data to provide an explanation of ‘why’ and ‘how’ the project has achieved the type and scale of results that are quantitatively observed.
    a. Assessing impact through experimental or quasi-experimental approaches
    To definitively attribute impact, then the establishment of a counter factual is required: e.g. What would have happened to beneficiaries in the absence of the intervention? Evaluators are encouraged to consider the extent to which approaches, such as measuring the difference between treatment and control groups, can be successful in capturing impact while also balancing concerns relating to proportionality. It is likely that experimental or quasi experimental approaches will not be appropriate for a large number of grantees.
    b. Assessing impact through contribution-based research
    Contribution-based approaches are helpful for overcoming the attribution issue of proving cause and effect. A contribution-based approach should result in a ‘plausible’ account of the difference that DFID’s funding has made to the impact of grantees. Such an approach is typically informed by a wide range of evidence sources that are brought together to produce a ‘plausible’ assessment of the ‘contribution’ of grantees to higher level outcomes and impacts. It is likely that such an approach will be appropriate for a large majority of grantees.
    c. Further guidance
    Additional guidance concerning the fundamental principles and techniques of impact and contribution research is presented in ‘Broadening the Range of Designs and Methods of Impact Evaluations’ available at:
    https://www.gov.uk/government/uploads/system/uploads/attachment_data/fil…
    And Annex 8.1 of the Evaluation Manager’s Evaluation Strategy for the GPAF and PPA which is also publically available: https://www.gov.uk/government/publications/evaluation-strategy-programme…
    3.2. Indicative materials to review
    Relevant to review may include review of:
    a) The commissioning organization’s original application for funding;
    b) MOU with DFID for funding;
    c) Updated versions of organisational project logframe;
    d) Monitoring data;
    e) Monitoring systems;
    f) Annual reports and comments provided by the Fund Manager;
    g) Organisational monitoring & evaluation strategy;
    h) Studies undertaken by the commissioning organization;
    i) Financial information / information on resources spent;
    j) Information on synergies / collaboration with DFID country programmes and other actors;
    k) Published material (e.g. to demonstrate sharing of learning with others);
    l) Additional relevant documents.
    3.3. Indicative methods for conducting primary and secondary research
    Relevant primary and secondary research may include:
    a) Interviews with staff at the commissioning organisation involved in the management and delivery of work;
    b) Interviews with various delivery partners;
    c) Focus group discussions with ultimate beneficiaries;
    d) Surveys with project partners and other relevant stakeholders where (possible and proportionate);
    e) Systematic reviews of secondary studies and sources, measuring impact where possible and proportionate through comparison groups and other quantitative methods;
    f) And verifying reported data through back checking and quality control assessments.


21. Contractual and reporting arrangements
    4.1. Profile of the Independent Evaluation provider
    The Independent Evaluator should be a suitably-qualified and experienced consultant or consulting firm. The consultant profile should include:
    Essential:
    a) An evaluation specialist with a minimum of seven years’ experience in programme/project evaluation in an international development context;
    b) Experience of results-based monitoring and evaluation;
    c) Ability to design and plan the evaluation approaches and research methodologies, including quantitative and qualitative research methods. Where feasible and proportionate, the person or team should include skills and expertise required to design, plan and conduct impact evaluation, potentially using experimental or quasi-experimental techniques;
    d) Relevant subject matter knowledge and experience with women’s economic and social empowerment through diversification of their livelihood opportunities to ensure the evaluation design and research methods are as relevant and meaningful as possible given the aims and objectives of the project and the context in which it is being delivered;
    e) Experience in developing and applying gender sensitive participatory research, evaluation and review methodologies in traditional Muslim cultural contexts
    f) Ability to manage a potentially large-scale and complex evaluation and research process, including interpreting baseline data and conducting a final evaluation;
    g) Ability to design, manage and implement primary research in challenging project environment. This may include the design of surveys, in-depth interviews, focus group and other research;
    h) Design and manage data and information systems capable of handling large datasets for monitoring and evaluation purposes;
    i) Excellent analysis and writing skills
    j) Ability to work independently with minimum supervision
    k) No conflict of interest with the ongoing activities of commissioning organization.
    Desirable:
    l) Appropriate country knowledge/experience or experience from other conflict affected countries;
    m) Local language(s) proficiency;
    n) Willingness to live and work in a very basic conditions during the assignment.

4.2 Management arrangements
The commissioning organization is responsible for the recruitment and briefing to the final evaluator(s), and will be the point of contact within the commissioning organisation for the duration of the evaluation process. The commissioning will also provide logistical and technical support to facilitate required meetings and interviews.
The consultant will report directly to the Monitoring and Evaluation Learning Manager and the Programme Director.
4.3 Deliverables and timeframe
The final evaluation consultant(s) will submit the final report to the commissioning organisation who, in turn, will submit it to the Fund Manager within three months of the project’s completion. In order to minimalize the risk of not being to access the required project staff and key stakeholders it is recommended that the reports should be completed before the project closes. The main body of the report (draft and final version) must be limited to 40 pages (this can include or exclude annexes). One of the annexes should consist of a table which summarises the findings according to the OECD-DAC criteria.
The consultant will be required to complete the work over a proposed period of 25 days as outline in the below table. The resource person will include following specific activities in the assignment and will indicate in their proposal number of days they propose for each activity to be discussed and agreed with Afghanaid. The consultancy expected to start in the first week of March, 2015.
Activity
Number of Days
Review of documents (literature and data from secondary sources and project related documents); development and agreement with Afghanaid on methodology, detailed planning e.g. survey, formats, meetings 3
Field visits: visiting project sites, meetings with project participants and communities in eight districts (if security permits); meeting with DoWA, DAIL, DoEDU and other stakeholders (GIZ, DCA); meetings with project teams in district offices, debriefings with the provincial project team 12
Debriefing meetings in Kabul with Afghanaid 1
Draft reports 4
Final reports 1
Travel time (actual) – international and local 4
Total 25

To ensure consistency across evaluation reports, the following structure should be used for reporting:
Executive Summary
Introduction

• Purpose of the evaluation


• Organisation context


• Logic and assumptions of the evaluation


• Overview of GPAF funded activities
  Evaluation Methodology


• Evaluation plan


• Strengths and weaknesses of selected design and research methods


• Summary of problems and issues encountered
  Findings


• Overall Results


• Assessment of accuracy of reported results


• Relevance


• Effectiveness


• Efficiency


• Sustainability


• Impact
  Conclusions


• Summary of achievements against evaluation questions


• Summary of achievements against rationale for GPAF funding


• Overall impact and value for money of GPAF funded activities
  Lessons learnt (where relevant)


• Project level – management, design, implementation


• Policy level


• Sector level


• GPAF management
  Recommendations
  Annexes (such as)


• Independent final evaluation terms of reference


• Evaluation research schedule


• Evaluation framework


• Data collection tools


• List of people consulted


• List of supporting documentary information


• Details of the evaluation team


• Commissioning organisaion management response to report findings and recommendations

How to apply:

Please send:
a) A covering letter explicitly referring to previous similar experiences, preferably in Afghanistan
b) CV
c) Detailed methodology
d) Work plan indicating dates and number of days for each task
e) Professional fee (in USD – either lump sum or daily rate)
f) Availability (date)
Your EoI must reach by COB on Friday 15th January 2016, clearly marked WEE Project Final Evaluation, Badakhshan, Afghanistan; mailed at: vacancies@afghanaid.org.uk.