Afghanistan: Consultant (WASH Project End of Evaluation)

1.1 Introduction

Afghanaid is a British international NGO (www.afghanaid.org.uk) that has been working in Afghanistan since 1983. Headquartered in Kabul and with 20 other offices across three provinces (Samangan, Badakhshan and Ghor). Afghanaid’s work helps community-based recovery and development in some of the most remote areas of the country. A small marketing and fundraising office is based in London.

Afghanaid works alongside poor Afghans to enhance their opportunities and capabilities to achieve sustainable and equitable economic and social development. Afghanaid does this through a range of programme interventions that support grass-roots development of basic services, such as water and sanitation, access roads, education facilities and other community infrastructure; improved livelihoods, through natural resource management, agricultural and small enterprise development, and improved access to markets; as well as humanitarian assistance. Gender mainstreaming, local governance and community development are integrated into all of Afghanaid’s work. The results achieved have earned Afghanaid the respect and recognition of stakeholders as an effective and collaborative charity.

1.2 Project Description

Afghanaid is currently implementing a DFID funded project on “Water, Sanitation and Hygiene (WASH) Improvement” in Hazrat-e-Sultan and Aybak districts of Samangan province. The project aims to reduce risk of diseases and mortality among poor Afghan women, men, girls and boys associated with lack of safe water, poor sanitation and hygiene practices. The project also aims to address the poverty and gender issues that poor access to WASH services causes to communities.
The overall objective of the project to contribute to the reduced risk of disease, mortality and vulnerability associated with lack of potable water, safe sanitation and hygiene practices in Samangan province, Afghanistan. More specifically this action is contributing to above objective through: Improved water, hygiene and basic sanitation behaviours among the people living in 60 villages of Aybak and Hazrat-e-Sultan. The project is about to complete in April 2016.

1.3 Purpose and Scope of the independent final evaluation

The purpose of final evaluation of the project is to measure the progress it has made in terms of achieving its stated outcomes and to highlight achievements. The evaluation should assist in measuring the impact that the project has had on the targeted population, and should also contribute to future programming with particular emphasis on appropriateness and effectiveness of the interventions carried out. Furthermore, the final evaluation should document lessons learnt and provide recommendation to help with other similar project locally and nationally. The achievement of the project will be measured against the following outcomes and impact on the lives of the local population:

a. Women, men, girls and boys in the villages covered by the project area have access to safe/potable water;
b. Women, men and girls and boys have increased knowledge and awareness about good hygiene methods, and make effort to practice those;
c. Women, men, girls and boys have access to basic sanitation services and are discouraged from defecating in the open;
d. Community structures have been created and strengthened to manage, maintain, and improve WASH schemes;
e. There is greater awareness about safe drinking water, hand washing, and sanitary latrines at the community, the provincial and regional levels, and related policies were being impacted.

The evaluation should also examine gender roles in WASH and propose guidelines for ‘engendering’ WASH related community structures for the future intervention. It should examine the different roles women, men, girls and boys are playing at present, and suggest ways to strengthen those roles for the similar future interventions.

The assignment shall include:

· Measuring performance (quantitative and qualitative) against the project’s specific objectives and outcomes approved in the project documents (primarily the LFA and the work plan);

· Identification and analysis of what has worked and what has not worked and why;

The evaluation will assess through measurable evidence (both qualitative and quantitative) and comment up on;

· The extent to which the programme has meaningfully targeted appropriate beneficiaries;

· The contextual relevance, appropriateness, and technical soundness of the project;

· The relationship between project costs, results so far, and its attaining the stated objectives;

· Respect for and adherence to the guidelines and regulations of DFID;

· The appropriateness of the project in the context of the Afghanaid’s current programme strategy;

· Whether there has been a plan for project sustainability.

1.4 Key objectives of the evaluation

The evaluation has three explicit objectives that are explained below:

1. To independently verify (and supplement where necessary), project’s record of achievement as reported through its Annual Reports and defined in the project Log Frame;

2. To assess the extent to which the project was good value for money, which includes considering:

· How well the project met its objectives;

· How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;

· What has happened because of DFID funding that wouldn’t have otherwise happened; and

· How well the project aligns with DFID’s goals of supporting the delivery of the MDGs.

3. To assess the extent to which the project helped the targeted beneficiaries financially and economically.

1.5 Verification of reporting

The first task of the final evaluation is to verify project’s achievement. The record of achievement is presented in past Annual Reports and its progress against the project Log Frame. This exercise could include verifying information that was collected by Afghanaid for reporting purposes and possibly supplementing this data will additional information collected through primary and secondary research.

Verifying the results from the project Log Frame will begin to capture what the project has achieved. However, there will be other activities and results that occur outside of the Log Frame that may require examination in order to respond to the different evaluation questions. Verifying reporting will also necessarily include a review of the data and systems that were used to populate results.

1.6 Evaluation questions

To ensure comparability across the final evaluation reports, the evaluator(s) should adapt and respond to the questions below:

1. 
   

   To what extent did the project target and reach the poor and marginalized?

   
   


2. 
   

   How well did the project respond to the needs of target beneficiaries, including how needs evolved over time?

   
   


3. 
   

   To what extent has the project used learning to improve delivery?

   
   


4. 
   

   What are the key drivers and barriers affecting the delivery of evidence that the benefits delivered by the project will be sustained after the project ends?

   
   

The evaluator is encouraged to structure his/her research questions according to the OECD-DAC criteria of relevance, effectiveness, efficiency, sustainability and impact.

Relevance

· To what extent did the project support achievement towards the MDGs, specifically off-track MDGs?

· To what extent did the project target and reach the poor and marginalized?

· To what extent did the project mainstream gender equality in the design and delivery of activities (and or other relevant excluded groups)?

· How well did the project respond to the needs of target beneficiaries, including how these needs evolved over time?

Effectiveness

· To what extent are the results that are reported a fair and accurate record of achievement?

· To what extent has the project delivered results that are value for money? To include but not limited to:

o How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;

o What has happened because of DFID funding that wouldn’t have otherwise happened; and

o To what extent has the project used learning to improve delivery?

o What are the key drivers and barriers affecting the delivery of results for the project?

Efficiency

· To what extent did the project deliver results on time and on budget against agreed plans?

· To what extent did the project understand cost drivers and manage these in relation to performance requirements?

Sustainability

· To what extent has the project leveraged additional resources (financial and in-kind) from other sources? What effect has this had on the scale, delivery or sustainability of activities?

· To what extent is there evidence that the benefits delivered by the project will be sustained after the project ends?

Impact

· How many people have received support from the project that otherwise would not have received support?

· To what extent and how has the project affected people in ways that were not originally intended?

2. Methodology
The methodology in this evaluation to be used must ensure the active participation of the beneficiaries and the intended respondents for men, women and children when necessary to best solicit the ideas and opinions of the beneficiaries. The method must respect gender, cultural diversity, individual and collective opinions.

Whatever the methodology used, it must also ensure that rights, safety and security of the respondents and beneficiaries are not overlooked. The Consultant is required to develop the evaluation methodology and provide an appropriate representative sampling in consultation with Afghanaid M&E and programme staff.

1. Deliverables
   The consultant will deliver one report for end of project evaluation, covering the scope and the specific objectives of this study described above, a comprehensive report incorporating lessons learnt and recommendations for similar future projects. To ensure consistency across evaluation reports, the following structure should be used for reporting:

Executive Summary maximum 4 pages of Arial font 11 with normal page margins.

Context and Introduction

• 
  

  Purpose of the evaluation

  
  


• 
  

  Organisation context

  
  


• 
  

  Logic and assumptions of the evaluation

  
  


• 
  

  Overview of GPAF funded activities

  
  

Study methodology

Findings

• 
  

  Overall results

  
  


• 
  

  Assessment of accuracy of reported results

  
  


• 
  

  Relevance

  
  


• 
  

  Effectiveness

  
  


• 
  

  Efficiency

  
  


• 
  

  Sustainability

  
  


• 
  

  Impact

  
  


• 
  

  Analysis of the project evaluation findings regarding each of the key sections of the approved LFA and proposal (both qualitative and quantitative analysis) including challenges and constraints.

  
  


• 
  

  Debriefing of the findings in the provincial office and HO.

  
  


• 
  

  Lessons learnt and to note two good practices of the project activities.

  
  

Conclusions

• 
  

  Summary of achievements against evaluation questions

  
  


• 
  

  Summary of achievements against rationale for GPAF funding

  
  


• 
  

  Overall impact and value for money of GPAF funded activities

  
  

Lessons learnt (where relevant)

Recommendations

Annexures (such as)

• 
  

  Independent final evaluation terms of reference

  
  


• 
  

  Evaluation research schedule Evaluation framework Data collection tools

  
  


• 
  

  List of people consulted

  
  


• 
  

  List of supporting documentary information

  
  


• 
  

  Details of the evaluation team

  
  


• 
  

  Grantee management response to report findings and recommendations

  
  

A draft report will be submitted for review by Afghanaid, after incorporated Afghanaid’s feedback and comments the final report will be submitted again to review for approval.

3.1 Evaluation Timeframe and Duration

Afghanaid estimates 30 days for the consultancy. However, the bidder will provide a schedule showing the number of days s/he requires to complete the assignment. She may include following activities and more and indicate the number of days she might require to complete each of the activities proposed. The final schedule will be based on availability of budget, mutual convenience and agreement.

Activities

Estimated number of
days

1. 
   

   Review project documents (proposal, LFA, progress reports and etc.)

   
   


2. 
   

   Developing evaluation methodology and sharing it with relevant Afghanaid staff.

   
   


3. 
   

   Discussions with Program Director, M&E Manager and Donor Coordination Coordinator in HO.

   
   


4. 
   

   Discussions with Provincial Programme Manager, Programme Manager and Research &Documentation Coordinator on in-house data availability (a.m.)

   
   


5. 
   

   Discussion on proposed evaluation methodology (p.m.)

   
   


6. 
   

   Planning and other preparations for information gathering

   
   


7. 
   

   Information gathering

   
   


8. 
   

   Sharing observations and inferences with the WASH team

   
   


9. 
   

   Debriefing to Afghanaid in province and HO

   
   


10. 
    

    Data analysis and drafting revaluation report

    
    


11. 
    

    draft report shared with Afghanaid staff for comments and feedback

    
    


12. 
    

    Finalizing report after incorporating comments and feedback from Afghanaid

    
    


13. 
    

    International and local travel, 4

    
    

Estimated total number of days, 30

3.2. Commissioning Manager / Reporting Line

The consultants will report directly to Project Manager DfID WASH project

Requirements

Consultant’s Expertise and Skills

Essential:

· Post-graduate degree in social science or other related fields;

· Demonstrable evidence of a strong background of quantitative and qualitative social research methodologies; and

· Past experience of conducting mid-term and end-term evaluations of large multi-year projects;

· Hands-on Project Cycle Management experience; and

· Knowledge of Gender Analysis and mainstreaming

Desirable:

· Previous experience of conducting assessments / evaluations of DfID/GPAF funded projects;

· Experience of evaluating / managing WASH / Health sector projects

· Experience of working in Afghanistan or other conflict affected countries