Indonesian Red Cross (Palang Merah Indonesia – PMI), supported by American Red Cross, has implemented a project since April 2016 that aims to provide assistance to drought affected populations and improved PMI disaster response capacity and improved coordination among key stakeholders during short-term El Niño related drought, and related and longer term disaster management situations. This project actually covers 2 sectors, that is WASH and Risk Management Policy and Practice. Funded by USAID/OFDA, this project covers two sectors that are Water, Sanitation, and Hygiene (WASH), and Risk Management Policy and Practice. The first sector was implemented under emergency response scheme that was concluded in December 2016, while the second sector is ongoing until the project closes by the end of 2018. To achieve the goal, the project targets two outcomes that each of them lies under each project’s sector. The first outcome is targeted communities adversely affected by severe drought conditions receive adequate clean water and hygiene promotion (for WASH sector); and the second outcome is PMI Regional and Sub-Regional hub strengthening capacity and management by developing SOPs, contingency plan, and improving personnel’s response skills and knowledge (for risk Management Policy and Practice sector). The project would to conduct final evaluation for each sector. This Term of Reference of Evaluation exclusively focuses on evaluating the WASH sector.
The WASH sector was implemented as response to predicted prolong drought exacerbated by El Niño. Indonesia suffered a serious drought during the 1997-1998 El Niño that was recorded as the country’s strongest drought in history. Following the nearly absence of rainfall during the first eight months in 2016, it was forecasted that prolong drought exacerbated by the El Niño weather phenomenon would again hit the majority of the country’s 34 provinces. This was predicted to contribute to significant water supply issues while the information of total water supply and demand by communities and service providers remained limited. Indonesian Disaster Management Agency (Badan Nasional Penanggulangan Bencana - BNPB) identified 14 provinces that would be most affected by the drought. Following this, PMI
determined 5 out of the 14 most affected provinces as the project’s targeted areas, that were: West, East, and Central Java, as well as West and East Nusa Tenggara.
PMI, as one of largest humanitarian organization in Indonesia with its volunteers spread nationwide and equipped with skilled personnel and significant WASH infrastructure facility; was able to deliver hygiene promotion during the drought response, water supplies, and repair/construct water points in 288 sites along with total 16.7 km of water pipe constructions and 4,844,000 liters of clean water distributed across the 5 provinces. The water supplies were distributed through different modes such as cash transfer using mobile wallet and card in urban (Bekasi) and semi-urban (Boyolali) areas for drinking water and water tankers for clean water. These interventions were still given to drought prone areas even though the El Niño exacerbated drought eventually did not happen as it had been predicted previously. As part of quality assurance, PMI worked with various stakeholders such as District Health Office to ensure low chlorine residual within distributed clean water, and village apparatuses to monitor the function of repaired/constructed water points after its 3-month completion. The WASH sector intervention ended in December 2016 and PMI at levels (Head Quarter, Province, and District) held lessons learnt and shared of best practices in October 2017. A mid-term project evaluation was conducted to examine how relevance, monitoring, evaluation, and reporting for the WASH sector. Since the project is going to close, the project would like to conduct final evaluation that
Project location and beneficiaries
After 10-month implementation (March-December 2016), these responses benefitted 214,939 people that significantly achieved initial target by 268%.
- USAID/OFDA: Donor
- American Red Cross: Supporting Partner (Operating Country, AMEE Region, Head Quarter)
- Indonesian Red Cross: Implementer (at three levels: Branch, Chapter, and Head Quarter)
Previous evaluation activities
Throughout implementation of the project, PMI and American Red Cross implemented survey, monitoring, and lessons learned as part of formative and process evaluation (performance monitoring) as follow:
- Drought survey, 2015
- Final Evaluation of Supporting Household Access to Clean Water Using Mobile Wallet Cash Transfers in Bekasi, 2016
- Evaluation of Cash Transfer Program (CTP) for Drought Disaster Affected Families in Boyolali, 2017
- Project Mid-term Evaluation Report, 2018
- Project Quarterly Report
- Indicator Tracking Table
2. Evaluation Overview
Objectives of final evaluation
The Final Evaluation objectives are to:
- Measure project’s WASH sector achievement, focusing on the changes (positive or negative, intended or unintended) contributed by the project.
- Determine the relevance, effectiveness, and sustainability.
- Document key lessons learned, best practices, and stories of changes.
Evaluation criteria and questions
The main evaluation question is: How did the WASH intervention contribute to the changes in communities?
Details evaluation criteria and question available at TOR and upon request by email to email@example.com
Evaluation, preferably, applies qualitative methodology supported by secondary data derived from previous assessment such drought survey, cash pilot evaluation reports, quarterly reports and indicator tracking tables. In the proposal, the consultant candidates are expected to be able to explain the proposed evaluation methodology with suggested measurement strategy (for qualitative method, the suggested measurement strategy could be varied such as key information interviews, focus group discussions, stories of changes, most significant changes, etc.), considering the project context that the water supplies were given to different 5 provinces that was concluded in 2016. The consultant candidates have to narratively explain their justification over the proposed evaluation methodology, whether they suggest for qualitative method only or mix methodologies including the measurement strategy and its benefits for evaluation, analysis plan, and strategy to reduce possible bias.
The evaluation will cover the WASH sector interventions that were operated in 22 Districts across 5 Provinces. As can be seen from Table 1, the WASH interventions were ranged from water dropping to filtering, storing, piping and water points repair/construction. Despite the water infrastructure was built within short time as this was intentionally designed for emergency response, the sustainability of the water infrastructure (its existence/usage, quality, and maintenance) would be assessed as part of Final Evaluation.
Scope of work
The selected consultant/firm will be responsible to ensure that the project final evaluation of the Indonesian Disaster Management and Logistics Readiness: Emergency Water Supply and Strengthening Regional Logistics Capacity Project meets standard evaluation and research. The scope of work must include:
- Provide a complete inception report (project final evaluation proposal and tools).
- Develop evaluation tools,
- Lead data collection and coordinate with PMI throughout the evaluation process.
- Perform data management, quality assurance, and data analysis.
- Write and present project final evaluation.
- Develop 2-3 stories of changes as contributed by the project.
- Develop a fact sheet of project’s key achievement.
Logistic and administrative support
The consultant is expected to use her/his own computer. Approved administrative and logistic costs will be reimbursed by the Red Cross Team. The consultant will be able to work remotely, after approval of the inception report.
The consultancy service agreement would be facilitated by the American Red Cross through IFRC. The consultant will report to Madiatri Silalahi, Monitoring & Evaluation Division from American Red Cross, and Hanna Kusumastuti as the Project Coordinator from PMI Disaster Management.
Future use of data
All collected data will be the sole property for the American Red Cross. The consultant may not use the data for their own research purposes, nor license the data to be used by others, without the written consent of the American Red Cross.
Expected activities and deliverable
- Inception Report (Final proposal and tools) prior data collection
- Conduct initial briefing and document review
- Finalized Evaluation tools
- Provide Training Manual for field staff (in Bahasa Indonesia)
- Perform data collection
- Implement data management, quality control, and analysis
- Provide updates of weekly/biweekly evaluation progress
- Submit draft report
- Submit signed inform consent (using American Red Cross form) and photos taken throughout Evaluation process
- Provide all data collected used for data analysis and reporting (sealed transcript and records, analysis table/working paper, etc)
- Submit Final Report (in English and Bahasa Indonesia).
Duration of consultancy
The consultancy will be provided within 40 working days, starting from 8 October to 3 December 2018.
Required qualifications for the consultant team is:
- Five years of experience conducting and leading qualitative researches/evaluations, 3 of which should be with international organizations.
- Excellent knowledge and skills in qualitative methodology. Be able to show experiences in designing evaluation, managing, and analyzing data.
- Experiences in leading evaluation, particularly in WASH related projects would be advantage.
- Experiences in story writing as part of Evaluation products shown by sample of stories from previous evaluations.
- Demonstrated expertise in WASH as determined by working experience and would be plus with supporting educational background.
- Demonstrated high quality report (in English and Indonesian language) as shown by sample of Evaluation reports.
- Previous evaluation experiences with PMI and American Red Cross are desirable.
The consultant candidates are expected to submit the proposed Project Final Evaluation Proposal with table of content as followed:
- Consultant profile
- Evaluation purpose
- Scope of work
- Evaluation methodology
- Data management and Quality control
- Analysis plan
- Evaluation time frame
- Evaluation budget plan
The Annex should be consisted of:
- CV of each Evaluation team members
- Sample of previous Evaluation report in English comprised of**: Executive Summary (1 page), Analysis results and key findings (4 pages), Recommendation (1 page), Conclusion (1 page), and a sample of stories.**
- Qualifications section
- Quality of proposal
- Experience in evaluation
- Expertise in WASH
- Number of Days and Timeline availability
- Rational budget plan
- Previous evaluations in PMI and American Red Cross projects
The payment will be made based on satisfactory deliverable and after approval from American Red Cross Regional Office. Consultant must submit invoice for the payment.
- 20% after contract signed by both parties with submission and acceptance of Inception report
- 40% after submission of and accepted draft report
- 40% after approval of Final Report and all deliverable
How to apply:
Please submit your proposal application in English not later than 30 September 2018 to firstname.lastname@example.org with title DM WASH Sector Project Final Evaluation Application in the subject line and label your CV, proposal, and sample of Evaluation report with your name and document type in the file names. Only short listed candidates will be notified and contacted for an interview. Applications received after the deadline or incomplete applications will not be considered.