LocationsThe UNDP Independent Evaluation Office is delighted to announce the winners of the 2023 Evaluation Excellence Awards.
These awards recognize and celebrate excellent quality decentralized evaluations conducted in 2022 across three categories: outstanding, innovative, and gender-responsive evaluation. This year’s selection process was highly competitive. The over 280 decentralized evaluations completed during the year and uploaded to the Evaluation Resource Centre by year’s end all merited consideration for the awards. Finalists were chosen by members of the IEO’s Evaluation Advisory Panel.
The Outstanding Evaluation Award was given to two country offices:
- UNDP Barbados and the Eastern Caribbean for Monitoring and assessment of MEA implementation and environmental trend in Antigua and Barbuda
- UNDP Libya for Final evaluation of the Strengthening Local Capacities for Resilience and Recovery
The Innovative Evaluation Award was given to UNDP Bangladesh for exceptional quality evaluations across the
board this year, especially when it comes to innovative methods and approaches. Two evaluations were chosen as prime examples:
- Final evaluation of the Agriculture and Food Security (AFSP III) project
- Final evaluation of the Community Cohesion in Cox's Bazar project
The Gender-responsive Evaluation Award was given to two country offices:
- UNDP Namibia for Final evaluation of the Strengthening Local Capacities for Resilience and Recovery
- UNDP Yemen for Rebuilding livelihoods and capacities of conflict-affected small-scale fisheries households in Aden and Hadhramout - Final Evaluation
Outstanding Evaluation: UNDP Barbados and the Eastern Caribbean
Monitoring and assessment of MEA implementation and environmental trend in Antigua and Barbuda
The Evaluation Advisory Panel said the following about the report:
"From a gender perspective, the greatest strengths of this evaluation are in its judgement and methodology, which provide gender responsive evidence to support questions addressing human rights, gender equality and social inclusion. Participant selection took into consideration the cross-cutting themes of gender, age and ability to ensure the voices of women, men, youth and people with disabilities were represented."
Statement by the winning country office
The UNDP Multi-Country Office (MCO) for Barbados and the Eastern Caribbean is proud to receive an Outstanding Evaluation Award for its evaluation of the GEF-funded project “Monitoring and assessment of MEA implementation and environmental trends in Antigua and Barbuda”.
The award serves as a reminder of UNDP’s commitment to creating positive change in Antigua and Barbuda, and across the Eastern Caribbean. It is a reflection of the collective commitment of the Barbados MCO and its government counterparts in the Department of Environment and Ministry of Health, Wellness and the Environment in Antigua and Barbuda to build sustainable and resilient solutions to the challenges arising from the climate crisis faced by the region.
Strong monitoring and evaluation practices are a cornerstone of achieving project sustainability and effectiveness. They help to identify areas where we can make course corrections and accelerate progress, so projects have the greatest impact possible. The MCO is also committed to learning lessons from our work and sharing that knowledge more broadly, to inform future projects. We will continue to seek out new approaches and solutions that can help us make an ever-greater difference in the Caribbean region.
During this evaluation we learned some important lessons, including three things that are at the heart of an effective evaluation: good alignment of the evaluator’s knowledge and expertise with project design, advanced planning and consistent communication. Choosing the right consultant was an important consideration, balancing a strong evaluation background with the education and experience needed to understand the project’s design and implementation. Advanced planning allowed sufficient time for development and feedback, including verification of the project context, evaluation criteria and initial expectations. Consistent communication between the evaluation team and key stakeholders allowed gaps and misconceptions to be identified and addressed early, and improve the evaluators’ understanding of the project and context.
We look forward to continuing along this path, in strong partnership with the Department of Environment, Antigua, and our partners throughout the Caribbean region. With receipt of this award, the Barbados MCO reaffirms its commitment to a better and more sustainable future for all.
Outstanding Evaluation: UNDP Libya
Final evaluation of the Strengthening Local Capacities for Resilience and Recovery
The Evaluation Advisory Panel said the following about the report:
"This remotely conducted evaluation shows exceptional awareness of contextual limitations and takes useful steps to mitigate them. It clearly links the findings and recommendations, and shows the positive role UNDP can play in fragile contexts, to help bridge the gap between crisis and recovery."
Statement by the winning country office
After a decade of conflict and instability, Libya faces significant economic, social and security challenges, and has suffered from an erosion of human and institutional capital. The country and its citizens are faced with diverse and complex challenges, including the collapse of public services, loss of livelihoods and major setbacks to the safety of citizens and the rule of law.
To meet these challenges and help restore trust between citizens and institutions, UNDP Libya is implementing programmes that support electoral processes, decentralized governance, build resilience and peace, and promote jobs and sustainable livelihoods. To ensure that programmes are context-sensitive and impactful, project design and implementation must be informed by the evidence that quality evaluations can provide.
UNDP’s Strengthening Local Capacities for Resilience and Recovery project focused on building the capacities of municipalities to restore livelihoods, infrastructure, and access to quality basic social services, particularly for the most vulnerable people. Compelling evaluative evidence was crucial to assessing the impact of the project on people’s lives.
Evaluation in fragile states needs to be swift, flexible, and practical to support programmatic decisions in a rapidly changing environment. Instability poses significant issues for data collection, which compromises evaluation. To overcome this obstacle, we used multiple data sources and accounted for data gaps in our analysis. As field surveys proved to be challenging to implement, other tools such as in-depth interviews, observations, focus groups and evidence generated by partners were used to collect and triangulate evidence. To avoid increasing tensions between rival groups, the “do no harm” principle was applied throughout the evaluation. Recognizing the significant challenges for evaluation in conflict contexts, contingency plans were put in place for sudden changes in the situation that could impact data collection.
In addition to the evaluation, UNDP has engaged a third-party monitoring firm for independent monitoring of projects and provide real-time feedback for the design and implementation of project outputs. Third-party monitoring will also support target municipalities to develop area-based peacebuilding and development plans that serve as a mechanism to identify priority projects, while promoting peace, resilience and economic recovery at the local level.
Innovative Evaluation: UNDP Bangladesh
Final evaluation of the Agriculture and Food Security (AFSP III) project and Final evaluation of the Community Cohesion in Cox's Bazar project
The Evaluation Advisory Panel said the following about the Final evaluation of the Agriculture and Food Security (AFSP III) project:
"This evaluation shows an excellent application of innovative methods to evaluate grassroots-level projects in hard-to-reach areas. The balanced use of quantitative and qualitative methods highlights the results achieved in a complex context. It contains solid learning points for evaluation practitioners."
The Evaluation Advisory Panel said the following about the Final evaluation of the Community Cohesion in Cox's Bazar project:
"A complex social setting necessitates innovative data collection methods for social groups, to enhance the robustness of the findings and mitigate sensitivities. This evaluation successfully utilized a careful articulation of evaluation questions, data collection and content analysis. It is recommended reading for evaluators of development projects and programmes."
Statement by the winning country office
UNDP Bangladesh is delighted to receive the Innovative Evaluation Award for 2023. At the heart of our new approach is the consideration of several factors which turned out to be game changers.
Firstly and most importantly, we put in place concrete steps to remove the language, cultural and gender differences that can affect the quality of the data collected. The first step was to hire people from the local community and train them in data collection. The new data collectors comprised men and women to fit the cultural context and the diverse groups of people consulted.
“We could communicate with the data collectors as they spoke the same dialect. We could also talk about problems with the ladies on the evaluation team. They were like us, and we could tell them everything openly,” said Nikunti Tripura, a female farmer from Khagrachari, a district in the Chittagong Division of Southeastern Bangladesh.
Secondly, a three-tier follow-up system was implemented to ensure the data collected was of high quality. Project staff and the office’s M&E Analyst oversaw data collection and ensured strict adherence to the data collection timeline with regular progress checks. The use of digital data collection systems was also a boon, as it allowed for real-time observation.
The third key factor was the integration of approaches that reflected the context. Gender analysis was included through the Harvard Gender Analysis Framework. The conflict-sensitive lens from the USAID guidelines added an additional dimension to the evaluation of the “Community Cohesion in Cox’s Bazar Project”.
Finally, the practice of peer review contributed significantly to the quality and relevance of the evaluation results. The initial findings were presented to and discussed with project managers, and their feedback was integrated.
“In UNDP Bangladesh, we take continuous learning seriously,” said Stefan Liller, Resident Representative of UNDP Bangladesh. “We do everything we can to ensure that findings and recommendations of project evaluations are shared widely and used both to improve ongoing projects as well as to inform the design of new ones.”
Gender-responsive Evaluation: UNDP Namibia
Final evaluation of the Strengthening Local Capacities for Resilience and Recovery
The Evaluation Advisory Panel said the following about the report:
"From a gender perspective, the greatest strengths of this evaluation are in its judgement and methodology, which provide gender responsive evidence to support questions addressing human rights, gender equality and social inclusion. Participant selection took into consideration the cross-cutting themes of gender, age and ability to ensure the voices of women, men, youth and people with disabilities were represented."
Statement by the winning country office
UNDP Namibia attaches absolute priority to gender equality, in accordance with the UN System-Wide Action Plan on Gender Equality and the Empowerment of Women,and takes gender responsiveness in its programming seriously. We are delighted to be awarded the 2023 Evaluation Excellence Award in the gender-responsive category for the mid-term evaluation of our Country Programme Document (CPD) covering the current programme cycle (2019-2023). This is also a result of internalising the UNDP evaluation policy guidelines on this issue.
The CPD is firmly anchored in the principle of ‘leaving no one behind’, including the commitment to strengthening gender equality. It focuses on reaching women and youth living in poverty, marginalized groups, and persons with disabilities, especially in Namibia's biodiversity-rich, sensitive, and disaster-prone areas. Gender markers are used to track country programme budgets, to improve planning and resource allocation for gender equality.
It was crucial to the Office that the evaluation methods and approach used for the CPD evaluation recognized and responded to this important lens. To incorporate the gender perspective, human rights, gender equality, disability issues and social inclusion were included as cross-cutting themes in the CPD. Further, the OECD-DAC evaluation criteria of relevance, effectiveness, efficiency, and sustainability were used as additional guidance. A participatory and inclusive approach facilitated giving voice to the different stakeholders. The methodology, and data collection tools were assessed minutely to ensure they adequately addressed gender equality, and dedicated questions helped assess UNDP Namibia’s contribution to gender mainstreaming in programming.
Using this gender lens, the evaluators could accurately assess that UNDP Namibia adopted a ‘gender-targeted’ approach to programming, including a gender action plan that is being implemented and monitored, along with a gender-responsive budgeting roadmap. Key issues the evaluation highlighted are that intersectionality can’t be appropriately addressed through a siloed approach. Cognisant of this, the Country Office is consistently engaged in ensuring that government counterparts and other implementing partners have the technical support they need for full gender mainstreaming.
Gender-responsive Evaluation: UNDP Yemen
Rebuilding livelihoods and capacities of conflict-affected small-scale fisheries households in Aden and Hadhramout - Final Evaluation
The Evaluation Advisory Panel said the following about the report:
"A gender-transformative lens is clearly visible in every aspect of this evaluation. The interview questions and process respect gender protocols, and the methods and analysis utilize gender-responsive success standards. Notably, the lessons and recommendations highlight implementable actions that promote gender empowerment."
Statement by the winning country office
Yemen has witnessed a series of catastrophic events and widespread unrest since 2011. Despite this challenging operational context, UNDP projects in Yemen meet their general goals to a significant extent, and achieve a high level of community satisfaction.
The project that was the subject of the winning evaluation – “Rebuilding Livelihoods and Capacities of Conflict-Affected Small-Scale Fisheries Households in Aden and Hadramout” (RELACC) - had a significant impact on the livelihood needs identified. The fisheries sector has been heavily impacted by the conflict, including a ban on fishing in many areas, which has caused a substantial decline in production and loss of livelihoods. In response, the RELACC project created an environment that enabled the reclamation of livelihoods through business and skills training as well as the provision of grants and assets.
To address gender inequalities and overcome related constraints, RELACC project activities were designed based on the understanding of gender roles and relationships in the coastal society, with a gender-inclusive approach that involved both men and women in the design and implementation stages. The project made additional efforts to enhance women’s empowerment in two ways. Firstly, through scheduling the project activities to make it easy for women to attend them without any disruptions, and secondly by promoting women’s enterprises to facilitate the sale of produce and subsequent improvement of household incomes.
Yemen is a very conservative society. The consultant approach was extremely useful to evaluate women’s empowerment from the customary perspective. In particular, the ways in which the project was both articulated and implemented took into consideration the reality faced by Yemeni women. By respecting privacy and local societal norms, women's participation was encouraged. According to respondents involved in the evaluation, the most powerful impact of the project was that those who rejected the idea of women's inclusion changed their perceptions after the project interventions proved that addressing women's needs and rights is necessary.
