Results – Synthesis evaluations education programmes | Policy and Operations Evaluation Department (IOB)

For decades, the Netherlands has supported various education initiatives in low- and middle in-come countries. In doing so, the Netherlands aims to contribute to improving the quality of education and research in these countries and employment opportunities for young people. This study synthesises existing evaluation reports of these initiatives from the period 2015-2023.

Background

The Dutch development cooperation policy on education has several objectives: to strengthen education in ODA countries in order to contribute to increasing opportunities and prospects for young people; to increase the number of well-trained professionals; and to promote policy-relevant research. To achieve these goals, the Netherlands (co-)financed a number of global and bilateral funds and programmes. Between 2015 and 2023, the total development cooperation expenditure on education was over €800 million.

Education initiatives evaluated in this study are the following multilateral programmes:

The Global Partnership for Education (GPE), two phases
Education Cannot Wait (ECW) First Emergency Response (FER) funding modality
Education Cannot Wait (ECW) Multi-Year Resilience Plan (MYRP) funding modality
Education Cannot Wait (ECW) Acceleration Facility (AF) funding modality

And the following bilateral programmes:

Middle East and North Africa Scholarship Programme (MSP)
Nexus Skills and Jobs Programme (NSJP)
Netherlands Fellowship Programme (NFP) II
Netherlands Initiative for Capacity Development in Higher Education (NICHE) II
Orange Knowledge Programme (OKP)

This synthesis was a building block towards the Periodic Review of Article 3, Social Development, of the Foreign Trade and Development Cooperation budget.

Central question

The overarching questions for the synthesis are:

What do existing evaluations of Dutch-funded initiatives for education in ODA countries say about the extent to which the initiatives were effective in achieving their goals and why?

What do existing evaluations of Dutch-funded initiatives for education in ODA countries say about the extent to which the initiatives were relevant, coherent, efficient, and sustainable, and why this was the case?

Based on this, what overarching lessons can be drawn?

Conclusions

Effectiveness

Results differed per instrument. Effectiveness of the GPE was mixed, and improved over time, with eventually a number of objectives being achieved as per plan. The effectiveness of ECW was reasonably good. As for the bilateral programmes, the evaluation of NSJP reported partial

achievement of objectives, with good results on youth employability, but less so in ensuring youth actually gets access to market opportunities. NICHE was only evaluated at output level, since the projects’ duration, at the time of evaluation, had been too short to evaluate against other objectives. NFP was found the programme to be effective both at individual level and the level of employing organisations.

Across the funds and programmes, capacity building was one of the main achievements reported. Factors influencing effectiveness included socio-economic circumstances, absence of conflict, fragility and security issues, and the presence of rule of law. Additionally, political factors in the target countries, and administrative capacity of the institutions receiving support had an influence on effectiveness. Another factor was the necessity of having sufficient time available to obtain organisational and institutional change. Also, most evaluations found that having a well-defined Theory of Change and a good quality results framework was essential.

Efficiency

Findings were reasonably positive, with a few caveats. Some interventions had not been timely, due to reasons that were deemed external by the implementers. In the bilateral programmes, some evaluations tried to assess value for money but with few robust findings. For the multilateral interventions, evaluations assessed transparency and governance as mixed but improving. Evaluators deemed that a clear division of roles and responsibilities between key stakeholders was conducive to efficiency. On the other hand, they saw difficult procedures for grantees, and funding recipients’ lack of capable human resources as hampering.

Education is widely acknowledged as crucial for human development in both the development and the humanitarian sector. Nonetheless, gaps in terms of available funds and capacity continue to exist, which makes all the interventions essentially relevant – provided their design is of good quality. The selected interventions were all perceived relevant or highly relevant, since they were seen to respond to the country needs related to education as well as the beneficiary needs and had adapted to changes in context.

Coherence was seen as contingent on stakeholder participation and national and local government ownership was seen as crucial. The potential contribution of some programmes to the humanitarian-development nexus was assessed as positive. Though sufficient capacity of key stakeholders is an important condition for coherence, it is not always sufficient; coordination and collaboration must also be a structural aspect of the intervention, as well as in-depth analyses, visibility of the project with stakeholders and pronounced policy dialogue with the government. As for Dutch government priorities, the bilateral programmes were reported to be aligned with those.

Not all evaluations reported on sustainability. Most of the reported findings concerned institutional sustainability, and the findings on this were mixed. Collaboration with public education institutions and government bodies and their capacity and frameworks were reported as essential, but the absence thereof was also a constraint in both the multilateral and the bilateral programmes. Most of the evaluation reports indicated that financial sustainability was unlikely. In general, under emergency programmes sustainability is often not pursued, as was the case here in the multilateral

programmes. For the bilateral programmes, positive observations were made on the sustainability of academic achievements, jobs created, and behaviour change. Nonetheless, sustainability of institutional and sector improvements was rated less positively.

The multilateral funds met the planned requirement of gender equal targeting. Some also included activities aimed at improved gender knowledge and awareness. A consistent observation for the multilateral funds was the lack of attention to addressing specific needs of male and female students and staff. Evaluators found that including gender into programme goals, policy, result frameworks and action plans was conducive to gender equality. The evaluations assessed the bilateral programmes as gender sensitive. Most bilateral programmes also met their gender equal targeting requirement, although limited reporting of gender disaggregated data at outcome level have challenged creating fundamental changes.

Lessons

Icon with a network of connections for recommendations

The evaluation does not present specific recommendations, but does share three broad lessons learned:

A strong Theory of Change and results framework are important to measure achievement and fine-tune a programme’s approach, which in turn further effectiveness.
Measuring impact at beneficiary level or improvement of the education system is essential but not part of the independent evaluations under study.
Most interventions do not sufficiently engage local actors.

Background

Central question

Conclusions

Effectiveness

Efficiency

Impact

Relevance

Coherence

Sustainability

Gender

Lessons