Results – Evaluation of the Reconstruction and SPCC programmes and the tender phase of ARC
IOB’s report "Less Pretension, More Realism" contains lessons and recommendations for improving programmes that aim to provide sustainable benefits to vulnerable people in situations of insecurity.
Two important programmes of the ministry’s policy priority Security and Rule of Law (SRoL) were the Reconstruction programme (2012 - 2015) and the Strategic Partnerships in Chronic Crises (SPCC) programme (2014 - 2016). Together, the Reconstruction and SPCC programmes supported 36 projects implemented by Dutch and international NGOs in 24 countries. The pooled budget was about EUR 154 million. This evaluation focused on the effectiveness of both programmes. Additionally, it looked at the tender phase of the subsequent Addressing Root Causes (ARC) programme (2016 - 2021).
Main question
Have the Reconstruction and SPCC programmes been effective and how can programmes that aim to address conflict and insecurity in fragile contexts be improved?
Conclusions
The main finding of the evaluation is that the Reconstruction and SPCC programmes achieved several tangible outputs but that in the longer term, the impact of these programmes has been limited. While concrete results were achieved, such as the establishment of peace committees or infrastructural works, outcomes were often restricted to individual cases or were local in scope. In only a few instances did outcomes ‘trickle up’ and have an effect on village or subnational socio-economic development or on reduced levels of conflict or insecurity. This is only partly attributable to the highly complex and dynamic environments, characterised by violence, political unrest and economic problems, in which projects were implemented. The subsequent findings shed light on other explanations.
Findings
Many of the project proposals were based on implicit and untested assumptions. On the ground, activities most commonly implemented included: supporting savings groups; providing business and vocational training; setting up peace committees; promoting peace dialogues; providing peace trainings; and stimulating local participation in community development. The available literature and our field research suggest that the results that can be expected from these types of activities are limited in scope.
Resources were spread out thinly over numerous small and geographically dispersed activities. Moreover, activities were rarely aligned with the programmes of embassies or of other donors or with government programmes, with the result that activities were often isolated and the funds per activity were often too small to achieve substantial effects at the outcome level.
Proposals and corresponding projects responded (and, in the case of the ARC programme, are still responding) to the overarching global policy objectives at the cost of alignment with local needs. Despite efforts to establish strategic partnerships with implementing partners, there were few strategic discussions between partners and the ministry after the projects started, especially in the SPCC and ARC programmes.
This came at the cost of critically reflecting on the relevance and effectiveness of activities. Pressure to report on standardised result indicators at outcome and impact levels within short time frames jeopardised critical assessments of the implemented activities. This contributed to a disconnect between project results on paper and project realities on the ground. In addition, the quality of project evaluations was often disappointing and unreliable beyond the output level.
Recommendations
Carefully consider the spheres of influence of the implementing organisations and formulate goals at output and outcome levels. Be wary of standardised progress indicators and overarching results frameworks. Partners should not be asked to monitor their own progress beyond intermediate outcome levels, especially not in the short term. However, this does not dispense with the need for ex post evaluation of projects’ outcome and impact results.
This entails more than reducing the number of individual activities. Centrally funded programmes should be better linked to embassy programmes and to other donors’ initiatives and government policies. In addition, IOB recommends limiting the geographic focus and increasing the duration of programmes. Consistency in thematic and geographic focus is warranted and more attention should be paid to following up existing effective activities in subsequent programmes.
This requires behavioural change from both the ministry and its partners. Developing effective partnerships with implementing partners requires the ministry not only to be flexible but also to provide additional resources and staff. The ministry should ensure that there are enough opportunities for strategic discussions and learning. Partner organisations should invest in critically reflecting on activities and in learning together with their implementing staff and (local) partners. IOB also recommends institutionalising the active involvement of embassies during the implementation of centrally funded projects.
Explore funding schemes that enable iterative programming and adaptive planning. For projects to be implemented by NGOs in a specific country or context, consider devolving budgets to embassies.
Where no evidence is available (for example, for innovative activities), a clear strategy should be articulated for testing and reflecting on assumptions. Improve individual project evaluations and stimulate implementing partners to strengthen their research methodologies. It is important to take evaluations and their requirements into account as early as the phase of project design.
Both the ministry and its partners should regularly reflect on implementation strategies and underlying assumptions. Iterative programming and planning are important to facilitate context-specific learning, especially when working in volatile contexts.
After publication report