Working method IOB
IOB adheres to several fixed steps in the research process and a quality control system. We work together with other departments and national and international organisations to increase the quality of evaluations and strengthen policy learning.
For most evaluations, the research process at IOB consists of the following steps:
- Exploratory phase
The preliminary study on the (policy) subject, the available data, the research questions, the scope of the research, the most suitable methodology and the deliverables – the research team records all of these aspects in the Terms of Reference (ToR).
- Adopting the ToR
After these aspects have been discussed in draft form with the internal ‘sounding board’ group and external reference group, the ToR are determined and published on IOB’s website. The ToR guide the entire research process and can be adjusted along the way if necessary.
- Carrying out the research
IOB assesses the policy in question on the basis of several criteria, namely relevance, effectiveness and efficiency and – if possible – also the coherence and sustainability of the results achieved.
This phase often generates several intermediate products. For example, we regularly hold workshops on sub-themes. Sub-themes can also be concluded with a partial report. We discuss these reports with the policy departments that are directly involved and any external experts.
- Adopting the report
The study concludes with a final report that distils all the themes. The director of IOB adopts this report.
- Drafting a policy response
The departments in question draft a policy response. Generally, this occurs within three months of their having received the report.
- Publication of the report
No later than three months after its adoption, the report will be published on IOB’s website and sent to parliament, together with the policy response.
After twelve months, there will be a follow-up discussion between the director of IOB, the director of Financial and Economic Affairs and the directors-general and directors who were most closely involved. In this meeting, we discuss how the departments are implementing or have implemented policy recommendations that have been adopted by parliament. The report on this meeting is shared with the ministry’s management board.
- Ready: Exploration
- Ready: Terms of Reference adopted
- Ready: Research in progress
- Ready: Report adopted
- Ready: Drafting policy response
- Ready: Report to parliament
To assure the quality of the research, IOB works with internal peer reviewers and an external reference group for each evaluation/policy. The internal group works with the research team, reading the reports and providing input. The reference group meets to discuss how the research is progressing and provides feedback on the researchers’ work. Members of the reference group always include staff from the Ministry of Foreign Affairs who are involved in the policy theme, as well as three independent external experts.
IOB liaises and works together with different departmental directorates in the areas of research, knowledge management and policy learning. Examples include the Strategy and Analysis Unit (ESA), the Financial and Economic Affairs Department (FEZ), the Work in Progress Programme Directorate (WiU) and the Academy for International Relations (AIB). IOB is also closely involved in the government-wide Insight into Quality operation, which aims to increase the quality of evaluations and strengthen policy learning.
In addition, we have partnerships with national and international organisations in the field of foreign policy impact measurement. Following are some examples of the organisations with which IOB cooperates:
3ie – International Initiative for Impact Evaluation
AfrEA – African Evaluation Association
Algemene Rekenkamer – The Netherlands Court of Audit
BMZ – Bundesministerium für wirtschaftliche Zusammenarbeit und Entwicklung (German Federal Ministry for Economic Cooperation and Development)
CLEAR – Centers for Learning on Evaluation and Results
DAC Network on Development Evaluation – Organisation for Economic Cooperation and Development (OECD DAC EvalNet)
DBE – Dienst Bijzondere Evaluatie, the Belgian Development Cooperation evaluation service
DEval – Deutsches Evaluierungsinstitut der Entwicklungszusammenarbeit (German Institute for Development Evaluation)
DFID – Department for International Development, UK
EES – European Evaluation Society
GEI - Global Evaluation Initiative
EuropAid Evaluation Department – European Commission
IDEAS – International Development Evaluation Association
IEG – Independent Evaluation Group, World Bank Group
International Cooperation and Development, European Commission
Vide – the professional association for regulatory, inspection, enforcement and evaluation officers