What’s Under the Bonnet … Getting into our Index Methodology
How do you measure international aid transparency?
We have gained rather a lot of experience developing, improving and testing a robust methodology for our Aid Transparency Index. So as we begin work on the 2020 Index, we have decided to opt for only a few minimal changes to our methodology, as set out in our new Technical Paper. These include:
- Updating the Capital Spend indicator to include testing for the OECD-DAC voluntary sector codes, for better budget alignment. This indicator has been re-named “Budget alignment”.
- Some minor changes to four indicators tests: project budget documentation, sub-national location and pre-project impact appraisals to exclude non-relevant activity status or project / aid types.
- We will no longer be reviewing data published in International Aid Transparency Initiative (IATI) version 1.0x since this version has now been deprecated by IATI.
The donor consultation
The last few months have seen us assessing the major donors against our criteria for inclusion in the Index to finalise the new 2020 donors list. Since the decisions about who to include, or not, were made, we turned our minds to the data collection itself and how we go about testing donors’ data for quality and consistency. While we are using the 2018 methodology again this time around, we do need to keep any technological and IATI standard changes in mind.
To do this we held an online consultation in September on the tests we run to score donors’ IATI data. This allowed any interested donors and organisations to comment on how we measure their transparency data and ensured that the tests we use are sense-checked against those who we are actually measuring. The consultation and organisations’ responses are openly available online.
The only substantive change to the tests comes in the use of the budget-aligned sector codes. In 2017, the IATI Technical Advisory Group recommended the use of the voluntary sector codes. These are a set of more detailed sector codes describing aid spending areas in order to make aid budgets more useful, and accessible, for recipient countries. Previous work on why this matters has been done by Aid on Budget with their own budget mapping initiatives. We are excited to support this work by testing across the selected donors how well these codes are now being used. Specifically, the codes provide more granular detail on what aid spending takes place and are in line with partner country classifications, making it possible to automatically map donor data against partner country budgets. For instance, distinguishing between non-formal youth spending or formal education spending in primary education. Or, in general health spending distinguishing between administrative spending and medical research.
Other minor changes have been made to the general tests to tighten them up and ensure they remain relevant.
Still want to know more about our methodology? Get the full updated 2020 Technical Paper by clicking the link below: