Guest post by Ruth Salmon, Research Assistant

Joining the data collection team for the 2014 Aid Transparency Index (ATI) has been an eye opening experience. The thing that struck me most is how the volume of data available for an organisation does not necessarily correlate with usability. As Publish What You Fund has been saying for a while now, more data is not necessarily better data.

I’m responsible for manually collecting and scoring individual organisations’ data – so whatever aid information is not published to the IATI Registry. This is no easy task, and I have to go through many steps:

First, I locate the main sources of an organisation’s project information. Project information comes in many shapes and forms; ranging from complex searchable databases to a single webpage listing projects. I familiarise myself with the organisations website, and compare what I can find this year to what was available last year. It’s exciting to see which organisations have made massive leaps forward, trying innovative approaches to present their data in more accessible ways – interactive maps using geo-coding, complex search filters, info graphics. And yet this is often matched by frustration as I find fancy databases with incomplete and fragmented data, and a lack of open licences or means of export to allow data to be reused freely.

Then I try to find organisation-level documents. These include the organisation’s strategy, annual report, allocation policy, procurement policy, budgets and audits. I check if they are published in regular cycles and contain the relevant detail. The main issue here has been language. Many smaller organisations have no English versions available, so I am left trying to run PDFs through Google translate with varying levels of success. Whilst I recognise that it may not be a high priority to publish this information in English (France’s strategy for engaging with Francophone Africa is more probably most useful in French, for example); the preference for PDFs highlights a lack of understanding of facilitating access for different users with different needs.

Next, I verify what activity-level information is available. I do this by picking five projects at random from those implemented in the country which receives the most money from the organisation, and look for the data available for the 28 activity-level indicators. These include information such as: project title, objectives and description, total and dis-aggregated budgets, dates, location and results. The organisations can only score for an indicator if the information is available for all five activities. I also look at the format the information is published in. PDF scores lowest as it is harder to extract information, followed by website, with machine readable data (Excel/CSV) scoring highest.

Again, my findings have been both encouraging and at times disheartening. The best examples have all the information for a specific activity in one place, with links to information across the lifetime of the project. Its easy to build up a picture of what activities the organisation is implementing, where, when and for whom. But for others the information is so fragmented it would take Sherlock on Red Bull to piece it together. An activity page may contain the summary information, results are found in a separate database, financial information is only found on a central government website. Even if the information for all activities is published, I doubt anyone would have the time or patience to try and join the dots.

Finally, I review and share findings. I first review my findings with my colleagues, to ensure consistency across organisations. Then we share the assessment with the organisation and request feedback. The feedback is reviewed and any necessary changes to the scores are made. This ensures that we haven’t missed any information that could score, which can happen, especially when  the information is in several different locations. The changes and decisions that come out of these conversations with organisations are exciting, and allow us to tackle the challenges the organisation faces with making its information more transparent as well as any misunderstandings about IATI. Lastly we send the survey to an independent reviewer, and make any necessary changes based on their comments.

All this just to establish what is or isn’t published for one organisation!

This highlights the importance of IATI  for presenting information in a comparable format which has all the data in one place, is easy to update and free to access, use and reuse.

Does IATI render an organisation’s website irrelevant then? By no means. The IATI Registry does not store data, so websites are still needed for this, and for turning raw XML data into useful, narrative text that is accessible to those of us that aren’t data experts. Instead, future websites could be IATI driven, as with the new Dutch and Swedish open data portals, soon to go live.

After reviewing the  information currently available for 68 of the world’s largest and most influential donors, I am convinced that if aid data is going to be truly useful  it should be published in a standard format so that it can be used by, civil society in both donor and partner countries, partner country governments, not forgetting the donors themselves.

Ruth Salmon is Publish What You Fund’s brand new Research Assistant. She is the lead on data collection for the 2014 Aid Transparency Index.