November 2017

Governing by rankings

How the Global Open Data Index helps advance the open data agenda

Mária Žuffová, PhD candidate in Politics at the University of Strathclyde, Glasgow

Key findings

This paper studies how the Global Open Data Index (GODI) mobilises different audiences and translates into open data policy and publication. As such, it urges for a broader vision of ‘impact’ that traces the users of governance indicators first, before describing how indicators translate into different actions. The findings should be read as a depiction of different types of impact, yet without claiming representativity across different contexts.

Based on the interview sample, GODI drives change primarily from within government. Its international visibility and country rankings may incentivise and maintain high-level support for open data. In the absence of open data policies, GODI has been used to support commitment when combined with soft-policy tools. GODI also lays a baseline for agencies to improve and is used at multiple government levels to highlight progress around open data, despite non-comparability of results across years. GODI could help individual agencies and communicate more clearly what a good open dataset looks like.

Being easily communicable does not come without risks. There is evidence that GODI was confounded with broader open government policies and used as an argument to reduce investment in other aspects of open government. This effect may possibly be exacerbated by superficial media coverage that reports on the ranking without engaging with the broader context of country’s information policies. In order to understand civil society’s engagement with GODI, more research is needed first to classify and sample civil society actors and reconstruct ways of engaging with governance indicators carefully. Our findings suggest that individuals and organisations working around transparency and anti-corruption make little use of GODI due to a lack of detail, and a misalignment with their work tasks.

Recommendations

On the basis of our case studies and interviews for this report, we suggest that all key GODI users who want to advance open data policy and publication should:

  • Share best practice examples of open data publication that go beyond rankings. When discussing GODI results, they should aim for providing ‘a full picture’ and explain specifics of different country contexts, e.g. potential barriers to publication, such as a weak data infrastructure, conflicting copyright legislation etc., and inform about open data progress over time.

We recommend that civil society organisations and journalists who are interested in influencing open data publication should:

  • Use GODI as data source to access government data and investigate reasons as to why government producers do not provide data.
  • Access datasets that are linked by GODI and document how useful the data is for them. Some countries have feedback channels to comment on datasets, or a responsible department that can take up the request.
  • Identify and actively engage in open consultations and policy development run by national and regional governments.
  • Submit data requests via data portals or data request forms on websites. Depending on the legal environment governments are required to react to these requests within a certain timeframe.
  • Engage with potential data users and help them to gain the skills necessary to use the data assessed by GODI. Increased demand can be an incentive for the government to publish key datasets.

We recommend that government agencies in charge of national open data policies and publication should:

  • Support open data champions (who support open data) in government agencies and strengthen their capacities. This can include equipping an agency with technical capacity, experienced workforce, and ongoing learning opportunities.
  • Provide government agencies with diverse incentives to publish open data. These can include using departmental scores, rankings for high-level political support or to respond to financial needs of agencies.
  • Engage with data users and identify what open data, beyond data assessed by GODI, is in demand.

To strengthen positive effects of GODI, Open Knowledge International should:

  • Keep assessment criteria consistent over time to allow comparison.
  • Solicit insights blog posts from submitters and government officials to explain country contexts.
  • Award achievement by clearly pointing out best practices of government.
  • Create more opportunities for global interactions and knowledge sharing.

1. Introduction

A key driver of the rapid rise of the global governance indices has been their capacity to value or measure different phenomena with a common metric. By ‘commensurating’ things which otherwise would have different qualities, these phenomena can be categorised, making them countable, calculable, and comparable.1 This applies to all indices, no matter whether they measure inputs, processes, or outputs. Global governance indices have become popular mainly due to their straightforwardness, easy communicability, and credibility attributed to numbers.2 They enable a non-expert audience to compare performance in different areas across countries and over time easily.

Governments started to use governance indices in the 1980s and early 1990s, in the era of the new public management, and measured their performance more rigorously to achieve higher efficiency. Outside of government, diverse actors ranging from investment banks and aid agencies to civil society organisations use governance indices to influence policy or guide allocation of resources.3

The literature on the effects of governance indices has been extensive. We know that people and institutions respond to being evaluated in different ways.4 While indices might encourage governments to adopt better policies, provide better services and improve their performance, challenges prevail. Some argue that indices might create perverse incentives and motivate governments to focus on enhancing their ranks solely.5 There has been evidence in some countries that important open data events or visits by a donor have a similar effect. Governments published datasets suddenly over a short period before these events or visits, but do not update them regularly afterwards.6

Others like the Corruption Perception Index warn that indices might have severe consequences for the poorest countries if the funding is tied to performance in governance indices. They argue that the countries which are in greatest need of resources are usually the ones who score lowest on different governance indices. In their view, indices might contribute to widening inequalities and divides.7

Open data indices face similar challenges since they operate as means to track progress and inform investment decisions. It is therefore paramount to study the ways open data indices translate into open data policy and publication.

What’s behind the Global Open Data Index (GODI)?

GODI is a global assessment of open data publication. It is run by OKI covering 94 countries in its most recent version. The index scrutinises the existence of key government datasets and whether they meet criteria for open data such as machine-readability, accessibility and legal openness. Starting in 2013, GODI was designed to help open data campaigners make an evidence-based case for further open data publication. To make recommendations, they needed to know the current state of affairs; i.e. how much data had been published and in which areas.8 Likewise, GODI was designed to help governments measure their performance and identify roadblocks to better open data publication.

Since then GODI has served as a tool for national governments and civil society to follow developments in open data publication. In the latest 2016/17 edition, the index measured 1410 datasets in 94 countries. GODI provides both quantitative and qualitative data on open data availability. Concerning quantitative data, it gives a score for each assessed dataset and scores for all datasets create a country score, based on which the country is attributed a global ranking. It also provides qualitative data for each dataset score. These are accompanied by a short narrative including reviewers’ comments and metadata on data location, license and format.

GODI has been conducted openly and collaboratively. Anyone interested in the topic of open data can become a submitter and contribute to the index. The methodology and all related quantitative and qualitative data is openly available on the GODI website. OKI engages with different stakeholders from government and civil society to discuss submissions and rankings. That said, not all GODI users perceive OKI’s engagement and GODI process as such. There have been complaints about an opaque assessment process in the past.9

Our research approach

The aim of this research is to explain in what ways the GODI process mobilises support for open data in countries with different degrees of open data policy and publication, and how this support manifests itself (research question 1). Furthermore, it asks how different elements of GODI help governments and civil society actors to drive progress in open data policy and publication (research question 2).

To answer these questions, it is necessary to know who the users of GODI are, and how and why they use it. Based on OKI’s previous experience with GODI, we assume that governments, civil society, and media would represent different sets of users who each use GODI in various ways. To address the research questions, we will examine the relationship between GODI and the mobilisation of support for open data, which might have different forms based on who is using GODI. For each group identified above, we formulate the following hypotheses:

  • Hypothesis 1: GODI motivates governments to achieve a good rank.
  • Hypothesis 2: Governments use GODI to assess their performance in open data publication.
  • Hypothesis 3: Governments use GODI to implement open data policy reforms.
  • Hypothesis 4: Civil society uses GODI as evidence to support their work pushing for better open data policies and publication.
  • Hypothesis 5: Media uses GODI to inform about open data progress, and subsequently mobilises support for better open data policies and publication.

Argentina, United Kingdom and Ukraine were selected as country case studies for this report. The aim was to ensure reasonable geographical representation of countries with a different degree of open data publication and economic development, and political set-up. The information sources differ across hypotheses, ranging from media reports referring to GODI results to references to GODI in the policy documents, minutes of government meetings, Open Government Partnership (OGP) National Action Plans and political party manifestos. In areas where the evidence was not straightforward and easily accessible, it was complemented with 12 qualitative semi-structured online interviews with different stakeholders from government, civil society and the media.

These stakeholders are a convenience sample and include civil servants in national government agencies responsible for open data initiatives, civil society members engaged with government transparency or open government data and data journalists. Members of civil society were selected based on their assumed use of GODI and included anti-corruption organisations, open data activists, as well as data journalists. Throughout the sampling process we included other user groups such as academics using GODI for research purposes.

Follow-up research can benefit from consultations with a broader range of stakeholders as well as a larger sample of interviewees. For instance, interviews with members of civil society provide accounts of third parties using GODI and some context might be left out. While this report provides insights into a diverse range of practices and approaches towards GODI by different stakeholders, the findings cannot be generalised, as the context may significantly differ from one country to another. Eventually, this is also one of the conclusions of this research that while some of GODI effects apply globally, some are very country-specific.

  1. Wendy Nelson Espeland and Michael Stevens, ‘Rankings and reactivity: How public measures recreate social world,’ American Journal of Sociology, 113, no. 1 (2007).
  2. Kevin Davis, Angelina Fisher, Benedict Kingsbury, and Sally Engle Merry Governance by Indicators: Global power through quantification and rankings. Oxford University Press (2012).
  3. There has been evidence that the indices inform decisions of the banks, investment banks and asset managers, in particular when low-income countries are concerned. In the past, Transparency International acknowledged that the Corruption Perception Index (CPI), their well-known index measuring perceptions of corruption could have caused a fall-off in foreign direct investment in Bangladesh. Also, aid agencies use the governance indices to allocate its funding. Thus, it matters how the indices are used all the more, as they have become powerful tools influencing policies and allocation of resources. See Christiane Arndt and Charles Oman, Uses and Abuses of Governance Indices, OECD Development Centre, (2006), Hugh Williamson, “Hazards of Charting Corruption”, https://goo.gl/BAzbie, Craig Burnside and David Dollar, ‘Aid, Policies, and Growth: Aid, Policies, and Growth: Revisiting the Evidence’, accessed October 27, 2017.
  4. Donald T. Campbell, ‘Factors relevant to the validity of experiments in social settings’, Psychological Bulletin, 54, no. 4 (1957), 299.
  5. Bjørn Høyland, Karl Moene, and Fredrik Willumsen, ‘The tyranny of international index rankings’, Journal of Development Economics, 97, no. 1, (2012).
  6. Alon Peled and Jennifer Shkabatur, ‘Sustaining the open government data movement worldwide: Insights from developing countries’, CEDEM16.
  7. Staffan Andersson and Paul M. Heywood, ‘The Politics of Perception: Use and Abuse of Transparency International's Approach to Measuring Corruption’, Political Studies, 57, no. 4 (2009).
  8. Danny Lämmerhirt, Mor Rubinstein and Oscar Montiel, ‘The State of Open Government Data in 2017’, https://goo.gl/iHcz68.
  9. Danny Lämmerhirt (Open Knowledge International), online conversation, October 17, 2017, See Acknowledgments.

2. Argentina

From local innovation to national policy

In Argentina two avenues were taken to modernise government information systems, starting with initiatives on a local level that were replicated on a national level later on. For instance, the City of Bahía Blanca launched its open data portal in 2012.10 In the City of Buenos Aires, open government and open data have both become political priorities under the mayor Mauricio Macri. In his second term in office, he initiated the creation of the Ministry of Modernisation, Innovation and Technology to reform the public sector. Public officials in charge of open data have developed a strategy to find an open data champion within each ministry, and create for them opportunities for interaction to push the agenda forward. In March 2012 the City of Buenos Aires adopted the Decree no. 156/201211, which has created a regulatory framework for launching an open data portal.12

While on the local level initiatives around open data were top-down, at the national level the picture was a bit different. Open data had not represented a salient issue for the government until 2013 when Argentina joined the OGP, and one of the commitments of its first National Action Plan was to develop a national open data portal.13 Although the commitment was fulfilled and the portal14 was launched, the quantity and quality of datasets were low. At that time, a major move towards open data came from media and civil society, which had collected data in closed formats from the government and then provided it publicly as open data. La Nación, a leading national daily, initiated several open data projects filling the gaps for relevant information that government did not supply, such as data on the inflation trends or Senate’s expenses.15 However, developments changed substantially with a new administration.16 Our interviewees agreed that it was the lack of political will and absence of open data policy which impeded advancements in open data, and that things changed when open data became a political and policy priority.17 The national government has replicated the idea of a highly specialised Ministry of Modernisation and put it in charge of open data, e-government and innovation policies. Also, it has adopted the Open Data Decree no. 117/2016, which sets the basis for open data publication on the national level. While it requires all national agencies to create a plan for open data publication, it is a soft policy tool and can only give recommendations for government.18 There are no hard-wired sanction mechanisms in place to support implementing the decree.

These changes, both on the national and local level, were reflected in GODI’s indicators. Argentina ranked 17th in the latest GODI19 survey moving up almost 40 places a year. There have been significant improvements. Many datasets that were previously published in closed formats, such as procurement data, company register, and weather forecast data, switched to fully open formats. Key national statistics have improved too but lack open licenses. Land ownership data, location data and data on government spending remain problematic.

GODI as a stamp of achievement and incentive for reforms

Government and GODI

In Argentina, we found support for many of the proposed hypotheses on both local and national level. As a representative of the national Ministry of Modernisation told us, GODI complements the Open Data Decree no. 117/2016 by being a monitoring and sanction mechanism. Furthermore, using a ranking helps to mobilise high-level political support. For instance, Andrés Ibarra, the Minister of Modernisation, praised its team publicly on Twitter after Argentina ranked 17th this year.20 He also claimed earlier that he wants Argentina to be among the top ten countries.21 That said, his team at the Ministry emphasised that they also encourage publication of datasets beyond those assessed by GODI.

The national Ministry of Modernisation perceives GODI as a useful political tool for motivating or pressing other government agencies to publish open data. Agustín Benassi from the ministry stated that some agencies are more cooperative when they know that if they do not publish the datasets, they will score poorly in GODI, which is closely followed by high-level politicians. This suggests that GODI can motivate government agencies to seek a good rank. Benassi added that GODI plays a crucial role in shaping open data publication mainly in the initial stages, when open data legislation is absent in a country and publishing data in open formats is recommended, but not yet obligatory for agencies.22

Benassi argued that GODI also facilitates discussions within government. For instance, the publication of land ownership datasets in Argentina is a responsibility of individual local governments. Since there are more than three thousand local governments, it is “close to impossible to have comprehensive data on land ownership”, but this “red flag” in GODI allows the Ministry of Modernisation to have a dialogue with the national government agencies working with local or regional governments. The Ministry of Modernisation underlined that even though GODI’s land ownership data requirements might be too ambitious, they provide a basis for dialogue and address blockages for open data publication.

GODI also provides the government with a readily available diagnosis of open data availability. Alváro J. Herrero from the Buenos Aires City Government stated that they are using GODI to see where the government is doing well and where it needs to improve in open data publication. Natalia Sampietro from the Argentinian Ministry of Modernisation also confirmed that GODI represents for them a simple way of measuring and understanding where open data publication is getting better or worse. For this simplicity, the Ministry uses GODI as a communicational tool too.23

“The index provides an incentive for public officials to implement reforms. It is also a useful source of information. We do not need to hire a consultant to see how we are performing.”

Álvaro J. Herrero, Department of Strategic Planning and Institutional Quality, The Buenos Aires City Government

A representative of OKI’s local Argentinian chapter pointed out a tendency of high-level politicians in the country to sell a good ranking in GODI as a proof of the government’s transparency and good governance even though GODI does not capture any of these in particular.24 For instance, the Minister of Modernisation Andrés Ibarra claimed that “we [Argentina] went from being 54th to 17th in the index of transparency and open government, and we seek to be among the top ten”.25

Civil society and GODI

While local and national governments in Argentina use GODI to inform their decisions on data publication, the interviewed civil society actors - with the exception of the OKI Argentinian chapter - do not engage with GODI much. That said, some interviewees were knowledgeable about GODI and used it in their reports to describe open data developments in Argentina.26 They also proposed several suggestions to improve GODI. Silvana Fumega from the Latin American Open Data Initiative suggested that including a particular dataset in the assessment declares it is an important issue. Thus decision-making on the datasets which are assessed could be more inclusive and open.27

Media and GODI

When the results of GODI were published, the media reported on them but not widely.28 Likewise, media outlets covered the Open Data Index for Argentina’s cities. By using Google News search, we found four results for the keywords “Índice de datos abiertos” and “Argentina” and five results29 for the English equivalent “Open data index” and “Argentina”. In all cases, it was basic reporting about the GODI rankings. Florencia Coelho from La Nación stated that they covered the rankings but also prepared a more comprehensive story on Cordoba, which ranked the first in Argentina’s local open data index.30

Future scenarios

In Buenos Aires, there is a demand for a more institutionalised relation with Open Knowledge Argentinia. Herrero expressed an interest in holding meetings to discuss the findings, best practices from other countries and learn how they can improve. Other ways of engagement, such as awarding achievement, helping to connect with local governments in other countries, creating thematic working groups and providing toolkits on data publication were also mentioned as desirable actions.31

Interviewees working in the central government suggested that GODI could include additional datasets on education or health, and consider how to incorporate an assessment of the metadata provided and whether datasets can be accessed via an application programming interface (API). Also, they believed that an ongoing evaluation capturing progress in open data publication over time would be more useful than running the assessment once a year and with a limited timespan.32

  1. ‘Bahia Blanca – Open government’, https://goo.gl/P12fJU (in Spanish).
  2. Buenos Aires City Government, ‘Official bulletin’, https://goo.gl/LAHR5E (In Spanish).
  3. ‘Buenos Aires City Government open data portal’, https://data.buenosaires.gob.ar.
  4. Presidencia de la Nación, ‘OGP National Action Plan, https://goo.gl/s7Z6BD.
  5. ‘Central open data portal’, https://datos.gob.ar/.
  6. Mor Rubinstein, Josh Cowls and Corinne Cath, ‘OpenData.Innovation working paper’, https://goo.gl/cTnzW1. Florencia Coelho (La Nación), October 11, 2017, See acknowledgements. La Nación Data: Open Data Journalism for Change (2016), https://goo.gl/W149AR.
  7. Ines M. Pousadela, ‘IRM Progress Report 2015–16: Argentina’, Open Government Partnership, https://goo.gl/j9rB9e.
  8. Florencia Coelho (La Nación), Silvana Fumega (ILDA), October, 2017, See acknowledgements.
  9. Allende and Brea, ‘El Estado Argentino abre sus datos públicos’, https://goo.gl/hMxG74 (in Spanish).
  10. Open Knowledge International, ‘GODI in Argentina’, https://index.okfn.org/place/ar/.
  11. Andrés Ibarra (@andreshibarra), ‘Argentina ascendio’, Twitter, May 3, 2017, https://goo.gl/y4WdYH (in Spanish).
  12. La Nación, ‘Andrés Ibarra: La meta es estar entre los diez países más transparentes’, https://goo.gl/ZvTPoV (in Spanish).
  13. Agustín Benassi (The Ministry of Modernisation), online interview, October 25, 2017, See Acknowledgements.
  14. Natalia Sampietro (The Ministry of Modernisation), online interview, October 25, 2017, See Acknowledgements.
  15. Guido López Avakian (Fundacion Conocimiento Abierte), online interview, October 23, 2017, See Acknowledgements.
  16. Lucrecia Lacroze, ‘El Ministerio de Ambiente abrió sus datos para los ciudadanos’, La Nación, October 25, 2017, https://goo.gl/KRtrrX (In Spanish).
  17. Ibid.
  18. Silvana Fumega (ILDA), Online interview, October 4, 2017, See Acknowledgements.
  19. La Nación, ‘El país escaló 34 lugares en un ránking de transparencia’, May 4, 2017, https://goo.gl/q5rSKN (in Spanish).
  20. Lucía Martínez, ‘Índice de Datos Abiertos: en qué mejoró y en qué no la Argentina para subir en el ranking’, Chequeado, May 10, 2017, https://goo.gl/D6aWca (in Spanish)
  21. Florencia Coelho (La Nación), October 11, 2017, See acknowledgements.
  22. Álvaro J. Herrero (The Buenos Aires City Government), online interview, October 12, 2017, See Acknowledgements.
  23. Agustín Benassi and Natalia Sampietro (The Ministry of Modernisation), online interview, October 25, 2017, See Acknowledgements.

3. United Kingdom

Paving the way for open data initiatives globally

The UK has been one of the innovators and leaders in open data. It has continuously performed well in international open data rankings including GODI.33 It was also one of the founding members of the OGP and is involved in other important global transparency partnerships.

Better access to government information and more effective use of new technologies in the public sector have been prominent topics in the political discourse for more than a decade now. In 2007 the Labour government commissioned an independent review34 to explore new developments and opportunities for the use of public information. The review triggered an intense debate within and outside the government that led to many substantial improvements in the field.35 Most notably, it helped to create the concept of the central repository of government data. Data.gov.uk, the central open data portal, was launched in February 2010 and contains over 42 000 datasets to this date. In 2010, open data had become a part of the electoral campaign too. David Cameron pledged that his government would publish an “unprecedented amount of government data”36 and he continued to push the open data agenda forward once he took office.

In 2011, the government created the Government Digital Service and tasked it with transforming the provision of government digital services. In 2012, the Cabinet Office presented the Open Data White Paper37, which reaffirmed government’s commitment to transparency. The government launched several advisory bodies, such as the Public Sector Transparency Board, Open Data User Group and Data Strategy Board, whose primary roles were to drive open data publication forward and advise what datasets should be prioritised. It also co-financed the Open Data Institute, which along fulfilling other roles, supports open data startups.38

Nonetheless, open data lost its momentum in 2015 as the government’s attention turned to the Brexit referendum and later on to the Brexit negotiations. Many of the open data advisory bodies ceased to exist or merged with others.39 Although these developments suggest that open data has moved down on the UK government’s list of priorities, there have still been many improvements in open data in different areas as the GODI demonstrates.40

GODI is a weak mechanism for shifting policies

Government and GODI

Regarding the government’s use of GODI, we have found support for several of our hypotheses. First, the national government pays attention to GODI. The rankings are important, and once they are launched, they are discussed on the government’s blog.41 Civil servants interviewed for this research claimed to use GODI and other indices in internal documents. They also discussed that an agency might use GODI as evidence that it should publish open data to inform an internal business case.42

High-level politicians and civil servants have often referred to GODI and other open data indices, in particular, the rankings in their speeches.43 Francis Maude, Paymaster General and Minister for the Cabinet Office in the 2010 to 2015 Conservative and Liberal Democrat coalition government, often used GODI and World Wide Web Foundation’s Open Data Barometer (ODB) to emphasise the progress the UK has made in open data publication. Interestingly, once the UK lost its primacy in GODI, the government preferred to refer to ODB only where the UK has still been the number one.44 This suggests a possible cannibalisation of indices which have a similar focus but slightly different methodologies.

Both GODI and ODB were also misused to make a case for redundancy of freedom of information (FOI) legislation.45 In 2015 the government set up the Independent Commission on FOI to review the FOI Act.46 In the end, the commission concluded that “there is no evidence that the Act needs to be radically altered, or that the right of access to information needs to be restricted”. However, the political discourse before setting up the Commission suggested that FOI should be made redundant based on the UK’s open data leadership and proactive data publication. This time the misuse of GODI and ODB did not have any consequences for FOI legislation, presumably thanks to the strong civil society’s opposition to any restricting amendments of the law.47 Nonetheless, it is important to acknowledge that GODI and other indices might be used to justify resource allocation or policy changes that do not advance open government more broadly.

GODI also has positive effects on open data publication. Our interviewees representing both government and civil society were reasonably confident that GODI triggered some improvements. For instance, the Environment Agency, which is responsible for the water data published in England, created a new repository for water quality data in open formats.48 As a result, the GODI score for water quality data went from 10% in 2015 to 85% in 2016.49

Our interviewees confirmed that they also use GODI “to measure performance […] and identify potential issues that need to be addressed”.50 GODI helps inform their decisions. They mentioned that it also is a useful mechanism to motivate government agencies which publish open data: “We can say that we are a world leader in open data and work with other departments to understand how we maintain that position”.51

On the other hand, GODI is not a strong enough mechanism to trigger more significant changes in some areas. For instance, in 2015, OKI’s assessment criteria changed for the election results data and included a condition that all data should be reported at the level of the polling station. As a result, the UK, which provides less granular data (on constituency level only), went from 100% score in 2014 to 0% score in 2015. However, GODI has not created significant pressure for the Cabinet Office to change the legislation.52 Similarly, a significant change towards greater openness in land ownership data is not expected, as this data generates government revenue at the moment.53

Civil society and GODI

Owen Boswarva, an open data campaigner interviewed for this research, claimed to make limited use of GODI data. He said that he is campaigning for open data on the departmental level mostly because open data activities are more intense on the departmental level now, in particular in the area of transport, agriculture and environment. In his view, open data is not a priority issue for the United Kingdom’s national government at the moment.54 He argued that the latest alterations of GODI have created animosities when countries “have not done any worse, but their score changed significantly”. Also, these modifications restrict opportunities to compare data categories across years. He also suggested that more submissions could be allowed per each category as crowdsourcing might decrease a possibility of submitting erroneous information.55

Civil society organisations specialised in governance, transparency and innovation, use GODI and other open data indices as a reference to set the landscape when describing open data.56 Some institutions also use GODI as a sub-index for their own indices. For instance, the Institute for the Government and the Blavatnik School of Government at the University of Oxford recently launched their Index measuring the effectiveness of international civil services, which uses GODI as one of the measures of government openness.57 The UCL Energy Institute58 was also considering using GODI for constructing its MaaS Maturity Index.59 However, they decided not to use it eventually, as the open data definition presented by GODI did not match their needs because “it does not cover sharing data through APIs and making software open source”.60

All in all, our findings suggest that GODI had some positive effects on open data publication in the UK. Yet it may also have had unintended adverse consequences on other open government policies - FOI legislation in particular. To avoid this in the future, OKI should disapprove such misuse of GODI results immediately and better explain the context of its results to the media and the public.

Media and GODI

To track the use of GODI by the British media, we examined various sources. We used Google’s search engine for news stories and LexisNexis, an online research database of legal documents and news coverage. Using the Google search engine for news coverage, the keywords “Open Data Index” and “UK” appeared in 29 instances and the keywords “Open Data Index” and “United Kingdom” appeared in 15 instances. Most times, the media informed about the results when these were launched and discussed rankings. The reporting “does not dig into details; it tends to be superficial”, as one of our interviewees noted.61 The analysis of the texts suggests that journalists tend to reproduce misconceptions of GODI that high-level politicians and civil servants put forward, and confuse an excellent ranking in GODI with government transparency and openness.62

Using LexisNexis, GODI came up in six unique articles. Again, GODI was used in the reporting to provide information how much open data is being released in different countries,63 but also authors of the commentaries used GODI to advocate for opening more data,64 higher user engagement65 and closing the data divide.66

  1. https://index.okfn.org/, http://opendatabarometer.org/, http://www.oecd.org/gov/digital-government/open-government-data.htm and http://odin.opendatawatch.com/
  2. Ed Mayo and Tom Steinberg, ‘The Power of Information: An independent review’, https://goo.gl/Epgkfm.
  3. Cabinet Office, ‘The Government’s Response to The Power of Information: An independent review by Ed Mayo and Tom Steinberg (2007)’, https://goo.gl/ivk7ug.
  4. The Conservative Party, ‘2010 Manifesto’, https://goo.gl/tzctmX.
  5. Cabinet Office, ‘Open Data White Paper’, https://goo.gl/gPz9bZ.
  6. https://theodi.org
  7. The role of the Public Sector Transparency Board became part of the Data Steering Group in November 2015. The Open Data User Group discontinued its activities in 2015. The Data Strategy Board has become a part of the Department for Business, Innovation and Skills.
  8. Open Knowledge International, ‘GODI in the UK’, https://index.okfn.org/place/gb/.
  9. Gov.uk, ‘Open Data - the race to the top’, https://goo.gl/MmQ6Fs.
  10. Civil servant who wished to retain anonymity, Online interview, See acknowledgements.
  11. Gov.uk, ‘Open Data Institute summit 2015: Matt Hancock speech’, https://goo.gl/fyDsVh
  12. Gov.uk, ‘Open data: Matt Hancock speech at GeoPlace conference 2016’ https://goo.gl/w313rk
  13. Gov.uk, ‘Francis Maude’s speech on open data and transparency’, https://goo.gl/hwcB3N.
  14. Gov.uk, ‘Independent Commission on Freedom of Information report’, https://goo.gl/7Sf1Kj.
  15. Ellen Broad, ‘Appeal to the FOI Commission: don’t confuse open data with FOI’, Open Data Institute, August 5, 2015, https://goo.gl/VvvaHh.
  16. Environment agency, ‘Water quality data archive’, https://goo.gl/G8ekzQ.
  17. Owen Boswarva (open data campaigner), Online interview, October 5, 2017, See Acknowledgements.
  18. Civil servants who wished to retain anonymity, Online interview, See acknowledgements.
  19. Ibid.
  20. ‘Open Government National Action Plan 2016-18 - Final network comments’, https://goo.gl/gT73J9.
  21. Owen Boswarva (open data campaigner), Online interview, October 5, 2017, See Acknowledgements.
  22. Ibid.
  23. Ibid.
  24. Mor Rubinstein, Josh Cowls and Corinne Cath, ‘OpenData.Innovation working paper’ (2015), https://goo.gl/cTnzW1.
  25. ‘The International Civil Service Effectiveness (InCiSE) Index’, https://goo.gl/TnoCEp.
  26. Maria Kamargianni (MaaSLab, UCL Energy Institute), E-mail conversation, October 26, 2017, See Acknowledgments.
  27. https://goo.gl/eSMcKg
  28. Maria Kamargianni and Richard Goulding, ‘The Mobility as a Service Maturity Index: Preparing the Cities for the Mobility as a Service Era’, (2017).
  29. Owen Boswarva (open data campaigner), Online interview, October 5, 2017, See Acknowledgements.
  30. Ralph Jennings, ‘How Taiwan Fostered The World's Most Open Government’, Forbes, December 15, 2015, https://goo.gl/94NoaC.
  31. Marc Ambasna-Jones, ‘Can data rebuild investors' confidence in Spanish property?’ The Guardian, January 19, 2016, https://goo.gl/ncsk6c
  32. Paul Ormerod, ‘Britain leads the open data revolution – but sloppy economists must keep up’, City A.M., July 22, 2014, https://goo.gl/q28cEU
  33. Jonathan Stoneman, ‘There is no shortage of open data. The question is, is anyone using it’? The Computer Weekly, January 2017, https://goo.gl/MuwF7s
  34. Michael Green, ‘We must end the world's data divide’, The Guardian, November 1, 2013, https://goo.gl/DsNLaa

4. Ukraine

The Euromaidan revolution as a catalyst of open data policies

In November 2013, Viktor Yanukovych government’s failure to sign a long-awaited trade-association agreement with the EU triggered the Euromaidan revolution and created a window of opportunity for substantial reforms.67 Since there was a public demand for government transparency and accountability, open data became one of the government’s priorities. All interviewees agreed that the current reforms widening access to government information would not be in place if not for the revolution.68

Although Ukraine has been an OGP member since 2011, the implementation of commitments was behind schedule in 2013, and while some commitments dealt with access to information, none were focused on opening government datasets. The post-revolution developments have affected Ukrainian participation in OGP too. In the second national action plan, the government pledged to develop a legal framework for public access to government information in open data formats.69 The legislation and other relevant regulations were adopted in 2015 and set guidelines for public agencies to follow open data standards.70 A central open data portal was launched too, and the State Agency of E-governance was mandated to manage it and coordinate the open data agenda.71 Since then many interesting open data initiatives have emerged. Some have seen collaboration between all the sectors (public, private and civil society) with the most prominent example being ProZorro, an award-winning e-procurement platform.72 Open data has become a topic for several ministries and public agencies, as well as for local governments. For instance, the Ministry of Finance has developed the platform E-Data making national spending data available in open formats. Budget data is planned to be published in the near future.73 The Ministry of Justice has recently launched an open beneficial ownership register, in partnership with OpenOwnership. In open data circles, these public agencies are informally known as open data champions.74

All these positive developments were reflected in the latest GODI results too. Ukraine jumped from 54th place in 2015 to 31st in 2016/17.75 However, interviewees noted that quality of data is still problematic.76 Kateryna Onyiliogwu from the Eurasia Foundation argued that many government agencies have poor data infrastructures. She stated that some agencies collect data by hand and are not in a position to improve their ranking anytime soon, due to limited human, technical and financial resources. Thus, the standards set by GODI are too high for some agencies.77

GODI is relevant for government, less so for civil society

Government and GODI

All interviewees confirmed that open data publication, along with e-government, is a political priority in Ukraine these days.78 They also agreed that the government is paying attention to GODI and other open data indices. In particular, the Prime Minister Volodymyr Groysman strongly supports open data. He routinely informed on his social media when Ukraine ranked well in both GODI and ODB. But the support for open data varies greatly among ministries and is not always motivated by GODI. For instance, the Ministry of Economy and Trade is collaborating in many important open data projects but tends to refer to other indices rather than GODI.

On the other hand, other agencies - for example the Ministry of Finance - uses GODI in its work. Oleksander Shchelokov, the manager of their E-Data platform, stated that open data indices help them to track which sectors of the economy remain closed to the public, and which have opened their datasets. He stated that they “actively monitor open data indices and work to improve the position of Ukraine substantially in next two years”. The Ministry of Finance has created working groups to ensure that requirements on datasets are met. He said that GODI and ODB represent international standards, acknowledged by a wide international community. However, he argued that legislation is a key driver for open data. The creation of the E-Data platform has also reacted to the need to comply with the new legislation on the open use of public funds.79

Onyiliogwu argued that GODI and other open data indices play an essential role in keeping the open data agenda alive politically. In her view, if Ukraine gets visibility internationally through GODI or ODB, political commitment to open data is likely to last longer. Therefore, for an organisation like hers which supports open data initiatives in the country, it is essential to feed the discussion on the top level.80

Civil society and GODI

Regarding civil society, there was no support for the proposed hypothesis. Like in Argentina and the UK, civil society actors interviewed for this report make a limited use of GODI. Ievgen Bilyk from Transparency International (TI) Ukraine could not recall the organisation referring to GODI in its reports. GODI seems to have limited added value and is not detailed enough for TI Ukraine’s needs. For instance, regarding procurement data, TI Ukraine needs to evaluate the legal framework aside the publication of open data and for that, it uses a specific index on procurement data.81 Onyiliogwu also mentioned that civil society actors, mainly those outside of the country’s capital, lack the necessary skills to harness benefits coming from open data publication.

Media and GODI

Before the Euromaidan revolution, open data was not a salient topic in Ukraine and thus, has not made many headlines either. However, the latest success in GODI and other open data indices caught media attention. Using Google search engine for news coverage, the keywords “Open Data Index” and “Ukraine” appeared in 11 instances. Ukrainian equivalents of these keywords “рейтингу відкритості державних даних” and “Україна” appeared in 43 instances. The GODI results were covered in respected media, such as Ukrayinska Pravda but also on Texty.org.ua, a popular data journalism platform.82 News reports primarily discussed Ukraine’s country ranking. According to Onyiliogwu, “open data is not necessarily a topic for journalists. Also, when they are reporting about GODI, they are not going deeper, they do not scrutinise the reasons behind the ranking”.83

Other interviews suggest that data journalists might use GODI differently. Andriy Gazin, a local data journalist, the head of Textura.in.ua, previously working for Texty.org.ua, mentioned that he has used GODI as a data source to find specific data for his writing. He sometimes still uses it during his data journalism training to teach journalists where government agencies publish datasets. “Lately, I conducted training in Romania, and I needed data to work with, and I used the index to look for government datasets in Romania”, he said during our interview. He also mentions GODI when training open data. He uses GODI as an example of measuring the progress in open data publication and datasets categories as an example of high priority datasets, as he considers it to be a good choice of datasets.84

  1. Joshua Yaffa, ‘Reforming Ukraine after the revolutions’, The New Yorker, September 5, 2016, https://goo.gl/rBkxxZ.
  2. Andriy Gazin, Ievgen Bilyk, Kateryna Onyiliogwu, Online interviews, October 2017, See Acknowledgements.
  3. Dmytro Kotliar, ‘IRM Progress Report 2014–15: Ukraine’, Open Government Partnership, https://goo.gl/sYbM1J.
  4. The National Council of Ukraine, accessed October 27, 2017, https://goo.gl/LgNjCJ (in Ukrainian).
  5. ‘Open data portal in Ukraine’, accessed October 28, 2017, http://data.gov.ua/.
  6. https://prozorro.gov.ua
  7. https://spending.gov.ua
  8. Andriy Gazin (Textura.in.ua), Online interview, October 16, 2017, See Acknowledgements.
  9. Open Knowledge International, ‘GODI in Ukraine’, accessed October 27, 2017, https://index.okfn.org/place/ua/.
  10. Andriy Gazin, Ievgen Bilyk, Kateryna Onyiliogwu, Online interviews, October 2017, See Acknowledgements.
  11. Kateryna Onyiliogwu (Eurasia Foundation), Online interview, October 5, 2017, See Acknowledgements.
  12. Andriy Gazin, Ievgen Bilyk, Kateryna Onyiliogwu, Online interviews, October 2017, See Acknowledgements.
  13. Oleksandr Shchelokov (The Ministry of Finance), E-mail conversation, October 27, 2017, See Acknowledgments.
  14. Kateryna Onyiliogwu (Eurasia Foundation), Online interview, October 5, 2017, See Acknowledgements.
  15. Institute for Development of Freedom of Information, ‘Transparent Public Procurement Rating’, https://goo.gl/pBd9Dk
  16. ‘Ukraine rose in the ranking of openness of government data’, Ukrayinska Pravda, May 3, 2017, https://goo.gl/BRgdkM. ‘Ukraine has climbed up 30 places in GODI, catching up with Austria’, Texty.org.ua, May 3, 2017, https://goo.gl/KJYcCn. ‘Ukraine climbs 30 positions up in GODI’, KyivPost, May 3, 2017, https://goo.gl/vaYzBb.
  17. Kateryna Onyiliogwu (Eurasia Foundation), Online interview, October 5, 2017, See Acknowledgements.
  18. Andriy Gazin (Textura.in.ua), Online interview, October 16, 2017, See Acknowledgements.

5. Conclusions

The goal of this research was twofold: understanding mobilisation processes of different user groups to support open data, as well as examining how elements of GODI help support open data policy development and publication. Based on our interviews, we have found mixed support for our hypotheses. Governments use GODI in different ways, for instance, to seek a good rank, measure performance and encourage improvements in open data publication. Nonetheless, there might be the risk that a positive GODI ranking is conflating open data publication with broader concepts such as open government, good governance and transparency.

Based on our interview sample, there is a lack of evidence that civil society uses GODI in its advocacy work. Nonetheless, interviewed civil society actors were knowledgeable about GODI. They argued that standards set by GODI might be too high for some countries, and too low for others, and suggested to include more contextual information when presenting the rankings.

And while the media sometimes uses GODI to set the scene when discussing open data issues, their reporting is often heavily focused on the country ranking without discussing individual dataset scores and without providing further background on policy or data quality. There is an absence of in-depth reporting on open data that would dig deeper into reasons why a particular country achieved good or bad results. Our analysis suggests that journalists may reproduce politicians’ misconceptions of open data and present countries that rank well in GODI as the most transparent. This observation can benefit from more in-depth research. More detailed findings are provided below:

Assumption Who uses GODI? What part of GODI? To do what?
H 1: GODI motivates governments to achieve a good rank.
  • Politicians and high-level civil servants
  • Country ranking/ league table
  • Build reputation at home
  • Get international visibility
  • Keep open data a political priority
  • Open-washing and conflation with the wider open government agenda
H 2: Governments use GODI to assess performance in open data publication.
  • Civil servants implementing open data
  • Dataset scores
  • Qualitative information for each dataset score
  • Measure progress despite non-comparability of results across years
  • Create a basis for improvement, as GODI requirements on datasets set standards
  • Help to identify and address blockages, including lack of technical expertise
H 3: Governments use GODI to implement open data policy reforms.
  • Politicians and high-level civil servants
  • Datasets in GODI
  • Country ranking
  • Encourage/pressure open data publication which can spark the development of new policies.
  • Use GODI as soft-policy tool to pressure open data publication in the absence of open data legislation
H 4: Civil society uses GODI to advocate for better open data policies.

Little evidence based on our interviewee sample. Lack of detail was one of the most commonly cited reasons for not using GODI.

Some civil society actors, including individuals and CSOs supporting freedom of information, anti-corruption or transparency mentioned that GODI does not provide additional information and value for their work, which is often narrowly specialised. On the other hand, academics show interest to GODI to develop new governance indicators. All interviewees from civil society were aware of GODI.

H 5: Media uses GODI to inform about open data progress.
  • Media
  • Data journalists
  • Rankings
  • Datasets scores
  • Qualitative information
  • Inform about results
  • Teach journalists about government data publication schemes

Acknowledgements

I would like to thank Danny Lämmerhirt from Open Knowledge International (OKI) for the guidance and useful feedback that I have received for multiple drafts of this report. I am also very grateful to the following people who have invested their time and agreed to be interviewed. Many of them have provided an insightful feedback to the draft of this report:

  • Guido López Avakian, Open Knowledge Foundation Argentina
  • Agustín Benassi, Ministry of Modernisation of the Argentine Republic
  • Ievgen Bilyk, Transparency International Ukraine
  • Owen Boswarva, open data campaigner
  • Florencia Coelho, LNData, La Nación
  • Silvana Fumega, Latin American Open Data Initiative
  • Andriy Gazin, Textura.in.ua
  • Álvaro J. Herrero, Department of Strategic Planning and Institutional Quality, Buenos Aires City Government
  • Maria Kamargianni, MaaSLab, UCL Energy Institute, UK
  • Kateryna Onyiliogwu, Open Data Team Lead at the Eurasia Foundation
  • Natalia Sampietro, Ministry of Modernisation of the Argentina Republic
  • Oleksandr Shchelokov, E-Data project, Ministry of Finance, Ukraine

This report was made possible thanks to a grant from the William and Flora Hewlett Foundation.

Reference and further reading

Alon Peled and Jennifer Shkabatur, ‘Sustaining the open government data movement worldwide: Insights from developing countries’, CEDEM16.

Bjørn Høyland, Karl Moene, and Fredrik Willumsen, ‘The tyranny of international index rankings’, Journal of Development Economics, 97, no. 1, (2012).

Cabinet Office, ‘Open Data White Paper’, https://goo.gl/gPz9bZ.

Cabinet Office, ‘The Government’s Response to The Power of Information: An independent review by Ed Mayo and Tom Steinberg (2007)’, https://goo.gl/ivk7ug.

Christiane Arndt and Charles Oman, Uses and Abuses of Governance Indices, (OECD Development Centre, 2006).

Craig Burnside and David Dollar, ‘Aid, Policies, and Growth: Aid, Policies, and Growth: Revisiting the Evidence’, accessed October 27, 2017.

Danny Lämmerhirt, Mor Rubinstein and Oscar Montiel, ‘The State of Open Government Data in 2017’, https://goo.gl/iHcz68.

Dmytro Kotliar, ‘IRM Progress Report 2014–15: Ukraine’, Open Government Partnership, https://goo.gl/sYbM1J

Donald T. Campbell, ‘Factors relevant to the validity of experiments in social settings’, Psychological Bulletin, 54, no. 4 (1957), 299.

Dr. Foster, ‘Uses and abuses of performance data in healthcare’, https://goo.gl/nftUCG.

Ed Mayo and Tom Steinberg, ‘The Power of Information: An independent review’, https://goo.gl/Epgkfm.

Jonathan Stoneman, ‘There is no shortage of open data. The question is, is anyone using it’? The Computer Weekly, January 2017, https://goo.gl/MuwF7s.

Joshua Yaffa, ‘Reforming Ukraine after the revolutions’, The New Yorker, September 5, 2016, https://goo.gl/rBkxxZ.

Kevin Davis, Angelina Fisher, Benedict Kingsbury, and Sally Engle Merry, ‘Governance by Indicators: Global power through quantification and rankings’, Oxford: Oxford University Press, 2012.

Maria Kamargianni and Richard Goulding, ‘The Mobility as a Service Maturity Index: Preparing the Cities for the Mobility as a Service Era’. Paper accepted for presentation at the 7th Transport Research Arena TRA 2018, April 16-19, 2018, Vienna, Austria.

Michael Power, ‘From risk society to audit society’, Soziale Systeme 3, no. 1, (1997).

Mor Rubinstein, Josh Cowls and Corinne Cath, ‘OpenData.Innovation working paper’, https://goo.gl/cTnzW1.

Open Knowledge International. (2015). Global Open Data Index. Available at: http://index.okfn.org/

Staffan Andersson and Paul M. Heywood, ‘The Politics of Perception: Use and Abuse of Transparency International's Approach to Measuring Corruption’, Political Studies, 57, no. 4 (2009).

The Conservative Party, ‘2010 Manifesto’, https://goo.gl/tzctmX.

Theodore M. Porter. Trust in numbers: The Pursuit of Objectivity in Science and Public Life. Princeton University Press, 1995.

Wendy Nelson Espeland and Michael Stevens, ‘Rankings and reactivity: How public measures recreate social world,’ American Journal of Sociology, 113, no. 1 (2007).