5. Principles 13–17: On The Openness Process
The above twelve principles essentially define open government data in its ideal form, but more can be said about the process of opening up government data. How should government agencies decide what to open and how to do it?
13. Public input: The public is in the best position to determine what information technologies will be best suited for the applications the public intends to create for itself. Public input is therefore crucial to disseminating information in such a way that it has value. As the Association of Government Accountants’ principles154. Association of Government Accountants. July 2009. Recovery and the Transparency Initiative (Annual CFO Survey). state, “Understand the information that people want, and deliver it. They may not be sure what they need, so help them define it.”
14. Public review: The Association of Government Accountants’ principles also note that not only should the data itself be open, but the process of creating the data should also be transparent: “Have a process for ensuring that data you disclose are accurate and reliable, and show that process to users.”
15. Interagency coordination: Interoperability makes data more valuable by making it easier to derive new uses from combinations of data. To the extent two data sets refer to the same kinds of things, the creators of the data sets should strive to make them interoperable. This may mean developing a shared data standard, or adopting an existing standard, possibly through coordination within government across agencies. The use of open data formats often, but not always, entails interoperability. However, we recognize that interoperability can come at a cost. Governments must weigh the advantages of distributing non-interoperable data quickly against the net gain of investing in interoperability and delaying a release of the data.
16. Technological choices can be a type of endorsement. Endorsements of technology created or controlled by the private sector can create a conflict of interest when regulating that sector, and creates an incentive for endorsed corporations to be involved in policymaking. Other things being equal, technological choices should be avoided that essentially endorse a single entity.
17. Prioritization. Although the principles of open government data are written in the form of a definition, it would be a mistake to treat openness as a binary value. Government agencies have limited resources with which to prepare data for public consumption, and it would be impossible for all government data to meet these standards of openness instantaneously and simultaneously. Cataloging, documenting, preparing an infrastructure for distributing updates, and in some cases reviewing and redacting all require significant effort, and prioritization. A timely release of data might outweigh the desire for greater precision in the data, for instance the difference between a scanned image versus a database. In our world of limited resources, an incremental plan for achieving best practices is needed.
In Government Data and the Invisible Hand (Robinson et al 2009), a controversial point was made:
[T]o embrace the potential of Internet-enabled government transparency, [one] should follow a counter-intuitive but ultimately compelling strategy: reduce the federal role in presenting important government information to citizens. Today, government bodies consider their own websites to be a higher priority than technical infrastructures that open up their data for others to use. We argue that this understanding is a mistake. It would be preferable for government to understand providing reusable data, rather than providing websites, as the core of its online publishing responsibility.155. Robinson, David G., Harlan Yu, William P. Zeller, and Edward W. Felten. 2009. Government data and the invisible hand. Yale Journal of Law & Technology, 11. p160.
Governments are limited in what they should do, and in how fast they can do it, but the private sector — which has an established role in promoting civic engagement — can move quickly into applications governments won’t. Robinson et al’s point was not that a government agency should not provide services over the Internet, only that the agency should first publish its data to encourage innovation in the private sector, which might develop services better than those the agency itself could.
The infrastructure that the agency creates to publish to the private sector can and should be used by the agency itself to access its own public information. It is a common practice in technology companies to use your own software in order to make sure it works:
Such a rule incentivizes government bodies to keep this infrastructure in good working order, and ensures that private parties will have no less an opportunity to use public data than the government itself does.156. ibid
These ideas naturally lead to the following order of priorities:
Government agencies should first establish a basic public-facing website to meet critical and mandated needs to service the public.
The agency’s policy regarding open data and web best practices should be established, for instance by the agency’s Chief Information Officer, in consultation with the public.
Comprehensive bulk data access to public records maintained by the agency should be made available with the target audience as mediators, such as journalists, researchers, and technologists.
The agency website should then be expanded to include non-critical functionality, such as advanced search capabilities, and should rely on the same technical infrastructure created in the last step for its own access to its public records.
The last priority is for the agency to develop APIs and web services, which allow for third-parties to automatedly search, retrieve, or submit information without first acquiring the bulk data.
Whether or not these priorities are sensible in any particular instance will depend on the resources and expertise available to the agency.