◀ Open Government Data: The Book

Transparency, Participation, and Collaboration

By Joshua Tauberer. August 2014.

The 2009 Memorandum re-framed the world-wide movement. This was so, in part, because it presented new principles of open government (which the Directive later that year re-framed as a definition of open government):

The three principles of transparency, participation, and collaboration form the cornerstone of an open government.

From the Memorandum, transparency was relevant to “accountability” and data as a “national asset.” Participation was the leveraging of the “dispersed knowledge” and “collective expertise” held by Americans through “public input” in government decision making. Collaboration, finally, was engagement in other aspects of government through the use of “innovative tools, methods, and systems to cooperate . . . across all levels of Government, and with nonprofit organizations, businesses, and individuals in the private sector.” While transparency was a core component of open government from the beginning of the open government movement in the 1950s, participation and collaboration were relatively new and less tested.

Transparency

Each of those three parts of the definition of open government was to be backed up by a new White House technology project. Data.gov, a dataset catalog, and the IT spending dashboard were launched early that year as new efforts to promote transparency. Vivek Kundra, the first U.S. chief information officer and the driving force behind Data.gov, claimed the IT spending dashboard saved taxpayers $3 billion and accelerated other programs by giving government administrators better access to performance measures of information technology projects.¹

The Directive, to my surprise at the time, also addressed nearly all of the 8 Principles of Open Government Data and added being pro-active about data release. It also proposed to increase accountability by designating an official responsible for data quality at agencies (which was suggested in a post-script of the 8 Principles).² The Directive also called for the creation of agency.gov/open web pages, changes to government culture, and a 10% reduction on FOIA backlogs per year.³

Data.gov has been criticized generally on two fronts. First, the apparent success of Data.gov has largely ridden on the inclusion of data sets that had already been available to the public. My favorite data set included early on in Data.gov was Federal Aviation Administration flight on-time statistics, which has been released in some form since 2003⁴. A fellow coder Josh Sulkin and I built FlyOnTime.us, which used historical flight on-time statistics and weather data from the National Weather Service to predict future delays, for instance to help fliers make better decisions about connecting flights. (FlightStats.com, a commercial website, and FlightCaster.com, a startup that raised nearly $1M, independently had similar ideas.)

The second line of criticism has been that whatever new and supposedly “high value” data that was released following the Directive was not very interesting for government transparency. The highest rated dataset on Data.gov as I write this is “Active Mines and Mineral Plants in the US” from the U.S. Geological Survey. Environmental and weather data comprise a large part of the data catalog. These are certainly important data sets for their connection to public safety. If journalists get a deeper perspective on mine safety and if that saves lives, then it would be hard to name an even more important data set. But the datasets don’t fulfill the promise of transparency. For that, we’re looking for open access to administrative records, records that tell us how decisions were made and that help the public stay informed about agency activity. It’s really no surprise that Data.gov has excelled in the sort of data sets it has since the Environmental Protection Agency and especially the National Oceanic and Atmospheric Administration in the Department of Commerce, the original source of most weather reports, have been leading the public dissemination of raw government data since well before there was an open government data movement.

Alon Peled, a professor of political science and public administration, explained in a review of Data.gov why federal agencies may have resisted open data:

Open Data architects failed to consider that datasets are valuable assets which agencies labor hard to create, and use as bargaining chips in interagency trade, and are therefore reluctant to surrender these prized information assets for free.⁵

Peled’s point highlights the need for open government advocates to stay grounded in reality.

Still, the benefit of Data.gov may be less in the catalog itself and more in the standards it sets for federal agencies and the cultural change it symbolizes. Harlan Yu, writing in 2011, pointed to what he says is under-appreciated infrastructure:

There’s a Data.gov manual that formally documents and teaches this process. Each agency has a lead Data.gov point-of-contact, who’s responsible for identifying publishable datasets and for ensuring that when data is published, it meets information quality guidelines. Each dataset needs to be published with a well-defined set of common metadata fields, so that it can be organized and searched. Moreover, thanks to Data.gov, all the data is funneled through at least five stages of intermediate review—including national security and privacy reviews—before final approval and publication. That process isn’t quick, but it does help ensure that key goals are satisfied.⁶

Participation

Law professor Beth Simone Noveck initially led the development of the participation and collaboration aspects of the Directive during her time as the U.S. deputy chief technology officer for open government (2009–2011) — she had previously invented Peer-to-Patent (see Consumer Products), which connects the U.S. patent office with expert volunteers to make the patent review process more informed. The White House’s We The People website at whitehouse.gov/petitions, launched in September 2011, was to fulfill the promises of participation. The site is a place for the public to submit petitions, which the White House pledges to respond to if it gathers more than a certain number of signatures within 30 days.

We the People is one of several similar experiments world-wide. The U.K.’s prime minister office began a similar system called e-petitions in 2007. In the current version at epetitions.direct.gov.uk, it takes 100,000 signatures to bring up an issue in the House of Commons. On August 11, 2011, a petition to cut social security benefits from rioters reached the threshold and was referred to a House of Commons committee, but the parliament has been out of session recently and has not formally debated the e-petition yet.⁷ Other participatory projects are occurring throughout the world. The Palestinian Prime Minister used Facebook to collect nominations for his cabinet, and the New York City council empowered neighborhood assemblies to determine local infrastructure projects.⁸ These sorts of petitioning are also closely related to participatory budgeting, which has been used experimentally many times throughout the world.⁹

2,916 petitions have been started on We the People to-date.¹⁰ The White House has posted 156 responses to those petitions.¹¹ There are no clear success stories, however, and many of those responses are unsatisfying.

The most successful petition — at least by its number of signatures — was one created in December 2012 asking the White House to recognize the Westboro Baptist Church as a hate group. It gathered 367,180 signatures and in July 2013 received a response. The response was not that the White House would recognize the Westboro Baptist Church as a hate group but instead an explanation that the White House does not maintain a list of hate groups. Many of the other responses to petitions crossing the signature threshold are explanations of existing White House positions and not very satisfying.

The most successful petition in the sense of actual policy change was one started in January 2013 supporting cell phone unlocking, the ability to use a cell phone with a new carrier. Strangely, cell phone unlocking had been a violation of copyright law under a ruling by the Library of Congress, and the petition asked that the White House ask the Library of Congress to reverse its decision. The petition gathered 110,028 signatures in its first 30 days, just 9 days later the White House responded, and in August 2014 the President in fact signed a law undoing the prior ruling and making cell phone unlocking legal again.

There is a sense in which this was “an unprecedented democratic process,” as Alex Howard wrote. But it would be wrong to credit the petition alone for the success. Howard explained:

[T]here’s a much deeper backstory to why Internet activism worked: the people behind the e-petition didn’t stop with a response from the White House. After making a lot of noise online, activists engaged Congress over a year and a half, visiting Capitol Hill, sitting in on phone calls and hearings, and being involved in the democratic process that led to this positive change. This is an important lesson in why “clicktivism” alone won’t be enough to make changes to laws or regulations emanating from Washington: people who want to shifts in policy or legislation have to learn how Congress works and act.

The petition was a signal of public support that enabled a broader advocacy campaign to succeed.

Yet I think it would be wrong to say that the White House was successfully petitioned. The White House already had asked the Library of Congress to rule in favor of unlocking, as explained in the response to the petition. In 2010, and again four months before the petition was created, the National Telecommunications and Information Administration, “the President’s principal advisor on telecommunications and Internet policies pertaining to the nation’s economic and technological advancement,”¹² had urged the Library of Congress to allow cell phone unlocking during the deliberative proceedings that lead up to the Library of Congress’s decision.¹³

The petition raised the profile of a position the White House had already taken. That the petition received a response (which is not always the case — see below), and that the response came so quickly, isn’t surprising in light of the White House’s existing position on the issue. The success of this petition does not demonstrate the success of a platform to petition the White House but rather the success of the White House in using a neutral platform to selectively draw attention to campaigns in line with White House policies.

The threshold for a White House response has gone up steadily as the popularity of We The People has increased. In 2012 the threshold was 25,000 signatures within 30 days, and now in 2014 the threshold is 100,000. Except when it isn’t.¹⁴ As of August 2014, 29 petitions had crossed the signature threshold without receiving a response (about 10% of all petitions that cross the threshold).

Two of those petitions hit close to home for the open government movement. In early 2013, Aaron Swartz, an early leader of the open government movement, committed suicide while under investigation for downloading research papers without permission.¹⁵ Two petitions¹⁶ were submitted shortly after his death calling for the firing of attorneys believed to be over-zealous in their prosecution of Swartz. The petitions each gathered more than the then-threshold of 25,000 signatures, but more than one year and 89,733 signatures later there had been no response. The White House issued a response on January 6, 2015, or more accurately a non-response that merely said that the White House cannot address “agency personnel matters.”

The earliest of the petitions still “pending response,” according to the White House’s own data, is a petition which has now reached ten times the (early) 5,000 signature threshold. It is a petition about GMO food labeling. At 211,925 signatures (181,479 in its first 30 days), another petition created in July 2013 asked the White House to declare the Muslim Brotherhood party in Egypt a terrorist organization. It remains not responded to as well.

Collaboration

The benefit of applying technology to participation and collaboration is in making government smarter, policy better, and citizens motivated, according to Noveck.¹⁷ But the flagship project for the collaboration component of the Directive never came to be.

Initially planned as ExpertNet, it was a program or process to connect experts in the public with public servants, similar to Noveck’s Peer-to-Patent. The White House canned the project without any part completed in 2013.¹⁸ Noveck later tweeted me, “Expernet was process not application. Software exists. Hard part is getting people to ask questions,”¹⁹ meaning ExpertNet was a project about transforming government culture, not in developing new technology or launching a new product. This is true of all aspects of open government. The cultural change is always the hardest.

Knowing now that the White House’s flagship product for collaboration would fail, I wonder whether the Administration would still stand by their original, three-part definition of open government.

The flagship product may have failed, but there has been a lasting legacy of the collaboration component of the Directive: “challenges.” Modeled after the 2008 Apps for Democracy contest in Washington, D.C., a challenge on Challenge.gov is a government-sponsored competition, typically with monetary prizes, in which participants are asked to solve a problem a government agency faces. Its purpose is to promote innovation, both in the private sector as well as within government, by introducing the agency to new ideas. Challenge.gov, which launched in 2010, lists 23 open challenges at the time of writing²⁰ with 5 federal agency sponsors. Challenges generally offer prizes between $0-$100,000.

The recent Space Apps Challenge, run by NASA, exemplifies how challenges are geared toward an agency’s mission:

The International Space Apps Challenge is an international mass collaboration focused on space exploration . . .. The event embraces collaborative problem solving with a goal of producing relevant open-source solutions to address . . . NASA’s mission directorates in five themes: Earth Watch, Technology in Space, Human Spaceflight, Robotics and Asteroids.

This challenge from the Department of Energy is targeting very specialized participants:

Microgrid 2014 MVP Challenge: The Competition will award prizes for operational microgrids in the U.S. with interconnected distributed energy resources (DER) that provide uninterruptible power to critical facilities and services during emergencies.²¹

The idea behind these challenges is that they are cheaper or more effective at producing ideas and solutions than government contracts or grants. Anecdotally, there are many instances of successful challenges. I mentioned several health-related apps in Introduction. In the White House’s 2013 report to congress on challenges, they described several other successes including the Federal Trade Commission’s Robocall Challenge:

The vast majority of telephone calls that deliver a prerecorded solicitation are illegal. The FTC receives between 125,000 to 200,000 complaints each month regarding these harassing calls. With a $50,000 prize purse, the Federal Trade Commission challenged the public to create innovative solutions to block illegal commercial robocalls on landlines and mobile phones. FTC received 798 eligible submissions from individual solvers and teams. . . . “Nomorobo” is one of the winning solutions – developed by citizen solver Aaron Foss, this service allows incoming calls to be routed to a second telephone line that can identify and hang up on illegal robocalls. As of March 2014, Nomorobo has over 86,000 users and has blocked over 2 million robocalls. The challenge engaged solvers who were not focused previously on helping consumers block illegal robocalls – none of the developers and engineers who created the winning solutions had been engaged with the problem before.²²

(For more, see Alex Howard’s recent roundup.)

Whether challenges are cost effective, I don’t think this is known. Past challenges are difficult to survey for effectiveness: determining whether the contest winners’ solutions were valuable to an agency’s mission may be impossible to measure. In some cases, such as the microgrid challenge, I suspect challenges are merely displacing or are a relabeling of existing funding mechanisms.

And as they relate to open government, challenges should not only be generating ideas, or even generating working solutions, but they must also be involving the public in the government’s implementation of those ideas. Many successful challenges demonstrate the public’s ability to innovate but not the government’s ability to collaborate, except in so far as the government writes the check. A study of Challenge.gov through 2012 showed that challenges had focused on collecting feedback and outreach:

[A]gencies mainly use Challenge.gov for so-called “low-hanging fruits”, those projects which are easy to outsource and implement such as information and education campaigns that help them better understand how to improve their service delivery, but not necessarily the service itself. . . . Challenge.gov [is also used] as an outreach mechanism to educate large parts of the citizenry that might not be reached through the traditional channels.²³

Collecting feedback and outreach are both one-way communication. These examples also fail to demonstrate collaboration. While challenges may be successful in their own right, Challenge.gov may not demonstrate a successful implementation of open government.

Was the definition too broad?

The breadth of open government created communications challenges within and outside of the movement. Beth Noveck, the professor and former U.S. deputy chief technology officer for open government, wrote in a blog post in 2011 of the trouble the ambiguity created for the goals that she had brought to the White House:

In retrospect, ‘open government’ was a bad choice. It has generated too much confusion. Many people, even in the White House, still assume that open government means transparency about government. (\ldots) The aim of open government is to take advantage of the know-how and entrepreneurial spirit of those outside government institutions to work together with those inside government to solve problems.²⁴

Harlan Yu and David Robinson wrote about the ramifications in their paper “The New Ambiguity of ‘Open Government’ ”:

The shift has real-world consequences, for good and for ill: Policies that encourage open government now promote a broader range of good developments, while policies that require open government have become more permissive. A government’s commitment to be more “open” can now be fulfilled in a wider variety of ways, which makes such a promise less concrete than it used to be. (\ldots) A government could commit to an open data program for economic reasons—creating, say, a new online clearinghouse for public contracting opportunities.²⁵