4. A Brief Legal History of Open Government Data
Let’s say we could start from scratch. What would a technologically-enabled open government look like? Would it involve mandatory open records? Live streaming of all public meetings? Maybe the right to information would be a constitutional right — something that new constitutions in the world are beginning to adopt. The new constitution of Kenya which was ratified in the summer of 2010 reads:
(1) Every citizen has the right of access to information held by the State. (2) Every person has the right to the correction or deletion of untrue or misleading information that affects the person.92. http://www.nation.co.ke/blob/view/-/913208/data/157983/-/l8do0kz/-/published+draft.pdfand for more: http://j.mp/98AtiE
Kenya is only the latest and not the first to make information a right. According to the website Right2Info.org93. http://right2info.org/constitutional-protections-of-the-right-to the right to government-held information is protected by the constitutions of roughly 40 countries. The Philippines has a particularly strong provision specifically mentioning records, documents, papers, and research data. The United States is conspicuously not among these countries.
But while hypothesizing an ideal right to information may be a useful exercise, the history of the right to government-held information has at least as often as not been born out of the practical, immediate needs of those in power.
Three Origins of Open Access to Law
It’s impossible to say exactly when and where open government data began, but a useful starting place is in open access to law. Three examples of open access and codification — in Archaic Greece, Visigothic Europe, and the American colonies — paint an interesting picture of how access to the law arose through anything but a democratic ideal.
When Athens codified its law in the 6th century BC, moving away from an oral tradition, it was a small part of a larger set of reforms implemented by the archon Solon. During this time, the public at large was of course not very literate. They could not have made much use of codified law. But according to author Jason Hawke, there is another reason to believe that the law was first written down not with the ideas of a participatory government in mind, but instead with the needs of the elites: The laws that were codified were those that kept wealth in place. Hawke wrote,
Plutarch states that Solon enacted a law forbidding dowries in all marriages save those which occurred between an heiress and her kinsman, the bride otherwise bringing with her only three changes of dress and inexpensive household goods. . . . These statutes regarding the control of property exhibit a common concern: the preservation and stability of the patterns of resource-ownership. . . . [T]he overall effect of such legislation was to conserve the distribution of wealth and resources and to prevent the easy movement of property and the de-stabilization of social and political arrangements within the community as a result.94. Hawke, Jason. 2011. Writing Authority: Elite Competition and Written Law in Early Greece. page 171–172.
Athens was undergoing significant socio-economic changes during Solon’s time. Codification appears to have been a part of a reactionary program of holding onto a social structure upended by population growth and new wealth. Democratizing or equal justice ideals were not a reason for the modernization of law that took place.
In colonial America, as in Archaic Greece, the codification of law emerged out of the needs of those in power. There, inexperience and a lack of information lead to gross economic confusion, and it was the need to end that confusion that lead to advances in government transparency. A Virginia critic wrote at the time that “the Body of their Laws is now become not only long and confus’d but it is a very hard Matter to know laws are in Force and what not.” As a result, “[t]owns were legislated into existence at inappropriate sites; . . . [and] the price of bread was assigned on sizes the bakers did not sell.” Codification arose out of frustration among legislators, as a means to attack other political offices, and perhaps also to rally their constituents.
The publicity probably began simply because legislators went home between sessions and found that their constituents had no copies of laws and no idea why the legislators were fighting the governors . . . [In 1710] the [Pennsylvania] assembly began publishing its acts and laws twice a week, partly so members could have up-to-date copies, partly so interested groups could appeal acts before the end of a session. Ten years after that, in 1720, the house began publishing its journals. In 1729 and 1740 it published codifications of its laws . . . Massachusetts began publishing its journals in 1715, again in a controversy with the governor, and put out a systematic compilation of the laws in 1742.95. Alison G. Olson. 1992. Eighteenth-century colonial legislatures and their constituents. In The Journal of American History, 79(2), 543–567.
While the effort to codify law may have not been aimed primarily to democratize access to the law, it was in retrospect a thankful consequence. Private citizens began making use of the text of the law in ways they could not before. According to Olson (1992), “Pennsylvanians annoyed with what they thought to be unfair practices on the part of flour inspectors in the 1760s confronted the inspectors with copies of the laws.”96. ibid
The law in 7th century AD Europe, under the Visigoths, provides the first instance of an open access law that I have found. At this point “data” was of course still 1,300 years into the future and even the printing press 900 years away. Written information in the 7th century was disseminated only at great pains through the scribe work of monks at a handful of monasteries.
The story of the first copyright law exemplifies the differences between then and today. When a monk copied the manuscript of a saint in 6th century Ireland, the reigning king decreed, “To every cow her calf, and consequently to every book its copy.” This decree ordered the monk to hand over his copy to the saint, and in so doing inventing the concept of copyright. But the more interesting event was the first appeal of a copyright ruling. The monk, unhappy with the decree, unleashed the military power of his family against the king, and he won.97. George Haven Putnam. 1896. Books and their Makers During the Middle Wages. Reprinted 1962, Hilary House Publishers. Page 46. In these times when the loss of a single manuscript could be cause for an attack on the king, an open records law seems as though it would have made little sense, as the cost would have easily outweighed the benefit.
And yet, the promulgation of the law using price control was a part of public policy of the time. The Visigothic Code, written in Western Europe around 649-652 (but based on a legal tradition dating back well before that), set a maximum price that the Code itself could be sold for: “it shall not be legal for a vendor to sell a copy of this book for more than four hundred solidi,” or some $100,000–$400,000 today, the Code read. In setting a maximum price the Code seemed to intend to create wider access to the Code than would have otherwise occurred.
And yet a hint of its rationale can be found in another part of the Code. It also directed “bishops and priests to explain to the Jews within their jurisdiction, the decrees which we have heretofore promulgated concerning their perfidy, and also, to give them a copy of this book, which is ordinarily read to them publicly.”98. Samuel Parsons Scott. 1910. The Visigothic Code: (Forum judicum). http://libro.uca.edu/vcode/visigoths.htm. Scott, the translator of the code, claimed 400 solidi was the equivalent of $17,600 in 1908, or approximately $400,000 today. As the solidi was 4.5 grams of gold, the same gold would have been worth just $110,000 in 1995 (the price of gold has fluctuated significantly since then). To give an idea for the length of the Code, it was translated and annotated within 500 pages. Given the enormous cost of creating a copy and low rates of literacy, I doubt this provision, the Code’s only provision for the free dissemination of the law, was carried out literally. But the law suggests that the purpose of the dissemination of the law was not to widen access to justice but instead to suppress dissidence. While this is the earliest open-records law that I’ve found, the legal tradition may have in any case ended when the Visigoths were replaced by the Moors less than a century later in 711.
The Origins of Government Spending Records
Open records practices can only occur if there can be an inexpensive but comprehensive infrastructure for information dissemination, and it was only 1,000 years later that we can start to trace a continuous history of open data. Modern open records laws probably drew more from 17th century China than any Western tradition of the time. Jean Baptiste du Halde wrote about the Chinese empire in the early 1700s:
Every three years they make a general review of all the Mandarins [officers] of the Empire, and examine the good or bad qualities that they have for government. Every superior Mandarin examines what has been the conduct of the inferior since the last informations have been given in, or since they have been in office, and he gives Notes to everyone containing praises or reprimands . . . They reward a Mandarin by raising him to a higher degree, and they punish him by placing him in a lower, or by depriving him of his office.
The reviews would then be passed up the chain of command, each officer adding his notes onto those of his subordinates. At the highest level, where an account of all of the officers of the empire was put together, the punishments and rewards would be set and instructions would be distributed back down the chain of command, all the way down to the common people.
[T]he Mandarins are obliged to put at the head of their orders the number of degrees that they are to be raised or depressed: For instance: I, the Mandarin of this city raised three degrees, or depressed three degrees, do order and appoint, etcetera. By this means the people are instructed in the reward or punishment that the Mandarin deserved.99. Jean Baptiste du Halde. 1736. The general history of China containing Geographical, Historical, Chronological, Political and Physical Description of the Empire of China, Chinese-Tartary, Corea, and Thibet. Second volume. John Watts: London. pp64–65.
This first practice that du Halde was aware of was an interesting application of open records. Officers of the court were required to announce their own promotion or demotion in each of their orders.
The second remarkable practice that du Halde learned of was the Peking Gazette, published in the capital and distributed throughout the provinces of the empire. The gazette recorded punishments of officers, “expenses disbursed for the subsistence of the soldiers, the necessities of the people [probably care for the old and poor], the public works, and the benefactions of the prince,” and “laws and new customs that have been established.”100. pp69–71. The gazette itself reportedly goes back 1,000 years, but this is the description of the earliest gazette I could find.
Although the Peking Gazette appears to be a form of government disclosure and might have been seen that way by Western Enlightenment thinkers, its true purpose was surely for the emperor to “instruct the Mandarins how to govern,” as du Halde wrote.101. ibid The gazette was most likely a form of government propaganda and control written in the form of an announcement.
The Freedom of Information in the Kingdom of Sweden and the United States
While the foundations of open records were present in the Visigothic Code, new practices in the American colonies, and the long tradition of the Peking Gazette, it was in the Kingdom of Sweden in 1766 that the wide dissemination of government records became a constitutional right.
Sweden was then a rare parliamentary government with a weak king, but it had not yet outgrown the common practice of the time of government-granted monopolies. Priest, farmer, doctor, and Enlightenment thinker Anders Chydenius, who would later promulgate offentlighetsprincipen, “the principle of publicity”102. Juha Mannine. 2006. Anders Chydenius and the Origins of World’s First Freedom of Information Act, in The World’s First Freedom of Information Act: Anders Chydenius’ Legacy Today, ed. Juha Mustonen, Anders Chydenius Foundation., had a problem with those monopolies. Chydenius’s home province in a remote part of the Kingdom had not received such a monopoly, and for the sake of free trade — or else for the sake of his own province’s well being — Chydenius demanded freedom of sailing at a local government meeting. His brief role in local politics was followed by a well-timed shift in political power which opened the door to his service in the national parliament in 1765–1766. There he continued to defend economic freedom and a new subject for him, the ability of the public to participate in the national debate.
Chydenius was inspired by the Chinese practices that he knew of through works by du Halde as well as the writing by one of his contemporaries, Anders Schönberg, who in the early 1760s called for the free publication of government documents, decisions, and voting records. But it was Chydenius as secretary of a committee on the freedom of the press that drafted what became the first known freedom of information act in history, combining both freedom of the press to publish as well as the right to access government information. These principles were enacted at the end of 1766 with much debate but no objection. Chydenius’s act guaranteed access to two types of government information, documents and records of votes:
6. [A]ll exchanges of correspondence, species facti, documents, protocols, judgments and awards . . . when requested, shall immediately be issued to anyone who applies for them.
7. [I]n order to prevent the several kinds of hazardous consequences that may follow from imprudent votes, likewise graciously decided that [judges] shall no longer be protected behind an anonymity that is no less injurious than unnecessary; for which reason when anyone, whether he is a party to the case or not, announces his wish to print older or more recent voting records in cases where votes have occurred, they shall, as soon as a judgment or verdict has been given in the matter, immediately be released for a fee, when for each votum the full name of each voting member should also be clearly set out . . . and that on pain of the loss of office for whosoever refuses to do so or to any degree obstructs it.103. Peter Hogg. 2006. Translation from Swedish to English of His Majesty’s Gracious Ordinance Relating to Freedom of Writing and of the Press (1766) in The World’s First Freedom of Information Act: Anders Chydenius’ Legacy Today, ed. Juha Mustonen, Anders Chydenius Foundation
The law only remained in effect until King Gustav III’s coup six years later. But it returned in various forms over the succeeding centuries. A Freedom of the Press Act is one of the four documents that comprise the current Swedish constitution.
Progress since 1766 has been relatively slow. Twenty-one years after Chydenius’s act we adopted our constitution here. It did not have a general provision for access to government information. It did call for each house of Congress to maintain a journal.
Each House shall keep a Journal of its Proceedings, and from time to time publish the same, excepting such Parts as may in their Judgment require Secrecy; and the Yeas and Nays of the Members of either House on any question shall, at the Desire of one fifth of those Present, be entered on the Journal.
Today called the Congressional Record, Congress’s journal has been known to contain records of events that never occurred.104. The Open House Project Report, 2007. http://www.theopenhouseproject.com/the-open-house-project-report/10-the-congressional-record/ The second country to enact a FOIA law, after Sweden, was Finland, which had been a part of Sweden in 1766. Finland kept Chydenius’s spirit going through Russian control in the 19th Century and enacted its own law as a new country in 1919.105. http://www.freedominfo.org/regions/europe/finland/
It wasn’t until 1966 — two hundred years following Sweden — that we had a Freedom of Information Act (FOIA) here in the United States, the third such law in world history. Inspiration for FOIA came not from the ideals of European Enlightenment but from the expansion of the federal government during and after World War II, the resulting bureaucracy and record-keeping, and the increasing skill of the federal government at manipulating public opinion.106. Parks, Wallace. 1957. The open government principle: Applying the right to know under the Constitution. The George Washington Law Review, 26.1. (For more on the initial development of the modern open government movement, see Yu and Robinson (2012)107. Yu, Harlan and David G. Robinson. February 28, 2012. The New Ambiguity of “Open Government.” .) The burgeoning open government movement of the 1950’s and 1960’s was driven by the newspaper industry and its trade associations quite possibly with no knowledge of the precedent in Sweden. The enactment of the Freedom of Information Act here was, unlike the previous examples, motivated by idealistic principles of having an informed citizenry.
FOIA’s advocates worked for several decades with Congress and the executive branch on a compromise that created a default state of openness for government records with a few enumerated exemptions. The first compromise was targeting the executive branch only. William Matthews, editor and publisher of Tucson, Arizona’s Daily Star, wrote at the time, “As a matter of getting such a law through Congress, should we threaten that body by demanding that all committee meetings be open?”108. Parks, Wallace. 1957. The open government principle: Applying the right to know under the Constitution. The George Washington Law Review, 26.1. Members of Congress had little to worry about the Freedom of Information Act, which ultimately passed unanimously in the House, as the law left them alone. A later compromise created the conspicuous exemption for “geological and geophysical information,” which may have been insisted on, indirectly, by President Johnson, who may have been protecting the interests of the oil industry in his home state of Texas.109. Kennedy, George Penn. 1978. Advocates of Openness: The Freedom of Information Movement. Doctoral dissertation, University of Missouri-Columbia.
Johnson only begrudgingly signed the bill later. In a signing statement he pushed back on public access to personal files and other sorts of documents. His worry, which I think was fair, was that some decisions could be made better if deliberations could remain private, essentially the doctrine of executive privilege.110. http://www.gwu.edu/nsarchiv/NSAEBB/NSAEBB194/index.htm
Delays and fees, such as documents copied at $1 per page, lead Congress to amend FOIA in 1974. But that wasn’t a good time for President Ford, who was in the middle of responding to The New York Times’ coverage of military leaks. Ford vetoed the changes to FOIA hoping to keep control over his office’s information, but Congress easily overrode the veto.111. http://www.gwu.edu/nsarchiv/NSAEBB/NSAEBB142/index.htm There have been new laws strengthening right-to-know since then — Who Needs to Know? by Patrice McDermott (Bernan Press, 2007) is a good reference. But FOIA, and at the state level freedom of information laws (FOIL), have not kept up with technology in the least (though that’s not to say they don’t still serve an important purpose), and as I explained in Chapter 1 serve a very different need from open government data.
Most other countries didn’t follow suit in developing FOI laws until the 1990’s and early 2000’s.112. Michener, Greg. April 2011. FOI Laws Around the World. Journal of Democracy.
The 21st Century: Data Policy
The open government movement that arose in the mid 20th century had a major shift in the first decade of the 21st century largely due to the infusion of technologists into the movement. From a legal perspective, Data.gov, the Open Government Directive (discussed in Chapter 1), and open standards laws passed in Vancouver, Canada and Portland, Oregon all in 2009 marked the shift. In Vancouver:
Open and Accessible Data - the City of Vancouver will freely share with citizens, businesses and other jurisdictions the greatest amount of data possible while respecting privacy and security concerns; Open Standards - the City of Vancouver will move as quickly as possible to adopt prevailing open standards for data, documents, maps, and other formats of media;113. http://vancouver.ca/ctyclerk/cclerk/20090521/documents/penv20090521min.pdf
(see section 7.3 for a longer excerpt) and in Portland, Oregon:
[T]he Council of the City of Portland . . . [d]irects the Bureau of Technology Services to . . . [d]evelop a strategy to adopt prevailing open standards for data, documents, maps, and other formats of media;114. http://efiles.ci.portland.or.us/webdrawer/rec/3675248/view/
Progress since 2009 has been steady. In 2010, several more city governments picked up on the movement. San Francisco enacted a change to its municipal administrative code regarding open data policy. It was the first law to adopt language from the 8 Principles of Open Government Data and it called for technical requirements to be created for the purpose of “making data available to the greatest number of users and for the greatest number of applications,” with “non-proprietary technical standards,” and a “generic license” such as a Creative Commons license.115. http://www.sfbos.org/ftp/uploadedfiles/bdsupvrs/bosagendas/materials/bag110910101155.pdf (See section 7.4 for a longer excerpt. Unfortunately, a quick perusal of DataSF.org, a sort of Data.gov, shows that the city’s GIS data are all still hidden by an innocuous but non-generic click-through agreement.116. http://gispub02.sfgov.org/website/sfshare/index2.asp) The City of Ottawa seemed to pass an open data motion that year as well,117. http://ottawa.ca/calendar/ottawa/citycouncil/occ/2010/05-12/englishminutes89.htm which in its accompanying report referred to the 8 Principles of Open Government Data. A similar bill like the San Francisco bill was introduced in the New York City Council118. Int 0029-2010. More at http://www.scribd.com/fullscreen/34124837, http://nycctechcomm.wordpress.com/opengov/, and Stirland, Sarah Lai. February 28, 2012. In New York, Landmark Open Data Legislation Will Soon Be Up for a Vote. TechPresident. — it was enacted into law more than two years later in March 2012. The law defines data and open standards, it incorporates ideas from the 8 Principles, and it requires public data sets to permit “automated processing and . . . to notify the public of all updates.”
The first law to my knowledge to incorporate the full 8 Principles is New Hampshire’s HB 418, originally introduced as HB 310 in 2011119. http://www.gencourt.state.nh.us/legislation/2011/HB0310.html by an open-source-coder-turned-state-representative. The bill was enacted as HB 418 in March 2012, requiring state software acquisitions to consider open source software, requiring new data standards created by the state to be open standards, and directing the commissioner of information technology to develop state-wide information policies based on the 8 Principles.120. http://www.gencourt.state.nh.us/legislation/2012/HB0418.html, http://legiscan.com/gaits/view/347605 (See section 7.7 for an excerpt.) The bill’s open standards language was copied into Oklahoma bill HB 2197 which was enacted in April 2012.121. http://www.oklegislature.gov/BillInfo.aspx?Bill=HB2197 California SB 1002 (2012)122. http://legiscan.com/gaits/view/390077, which is currently pending in the California Senate, also defines open data drawing on the 8 Principles.
At the federal level progress is much slower. In 2010, H.R. 4858: The Public Online Information Act of 2010, or POIA, was introduced by New York’s Rep. Steve Israel, though in reality the brainchild of the Sunlight Foundation. Under POIA, records in the executive branch would be put online and an advisory committee for government-wide guidelines on Internet-enabled transparency would have been created. (POIA was re-introduced in 2011 by Montana Senator John Tester.) At the end of the year then-Illinois congressman Bill Foster, who lost reelection, introduced H.R. 6289: Legislative Data Transparency and Public Access Act of 2010. This bill should have been a no-brainer: It would have required the Library of Congress to publish its database that powers the THOMAS.gov website, and then I could get out of the business of screen-scraping THOMAS. Of course this bill, like the others, went nowhere. The executive branch renewed its commitment to open data in its 2012 Digital Government Strategy.123. http://www.whitehouse.gov/sites/default/files/omb/egov/digital-government/digital-government.html
In 2011 the New Zealand government approved comprehensive general principles for data management (including “open”, “well-managed”, and “reasonably priced”), which draws from several aspects of the 8 Principles for Open Government Data.124. New Zealand Data and Information Management Principles. August 8, 2011. A cooperation of the federal and local governments in Austria has endorsed the Creative Commons Attribution License for government data125. http://blog.okfn.org/2011/08/15/austria-adopts-ckan-and-cc-by-as-nation-wide-defaults/, which requires attribution (and nothing else) for reuse of data. Data is being released on websites like Data.gov throughout the world, such as in Montevideo, Uruguay.126. http://blog.okfn.org/2011/08/09/montevideo-proud-of-our-data/
Open government and data policy legislation is still evolving. Interestingly, it is the cities and states in the United States leading the legal frontier of open data here. And abroad, licensing rather than openness per se has seen the most traction.