Sunlight Foundation Reporting Group

OGD: Transparency for Whom?

Most of the raw data released by the Open Government Directive on Friday was released in an XML format.  For those of you who dont know what that is or what to do with it, thats because it most likely isnt for you.  Common knowledge about XML is that its a platform only Web developers and programmers use.  And thats what it looked like after opening up the datasets in XML on Data.gov. The unfamiliar code can be off-putting and might cause people to not try to figure out whats in these high value datasets.  In order to use XML one has to know how to convert it or how to use an XML reader. Its also possible to import XML to Microsoft Excel, and thats only if the file isnt so big it crashes the gentle Microsoft application. (We've had marginal success using Microsoft Access 2007 to look at some of the XML data.)

After finding an XML coded dataset that could be successfully imported to Excel, it became clear that the XML released in the name of transparency was delivered in bad XML. Take for example the Targeted Investment Program data, a program under TARP. Not only is that information already available in PDF and Excel formats on Financialstability.gov, those formats provide a more accurate and clear version of the data. The columns and rows are confused in the XML converted version, omitting the meaning of the data.

Most reporters that deal with data will say the best way to receive data files so they can be studied and displayed is in a CSV format. Luckily some of the raw data released on Data.gov is in CSV format, and not XML, making for an easy import to Excel. But even the CSV files arent entirely sufficient to accomplish transparency.  The truth is not every American is tech savvy enough to navigate raw data. 

After a lot of toying with the information released on data.gov, it doesn't seem that the raw datasets provide much use for regular people.  The tools provided on data.gov, however, have shown some promise. The FAFSA application data released by the Department of Education is a good example. The tool makes it easy to see where the bulk of the subsidized loans are going. Make sure to check out the Myfood-a-Pedia put out by the Department of Agriculture, Its at least fun.