open data

I spend a good deal of time exploring apps and data visualizations that use government data. Unfortunately, most of that time is spent yelling at my monitor about how they are missing some critical element or don’t provide me with enough information. In that spirit, here is some unsolicited advice.

1. show your work (and if you can’t, stop what you are doing)

First and foremost, your project needs a clear and concise explanation of what you did. What data did you use, how did you transform / normalize / prepare it, and how did you represent that data visually? If you can’t answer all of these questions, then you need to stop what you are doing and get some help; feel free to ping me and I can point you in the right direction.

You should know what every chart, map dot, thematic layer, and/or graph in your app means and be able to clearly communicate that to your users through concise, narrative text.

2. link to the source data and include the proper disclaimers

There’s a ton of public government data now available. Lots of it is from the same agency and multiple data sets will have similar sounding titles. You should, whenever possible, clearly explain what data set you used, where someone can find and download it for themselves, and re-present any and all disclaimers that the data providers attach to the data. It’s hard for me to count the number of projects that completely ignore this when it comes to crime data.

Everyblock.com does a really fantastic job on this.

3. label everything

It kills me when I see a chart w/o the x and y axes clearly labeled or labeled but lacking units of analysis. Come on people, this is an easy one.

4. provide an easy method for user feedback

I’m the kind of person who would love to ask you questions or point out awesome (and sometimes bunk) things about your project, but far too frequently there’s no easy way to send you my thoughts. Everything should include a feedback link. While it should go without saying, you need to respond to the feedback you do receive.

There’s more – much more – I can write here, but I think that these are the most critical basics. If you app meets these four tests, give yourself a pat on the back.

Please add your suggestions / ideas / comments below – my hope is that this list becomes a resource for folks over time; bookmark this and refer back when you start working on that next project.

I had the privilege to join my colleagues Paul Baker of Webitects, Harper Reed, and Dan O’Neil of EveryBlock for a talk about open gov data at Geekfest, a series of nerd talks at web dev firm Obtiva. Listen closely for a thinly veiled reference to what will be a major focus of my work in 2011.

Open Gov. Discussion at Obtiva Geekfest from Obtiva on Vimeo.

This month, I began working with the Metro Chicago Information Center (MCIC) to help them figure out how to share their treasure trove of data about the Chicago region with the rest of the world – more than 100 data sets covering more than a decade. They’ve seen the promised land on the horizon, a rich world of “public purpose applications” powered by data and empowering all sorts of folks to make better decisions in a wide variety of contexts.

But MCIC knows that without access to quality data and technical assistance to understand what the data can and can’t ‘say’, the dangers Lessig has pointed out will limit the impact of this emerging field. Developers should be free to focus on building rich, engaging, and useful applications rather than poring over non-standard metadata or trying to match seasonally with non-seasonally adjusted data sets.

MCIC is uniquely situated to provide both data and technical assistance for many of these applications; from their boilerplate description:

Metro Chicago Information Center (MCIC) is an independent, non-profit organization that provides the highest quality data collection, analysis and consultation to institutions dedicated to investing in communities and enriching lives. Though MCIC does not advocate specific public policies or policy choices, we strive to provide critical information and insight necessary to support human services, cultural programs and overall economic growth. MCIC was founded in 1990 by a consortium of regional business and philanthropic leaders at the Commercial Club of Chicago. MCIC works from a fundamental philosophy that better information produces better decisions.

As a self-described ‘legacy organization’, MCIC has a great deal of work to do in order to start sharing their data library. Figuring out the various licensing requirements based on their sources, standardizing all of the metadata, and building and creating documentation for an API will take a while. There’s also the matter of figuring out how to pay for this new and as of yet unfunded program (suggestions welcome).

MCIC is actively seeking your feedback on what to share, how to share it, and how to provide technical assistance. Below is a list of the data keywords from their catalog – please vote for which you think should be shared first.

Additionally, please leave a comment about how you think the mechanics of this sharing should work; as a developer of public purpose applications, what would make your life easier? How should we provide technical assistance on using this data? Finally, if you have a specific data set you’ve been searching for, drop us a line. If we have it and can, we’ll be happy to share.

A new post up at Rooflines.org on the missing priorities of the new HUD/DOT Sustainable Comunities Initiative:

At a US House of Representatives hearing last week on “Livable Communities, Transit Oriented Development, and Incorporating Green Building Practices into Federal Housing and Transportation,” HUD Secretary Shaun Donovan and Department of Transportation Secretary Ray LaHood announced an new partnership. A joint task force, the “Sustainable Communities Initiative,” will address the intersection of transportation and housing affordability.

While any genuine effort to address housing affordability is welcome, this new HUD/DOT task force is particularly exciting because it reflects a new understanding of the complex issues that confront metropolitan regions and the sometimes surprising ways they intersect. Encouraging smarter planning, expanding the definition of affordability, and researching the livability of communities work in tandem to increase the use of public transportation, decrease our carbon footprint, reduce urban sprawl, enable the smarter use of regional resources, and improve affordable housing options for families. In short, this is one of those rare initiatives that seem to naturally align a multitude of interests.

Read the full post…