geekfest video: open gov data

I had the privilege to join my colleagues Paul Baker of Webitects, Harper Reed, and Dan O’Neil of EveryBlock for a talk about open gov data at Geekfest, a series of nerd talks at web dev firm Obtiva. Listen closely for a thinly veiled reference to what will be a major focus of my work in 2011.

Open Gov. Discussion at Obtiva Geekfest from Obtiva on Vimeo.

new article: Why is the Internet so slow?

The great folks over at the Community Media Workshop asked me to write the below introduction to Net Neutrality and why neighborhood bloggers / journalists should care about it for their latest report, “Realizing Potential: What Chicago’s Online Innovators Need”. This is a follow-up to last year’s report, “The NEW news: The Journalism We Want and Need”, to which I also contributed an article. Enjoy.

Imagine if, when shopping for appliances, only GE microwaves could nuke your food on high power while other brands could only operate at 75 percent. Or, imagine if only calls from certain telemarketers rang through to your mobile phone while your friends had to pay an extra, per-call fee in order to reach you. Sounds crazy, right?

Unfortunately, there are a growing number of major corporations lobbying for just this approach to data on the internet.

Telephone and power lines are, in a word, dumb. They don’t pay attention to who is using them for what purpose or what devices they are connected to, only that the user has paid the bill. Until very recently, the internet operated in much the same manner; while your specific connection speed might vary based on your individual plan, the actual content that came to your device and the specific make/model of your computer, phone, radio or car didn’t matter. All websites loaded at (roughly) the same speed and you have been free to connect any device to the web. This is the core tenet of net neutrality: your connection to the internet should be ‘dumb’ and deliver whatever content you request to whatever device you use at the same speed, regardless of what the content is.

But this principle has recently been called into question by both the courts and major corporations. In April, the U.S. Court of Appeals ruled that the Federal Communications Commission (FCC) does not have the authority to regulate an internet provider’s network management practices and policies; in short, the FCC can’t enforce regulations to protect network neutrality.

Then, in August, Google and Verizon released “A Joint Policy Proposal for an Open Internet,” laying out a set of seven principles they believe should guide federal regulation. In their proposal, there is a clear distinction between the rules for “wireline” and “wireless” services. While wired broadband access (such as through a cable modem or an office’s network) would be governed by a weakened set of network neutrality principles, wireless broadband—which includes every connection to the web from a mobile phone—would only be required to disclose the exact nature of their services and would be allowed to control how fast various services were able to communicate data back and forth. Under their proposal, Verizon would be able to allow, for example, USA Today to display stories three times as fast as the Sun-Times in a mobile web browser, for the right price.

University of Illinois at Chicago Prof. Karen Mossberger’s research highlights the importance of network neutrality over wireless broadband for hyperlocal journalists in Chicago. In her “Digital Excellence in Chicago” report for the City of Chicago, she writes, “Over a third of Chicago residents have accessed the internet through some type of wireless device, and the concentration of such use among residents under 30 suggests that this trend is likely to increase in the future, especially with advances in technology.”

As the Workshop’s NEW News report suggests, the vast majority of Chicago’s neighborhood news sources are passion projects and few are generating much revenue. And, as we all know from our own internet use, speed is everything: waiting too long for a page to load simply means you will look elsewhere for the information. If wireless broadband providers are allowed to require that hyperlocal journalists pay for top-tier access—fees that many likely cannot afford—the inevitable result will be fewer sources for neighborhood news.

You can download CMW’s full report here (.pdf).

Come Work With Me

MCIC is hiring and you should totally come work with me.

Web and Database Programmer

MCIC (Metro Chicago Information Center) is a non-profit corporation that provides research and information services(surveys, maps, data analysis, focus group) to those concerned with public policy issues, community needs, and quality of life. Our products include both specific deliverables for client-based projects, including visuals and text (maps, graphs, tables, charts, written text, slide presentations, etc.) and “free” data products disseminated primarily through the web.

DESCRIPTION: The Web and Database Programmer is responsible for providing technical services as a member of proposal and project development teams for internal and external projects. S/he takes lead responsibility for implementing website and web survey projects, and for programming in areas including database development and management, custom data entry programs, etc. Updates and maintains MCIC website. Performs data processing and data analysis in a project team environment.

Key Responsibilities Include:

  • Participating in proposal development and project deliverable teams;
  • Implementing website and web custom survey projects for non-profit and corporate clients
  • Participating in project teams in areas of secondary research, database programming, statistical analysis(SPSS), and presentation graphics
  • Maintaining MCIC GIS-based websites and Intranet
  • Maintaining clients’ websites
  • Assisting in web server administration
  • Developing integrated databases from multiple sources
  • Other duties as assigned

Specific Skills:

  • Proficient in .NET programming (ASP.NET, C#, HTML, XML, JavaScript, VBA, and Visual Studio.NET) in Windows and web applications
  • Experience with SQL Server programming
  • Experience with PHP programming
  • Working knowledge of other relational database programs (FoxPro and Access)
  • Experience with statistical SPSS and spreadsheet software packages
  • Experience with IIS web hosting
  • Knowledge of GIS Software: ArcGIS 9.2 and Geocoder
  • Knowledge of US Census geographic and demographic data
  • Techniques for explaining technical concepts and procedures to non-technical users

Status:

This position is a half time position that will grow into a full time position. The position is salaried, and exempt from the overtime provisions of salary and wage administration. Reports to Director of Information Services.

Qualifications:

BA/BS or equivalent in geography or computer science or commensurate experience; excellent web, database, and desktop application programming skills; good communication skills; strong positive interpersonal and team skills; reliable follow-through habits to function in a deadline-driven team environment; GIS experience a big plus.

We know this is a big list of skills; while the ideal person will have all of them in their toolbox we’re also interested in hearing from people with some of them and an interest / excitement for learning the rest. MCIC is an awesome place to work; you’ll have a challenging and fast-paced workload and get to work with an amazing group of very, very smart people. To boot, in our office culture there’s a huge emphasis on learning new skills and innovating new approaches to data preparation and manipulation.

Send resumes and links to your work to jmassa@mcic.org.

How Not to Search for Housing

Nate Silver and New York Magazine have posted a neighborhood ranking article and interactive widget that make some awful assumptions and miss a huge opportunity.

From the article:

Our goal was to take advantage of this wealth of data and apply a little bit of science to the question. If there was anything that could plausibly affect one’s quality of life in a particular neighborhood, we tried to incorporate it. We sorted the dozens and dozens of statistics we compiled into twelve broad categories: housing cost (as measured on a price-per-square-foot basis, for both renters and buyers), housing quality (historic districts, code violations, cockroaches), transit and proximity (commute times to lower Manhattan and midtown, the density of subway coverage), safety (as measured by violent- and nonviolent-crime rates), public schools (test scores and parent satisfaction), shopping and services (the number of neighborhood amenities, especially supermarkets), food and restaurants (judged by density and quality of options), bars and nightlife (ditto), creative capital (arts venues as well as the number of residents engaged in the arts), diversity (in terms of both race and income), green space (park and waterfront access, street trees), and health and environment (noise, air quality, overall cleanliness).

Silver goes on to rank 50 of New York’s neighborhoods and includes an interactive Livability Calculator. The article rankings and the calculator, with its preset options of  “Young, Single, and Cash-Strapped”, “Double Income, No Kids”, “Married with Children”, and “Empty-Nested and Retired” as well as a customizable version, suffer from a number of flaws:

  • assuming that these profiles represent the best way to understand neighborhoods;
  • making some absolutely awful assumptions about what these profiles value and devalue (see next bullet list);
  • providing only a partial methodology, no justification for the assumptions of the parts of their methodology described, and no listing of source data;
  • assuming that everyone works in midtown or Lower Manhattan;
  • and, by ignoring the role that race plays in housing choice, perpetuating segregation.

Specifically, the index makes the following faulty assumptions:

  • the “Empty-Nested and Retired” aren’t interested in diversity,
  • the “Married with Children” are more interested in shopping and ‘creative capital’ than diversity;
  • the “Double Income, No Kids” care more about shopping than safety/crime and green space;
  • and the “Young, Single, and Cash-Strapped” don’t care at all about schools and barely care about safety/crime or housing quality.

These profiles are grossly over-simplified and, such as in the case of “Young, Single, and Cash-Strapped”, make implicit assumptions about folks within that profile – there are many young, poor, single parents that would benefit from some help finding a better neighborhood. Despite our apparent fascination with lists of this type, they provide little help when it comes to actually making a choice about neighborhoods. Every neighborhood search metric should be unique, tailored to each of our families’ needs and the resources we have to share.

But the worst part is that articles like this don’t take advantage of available technologies.  Whereas for decades the mainstream press was limited by the printed word to one-size-fits-all-lists, the barriers that formerly existed to unleashing the long tail of opportunity-based housing search are now nearly gone. Rather than crunch all of the data into rankings and sliders, Silver and NYM should have empowered their readers to search and explore the source data in an intuitive way, identifying specific neighborhoods that meet their specific needs and providing details as to how they might take advantage of those opportunities. Moving a slider along a bar with no units of analysis and then providing no maps, amenity listings, nor other visualization is far from helpful (although I’ll admit it is kind of fun and interesting).

Especially in New York, where the NYC Data Mine provides journalists with a great resource of data, stories and widgets like this do little more than drive speculative real estate investments and spread stereotypes about neighborhoods.

Where we live has an enormous impact on our lives and far, far too many make that decision based on shoddy information.

Notes:

  • I am the co-founder of MoveSmart.org, an opportunity-based neighborhood search system.
  • I’ve only visited NYC a handful of times and have never fully explored its diverse and amazing neighborhoods, so this post intentionally leaves out any comment on the actual list created. That said, that their top choice is near the bottom of the pack in affordability and diversity says a great deal about their assumptions and intended audience.