Weaving a Patchwork Map ... in Real Time

We did something a little creative and unique at WNYC this past election night: We mapped the vote by "community type."

This revealed the diversity of the vote across New York State -- from the cities to the suburbs, boom towns and "service worker" centers -- in real time, on the air and on the WNYC home page.

And the diversity is striking. Despite Democratic wins in every statewide race, the Republicans running for state attorney general and comptroller "won" every community type outside "Industrial Metropolis" and "Campus & Careers" counties.

Patchwork Nation's Dante Chinni talked about this on air during WNYC's coverage election night, and has written more about it since.

The live map was a mashup of Patchwork Nation's unique take on the nation and the Associated Press's live vote totals. At the request of WNYC, Patchwork Nation programmers dove into the AP test results and quickly wove them into a new map based on PN's existing county maps -- customizing them for the event and adding real-time percentages by community type.

Bringing the Threads Together

In the months before the election, I had wondered how we might better understand the early returns -- those that come in typically between 9 p.m. and 10:30 p.m. -- which often don't match the final results. I wanted more clarity.

At a Hacks/Hackers Open-Source-a-Thon, I started playing with the election data with help from Al Shaw (then at TalkingPointsMemo, now at ProPublica) and Chrys Wu (of Hacks/Hackers and ONA fame).

That evolved into a little program I wrote in Sinatra that generated vote-total map at the left, shading counties darker as more of their precincts reported. It also helped me better understand how the data were structured, how to retrieve the numbers and what it might take to make a live map.

So when Chinni asked if WNYC had any county-level data sets we'd like to put through the Patchwork Nation treatment, I had the perfect candidate.

Open All Night: The Great Urban Hack NYC

For 26 hours this weekend, a bunch of journalists and coders got together to make lots of great things designed to help the citizens of New York City.

My blog post summarizing the event and all of the resulting projects is upon Hacks/Hackers.

I helped Jenny 8. Lee, Chrys Wu and Stephanie Pereira organize the event. Then I joined a team working with digital heaps of NYC taxi trip data to make data visualizations and start some other projects. My favorite one is here (with a detail below), which is a representation of taxi usage for 24 hours, set around a clock. Beautiful. It was built by Zoe Fraade-Blanar using Processing and data crunched by the other teammates.

Click image for full view

I came away from the event with many new connections, excitement about learning Processing, some more skills in Sinatra and a note to check out Bees with Machine Guns(!)

NYC Taxi Visualizations: The Pulse of the City

Fantastic presentation of GPS data from New York City Taxis in the New York Times today. So many stories embedded here, giving a wonderful sense of the city.

The ability to "play" the data is especially key. And the treatment itself hits all of my buttons. You just get it.

Beautiful.

Paint Sticky Data (Please)

I'm into info. I want it accurate, relevant and clear.

On the radio, we try to paint clear, understandable, and journalistically-sound images of the mind -- the vivid mental pictures you see while listening to good storytelling.

Actual images can tell rich stories, too. The best photojournalism certainly does. Some pictures hit you in the chest.

But images drawn from data -- infographics, or visualizations -- rarely tell a story so well.

And they almost never hit me in the chest.

Why not? With all of the technology available, why can't we create really good visualizations that project understanding, timeliness, utility and ... dare I say ... stories?

I'm on the lookout. And I'm defining what I want to see.

For that definition, I've made a checklist based on one of my all-time favorite books, Made to Stick, by Chip Heath & Dan Heath (Random House, 2007). The initial words come from their Six Principles of Sticky Ideas; the rest is my application of their concepts.

For me, the best information images are ...

Simple: Non-geeks can absorb it within a few seconds
Unexpected: It fills a gap in our knowledge
Concrete: It takes advantage of our senses and understandings
Credible: It is journalistically sound, from a trusted source, without bias
Emotional: It hits you in the chest, you feel the data
Story: It tells one

And I'll add one more:

Relevant: It is timely, current and useful

Got examples that ring all seven bells? Maybe even four? Share them in the comments here or email me: john (at) designAgitator.com.

-----

The map detail above is from my favorite example at the moment, which is on Slate. Jump over there, take a look, and then run it through the checklist:

Simple? Once you know that blue is jobs gained and red is jobs lost, you're set. Just press play.
Unexpected? Seriously so. The speed of change is amazing.
Concrete? The familiar map orients me at a glance; I respond quickly to the circle sizes, colors and densities.
Credible? Bureau of Labor Statistics, Slate.
Emotional? Oh yeah. I saw someone actually shiver while watching it.
Story? Definitely.
Relevant? Yup.

Rings my bell.

Rapid News Visualizations: Prototype 1

In my quest for timely, interesting, understandable info-graphics, I've set up a prototyping challenge for myself: Upon finding news data, turn it into something visual, compelling and useful ... as fast as possible.

I'm prepared to fail quickly and often.

In this case, "as fast as possible" was three days to make, another two to find time to post it. The result is not wholly useful. And you can't absorb it quickly. And it's a little misleading.

But it's a start.

Here is what I made. It's a visual representation of the attendance rates for every public school in New York City on Thursday, May 21, 2009. The New York City Department of Education started posting this data the previous day as the Swine Flu/H1N1 outbreak was causing kids to stay home.

It tells me two things off the bat:

1) Queens and Brooklyn schools had much lower attendance rates than Manhattan and Staten Island schools.

2) Teens skip school on nice May days.

No. 2 is apparent because almost every red square is a high school, which have notoriously low rates this time of year. For a better indication of potentially flu-related absences, I'd chart the difference between these absentee rates and a typical May day at each school ... which is info I don't have. Yet.

Initially I published this in Google Maps, which was interactive and allowed you to click on schools for specific info. But Google Maps only plotted about 100 or so schools, and there are more than 1,000 here. Instead, I did it in Google Earth on my own computer and took a snapshot. Here's another.

Kinda cool. Was fun to do.

Next!

----------

Anatomy of the process:

Daily absentee data from the school system is here.
An Excel spreadsheet with general data on each school is here.
I crossed these two data sets in Access to match school numbers with addresses.
I got the latitude and longitude for each address, in 500-line batches, here.
I spent a lot of time learning about KML files, writing them, failing, trying again.
I made colored icons in Photoshop, and used Excel to assign each school the correct icon.
I put all of the relevant data into one spreadsheet and fed it to this little helper ...
Which gave me this KML file ...
Which I fed to Google Earth, running on my Mac.

Feeling Information

Information and raw data are piling up faster than our ability to absorb it. And the tools available to access, understand, visualize and feel that information are woefully inadequate.

I believe journalism, design thinking and information technology can be wielded to express these stories in ways never before considered. And I'm part of a small posse poised to do just that.

If this interests you, or if you'd like to join our rag-tag group, write me: john (at) designAgitator.com.

In the meantime, assume some of the gaps in designAgitator postings mean we're hard at work helping to explain the world(!)