Don't believe your eyes: Learning how to be critical with Alberto Cairo

A previous version of this story misstated Alberto Cairo's position on the proportion of people who oversimplify infographics. We've removed the number. Read Cairo's take on thinking critically about data visualizations, including his reaction to this piece, here.

Not 15 minutes into the first session at my first NICAR conference, I felt utterly mortified. Here was Alberto Cairo, author of “The Functional Art,” telling me the graphic I retweeted not two weeks ago with the comment, “wow,” sucked.

There goes my journalistic credibility on Twitter.

The graphic in question: a map of the United States demonstrating how half of the country’s gross domestic product (GDP) is concentrated in only a sprinkling of areas.

I didn’t think about it twice. Actually, I don’t think I thought about it at all. The tweet linked to a blog post from The Washington Post’s Chris Cillizza about why we need big cities. The post was based on the graphic, credited to a user on the forum Reddit named Alexandr Trubetskoy.

What could be a wrong in a graphic cited by The Washington Post?

‘Sucked’ is an exaggeration. It was nowhere near the standards of Rather, it was “oversimplified” because it failed to illustrate the nuances of the information presented.

“This map is just recreating a population map,” he said. “A number in isolation doesn’t tell anything.”

He then pulled up three additional maps: one of urbanized areas and two representing the migration of college graduates. All were remarkably similar.

I realized I had been allured by the complementary blues and oranges and forgot the graphic was supposed to show me something other than the obvious fact that large cities are usually hubs of the national economy.

And thus, we have Cairo’s five tenets of a great data viz. It must be:

  1. Truthful
  2. Functional
  3. Beautiful
  4. Insightful
  5. Enlightening

Trubetskoy’s graphic hit only a few of these.

I embarrassedly thought about my retweet and the numerous other times I must’ve tweeted a data viz project that came together beautifully on my screen. While I opted not to take down these incriminating tweets in true self-deprecating humor, I decided to test out my improved data viz literacy.

I considered a 2012 project from The New York Times called “Girls Lead in Science Exam, but Not in the United States,” something that supported my notions that in the U.S., girls are less likely to be interested in science than boys are.

Girls Lead in Science Exam, but Not in the United States, 2012, The New York Times

On my second look, I noticed I had neglected the note on the bottom that the data from China, Taiwan and the United Arab Emirates were only representative of a single city in those countries.  This means the visualization is comparing mostly national numbers to some city numbers, which would undoubtedly yield a different comparison.

Next question. What exactly was the interactive telling me that we have not been discussing already? I had fallen into the second trap Cairo mentioned: “The problem with preconceived ideas and patterns is that you have to test them somehow.”

We are not told what the perfect score is. Yet, the 14-point difference between boys’ average of 509 and girls’ average of 495 is the basis for both the headline and the graphic. What if the perfect score is 600? What is the statistically significant difference?

Additionally, the graphic is so focused on boys vs. girls that it fails to tell us how the countries perform overall. In Jordan, a country used in the accompanying article as an example in which girls do much better on the exam than boys, the average scores were 398 and 433, respectively. Yes, girls had done better. But neither boys nor girls from Jordan performed as well on average as the boys and girls did in the U.S.  Thus, U.S. is being criticized for a ‘gender gap’ in comparison to a country in which its students collectively perform worse on the exams.

And what about the fact that the Organisation for Economic Co-operation and Development, the data source for the graphic, said in its 2009 executive summary that “on average across OECD countries…gender differences in science performance tend to be small,” and that the gender gap of concern is among reading performance?

But The New York Times was factually accurate. It reported the numbers as they are. It was just that multi-dimensional data was only represented by only one of its dimensions.

“Visualization is not something to be seen, it is something to be read,” he said.

Yet, here I was, seeing and not reading.

The interactive is truthful. It is functional. It is beautiful. It is functionally beautiful. I just learned that sometimes, we lose the details in the beauty – just as we are all guilty of retweeting articles only by their headlines without even clicking through to understand the complexity of the matter.

Every year New York Times' Chrys Wu curates 'Slides, Links & Tutorials' from the annual CAR conference: 2014, 20132012 and 2011. They are a fantastic resource for all journalists and investigators of all skill levels.

About the author

Suyeon Son

Undergraduate Fellow

Latest Posts

  • Prototyping Augmented Reality

    Something that really frustrates me is that, while I’m excited about the potential AR has for storytelling, I don’t feel like I have really great AR experiences that I can point people to. We know that AR is great for taking a selfie with a Pikachu and it’s pretty good at measuring spaces (as long as your room is really well lit and your phone is fully charged) but beyond that, we’re really still figuring...

    Continue Reading

  • Capturing the Soundfield: Recording Ambisonics for VR

    When building experiences in virtual reality we’re confronted with the challenge of mimicking how sounds hit us in the real world from all directions. One useful tool for us to attempt this mimicry is called a soundfield microphone. We tested one of these microphones to explore how audio plays into building immersive experiences for virtual reality. Approaching ambisonics with the soundfield microphone has become popular in development for VR particularly for 360 videos. With it,...

    Continue Reading

  • How to translate live-spoken human words into computer “truth”

    Our Knight Lab team spent three months in Winter 2018 exploring how to combine various technologies to capture, interpret, and fact check live broadcasts from television news stations, using Amazon’s Alexa personal assistant device as a low-friction way to initiate the process. The ultimate goal was to build an Alexa skill that could be its own form of live, automated fact-checking: cross-referencing a statement from a politician or otherwise newsworthy figure against previously fact-checked statements......

    Continue Reading

  • Northwestern is hiring a CS + Journalism professor

    Work with us at the intersection of media, technology and design.

    Are you interested in working with journalism and computer science students to build innovative media tools, products and apps? Would you like to teach the next generation of media innovators? Do you have a track record building technologies for journalists, publishers, storytellers or media consumers? Northwestern University is recruiting for an assistant or associate professor for computer science AND journalism, who will share an appointment in the Medill School of Journalism and the McCormick School...

    Continue Reading

  • Introducing StorylineJS

    Today we're excited to release a new tool for storytellers.

    StorylineJS makes it easy to tell the story behind a dataset, without the need for programming or data visualization expertise. Just upload your data to Google Sheets, add two columns, and fill in the story on the rows you want to highlight. Set a few configuration options and you have an annotated chart, ready to embed on your website. (And did we mention, it looks great on phones?) As with all of our tools, simplicity...

    Continue Reading

  • Join us in October: NU hosts the Computation + Journalism 2017 symposium

    An exciting lineup of researchers, technologists and journalists will convene in October for Computation + Journalism Symposium 2017 at Northwestern University. Register now and book your hotel rooms for the event, which will take place on Friday, Oct. 13, and Saturday, Oct. 14 in Evanston, IL. Hotel room blocks near campus are filling up fast! Speakers will include: Ashwin Ram, who heads research and development for Amazon’s Alexa artificial intelligence (AI) agent, which powers the...

    Continue Reading

Storytelling Tools

We build easy-to-use tools that can help you tell better stories.

View More