Don't believe your eyes: Learning how to be critical with Alberto Cairo

A previous version of this story misstated Alberto Cairo's position on the proportion of people who oversimplify infographics. We've removed the number. Read Cairo's take on thinking critically about data visualizations, including his reaction to this piece, here.

Not 15 minutes into the first session at my first NICAR conference, I felt utterly mortified. Here was Alberto Cairo, author of “The Functional Art,” telling me the graphic I retweeted not two weeks ago with the comment, “wow,” sucked.

There goes my journalistic credibility on Twitter.

The graphic in question: a map of the United States demonstrating how half of the country’s gross domestic product (GDP) is concentrated in only a sprinkling of areas.

I didn’t think about it twice. Actually, I don’t think I thought about it at all. The tweet linked to a blog post from The Washington Post’s Chris Cillizza about why we need big cities. The post was based on the graphic, credited to a user on the forum Reddit named Alexandr Trubetskoy.

What could be a wrong in a graphic cited by The Washington Post?

‘Sucked’ is an exaggeration. It was nowhere near the standards of Rather, it was “oversimplified” because it failed to illustrate the nuances of the information presented.

“This map is just recreating a population map,” he said. “A number in isolation doesn’t tell anything.”

He then pulled up three additional maps: one of urbanized areas and two representing the migration of college graduates. All were remarkably similar.

I realized I had been allured by the complementary blues and oranges and forgot the graphic was supposed to show me something other than the obvious fact that large cities are usually hubs of the national economy.

And thus, we have Cairo’s five tenets of a great data viz. It must be:

  1. Truthful
  2. Functional
  3. Beautiful
  4. Insightful
  5. Enlightening

Trubetskoy’s graphic hit only a few of these.

I embarrassedly thought about my retweet and the numerous other times I must’ve tweeted a data viz project that came together beautifully on my screen. While I opted not to take down these incriminating tweets in true self-deprecating humor, I decided to test out my improved data viz literacy.

I considered a 2012 project from The New York Times called “Girls Lead in Science Exam, but Not in the United States,” something that supported my notions that in the U.S., girls are less likely to be interested in science than boys are.

Girls Lead in Science Exam, but Not in the United States, 2012, The New York Times

On my second look, I noticed I had neglected the note on the bottom that the data from China, Taiwan and the United Arab Emirates were only representative of a single city in those countries.  This means the visualization is comparing mostly national numbers to some city numbers, which would undoubtedly yield a different comparison.

Next question. What exactly was the interactive telling me that we have not been discussing already? I had fallen into the second trap Cairo mentioned: “The problem with preconceived ideas and patterns is that you have to test them somehow.”

We are not told what the perfect score is. Yet, the 14-point difference between boys’ average of 509 and girls’ average of 495 is the basis for both the headline and the graphic. What if the perfect score is 600? What is the statistically significant difference?

Additionally, the graphic is so focused on boys vs. girls that it fails to tell us how the countries perform overall. In Jordan, a country used in the accompanying article as an example in which girls do much better on the exam than boys, the average scores were 398 and 433, respectively. Yes, girls had done better. But neither boys nor girls from Jordan performed as well on average as the boys and girls did in the U.S.  Thus, U.S. is being criticized for a ‘gender gap’ in comparison to a country in which its students collectively perform worse on the exams.

And what about the fact that the Organisation for Economic Co-operation and Development, the data source for the graphic, said in its 2009 executive summary that “on average across OECD countries…gender differences in science performance tend to be small,” and that the gender gap of concern is among reading performance?

But The New York Times was factually accurate. It reported the numbers as they are. It was just that multi-dimensional data was only represented by only one of its dimensions.

“Visualization is not something to be seen, it is something to be read,” he said.

Yet, here I was, seeing and not reading.

The interactive is truthful. It is functional. It is beautiful. It is functionally beautiful. I just learned that sometimes, we lose the details in the beauty – just as we are all guilty of retweeting articles only by their headlines without even clicking through to understand the complexity of the matter.

Every year New York Times' Chrys Wu curates 'Slides, Links & Tutorials' from the annual CAR conference: 2014, 20132012 and 2011. They are a fantastic resource for all journalists and investigators of all skill levels.

About the author

Suyeon Son

Undergraduate Fellow

Latest Posts

  • Building a Community for VR and AR Storytelling

    In 2016 we founded the Device Lab to provide a hub for the exploration of AR/VR storytelling on campus. In addition to providing access to these technologies for Medill and the wider Northwestern community, we’ve also pursued a wide variety of research and experimental content development projects. We’ve built WebVR timelines of feminist history and looked into the inner workings of ambisonic audio. We’ve built virtual coral reefs and prototyped an AR experience setting interviews...

    Continue Reading

  • A Brief Introduction to NewsgamesCan video games be used to tell the news?

    When the Financial Times released The Uber Game in 2017, the game immediately gained widespread popularity with more than 360,000 visits, rising up the ranks as the paper’s most popular interactive piece of the year. David Blood, the game’s lead developer, said that the average time spent on the page was about 20 minutes, which was substantially longer than what most Financial Times interactives tend to receive, according to Blood. The Uber Game was so successful that the Financial...

    Continue Reading

  • With the 25th CAR Conference upon us, let’s recall the first oneWhen the Web was young, data journalism pioneers gathered in Raleigh

    For a few days in October 1993, if you were interested in journalism and technology, Raleigh, North Carolina was the place you had to be. The first Computer-Assisted Reporting Conference offered by Investigative Reporters & Editors brought more than 400 journalists to Raleigh for 3½ days of panels, demos and hands-on lessons in how to use computers to find stories in data. That seminal event will be commemorated this week at the 25th CAR Conference, which...

    Continue Reading

  • Prototyping Augmented Reality

    Something that really frustrates me is that, while I’m excited about the potential AR has for storytelling, I don’t feel like I have really great AR experiences that I can point people to. We know that AR is great for taking a selfie with a Pikachu and it’s pretty good at measuring spaces (as long as your room is really well lit and your phone is fully charged) but beyond that, we’re really still figuring...

    Continue Reading

  • Capturing the Soundfield: Recording Ambisonics for VR

    When building experiences in virtual reality we’re confronted with the challenge of mimicking how sounds hit us in the real world from all directions. One useful tool for us to attempt this mimicry is called a soundfield microphone. We tested one of these microphones to explore how audio plays into building immersive experiences for virtual reality. Approaching ambisonics with the soundfield microphone has become popular in development for VR particularly for 360 videos. With it,...

    Continue Reading

  • Audience Engagement and Onboarding with Hearken Auditing the News Resurrecting History for VR Civic Engagement with City Bureau Automated Fact Checking Conversational Interface for News Creative Co-Author Crowdsourcing for Journalism Environmental Reporting with Sensors Augmented Reality Visualizations Exploring Data Visualization in VR Fact Flow Storytelling with GIFs Historical Census Data Information Spaces in AR/VR Contrasting Forms Of Interactive 3D Storytelling Interactive Audio Juxtapose Legislator Tracker Storytelling with Augmented Reality Music Magazine Navigating Virtual Reality Open Data Reporter Oscillations Personalize My Story Photo Bingo Photojournalism in 3D for VR and Beyond Podcast Discoverability Privacy Mirror Projection Mapping ProPublica Illinois Rethinking Election Coverage SensorGrid API and Dashboard Sidebar Smarter News Exploring Software Defined Radio Story for You Storyline: Charts that tell stories. Storytelling Layers on 360 Video Talking to Data Visual Recipes Watch Me Work Writing and Designing for Chatbots
  • Prototyping Spatial Audio for Movement Art

    One of Oscillations’ technical goals for this quarter’s Knight Lab Studio class was an exploration of spatial audio. Spatial audio is sound that exists in three dimensions. It is a perfect complement to 360 video, because sound sources can be localized to certain parts of the video. Oscillations is especially interested in using spatial audio to enhance the neuroscientific principles of audiovisual synchrony that they aim to emphasize in their productions. Existing work in spatial......

    Continue Reading

Storytelling Tools

We build easy-to-use tools that can help you tell better stories.

View More