ONA15: How news organizations build simple bots to help report the news

It’s no secret that newsrooms are increasingly using bots to cut down on busy work. Software now routinely churns out quarterly earnings stories for The Associated Press and earthquake alerts for Los Angeles Times, freeing reporters to pursue more in-depth projects. And while no bot can write 3,000-word investigative stories, it can assist reporters by alerting them to new data and filtering the information for them.

While the idea of creating a bot may sound daunting, automating parts of the news gathering process can also be as simple as clicking a few buttons, according to some industry pros. At ONA’s Build Bots to Boost Your Reporting session John Keefe, Jenny Ye, Justin Myers, Philana Patterson, and Honest Charley Bodkin showed attendees how common software can help reporters find news.

In just five minutes, Keefe, senior editor at WNYC’s data news team, showed us how his team built a news bot that scored a scoop on a federal court case, using IFTTT.

IFTTT, for the uninitiated, is an acronym for If This Then That, and it’s a web service that allows users to create connections between apps such as Twitter, Dropbox, and Gmail through “recipes.” These recipes can automate a range of digital chores, from sending users a text message when the weather changes to logging their travels in Google Spreadsheets.

In WNYC’s case, Keefe set up the bot to automatically scan PACER, a publicly accessible electronic repository of federal court documents, for new feed items related to the United States v. Silver case. He then asked the bot to send him an email whenever there were new PACER files on the case. And he did it all through IFTTT’s point and click interface.

The result? A scoop for WNYC when an email alerted them to new development on indictment against former New York State Assembly Speaker Sheldon Silver, who faced corruption charges.

For the more code-savvy, WNYC’s Ye demonstrated a JavaScript scraper that fuels Nailbiter Bot, a Twitter account that tweets automatically when the score of a NCAA tournament game is within eight points with less than three minutes left in the game.

Lego block approach

The key to efficient automation is breaking down each project into units that can be built, reused, and mixed like Lego blocks, said Myers, news automation editor at AP. There are four essential parts to building a news bot, he said.

Input: The bot needs to react to new development and gather data. For example, when The Associated Press was investigating health care data breaches, Myers set up a bot to scrape the U.S. Department of Health and Human Services website for new reports of data breaches. He also set up a subscriber list for interested editors and reporters using Google Spreadsheets.

Processing: The bot needs to filter the data and surface useful information. In this case, Myers programmed the bot to filter reports of new breaches according to each reporter and editor’s preferences. For example, the bot might alert an editor only when the breach affected a large number of patients.

Output: The bot needs to tell others about the data. Myers’ bot emailed a customized list of breach reports to each subscriber.

Scheduling: The bot needs to perform the regularly. Bots can run nearly continuously or can be scheduled to run at certain intervals. Myers ran his bot once a day to scan for new breaches.

While bots can be invaluable in saving journalists time and effort in reporting, the danger of “overbot” exists, Myers said. To avoid inundating reporters with email alerts, Myers personally receives a copy of every reporter’s alert to gauge the extent of information overload.

At the end of the day, he said, bot building requires continued conversations between developers and reporters. That’s something bots can’t replace.

Here are the slides from the Build Bots to Boost Your Reporting session at ONA.

 

Latest Posts

  • A Google Spreadsheets change affecting TimelineJS users

    Google recently changed something about their Sheets service which is causing many people to run into an error when they are making a new timeline. Note: there should be no impact on existing timelines! After this change, many of you click on the "preview" and get this message: An unexpected error occurred trying to read your spreadsheet data [SyntaxError] Timeline configuration has no events. There is a straightforward work-around, but it requires those of you who have...

    Continue Reading

  • How Americans think and feel about gun violence

    A man killed his wife, then himself. I want you to see his face and learn that he enjoyed fishing with his grandchildren. A small-time drug dealer is shot by two men in a parking lot. I find his Facebook profile and a photo shows him striking a playfully irreverent pose, giving the camera the middle finger. The photo’s comments take a mournful turn after a certain date. “Rest easy bro ???” Gun Memorial runs...

    Continue Reading

  • Software developers interested in journalism: Northwestern and The Washington Post want you!

    Northwestern University and The Washington Post are offering a unique opportunity for two talented software developers interested in applying their programming skills in media and journalism. Here’s the proposition: (1) a full-tuition scholarship to earn a master’s degree in journalism at Northwestern University, followed by (2) a six-month paid internship with The Post’s world-class engineering team, with the possibility of subsequent full-time employment. These opportunities are made possible by the John S. and James L....

    Continue Reading

  • What happened when Gun Memorial let anyone contribute directly to victim profiles

    If you’re reporting local or niche news, there’s a good chance that your audience collectively knows more about the story than you do. That’s especially true for us at Gun Memorial, a small publication with a nationwide mission of covering every American who is shot dead. In our latest, mostly successful, experiment, we let readers add to our stories without editor intervention. This article shares some lessons from that experience. Asking for reader contributions A...

    Continue Reading

  • How conversational interfaces make the internet more accessible for everyone

    This story is part of a series on bringing the journalism we produce to as many people as possible, regardless of language, access to technology, or physical capability. Find the series introduction, as well as a list of published stories here. In 2004, human-computer interaction professor Alan Dix published the third edition of Human-Computer Interaction along with his colleagues, Janet Finley, Gregory Abowd, and Russell Beale. In a chapter called “The Interaction,” the authors wrote...

    Continue Reading

  • Three tools to help you make colorblind-friendly graphics

    This story is part of a series on bringing the journalism we produce to as many people as possible, regardless of language, access to technology, or physical capability. Find the series introduction, as well as a list of published stories here. I am one of the 8% of men of Northern European descent who suffers from red-green colorblindness. Specifically, I have a mild case of protanopia (also called protanomaly), which means that my eyes lack...

    Continue Reading

Storytelling Tools

We build easy-to-use tools that can help you tell better stories.

View More