Scraping web pages is a well documented process. There are plenty of guides on how to pull information using plugins like Python’s Beautiful Soup or browser extensions like Kimono. Many web applications even provide public APIs for gathering information, such as Facebook’s Graph API. Yet, there is a growing set of popular mobile apps that do not have a public API. Apps like Yik Yak, Tinder, and others contain a wealth of information about the...
We know data tells us a lot. We write programs to automate data scraping. We spend hours creating data visualizations that help readers see what they need to see. We use data to make claims and generate stories that are reliable and have impact. Data is important and we seem to be surrounded by it. But that's not quite true. Sometimes, there is no data? A session at NICAR that really resonated with me was Data from Scratch:...
NICAR |
A big mistake when dealing with data can ruin your day. Luckily there are simple ways to avoid big mistakes and maintain credibility with your colleagues and your audience. At NICAR 2015, a panel of data journalists from The New York Times, Wall Street Journal and Atlanta Journal-Constitution discussed the road blocks they've encountered when working with data for a story. The panel, moderated by MedPage Today reporter Coulter Jones, featured advice and cautions to keep in mind when dealing with numbers,...
Attending NICAR14 as a computer science student without a journalism background was an interesting experience, to say the least. Never have I been surrounded by so many journalists (and developers) who were so passionate about data and the tools that can help them attain it. As the journalism and developer worlds are converging and as access to information is becoming ever more important, the question of “when it is appropriate to access data and when is...
Slides from "NewsCamp::The next generation of data viz" Alberto Cairo gave the first talk I saw at NICAR. The room was packed. He is the author of The Functional Art, maintains a blog the same name, and has become a highly respected expert in data visualization. Cairo’s talk was titled "NewsCamp::The next generation of data viz," and he made the slides available. Cairo said something that really stood out for me. “We should not focus...
This post has been updated with additional links to MozFest 2013 content. Kicking off the festival, Friday night begins with a Science Fair at Ravensbourne College. Just like last year, the Knight Lab team took a jump across the pond a couple weeks ago to attend the Mozilla Festival (October 25-27) in London. Still running on fumes from the remaining high felt from our attendance in 2012, the eleven of us were really excited to be there. This incredible...
Journalists who want to learn more technology often jump into HTML, CSS and Javascript. Those are great places to start (as Knight Lab and others have written before), but if you want to maximize the potential of your computer, one of the first things you should learn is the command line! Some quick background: Regular computer users access the computer via a graphical user interface (GUI). This interface allows you to interact with the machine...
One of Mozfest’s most prominent themes this year has been “Build + Teach the Web.” Throughout the keynotes and sessions, Mozilla has pushed its new initiative, Webmaker, as a rallying point for all of us interested in educating the world in becoming creators of the web rather than users. It’s a great initiative. Nothing is more important for us as makers than getting more people onboard, especially in journalism. But journalism has a specific set...
Can new software help journalists do a better job of gathering, organizing and making sense out of newsworthy information? That's the question the Knight Lab has set out to answer through our Untangled project. When Joe Germuska wrote about Untangled in April, we were imagining a multipurpose "browser-based knowledge management" tool that would help journalists keep track of information they found online. Since then, we have explored this idea through a variety of approaches: by...
Using the theory ‘Hire humans. Not skills. Not roles.’ as inspiration, the Lab’s profiles are Q&As with smart people who are shaping the future of media. Catch up and/or follow the series here. Meeting the lovely Tasneem Raja was the highlight of my week during 2012's South by Southwest Interactive Conference. Formerly a staff writer at The Chicago Reader and the news apps editor at The Bay Citizen, she is now an editor at Mother Jones leading their interactive storytelling team. She is...
Using the theory ‘Hire humans. Not skills. Not roles.’ as inspiration, the Lab’s profiles are Q&As with highly-impressive makers and strategists from media and its fringes, each with unique perspectives on journalism, publishing and communications technology. We’re talking to smart people who are shaping the future of media. Not all of them work in a newsroom, not all are big names, not all have fancy titles, but each is a bright person with something to...
Events |
Travis Swicegood Travis Swicegood, director of technology at Texas Tribune, spoke this week at the latest Hacks/Hackers Chicago Meet-up about the challenges of working with public data — real world data, as Swicegood calls it. There are plenty of challenges in collecting, managing and presenting data from a state the size of Texas — 26 million people, 254 counties, five major cities and a gross state economy of $1.2 trillion. Swicegood shared just a few...
Photo by Pete Karl II At the beginning of the month, most of the Knight Lab team participated in the Investigative Reporters and Editors annual computer-assisted reporting conference, NICAR, in Louisville, Feb. 28 - March 3. In its twentieth year, this conference educates journalists on tools and skills for digging deeper into stories and online publishing. Throughout the conference, the team participated in a bunch of sessions. Rich Gordon and Larry Birnbaum reviewed open source licenses and talked with DocumentCloud's Ted...
NICAR 2013 was my first journalism conference and, as a computer scientist, I can't help but wonder what the implications are of this unprecedented NICAR showing — the largest ever attendance at the conference. The world is changing, to be sure. We are no longer in an age where it is okay to be afraid of mathematics. Everyone from the worlds of journalism, government, and business must become math savvy and computer literate. Today, even some...
This is my first NICAR, but someone mentioned that NICAR 2012 was “The Year of the Map”, with panels, tutorials, and discussion otherwise centered around new, exciting tools like Mapbox and Leaflet. We invented new terms like BoyerMaps and used the rest of the year to make some incredible map-based news apps. If NICAR 2013 has a theme among its divergent, widely varied sessions, it is the resurgence of an antiquated tool: the spreadsheet. From...
This morning’s Year in CAR presentation at NICAR 2013 provided a great look back not only at some of the great investigative work of the last year, but also some really good visualizations and presentations. The full slide deck from Mark Horvit and Megan Luther’s presentation should be on the IRE site soon, but below are some of my favorites from their list of 2012 data-driven highlights. Click through for some of the great work...
Ideas |
Over the past few months something unusual has happened to public data projects: they’ve made national headlines. For journalists the most well known project was the gun permit holder map the Journal News in White Plains, New York published late last year featuring names and addresses of all registered gun owners in two New York counties. The map was controversial and inspired journalists and journalism pundits to weigh in on the project’s virtues and faults...
Ideas |
It's not as though 2012 was the year in which a digital journalism community popped fully-formed into the world. However, looking back, there are some developments in our world which deserve to be called out. For this installment of our Moving the Needle 2012 series, we take a look at some of the best. As always, we expect we've missed a few, so please fill in the gaps in the comments below, or on Twitter...
mozfest , Team news , Events |
Last week, a significant portion of the Knight Lab team was fortunate enough to join the Mozilla Festival in London, November 9-12. This incredible event, hosted by the Mozilla Foundation, is in its third year and intends to motivate an entire generation of web makers. As far as we can tell, they are doing this job well. In fact, during the Sunday morning keynote, Mozilla's executive director, Mark Surman shared an anecdote that an attendee had likened #MozFest to #SXSWi in its early...
mozfest , Team news , Events |
This year at MozFest, I responded to a “call for help” from three African nations to solve the mysterious drop in life expectancy they experienced. In a session called “Data Expeditions: Scout the Data Landscape with our Data Sherpas” (organized by a caped Michael Bauer) all participants split into groups to research and tell a story. My group, made of people with diverse backgrounds, reminded me how valuable a wide variety of skill sets can...
mozfest , Team news , Events |
I spent a lot of time at MozFest thinking about data and how we can use it as journalists. Here’s a quick recap of the sessions I attended and the lessons I learned. Data Expeditions More than 50 journalists and engineers followed a group of “data sherpas” in to a role-playing game-style hack on datasets in the “Data Expeditions” session. The three-hour session was intended to be a hack and teams — consisting of storytellers,...