Six things to know about successful open-source software

In the community of media and journalism innovators, it is commonly accepted that releasing software with an open-source license is the best way to maximize the chance that others will use your code. Yet, by any measure, the vast majority of open-source software goes nowhere.

That's why we've spent some time at Knight Lab trying to understand the dynamics of software adoption — especially the factors that cause open-source software to be widely adopted. After all, our mission is to develop software for journalists, publishers and media consumers -- software that gets used.

In researching the topic, I discovered that two faculty members at the University of Massachusetts literally wrote the book on this topic. In their book, "Internet Success: A Study of Open-Source Software Commons," Charles Schweik and Robert English studied more than 174,000 open-source projects shared on SourceForge, one of the largest and oldest (founded in 1999) repositories for open-source code.

"From 1998 to 2005, there was a lot of hype about high-profile success stories like Linux or Apache, that were really anomalies," Schweik said.  "We were trying to get a better handle on the broader population of open-source projects, and at the time Sourceforge was arguably the dominant open-source hosting site."

I'll summarize their key findings below, but first I think it is important to understand a little about how they approached their research.

The researchers based their work on an analyisis projects hosted on SourceForge as of 2009. They classified projects based on what "stage" the software had reached as of that time:

  • Initiation stage — the period of time before the first public code release
  • Growth stage — the period after the first release until work on the software appeared to have stopped.

A successful product, as defined by Schweik and English, achieves at least three software releases and has value for "at least a few users." As evidence of user value, they looked at project attributes such as the number of downloads and installations, the presence of continued development activity, posts to project discussion boards or email lists and the addition of one or more developers.

Schweik and English supplemented their analysis of the SourceForge data with a survey of 1,400 open-source developers. This enabled them to ask questions about a subset of open-source projects that could not be answered just by looking at SourceForge data. Using this data they were able to identify characteristics of open-source projects that correlate statistically to greater likelihood of success.

Here's a summary of what they found:

1. Most open-source projects are not successful

Among the 174,333 projects they reviewed, Schweik and English found they could assess success or abandonment for 145,475. Of that total, only one in six -- 17 percent -- were successful. Almost half the projects (46 percent) were abandoned in the initiation stage -- before the first software release. More than a third (37 percent) were abandoned after the initial release.

2. Successful projects have some common characteristics:

  • A "relatively clearly defined vision and a mechanism to communicate the vision early in the project's life"
  • A clearly defined set of users who have a need that can be met by the software
  • Well-articulated and clear goals established by the project's leaders
  • Good project communication -- a quality website, good documentation, a bug-tracking system and a communication system such as an email list or forum.
  • Once a project has achieved its initial release, a software architecture that is modular -- so future development tasks can be carved out at different levels of complexity for other developers to work on. (Modular architecture alone isn't enough -- many abandoned projects were also modular, Schweik said.)

Most of these characteristics come down to effective leadership, Schweik told me in an interview. "There is someone who was more diligent, who was trying both to describe their project to the world and to provide a clear vision of where the project was going, and articulating those goals," Schweik said.

3. Open-source projects flourish when developers are also users of the software

"What we found across the board was the motivation of a user-centered need," Schweik said. "I need this software, so I want to work on it."

This is consistent with evaluations done by the Knight Foundation for software projects funded under the Knight News Challenge in 2007-08 and 2009. Successful projects, such as DocumentCloud and Ushahidi, had developers who had reason to use the software after its initial release.

4. The global Internet has made it easier to find software collaborators

While open-source projects can be successful without building a large community of code collaborators, Schweik and English were surprised to discover where additional developers were coming from. When a project added a developer to the team, 58 percent of those developers came from a different continent than the original lead developer -- most commonly, Europe. Schweik said that for many successful projects, developers from multiple continents had never met in person.

The Internet has enabled "intellectual matchmaking," Schweik said. "By having a hub where people go looking for things, it's allowing people with a passion and interest to go connect with each other, and it's driven by this user-centered need."

Adding one or more developers was an indicator of software success, the research found.  The addition of even one developer was meaningful, since most open-source projects are relatively small, Schweik said.

5. Some characteristics thought to be important in the spread of open-source software turn out not to matter

Schweik and English looked at many different characteristics that researchers had suggested could be important for open-source software success. Here are some of the characteristics they found do not matter:

  • which operating system (Apple, Windows, Unix) the code was written for
  • how many developers were involved
  • whether the project has a formalized system of governance (Schweik suggested this doesn't matter much because most open-source projects are so small)
  • which type of open-source license was used
  • whether the project has a source of funding

"In our research, 75 percent of the open-source projects had no funding of any kind," Schweik said. "Investment doesn't appear to drive open-source projects. Rather, the need for the software drives development."

That's not to say funding is unimportant -- "projects that are funded have higher success rates," Schweik said.  But he said it may well be that good projects attract funding, rather than the other way around.

6. Success doesn't have to mean large-scale adoption

Much of the attention to open-source software has gone to massive projects such as Linux, which has had more than 1,000 code contributors and is used throughout the technology industry and businesses large and small. But a project can also be successful, Schweik said, if it meets the ongoing needs of a small number of users.

"A clearly identifiable need, even without a big user base, can be successful," Schweik said.

* * *

I asked Schweik to put himself in the position of a funder, such as the Knight Foundation, that wants to invest in making open-source projects successful. What criteria should such a funder use to determine which projects to support financially?

For projects in the initiation stage (no software has yet been released), Schweik said, a funder should look for:

  • how well-defined the vision for the project is;
  • whether the project's developers have a track record of contributing productively to open-source projects or "leading through doing";
  • how professional and up to date the project's web presence is;
  • evidence that the project leaders have thought about a marketing plan;
  • evidence that the people involved in starting the project have a need for it that will extend beyond the first release.

For projects in the growth stage (after the first code release), Schweik said, a funder should consider:

  • whether the project has a well-defined set of users;
  • whether there is a natural constituency of developers interested in continuing to use the software;
  • whether the developers have prior open-source experience;
  • as with the initiation stage, whether the people leading the project have demonstrated leadership by articulating a clear vision, having a professional web presence and maintaining an active bug-tracking system or other communication platform for interacting with the user community.

About the author

Rich Gordon

Professor and Director of Digital Innovation

Journalism/tech intersection, my passion for 25 years, data journalism, Miami Herald web director, now hacker journalism.

Latest Posts

  • Prototyping Augmented Reality

    Something that really frustrates me is that, while I’m excited about the potential AR has for storytelling, I don’t feel like I have really great AR experiences that I can point people to. We know that AR is great for taking a selfie with a Pikachu and it’s pretty good at measuring spaces (as long as your room is really well lit and your phone is fully charged) but beyond that, we’re really still figuring...

    Continue Reading

  • Capturing the Soundfield: Recording Ambisonics for VR

    When building experiences in virtual reality we’re confronted with the challenge of mimicking how sounds hit us in the real world from all directions. One useful tool for us to attempt this mimicry is called a soundfield microphone. We tested one of these microphones to explore how audio plays into building immersive experiences for virtual reality. Approaching ambisonics with the soundfield microphone has become popular in development for VR particularly for 360 videos. With it,...

    Continue Reading

  • How to translate live-spoken human words into computer “truth”

    Our Knight Lab team spent three months in Winter 2018 exploring how to combine various technologies to capture, interpret, and fact check live broadcasts from television news stations, using Amazon’s Alexa personal assistant device as a low-friction way to initiate the process. The ultimate goal was to build an Alexa skill that could be its own form of live, automated fact-checking: cross-referencing a statement from a politician or otherwise newsworthy figure against previously fact-checked statements......

    Continue Reading

  • Northwestern is hiring a CS + Journalism professor

    Work with us at the intersection of media, technology and design.

    Are you interested in working with journalism and computer science students to build innovative media tools, products and apps? Would you like to teach the next generation of media innovators? Do you have a track record building technologies for journalists, publishers, storytellers or media consumers? Northwestern University is recruiting for an assistant or associate professor for computer science AND journalism, who will share an appointment in the Medill School of Journalism and the McCormick School...

    Continue Reading

  • Introducing StorylineJS

    Today we're excited to release a new tool for storytellers.

    StorylineJS makes it easy to tell the story behind a dataset, without the need for programming or data visualization expertise. Just upload your data to Google Sheets, add two columns, and fill in the story on the rows you want to highlight. Set a few configuration options and you have an annotated chart, ready to embed on your website. (And did we mention, it looks great on phones?) As with all of our tools, simplicity...

    Continue Reading

  • Join us in October: NU hosts the Computation + Journalism 2017 symposium

    An exciting lineup of researchers, technologists and journalists will convene in October for Computation + Journalism Symposium 2017 at Northwestern University. Register now and book your hotel rooms for the event, which will take place on Friday, Oct. 13, and Saturday, Oct. 14 in Evanston, IL. Hotel room blocks near campus are filling up fast! Speakers will include: Ashwin Ram, who heads research and development for Amazon’s Alexa artificial intelligence (AI) agent, which powers the...

    Continue Reading

Storytelling Tools

We build easy-to-use tools that can help you tell better stories.

View More