I wrote recently about how to select a partner when deploying Open Source projects within an enterprise — basically you want someone who is engaged with the community and can represent your interests and needs to that community. Now that we’ve covered how to find someone to help you along your journey, how do you decide what Open Source projects are ready for use in your organization?
First off, unless you’re a super agile software user, you need stability in the projects you deploy. It is cool that some organizations are willing to deploy code from upstream weekly (or even daily), but that’s not everyone. It is likely that you’ll want to run whatever code you deploy today for a fair while — perhaps years. You need the projects you deploy to understand and support that. This doesn’t mean you never deploy, but it means you want to deploy small incremental bug fix releases, not something with major changes in it. When you do get around to upgrading between major releases, you also want projects that understand that compatibility between major releases is super important — redeveloping your apps because a component shipped a major release isn’t desirable for most organizations.
Some things to look for:
- Breaking changes to user interfaces and APIs are minimized between releases.
- Large changes have a clear rationale as to how they are in the best interests of the community as a whole, and provide a clear and well tested transition plan. So for example, if the project’s main corporate sponsor makes a strategic decision about what’s right for their product and changes things despite the wishes of the community, then that’s a huge warning sign.
- The project must maintain a stable release branch, have a clear policy about what is backported to that branch, and follow that policy even when it is inconvenient for the developers. I would recommend stable branches receive bug fix backports for at least six months, and security fix backports for at least twelve months. These periods need to be in line with your organization’s preferred rate of change however.
- The project should be well tested. For modern projects that probably means Continuous Integration testing is enforced before changes are merged into the source code, but there are other valid ways to test projects.
- In this day and age it is also mandatory that code changes should be reviewed and approved by other developers before they are merged.
You should also look for projects with a regular release cadence. You want to be able to predict when major releases will ship, as well as when they will be stable, well before you deploy them. This helps with forward planning. Regular releases also mean that there isn’t a ton of new features sitting unreleased all the time. Unreleased code is generally less tested, and introducing a lot of it at once is more risky than smaller incremental changes. Similarly, checkout the project’s bug tracker and see if issues that at reported are regularly being addressed. You need to know that the project cares about the experiences of their users and is listening.
Diversity of contributions is really important too. It is obvious that there are forms of contribution to an Open Source project other than code, especially in terms of a community of users for the project and a solid documentation team. That said, the project dies if its developers all wander off. Having most of the developers work for a single employer increases the risk of project abandonment massively.
One of the ways that you attract other developers to your project is to make the project welcoming. Another list of things to look for to judge if the project is trying to be welcoming to other developers:
- Code is developed in public and anyone can leave review comments.
- There is a public bug tracker that anyone can file bugs in.
- The project has a reasonable code submission policy that welcomes public contributions. Sometimes projects require copyright assignment, which isn’t an unreasonable requirement although it is unpopular with a large number of Open Source developers.
- Upstream code and reviews are coming from more than one or two sources.
- The code base has existed for long enough that other people have deployed it in production (those people can be from the sources of the code though).
An active community of users and a documentation team which is keeping up with the overall level of change in the project is just as important as the code itself. If new code is landing, but no one can tell you how it works or how to use it, then we have a problem. User communities take different forms, but I’d be looking for active mailing lists, chat forums (IRC, slack, whatever), and blogs from larger deployers discussing what problems they’ve seen and how they’ve solved them.
Some evaluation criteria:
- There is an active user community separate to the developer community (i.e. people other than the devs are able to deploy the code and have formed into a self-supporting community)
- There is solid operations documentation — you can install and operate the project without having to engage the services of the software developers to make it go. While freemium support models exist and are sometimes the way to go, in general being held hostage to get help from the developers isn’t something that makes me comfortable.
An inclusive governance model is next on our shopping list. The governance model of a project is how big decisions about the overall direction of the project are made. There are a few common models seen in the wild — some sort of elected Technical Committee structure, a “Benevolent Dictator For Life” (BFDL), as well as others. You could argue that this criteria is very similar to the diversity criteria above, but this is more explicit — the project should have enshrined a clear process to deal with technical disputes as well as producing an overall roadmap so you can predict where the project will be in a few years.
- While BDFL projects might be the stereotype, it is generally better to have a community where the contributors (developers, operators, users, supporters, et cetera) have some say in the future direction of the project.
- That said, don’t confuse a lack of direction with being “inclusive”. You still need some set of people that is defining the overall long term direction of the project so that its still meeting your needs in a few years time.
There’s no clear answers here, you’re going to have to assess each project on their merits — for example some projects get their governance models partially “right”. OpenStack for example has a good elected structure for their Technical Committee, but they have a Board of Directors stacked with “pay for play” seats. Worse still, there hasn’t been much success with defining a long term road map for OpenStack as a whole. Despite this, I should be clear that OpenStack is still probably one of the best projects when rated against all the criteria in this post, it just doesn’t rate well for corporate governance.
Finally, a personal bugbear of mine. Look for projects that make technical decisions based on technical merit. For example, one of the real strengths of OpenStack is that they didn’t implement another hypervisor, but instead chose to orchestrate existing hypervisors. A project needs to know where their unique value add is, and avoid stepping outside the boundaries of that.
There’s been a lot to digest in this post, because these are complicated issues. Need a hand assessing if an Open Source project is ready for your organization? Why not drop us a line and have a chat?
One final personal note — this post and its predecessor are based on extensive conversations with many colleagues with Open Source experience adding up to hundreds of years. There are too many of them to list here individually, but I appreciate all the wisdom they were so happy to share. It is a great example of the Open Source community at work that people are willing to help out on non-technical exercises such as this post as well as just code or documentation.
Previous post in this series… Selecting a well engaged Open Source vendor.
