It's all about knowing how to build an open source community -- plus experience running applications in Linux containers, which Google invented
The tech world is increasingly awash in containers, and despite stiff competition, the industry seems to be settling on Kubernetes as the default container orchestration engine. Given the options, including Docker’s own Swarm, it’s worth looking at what makes Kubernetes so popular, especially given its comparative complexity.
As with other popular open source software, such as MongoDB and Linux, the answer seems to come down to smart plays in community building—and 15 years of Google R&D doesn’t hurt. Ultimately, this unique blend of exceptional engineering and a willingness to let others take over make Kubernetes such an impressive open source project.
Kubernetes’s community is real
It shouldn’t be this way. After all, Kubernetes is a mere two years old (as a public open source project), whereas Apache Mesos has clocked seven years in market. Docker Swarm is younger than Kubernetes, and it comes with the backing of the center of the container universe, Docker Inc.
Yet the orchestration rivals pale in comparison to Kubernetes’s community, which—now under management by the Cloud Native Computing Foundation—is exceptionally large and diverse.
- Kubernetes is one of the top projects on GitHub: in the top 0.01 percent in stars and No. 1 in terms of activity.
- While documentation is subpar, Kubernetes has a significant Slack and Stack Overflow community that steps in to answer questions and foster collaboration, with growth that dwarfs that of its rivals.
- More professionals list Kubernetes in their LinkedIn profile than any other comparable offering by a wide margin.
- Perhaps most glaring, data from OpenHub shows Apache Mesos dwindling since its initial release and Docker Swarm starting to slow. In terms of raw community contributions, Kubernetes is exploding, with 1,000-plus contributors and 34,000 commits—more than four times those of nearest rival Mesos.
What accounts for these wildly disparate community results? In a word: Google—or rather, the relative lack of Google. While each of the other orchestration projects comes with a heavy dose of single-vendor influence, Kubernetes benefits from Google’s hands-off approach to ongoing development, as well as its original engineering.
Meanwhile, Docker, which owns the de facto container standard, has struggled to build a community as broad and deep as Kubernetes has. The reason, as Google’s Kelsey Hightower points out, is Docker itself, which has blocked competitors from collaborating around a Docker standard.
Docker, of course, doesn’t need anyone’s input on standardization, but it would appear the market favors less concentration of power at the orchestration layer of containers. Given its need to come up with a viable business model, we should expect Docker Inc. to compete hard to own ops and, hence, orchestration, for Docker containers. But unless the company can turn Swarm into a true industry effort, it’s likely to win the container battle but lose the orchestration war, even if it could credibly claim better performance.
Kubernetes has deep roots at Google
Behind the community uptake of Kubernetes is exceptional technology. As Google’s Borg technology, Kubernetes enjoyed more than 15 years of active development and heavy production. The technology was so good that Urs Holzle, then Google’s head of technical infrastructure, reacted with disbelief when a few Google engineers suggested building a version of Borg and open-sourcing it:
So let me get this straight. You want to build an external version of the Borg task scheduler. One of our most important competitive advantages. The one we don’t even talk about externally. And, on top of that, you want to open source it?
Those engineers used Borg as a cluster management tool that powered the infrastructure behind Gmail, YouTube, Google Search, and other popular Google services. Later it was built into the Google Compute Engine, but the engineers noticed that customers were spinning up CPUs with terrible utilization rates. A container management system was needed, and they had one running behind the scenes. The trick was to get it out in the open.
According to Google product manager Martin Buhr, Google’s motivation behind Kubernetes comes down to noble intentions … and enlightened self-interest. Google hopes an open source Kubernetes will dramatically scale developer productivity and thereby make the world a better place. More selfishly, “We think over time that our deep, comprehensive support for containers on Google Cloud Platform will create a gravity well in the market for container based apps and that a significant percentage of them will end up with us.”
In other words, let developers and operations teams get comfortable fiddling with Kubernetes so that they’ll opt to let Google Cloud Platform make it easy to run. None of this works, however, if Google has a direct monetary interest in Kubernetes, which is something of a poison pill for open source communities.
In a nutshell, Kubernetes is rocking because of the 15-plus years of Google R&D that goes into its code, but also because of Google’s eagerness to encourage a community to take charge of the next 15-plus years of Kubernetes development.
