Docker, Containers


A Whirlwind of a Conference

I’m back, and I’m exhausted!

I’m a Docker Captain and I just spent three days in Austin, Texas, checking out the action at DockerCon 2017! Now that I’ve had a day to recover, I’m here to give you a behind the scenes peek at one of the year’s most exciting technology conferences. From on-stage open sourcing to block-long parties, I’ve got all the essential details.

In part one, I’ll dive into the two major DockerCon announcements, with part two to follow in a few days, in which I’ll cover the breakout sessions, networking activities, cool hacks, and the summits.

For now, onto the keynotes!

Keynotes

At DockerCon 2017, Docker has doubled down on their commitment to the open source world with two important announcements.

  • The Moby Project, which is an open source push to advance the software containerization movement by providing a set of components, frameworks, reference assemblies, and place for “container enthusiasts to experiment and exchange ideas.”

  • LinuxKit, a purpose built toolkit that enables engineers to create secure, immutable, and minimal Linux distributions.

With this move, Docker is creating a clear divide between itself as a company and the open-source community that has grown up around it. Between product and project, if you will!

Embracing Upstream: The Moby Project

First, let’s start with the most important open source software (OSS) project in Docker’s history.

Docker’s first push is towards the Moby Project, which can be seen as an incubator for exceptional software components. Those components are designed to allow teams to create container-focused platforms. Having standard components is important because they help define the vital aspects of a container system:  a minimal OS, container runtime, orchestration, infrastructure management, networking, storage, security, build, and image distribution, among others.

This means that Docker is a product that is built on Moby,  which itself is an open source project. In Moby, innovative software can be matured and integrated with Docker products as they’re distributed around the open source community. OSS developers now also know that the community participates in the Moby Project, while prospective customers can focus on Docker.

What’s important to note here is that while a set of reference assemblies will be provided with the Moby Project, the end user is free to switch out any piece, such as the OS, container runtime, or orchestration layer. By design, Docker will utilize the Moby Project to produce a set of software products to create their own opinionated set of container tools such as Docker for Mac (DfM) and Docker Datacenter (DDC).


Both DfM and DDC incorporate major open source pieces such as hyperkit and vpnkit, but they’ll also include that special sauce from Docker in the form of closed source bits. Through utilizing the common assemblies of the Moby Project, these projects and others can take advantage of a set of best practices and shared components to reduce the amount of replicated code and build systems.

This speeds up innovation.

Solomon and team showed us a neat example during the keynote in the form of RedisOS, which was composed of three main pieces:

  • LinuxKit: in charge of assembling the required operating system pieces required for different platforms

  • Containerd: the container runtime engine itself

  • Redis: the database program being re-platformed

With this simple set of building blocks, Moby is able to quickly build ISO-like images that can be run on Mac, Windows, and platforms such as Google Cloud Platform (GCP).

Good stuff, right? This might seem familiar to some of you who were paying attention to Redhat in the early 2000s, when they announced a collaborative project named Fedora that would help scale the Linux community into the juggernaut we’re all part of today.

The Moby Project is partly Docker’s acknowledgement of the success of Fedora, and Solomon hopes that it’s a “good time to evolve from a one-size-fits-all approach to innovation to a ‘let a thousand flowers bloom’ approach.” With Docker’s engagement on the CNCF and participation in the OCI formation, they’re hitting their stride in the container community with strong opinions, loosely held.

I’m excited to see what community the Moby Project brings, so I’ll see you on the project!

Secure by Design: LinuxKit

Docker has also introduced LinuxKit, which is designed to help build “custom, minimal, immutable Linux distributions.” This is another step along the path Docker began down back in 2014, when they open-sourced libcontainer (now known as runC). LinuxKit has the potential to be a key open source project that has the explicit goal of being used by Docker the product, while also extensible to all container systems and orchestration engines.

This is important, because it stresses the ‘boring’ nature of infrastructure plumbing that becomes a reality with shared components incubated by the Moby Project. No longer having to worry about infrastructure, this allows the operators of container platforms to drive user-facing innovation at maximum velocity.

LinuxKit embodies a four-tier philosophy:

  • Immutable, secure infrastructure that is atomic and replaceable.

  • A stateless, container-focused structure with easy tooling that can be run by higher-level tools such as Infrakit.

  • Designed for the plan, build and run of clustered applications, including but not limited to container orchestration tools such as Swarm and Kubernetes.

  • Inclusion and encouragement of longer-term projects focused on kernel and userspace changes.

This means that the end user or team will be able to build minimal Linux distributions designed to run containers, and output various images (ISO, AMI, VMDK, etc.) via a build system with YAML-based manifests. There are similarities to RancherOS, CoreOS, Atomic, and, for those that have worked with them, Unikernels, libOSTree and NixOS.

LinuxKit is based on containerd, which is part of the CNCF project. It allow you to build specific Linux subsystems for any container in order to take advantage of your own Linux distribution, or special hardware capabilities. With a minimalistic boot environment and a read-only root file system, LinuxKit will present a very small attack surface for cluster computing systems. This could be particularly important for IoT, which has been extremely vulnerable to security breaches and hacking.

Docker’s partnered with vendors like HPE, ARM, IBM, Microsoft, the Linux Foundation, and Intel in order to ensure that a rich stream of innovation flows through the build pipelines of LinuxKit.

If you were watching the keynote, you were treated to a particularly interesting demo: LinuxKit creating a platform that ran Kubernetes! Check out that example here.

Evolving and Flourishing

Over the past several years we’ve seen Docker learn how to participate in the open source and container community. For those of you that remember the beginning years of Docker’s journey, it’s important to recall that they were a product company named dotCloud. Solomon and team didn’t set out to build and live in an open source world with Docker, but they saw an incredible opportunity to build software around containers during a critical time in our technology history. Breaking off and focusing on the Docker engine as a core concept set off a series of events that would take years to play out.

In my mind, this DockerCon represents an inflection point for Solomon and team: they have recognized their power to shape the ecosystem, while also coming to terms with the innate limits of product companies to explicitly shape certain dimensions of an open source community.

With the creation of the Moby Project and the continued spin-off of valuable ‘boring infrastructure plumbing’ such as LinuxKit, Docker is just starting to hit its stride as an active facilitator, innovator, and incubator of the container world. A clear delineation between product and project is going to provide massive benefits for everyone.

I’m excited to see what this year brings from Docker the product company, as well as the innovations built in the community that will inevitably spring up around the open source project, Moby.

I’ll see you in part two of “A Docker Captain goes to DockerCon 2017!

  • Jesse White

    Account Principal

    Jesse White is a Account Principal with more than 15 years of technology industry experience across financial services, healthcare, advertising, e-commerce, and IoT verticals. As a 10 year veteran of New York City’s vibrant “Silicon Alley”, he’s intimately familiar with delivering cloud solutions, effective team building, and Agile process adoption. As an early contributor to the Container ecosystem, he plays a foundational role in New York City’s open source community as founder of DockerNYC. Jesse has spent the last 5 years focusing on the intersection of automation, cloud computing, security and DevOps methodologies.

    More Articles by Jesse