We think this is a great fit and I will tell you why.

The Problem

Modern apps are made of so many services. They’re everywhere. Every team we talk to is trying to figure out how to set up environments to run their apps in dev.  Simple `start.sh` scripts inevitably grow into mini bespoke orchestrators. They need to start servers in the right order, update them in-place, and monitor when one is failing. We built Tilt, a dev environment as code for teams on Kubernetes, to help solve these problems. Whether your dev env is local processes or containers, in a local cluster or a remote cloud, Tilt keeps you in flow and your feedback loops fast.

So how does this make sense at Docker?

When we started building Tilt in 2018, we thought of Docker as the container company selling Swarm to enterprises. In 2019, the Docker’s Next Chapter blog post announced a change in focus to invest more in great tools for developers and development teams to help them spend more time on innovation, less time on everything else. 

Tilt interoperates with Docker Buildkit, Docker Desktop, and Docker Compose. Improvements to these tools help Tilt users too! We always had a hunch that our product roadmaps might overlap. And in the years since Docker focused on developers, we’ve been converging more and more.

Once we started talking more with Docker, we found more in common than just a problem space including:

• A product philosophy around deeply understanding devs’ existing workflows, so we can make dramatic improvements in user experience that feel magic;
• An engineering philosophy around patterns and flexibility so devs can adapt their tools to their needs;
• A business philosophy around building a sustainable company so we can continue to make great free, open-source tools for every developer.

So you could say we got along. What’s next?

What Does a Combined Tilt + Docker Look Like?

Tilt will remain open-source. It’s great! You should try it! We’ll still be responding to issues and hanging out in the community slack channel. But this has never been about Tilt the technology. Or even about Kubernetes. Our history is full of experiments. Dan Bentley and I started hacking on ideas in 2017. We knew we were unhappy about microservice dev. But we weren’t sure what the first stepping stone might be.

This was more of a research project than a company. Some examples:

• Our first prototype was a more interactive, developer-focused CI.
• We almost trolled ourselves into becoming a Bazel company.
• We bought two lab coats, poster board, glue, and glitter so we could show off our prototypes at the GothamGo conference.
• We built many weird demos: (1) Mishell (an interactive multi-service shell) represented by a French-speaking hermit crab named Michel, and PETS (Process for Editing Tons of Services) represented by three cats overwhelmed by microservice dev. Our teammate Han Yu had a blast with mascot design (yeah to Docker for animal mascots):

The first version of Tilt was a bare-bones terminal app to update containers in a Kubernetes cluster. It resonated immediately. Tilt has grown a lot since then. Running all or just a few of your services is easier than ever. Why did we focus on Kubernetes? Kubernetes contains a few simple, brilliant ideas for how to operate apps. Tilt borrows a lot of ideas (and a lot of core libraries) from Kubernetes on how to be scriptable and adaptable. But more importantly, the Kubernetes community is lovely. They appreciate Goose-themed trolling. We found a worldwide community of people enthusiastic about building better tools.

We’re sad we missed everyone at Kubecon EU this year! We didn’t know if this deal would finish before, during, or after the conference.

That said, over the next couple months, we’ll be swapping notes with the Docker team about what we’ve both learned and what we’ve both tried.  We don’t know yet where this will take us. Maybe you’ll see Tilt & Kubernetes features in Docker Compose. Or maybe you’ll see Docker Desktop features in Tilt. There will be research and tinkering, where we’ll be in our lab coats and glitter, but do expect us to bring the power of Tilt to Docker.

As a developer using Docker Desktop, do I need to do anything?

No, there is no need to take any action, everything continues to work as before. There is no need to change your images or your developer workflows. Your images that ran on Kubernetes yesterday with dockershim will run unchanged on Kubernetes 1.24 without dockershim!

As a developer using Docker Desktop, do I need to care which container runtime implementation is used in production?

No, the container “runtime” is an implementation detail of Kubernetes and you don’t need to think about it. There is no need to rebuild any images or containers. Docker pioneered the industry-standard OCI specifications to ensure that containers can run anywhere on any runtime. 

What if I need to bind-mount the Docker socket in my container in a remote Kubernetes cluster?

Provided the Docker engine is still installed on the Kubernetes worker node, the Docker socket can be bind-mounted as before. It’s not necessary for the remote Kubernetes cluster to use the Docker runtime.

What did the dockershim do exactly?

Early versions of Kubernetes created containers directly using the Docker API, through an internal Kubernetes package called “dockershim” (“shim” meaning a small glue layer between Kubernetes internal APIs and the Docker API). Later the Kubernetes project gained the ability to create containers through a new interface called the Container Runtime Interface (CRI) . Until version 1.24, Kubernetes could use the Docker Engine to manage containers in two different ways: one way using the old “dockershim” and a second way using the new Docker CRI implementation from Mirantis (see the post “The future of dockershim is cri-dockerd”). To make Kubernetes easier to manage and maintain, the old “dockershim” has been removed, so all container management is now through the new CRI.

Docker Desktop has already switched from the old dockershim to the new Docker CRI implementation in version 4.7.0, so the removal of dockershim from Kubernetes does not affect Docker Desktop.

What about containerd?

Back in 2016, Docker started working with Google on “containerd”: a container runtime that could be shared by both Docker and Kubernetes. The low-level parts of dockerd were moved to containerd. Today there is a dockerd and containerd side-by-side in every instance of Docker Desktop. Here at Docker we’re committed to improving both dockerd and containerd to bring great new features to developers.

Benefits of Docker’s built-in Kubernetes distribution

Docker Desktop saves developers time and effort by including an optimized distribution of the latest Kubernetes, which can be enabled with a single checkbox.

Developers benefit from a fast “inner-loop” where they can “docker build” an image and then immediately test it from Kubernetes without having to push and then re-pull, all thanks to Docker’s shared image store and cri-dockerd.

Docker Desktop automatically binds network ports on the host and tunnels traffic to Kubernetes, allowing developers to immediately interact with their applications without remembering internal IP addresses or reconfiguring their DNS.

Docker Desktop manages all the complexity of running containers and Kubernetes, allowing developers to focus on their code rather than cluster administration. There is no need to worry about local cluster upgrades, choosing CNI plugins or container runtimes. If you’d like to know more about Docker’s built-in Kubernetes then check out the blog post “How Kubernetes works under the hood with Docker Desktop”.

Managing a custom Kubernetes installation is a complicated task. The advantage of using Docker Desktop over a DIY solution is that the complexity is handled for you. The removal of dockershim from Kubernetes is an example. With a DIY solution you would have to sort out the migration yourself, and then deploy the solution to all your developers’ desktops: with Docker Desktop it’s all done for you. So your developers can focus on developing, while Docker Desktop takes care of the admin. 

Keep calm and update Docker Desktop with confidence: you’ll get the latest Kubernetes, integrated with the latest Docker tools, all fully tested and ready to go!

As you may have heard, the Notary project has been invited to join the Cloud Native Computing Foundation (CNCF). Much like its real world namesake, Notary is a platform for establishing trust over pieces of content.

In life, certain important events such as buying a house are facilitated by a trusted third party called a “notary.” When buying a house, this person is typically employed by the lender to verify your identity and serve as a witness to your signatures on the mortgage agreement. The notary carries a special stamp and will also sign the documents as an affirmation that a notary was present and verified all the required information relating to the borrowers.

In a similar manner, the Notary project, initially sponsored by Docker, is designed to provide high levels of trust  over digital content using strong cryptographic signatures. In addition to ensuring the provenance of the software, it also provides guarantees that the content is not modified without approval of the author anywhere in the supply chain.  This then allows higher level systems like Docker Enterprise Edition (EE)  with Docker Content Trust (which uses Notary) to establish clear policy on the usage of content.  For instance, a policy can be set that only signed content can be used at runtime and deployed by the orchestrators in the Docker platform. Overall Notary is a core piece of plumbing in Docker’s  approach to the secure supply chain whereby security is seamlessly and uniformly embedded into a workflow from development all the way through to operations.

Notary is an implementation of The Update Framework (TUF) written in Go. TUF was developed at the NYU Tandon School of Engineering. TUF was submitted to join CNCF in partnership with Notary. The combined nature of these two projects makes for a particularly compelling donation– both the specification and most widely deployed implementation are coming in together under the auspices of the CNCF.

With technologies such as containerd and Kubernetes already members of CNCF, Notary and TUF are the first security-related projects to be added to the CNFC. This year has seen a significant uptick in data security compromises and we believe the CNCF is positioning itself ahead of the curve by inviting Notary and TUF to join. We hope that more security-focused projects are added to the CNCF over time.

Notary is already used in production environments beyond container distribution with Cloudflare integrating it into their PAL tool for container identity bootstrapping and Kolide using it to secure their autoupdater for the osquery tool. If current trends continue, there will be many more users in search of tools to secure their software distribution channels in the near future and Notary, TUF, and the CNCF will be well positioned to meet that need.

What is #Notary and why is it important to @CloudNativeFdn ? @moby #security
Click To Tweet