Ákos TakácsIntroduction
Usually I start with a blogpost and then I create a video. In this case I made a video in two languages, and now I'm here for you, who prefer reading. As the title suggests, I want to clear up a misunderstanding, and in the meantime explain how many kind of Docker exists and what components Docker has. We will also cover some of the history of Docker.
I often see that people think they are running "dockers". Keep in mind that "Dockers" is a garment company and has absolutely nothing to do with "Docker", the software.
If you prefer watching videos, most but not all of what I explain here can be watched on Youtube
For more videos, you can subscribe to my YouTube channel: @akos.takacs
Table of contents
- What I call Docker
- Docker as a company
- Docker as a software
- Conclusion
What I call Docker
In short, the Docker daemon. I could stop this post hear, since when I say "I run Docker on a server" I mean I run the Docker daemon, even if I don't have the client on that server. The daemon is required for running containers, but a container is not a docker, Docker just creates a container. Similarly to how bread is not a baker.
Unfortunately, it is more complicated, so let's not stop this post here just yet.
Docker as a company
The company who made Docker as a software was "dotCloud, Inc". The website doesn't exist anymore, but you can still find the GitHub organization. Later the company was renamed to "Docker, Inc" which was big news back then, and you can find some articles linked on Docker's website.
- Gigaom: The link takes you to an error page, so I don't even quote it.
- InfoQ: https://www.infoq.com/news/2013/10/dotcloud-renamed-docker/
- Forbes: https://www.forbes.com/sites/benkepes/2013/10/29/docker-and-the-timely-pivot/
Now, in a conversation we just say "Docker". For example "Docker Desktop is Docker's product." or "I know someone working at Docker". In the rest of this blog post we will talk about Docker as a software.
Docker as a software
The beginning of Docker as a software and where we are now
The first commit of Docker happened on January 19, 2013. You can still find it on GitHub: https://github.com/moby/moby/commit/a27b4b8cb8e838d03a99b6d2b30f76bdaf2f9e5d
Nowadays, Docker has different variants and multiple components. So when we say Docker, we could mean the entire software family. Collection of different products. That is what makes the confusion.
Two main ways of running the Docker daemon in terms of isolation
You can run Docker (meaning the Docker daemon) directly on a server or even on your machine, but there are two kind of containers. Linux containers require a Linux host. Windows containers require a Windows host. This is because a container is not a virtual machine. A container is just a process running on your host in an isolated way, so the process can't see everything on the host, only what it is allowed to see.
This means if you have a Windows machine, and you want to try Linux containers, you are in trouble. No... I'm kidding. But it is true that in this case you could not run Linux containers without the help of virtual machines.
I haven't mentioned macOS yet, because there is no such thing as "macOS container" at the moment, but you can install Docker in a virtual machine even on macOS.
Docker running in a virtual machine
Docker in a VM in general
If you want to run Linux containers on Windows or macOS, you need a virtual machine with a Linux operating system inside. If you want to run Windows containers on Linux or macOS, you could probably try a Windows VM, but I have never tried and Windows containers are different so you would also need to deal with two different isolation modes and the HyperV isolation mode would require nested virtualization enabled for your virtual machine.
Docker in a VM using Docker Desktop
A special way of using Docker in a virtual machine is when you have the client on your host and the Docker daemon is in the virtual machine. You could configure it manually similarly to how you would configure a remote Docker daemon for your local client, but Docker Desktop solves it for you.
Docker Desktop can run on Windows, macOS and also on Linux. Yes, it runs on Linux, but it still creates a virtual machine instead of accessing the Docker daemon on your host.
When you run Docker Desktop on Windows, you can also switch between Windows containers and Linux containers, but Windows containers are supported by Docker Desktop only on Windows. You could try running Docker Desktop in a Windows VM, but that would also require nested virtualization if you want to use Docker Desktop for running Linux containers, or Windows containers with HyperV isolation. Not to mention that as far as I remember, you could not even install Docker Desktop if you don't have HyperV or WSL2 enabled, which requires nested virtualization in a VM.
Docker is actually Docker Desktop on macOS
One problem on macOS that adds more confusion is that the application which you can install on macOS is called "Docker". When you open the launchpad, the label below the Docker icon is "Docker". Don't forget that you are still using Docker Desktop and the Docker daemon is running inside the virtual machine as well as all your containers.
Docker running directly on a physical machine
The source code of Moby
I shared the first commit of Docker's source code at the beginning of the "Docker as a software" section. The source code is on GitHub in the moby/moby repository. During the years since Docker was created, many repositories were used. "moby/moby" was originally called "dotcloud/docker" and the URL still works (http://github.com/dotcloud/docker). When the company was renamed, the repository was renamed again to "docker/docker" (https://github.com/dotcloud/docker), but both are redirected to Moby today. Moby is the base source code of the Docker variants we have today.
Docker Enterprise Edition
Years ago there was a Docker EE (Enterprise Edition), but it was bought by Mirantis, who made the Mirantis Container Runtime. You can even read it on their website:
Mirantis Container Runtime (formerly Docker Engine - Enterprise)
Since I'm focusing on Docker, this is all I wanted to say about an edition that does not exist today.
Docker IO
docker.io is based on Moby, but not supported by Docker, Inc. You could find this package on a Debian or Ubuntu Linux distribution. On Ubuntu, running apt info docker.io reveals that it is maintained by "Ubuntu Developers". The output of the command contains exactly
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
and the following on Debian
Maintainer: Debian Go Packaging Team <team+pkg-go@tracker.debian.org>
The Internet Archive's "Wayback machine" reveals that the earlier installation guide for Ubuntu started with installing docker.io, but recent guides start with removing docker.io.
I don't recommend using docker.io today as it will install an older version (at the time of writing this post, it is 24.0.7 on Ubuntu 22.04) and most of the tutorials will show you features based the official version and the official documentation.
Snap package
There is a snap package which is also based on Moby, but built by Canonical. I quote:
Authors
This snap is built by Canonical based on source code published by Docker, Inc. It is not endorsed or published by Docker, Inc.
So I don't recommend this variant either, unless the installation guide of the official version doesn't work on your distribution but Snap does. A snap is basically a kind of container with possible restrictions like you can only mount files from your home directory.
Docker CE
Docker CE is the Docker Community Edition based on Moby and officially supported by Docker, Inc. So when you want to install Docker on Linux without a virtual machine, this is what I recommend. This is also what you can install on a Windows server, but the installation is described by Microsoft.
Docker in Docker
The term "Docker in Docker" is misleading. You are not running Docker in a Docker. As I explained, the word Docker should be used for the Docker daemon, but you are not running a Docker daemon in a Docker daemon.
Then what is that you are doing when using Docker in Docker? You are running a Docker daemon inside a Docker container. Since calling it "Docker daemon in Docker container" would be long and wouldn't sound as good, we call it "Docker in Docker" in short.
If you want to try it, you can find Docker images on Docker Hub and you can use a Docker image downloaded (pulled) from Docker Hub to run a Docker container in which you run a Docker daemon to run isolated processes (containers) in an already isolated environment.
Docker CE components
Two main parts of Docker CE
As I pointed out earlier, you can run a Docker daemon in a virtual machine or on a remote machine and still keep the "docker" command on your local machine. This is possible because we can talk about two main parts of Docker CE. The client and the daemon. The client can be anywhere, but the daemon has to be where the containers will be running.
The client can instruct the Docker daemon through a TCP or Unix socket using the API service, but a "client" can mean multiple things.
Docker Client
The docker command
Often when we talk about the Docker client, we think of the "docker" command used in a terminal. For a beginner, who sees the name of the command, it could be confusing, but this is not the entire Docker, and it will not run "dockers", but Docker containers.
Docker SDK
The client can also be SDK, although the SDKs are for the core Docker and not for the plugins like Buildx.
Docker Compose
There is another plugin called Docker Compose. There was a Compose v1 originally written in Python for which we used the docker-compose command, but now we need to use docker compose without the dash.
Both versions require a so-called "compose file", but v1 required docker-compose.yml while the new version accept compose.yml as well. The other difference is that v1 required
version: XY
at the beginning of the file to specify which version of the compose syntax we wanted to use and the XY was a number like 3.8. So this was never the version of Docker Compose, but the version of the compose schema in the yaml file. For more information about the history of Docker Compose, you can visit the description in the official documentation.
Docker Daemon
The history of dockerd
The Docker daemon can be started by running dockerd. Until Docker 1.8.0, the daemon could be started with the docker command using the -d flag, but it was changed to docker daemon replacing the flag with a subcommand. Then that became deprecated too in Docker 1.13 after the dockerd binary was introduced in Docker 1.12.
Rootless vs rootful Docker
Originally the Docker daemon required a root user. Running it as a non-root user became possible in Docker CE 19.03 as an experimental feature which was already one of the core features of Podman (we will talk about it later) from the beginning. This is what we call "rootless mode", so the original mode became the "rootful mode".
Docker daemon dependencies
Originally, Docker was a single binary. Before introducing the "dockerd" command for the daemon, docker-containerd, docker-containerd-shim and docker-runc was introduced in Docker 1.11.
So once we had a single binary, then "Docker, Inc" started separating the functionalities into multiple binaries on Linux. That was the beginning the of dependencies and components we have today, except that these dependencies are now not limited to Docker. containerd can also be the container runtime of Kubernetes.
As the "docker" command is the client for "dockerd", nerdctl is the client for containerd, although the project was started only at the end of 2020.
Now we have dockerd which uses containerd, but containerd will not create containers directly. It needs a runtime and the default runtime is runc, but that can be changed. containerd actually doesn't have to know the parameters of the runtime. There is a shim process between containerd and runc, so containerd knows the parameters of the shim, and the shim knows the parameters of runc or other runtimes.
Podman pretending to be Docker
I admit the title of this section sounds worse than it is, but the fact is that sometimes when you install Podman, you can also have an alias called "docker" pointing to "podman". That can make you believe that you are running Docker and come to the Docker forum asking about an issue which is actually not related to Docker. The alias exists because Podman tries to keep a similar command line interface to the interface of Docker, so when someone relies on an existing docker command, they don't have to rewrite their scripts if they are lucky.
But again, Podman is not Docker and Podman Desktop is not Docker Desktop!
The docker version and docker info commands can give you an idea of what you are actually using.
Packages and versioning
In my mind the core components of Docker CE are the Docker daemon and the Docker client. When you install Docker CE on Linux, after you added the official APT repository provided by "Docker, Inc", the following packages could be installed:
- docker-ce: The daemon
- docker-ce-rootless-extras: Scripts to run rootless Docker
- docker-ce-cli: The docker command
These packages have the same version. You can use a different version of the command line interface (cli), but if you want to make sure that the docker command is compatible with the API of the Docker daemon, use the same versions.
Everything else listed below has completely independent version numbers:
- docker-compose (Compose v1)
- Plugins
- docker-compose-plugin (Compose v2)
- docker-buildx
- docker-init (only in Docker Desktop)
- Docker Desktop
- Docker daemon dependencies
- containerd
- containerd-shim*
- runc
Conclusion
Docker has a long history, and it changed a lot during the years, but one thing was always true. You always ran containers and not dockers. If you say "I want to run multiple dockers" on the Docker forum, we will think that you want to run multiple Docker daemons. Well... since we are kind of trained by our users, we will eventually realize what you mean, but knowing the right terms helps you communicate your issue better and get an answer faster.
Before you ask a question, always think about what you are using and communicate that to the potential helpers.
And finally, you can find an important screenshot from the video below that shows all the variants and components I described in this post. Click on the image to download it in full size.
