What are Docker containers and how to use Docker images.
In this article we introduced Docker and how it works, introducing the essential components of its architecture. Let us now see how we can work with Docker images and containers. It is worth mentioning that, in AIknow, the use of Docker reaches more complex levels than those presented here; anyway, we hope that this documentation will help you take your first steps into the world of Docker.
Creating and running a docker container
The process of creating and running a container usually follows a few steps, which we briefly present below.
1.Image creation.
- The developer defines the specification of the application environment in a file called Dockerfile. This file contains instructions for building an image.
- Using the docker build command, Docker reads the Dockerfile, downloads the necessary dependencies, and creates an image.
2.Storage on Docker Registry.
- The image can be stored locally or published to a Docker registry, such as Docker Hub, using the docker push command.
- The registry serves as an image repository, accessible to both developers and production servers.
3.Running the container
- Using the docker run command, a container is started from an image.
- During execution, the container has its own isolated file system space, but shares the kernel of the host operating system
4.Resource Isolation
- Docker uses Linux kernel features such as namespaces and cgroups to isolate processes belonging to containers. Namespaces ensure that processes within a container are not visible to others, while cgroups restrict access to system resources.
5.Networking.
- Docker provides a networking model that allows containers to communicate with each other and with the outside world. Each container can have its own IP address and ports assigned.
Example
Using Docker Desktop, we present a brief guide to running a container. We choose to run a ubuntu container, which is available on Docker Hub.
As a first step, we execute the pull ubuntu:latest image:
docker pull ubuntu:latest
If the ubuntu image is not present on the host machine, Docker pulls it from the configured registry.
Now we can run the /bin/bash program using the downloaded image:
docker run -i -t ubuntu:latest /bin/bash
- Docker creates a new container.
- Docker assigns a read-write filesystem to the container as the final layer. This allows the running container to create or modify files and directories in its local filesystem.
- Docker creates a network interface to connect the container to the default network, since no network option has been specified. This includes assigning an IP address to the container. By default, containers can connect to external networks using the host machine’s network connection.
- Docker starts the container and executes the /bin/bash command. Because the container runs interactively and is connected to the terminal (thanks to the -i and -t flags), you can provide input to the container using the keyboard while Docker sends output to the terminal.
- When you execute exit to terminate the /bin/bash command, the container stops, but is not removed. You can restart it again or remove it.
We can create our own Docker image by writing a file called Dockerfile. Inside the file we write:
FROM ubuntu:latest RUN echo "Hello" > /hello.txt CMD ["/bin/bash"]
Our image will be based on ubuntu:latest, as specified by FROM, customizing it.
We run a command with RUN: the cat directive “Hello”> /hello.txt creates a layer in our image where we write Hello to a file located in / (in the container’s relative filesystem) and named hello.txt.
Our image uses /bin/bash as its boot command, which allows us to get a Bash shell in the container if it is started with the -i and -t flags.
Let’s run the build of the image:
docker build -t my-ubuntu:latest .
We then run the command:
docker run --rm my-ubuntu cat /hello.txt
The -rm flag causes the container to be removed when it terminates. The cat /hello.txt command will cause us to display the contents of the /hello.txt file created in our image.
Docker Compose: Managing Multi-container Applications.
Docker Compose is a tool that simplifies the management of multi-container applications. With Docker Compose, you can define the configurations of multiple services, networks, and volumes in a YAML file. This file describes the entire application, allowing all containers to be created and started with a single command. It is especially useful for developers and teams managing applications with multiple components.
Here is an example of a Docker Compose file:
version: '3'
services:
web:
image: nginx:latest
database:
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: example
The default names to give the file are compose.yml or docker-compose.yml. This way, the docker compose commands automatically detect the configuration in the file. We refer to the latest V2 functionality (Compose V2 functionality, integrated in 2023 into the Docker CLI, see here for more information).
With this file, you can simultaneously start an Nginx-based web service and a MySQL database using the docker compose up command.
If you want to give the file a different name, you must then specify the file itself when running docker compose commands. For example, if we named the file stack.yml, to run the applications we would need to issue the command:
docker compose -f stack.yml up
where -f stack.yml tells Docker Compose to reference the stack.yml file.
For a reference to the functionality of Compose in the Docker CLI (Command Line Interface) see here.
Multi-Container Application
Let’s create a simple multi-container application to understand the basics of workflow with Docker Compose. The first steps in this procedure are the same as we would also use if we chose to use Docker Swarm (see below).
The Application
We create an application that returns a response to a GET call to some API. We release the API behind a reverse proxy. The basic idea is that the proxy handles communication with clients, any TLS certificates (which we do not introduce here) and possibly acts as a load balancer, while the API is isolated. Possibly, alongside the API we can have a database, which would be equally isolated in the Docker network.
We could also have multiple API components, each dealing with a specific task. In short, the example we introduce is the basic connection that could be worked on to achieve a microservices architecture.
API
We create APIs that will provide us with an example of a backend application. We use NodeJS and Express.
In the api directory we create an app.js file with the source code for the Express API, the key part of which is:
app.get(
'/',
(req, res) => { res.send('Example API!'); }
);
This API server will be put listening on TCP port 3000, returning the string Example API! to an HTTP GET call to the path /.
We now create a file that allows us to generate a Docker image that we will then use to run our API: in the same api directory we create the file called Dockerfile:
FROM node:16.13.2-alpine WORKDIR /usr/src/app COPY package.json . COPY app.js . RUN npm install EXPOSE 3000 CMD node app.js
What operations does the Dockerfile perform?
- First, we specify in the FROM directive to start from an existing image, node:16.13.2-alpine, which allows us to avoid having to configure a NodeJS server ourselves.
- Next, we indicate in the WORKDIR directive that the containerized filesystem directory in which we are to execute commands and save code is /usr/src/app.
- We then run the npm ci command.
- With COPY we copy the necessary files from the current directory (api) on the container filesystem, to the directory previously specified with WORKDIR.
- We expose TCP port 3000 with the EXPOSE specification.
- Finally, we finally indicate that the command to be executed by the container is node app.js, which executes our API.
Proxy
First, we configure our proxy, which will be Nginx, by creating in the proxy directory a file called nginx.conf that configures the reverse proxy. The key directive is:
server {
listen 8080 default_server;
location / {
proxy_pass http://api:3000/;
}
}
The configuration presented here is clearly incomplete and essential; it is not suitable for a production environment. See the Nginx documentation for more guidance on how to configure the proxy server.
Without going into too much detail, this file configures Nginx to listen on port 8080 and forward all requests it receives with location / to the URI http://api:3000/.
Docker Compose will resolve the api name to the internal network address corresponding to the API component.
We now create a file called Dockerfile that will contain:
FROM nginx:1.23.4-alpine COPY nginx.conf /etc/nginx/nginx.conf EXPOSE 8080
Let’s look at the syntax of the Dockerfile:
- we start by specifying a base image, in this case nginx:1.23.4-alpine, which is already configured to use an Nginx server (thus saving us a lot of work);
- we continue by copying our server configuration, contained in the nginx.conf file;
- finally, we instruct the container to expose port 8080.
Docker Compose File
As last step, we create in the multi-container-app directory a file called compose.yml and containing the configuration of the application stack to be run:
version: '3'
services:
api:
build:
context: api
dockerfile: Dockerfile
proxy:
build:
context: proxy
dockerfile: Dockerfile
ports:
- 8080:8080
This file instructs Docker Compose to run two services: one, called api, which launches a container based on the image specified in the Dockerfile in the api directory (indicated by build and context); the other, called proxy, launches a container based on the image specified in the directory with the same name (the corresponding Dockerfile indicates how to build that image). Finally, the proxy service exposes port 8080 on the same port as the host machine (the machine running the Docker daemon).
Running all the things on
We can execute the stack by running in a terminal (in the multi-container-app directory) the command:
docker compose up
or, if we prefer not to block the output stream with container logs, we can run
docker compose up -d
Docker Compose automatically locates the file called compose.yml and performs a build and run of the containers listed there, creating an isolated Docker network to which they have access.
The stack Docker network allows the containers to communicate with each other, while still remaining isolated from the host machine’s network: the only access to the containers is configured through the ports directive associated with the proxy service.
A very convenient fact is that Docker Compose is able to independently run a build of the necessary images, if this is indicated to it by the build specification.
If we access via a Web browser the http://localhost:8000/ page, we get the API response:
If we ran the docker compose up command, we can see in the container logs access to the proxy:
✔ Container compose-proxy-1 Created 0.1s ✔ Container compose-api-1 Created 0.1s Attaching to api-1, proxy-1 api-1 | Listening on port [3000]... proxy-1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration proxy-1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ proxy-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh proxy-1 | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf proxy-1 | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf proxy-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh proxy-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh proxy-1 | /docker-entrypoint.sh: Configuration complete; ready for start up proxy-1 | 172.23.0.1 - - [08/Feb/2024:09:45:18 +0000] "GET / HTTP/1.1" 200 12 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" proxy-1 | 172.23.0.1 - - [08/Feb/2024:09:45:18 +0000] "GET /favicon.ico HTTP/1.1" 404 150 "http://localhost:8080/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
Docker Swarm: Integrated Orchestration
Docker Swarm is an integrated orchestration tool in Docker that allows you to manage and scale containers across multiple nodes. With Swarm, you can create a cluster of Docker hosts and deploy containers on them. Swarm manages scalability, availability and automatic application deployment.
How does Docker Swarm work?
Docker Swarm uses a leader-follower approach, in which a leader node coordinates the activities of other follower nodes. You can define the desired services in a YAML file (similar to Docker Compose seen above, many of the options are the same); using the docker stack command, you can deploy the services on a Swarm cluster.
Swarm constantly monitors the status of services and ensures that the desired number of replicas are always running
Example
We can run the previously presented stack using Docker Swarm. There are some minor differences to take into account: commands will need to be launched from an orchestrator node, as explained here.
Also, we need to run a build of the Docker images using the command
docker build -t [image_name]:[image_tag] [build_context]
In addition, we will have to modify the compose.yml by specifying instead of the build directive the image directive, which specifies the Docker images to be used to create the services: while, on the one hand, Docker Compose is able to automatically run an image build, Swarm needs ready-made Docker images.
Then, instead of running docker compose up, we run in a terminal the docker stack deploy command indicating the necessary options (see here for more information).
Kubernetes: Orchestrating Docker Containers on a Larger Scale.
Kubernetes is an open-source system for automated orchestration, deployment and management of containers. Although it may look similar to Docker Swarm, Kubernetes is more powerful and suitable for more complex, large-scale environments.
Kubernetes features:
- Advanced orchestration: Kubernetes manages the deployment, update and rollback of containerized applications on a cluster of nodes.
- Scalability: Kubernetes can *automatically* scale containers according to workload needs.
- State management: Kubernetes supports stateful applications, enabling management of databases and other services that require persistence.
- Resource management: Kubernetes allocates resources to containers, monitors performance and balances load to ensure efficient use of resources.
Conclusion
In conclusion, Docker and its related technologies have changed the way applications are developed, deployed and managed. The use of containers offers greater portability and scalability.
If you want to simplify container management on a single host, Docker Compose is an ideal choice.
For more complex and large-scale environments, Docker Swarm and Kubernetes provide powerful orchestration tools.
The choice between these technologies depends on the specific needs of the project and the desired scale of deployment.
