Vue normale

Il y a de nouveaux articles disponibles, cliquez pour rafraîchir la page.
À partir d’avant-hierFlux principal

Enhancing Container Security with Docker Scout and Secure Repositories

Par : Jay Schmidt
25 novembre 2024 à 14:43

Docker Scout simplifies the integration with container image repositories, improving the efficiency of container image approval workflows without disrupting or replacing current processes. Positioned outside the repository’s stringent validation framework, Docker Scout serves as a proactive measure to significantly reduce the time needed for an image to gain approval. 

By shifting security checks left and integrating Docker Scout into the early stages of the development cycle, issues are identified and addressed directly on the developer’s machine.

2400x1260 generic scout blog d

Minimizing vulnerabilities 

This leftward shift in security accelerates the development process by keeping developers in flow, providing immediate feedback on policy violations at the point of development. As a result, images are secured and reviewed for compliance before being pushed into the continuous integration/continuous deployment (CI/CD) pipeline, reducing reliance on resource-heavy, consumption-based scans (Figure 1). By resolving issues earlier, Docker Scout minimizes the number of vulnerabilities detected during the CI/CD process, freeing up the security team to focus on higher-priority tasks.

Sample secure repo pipeline showing images are secured and reviewed for compliance before being pushed into the continuous integration/continuous deployment (CI/CD) pipeline, reducing reliance on resource-heavy, consumption-based scans.
Figure 1: Sample secure repository pipeline.

Additionally, the Docker Scout console allows the security team to define custom security policies and manage VEX (Vulnerability Exploitability eXchange) statements. VEX is a standard that allows vendors and other parties to communicate the exploitability status of vulnerabilities, allowing for the creation of justifications for including software that has been tied to Common Vulnerabilities and Exposures (CVE).

This feature enables seamless collaboration between development and security teams, ensuring that developers are working with up-to-date compliance guidelines. The Docker Scout console can also feed critical data into existing security tooling, enriching the organization’s security posture with more comprehensive insights and enhancing overall protection (Figure 2).

Sample secure repo pipeline with scout: The Docker Scout console can also feed critical data into existing security tooling, enriching the organization’s security posture with more comprehensive insights and enhancing overall protection.
Figure 2: Sample secure repository pipeline with Docker Scout.

How to secure image repositories

A secure container image repository provides digitally signed, OCI-compliant images that are rebuilt and rescanned nightly. These repositories are typically used in highly regulated or security-conscious environments, offering a wide range of container images, from open source software to commercial off-the-shelf (COTS) products. Each image in the repository undergoes rigorous security assessments to ensure compliance with strict security standards before being deployed in restricted or sensitive environments.

Key components of the repository include a hardened source code repository and an OCI-compliant registry (Figure 3). All images are continuously scanned for vulnerabilities, stored secrets, problematic code, and compliance with various standards. Each image is assigned a score upon rebuild, determining its compliance and suitability for use. Scanning reports and justifications for any potential issues are typically handled using the VEX format.

Key components of the repository include a hardened source code repository and an OCI-compliant registry
Figure 3: Key components of the repository include a hardened source code repository and an OCI-compliant registry.

Why use a hardened image repository?

A hardened image repository mitigates the security risks associated with deploying containers in sensitive or mission-critical environments. Traditional software deployment can expose organizations to vulnerabilities and misconfigurations that attackers can exploit. By enforcing a strict set of requirements for container images, the hardened image repository ensures that images meet the necessary security standards before deployment. Rebuilding and rescanning each image daily allows for continuous monitoring of new vulnerabilities and emerging attack vectors.

Using pre-vetted images from a hardened repository also streamlines the development process, reducing the load on development teams and enabling faster, safer deployment.

In addition to addressing security risks, the repository also ensures software supply chain security by incorporating software bills of materials (SBOMs) with each image. The SBOM of a container image can provide an inventory of all the components that were used to build the image, including operating system packages, application specific dependencies with its versions, and license information. By maintaining a robust vetting process, the repository guarantees that all software components are traceable, verifiable, and tamper-free — essential for ensuring the integrity and reliability of deployed software.

Who uses a hardened image repository?

The main users of a hardened container image repository include internal developers responsible for creating applications, developers working on utility images, and those responsible for building base images for other containerized applications. Note that the titles for these roles can vary by organization.

  • Application developers use the repository to ensure that the images their applications are built upon meet the required security and compliance standards.
  • DevOps engineers are responsible for building and maintaining the utility images that support various internal operations within the organization.
  • Platform developers create and maintain secure base images that other teams can use as a foundation for their containerized applications.

Daily builds

One challenge with using a hardened image repository is the time needed to approve images. Daily rebuilds are conducted to assess each image for vulnerabilities and policy violations, but issues can emerge, requiring developers to make repeated passes through the pipeline. Because rebuilds are typically done at night, this process can result in delays for development teams, as they must wait for the next rebuild cycle to resolve issues.

Enter Docker Scout

Integrating Docker Scout into the pre-submission phase can reduce the number of issues that enter the pipeline. This proactive approach helps speed up the submission and acceptance process, allowing development teams to catch issues before the nightly scans. 

Vulnerability detection and management

  • Requirement: Images must be free of known vulnerabilities at the time of submission to avoid delays in acceptance.
  • Docker Scout contribution:
    • Early detection: Docker Scout can scan Docker images during development to detect vulnerabilities early, allowing developers to resolve issues before submission.
    • Continuous analysis: Docker Scout continually reviews uploaded SBOMs, providing early warnings for new critical CVEs and ensuring issues are addressed outside of the nightly rebuild process.
    • Justification handling: Docker Scout supports VEX for handling exceptions. This can streamline the justification process, enabling developers to submit justifications for potential vulnerabilities more easily.

Security best practices and configuration management

  • Requirement: Images must follow security best practices and configuration guidelines, such as using secure base images and minimizing the attack surface.
  • Docker Scout contribution:
    • Security posture enhancement: Docker Scout allows teams to set policies that align with repository guidelines, checking for policy violations such as disallowed software or unapproved base images.

Compliance with dependency management

  • Requirement: All dependencies must be declared, and internet access during the build process is usually prohibited.
  • Docker Scout contribution:
    • Dependency scanning: Docker Scout identifies outdated or vulnerable libraries included in the image.
    • Automated reports: Docker Scout generates security reports for each dependency, which can be used to cross-check the repository’s own scanning results.

Documentation and provenance

  • Requirement: Images must include detailed documentation on their build process, dependencies, and configurations for auditing purposes.
  • Docker Scout contribution:
    • Documentation support: Docker Scout contributes to security documentation by providing data on the scanned image, which can be used as part of the official documentation submitted with the image.

Continuous compliance

  • Requirement: Even after an image is accepted into the repository, it must remain compliant with new security standards and vulnerability disclosures.
  • Docker Scout contribution:
    • Ongoing monitoring: Docker Scout continuously monitors images, identifying new vulnerabilities as they emerge, ensuring that images in the repository remain compliant with security policies.

By utilizing Docker Scout in these areas, developers can ensure their images meet the repository’s rigorous standards, thereby reducing the time and effort required for submission and review. This approach helps align development practices with organizational security objectives, enabling faster deployment of secure, compliant containers.

Integrating Docker Scout into the CI/CD pipeline

Integrating Docker Scout into an organization’s CI/CD pipeline can enhance image security from the development phase through to deployment. By incorporating Docker Scout into the CI/CD process, the organization can automate vulnerability scanning and policy checks before images are pushed into production, significantly reducing the risk of deploying insecure or non-compliant images.

  • Integration with build pipelines: During the build stage of the CI/CD pipeline, Docker Scout can be configured to automatically scan Docker images for vulnerabilities and adherence to security policies. The integration would typically involve adding a Docker Scout scan as a step in the build job, for example through a GitHub action. If Docker Scout detects any issues such as outdated dependencies, vulnerabilities, or policy violations, the build can be halted, and feedback is provided to developers immediately. This early detection helps resolve issues long before images are pushed to the hardened image repository.
  • Validation in the deployment pipeline: As images move from development to production, Docker Scout can be used to perform final validation checks. This step ensures that any security issues that might have arisen since the initial build have been addressed and that the image is compliant with the latest security policies. The deployment process can be gated based on Docker Scout’s reports, preventing insecure images from being deployed. Additionally, Docker Scout’s continuous analysis of SBOMs means that even after deployment, images can be monitored for new vulnerabilities or compliance issues, providing ongoing protection throughout the image lifecycle.

By embedding Docker Scout directly into the CI/CD pipeline (Figure 1), the organization can maintain a proactive approach to security, shifting left in the development process while ensuring that each image deployed is safe, compliant, and up-to-date.

Defense in depth and Docker Scout’s role

In any organization that values security, adopting a defense-in-depth strategy is essential. Defense in depth is a multi-layered approach to security, ensuring that if one layer of defense is compromised, additional safeguards are in place to prevent or mitigate the impact. This strategy is especially important in environments that handle sensitive data or mission-critical operations, where even a single vulnerability can have significant consequences.

Docker Scout plays a vital role in this defense-in-depth strategy by providing a proactive layer of security during the development process. Rather than relying solely on post-submission scans or production monitoring, Docker Scout integrates directly into the development and CI/CD workflows (Figure 2), allowing teams to catch and resolve security issues early. This early detection prevents issues from escalating into more significant risks later in the pipeline, reducing the burden on the SecOps team and speeding up the deployment process.

Furthermore, Docker Scout’s continuous monitoring capabilities mean that images are not only secure at the time of deployment but remain compliant with evolving security standards and new vulnerabilities that may arise after deployment. This ongoing vigilance forms a crucial layer in a defense-in-depth approach, ensuring that security is maintained throughout the entire lifecycle of the container image.

By integrating Docker Scout into the organization’s security processes, teams can build a more resilient, secure, and compliant software environment, ensuring that security is deeply embedded at every stage from development to deployment and beyond.

Learn more

Docker Best Practices: Using ARG and ENV in Your Dockerfiles

Par : Jay Schmidt
16 octobre 2024 à 14:35

If you’ve worked with Docker for any length of time, you’re likely accustomed to writing or at least modifying a Dockerfile. This file can be thought of as a recipe for a Docker image; it contains both the ingredients (base images, packages, files) and the instructions (various RUN, COPY, and other commands that help build the image).

In most cases, Dockerfiles are written once, modified seldom, and used as-is unless something about the project changes. Because these files are created or modified on such an infrequent basis, developers tend to rely on only a handful of frequently used instructions — RUN, COPY, and EXPOSE being the most common. Other instructions can enhance your image, making it more configurable, manageable, and easier to maintain. 

In this post, we will discuss the ARG and ENV instructions and explore why, how, and when to use them.

2400x1260 best practices

ARG: Defining build-time variables

The ARG instruction allows you to define variables that will be accessible during the build stage but not available after the image is built. For example, we will use this Dockerfile to build an image where we make the variable specified by the ARG instruction available during the build process.

FROM ubuntu:latest
ARG THEARG="foo"
RUN echo $THEARG
CMD ["env"]

If we run the build, we will see the echo foo line in the output:

$ docker build --no-cache -t argtest .
[+] Building 0.4s (6/6) FINISHED                                                                     docker:desktop-linux
<-- SNIP -->
 => CACHED [1/2] FROM docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e  0.0s
 => => resolve docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e34a9ab6  0.0s
 => [2/2] RUN echo foo                                                                                               0.1s
 => exporting to image                                                                                               0.0s
<-- SNIP -->

However, if we run the image and inspect the output of the env command, we do not see THEARG:

$ docker run --rm argtest
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=d19f59677dcd
HOME=/root

ENV: Defining build and runtime variables

Unlike ARG, the ENV command allows you to define a variable that can be accessed both at build time and run time:

FROM ubuntu:latest
ENV THEENV="bar"
RUN echo $THEENV
CMD ["env"]

If we run the build, we will see the echo bar line in the output:

$ docker build -t envtest .
[+] Building 0.8s (7/7) FINISHED                                                                     docker:desktop-linux
<-- SNIP -->
 => CACHED [1/2] FROM docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e  0.0s
 => => resolve docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e34a9ab6  0.0s
 => [2/2] RUN echo bar                                                                                               0.1s
 => exporting to image                                                                                               0.0s
<-- SNIP -->

If we run the image and inspect the output of the env command, we do see THEENV set, as expected:

$ docker run --rm envtest
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=f53f1d9712a9
THEENV=bar
HOME=/root

Overriding ARG

A more advanced use of the ARG instruction is to serve as a placeholder that is then updated at build time:

FROM ubuntu:latest
ARG THEARG
RUN echo $THEARG
CMD ["env"]

If we build the image, we see that we are missing a value for $THEARG:

$ docker build -t argtest .
<-- SNIP -->
 => CACHED [1/2] FROM docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e  0.0s
 => => resolve docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e34a9ab6  0.0s
 => [2/2] RUN echo $THEARG                                                                                           0.1s
 => exporting to image                                                                                               0.0s
 => => exporting layers                                                                                              0.0s
<-- SNIP -->

However, we can pass a value for THEARG on the build command line using the --build-arg argument. Notice that we now see THEARG has been replaced with foo in the output:

 => CACHED [1/2] FROM docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e  0.0s
 => => resolve docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e34a9ab6  0.0s
 => [2/2] RUN echo foo                                                                                               0.1s
 => exporting to image                                                                                               0.0s
 => => exporting layers                                                                                              0.0s
<-- SNIP -->

The same can be done in a Docker Compose file by using the args key under the build key. Note that these can be set as a mapping (THEARG: foo) or a list (- THEARG=foo):

services:
  argtest:
    build:
      context: .
      args:
        THEARG: foo

If we run docker compose up --build, we can see the THEARG has been replaced with foo in the output:

$ docker compose up --build
<-- SNIP -->
 => [argtest 1/2] FROM docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04  0.0s
 => => resolve docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e34a9ab6  0.0s
 => CACHED [argtest 2/2] RUN echo foo                                                                                0.0s
 => [argtest] exporting to image                                                                                     0.0s
 => => exporting layers                                                                                              0.0s
<-- SNIP -->
Attaching to argtest-1
argtest-1  | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
argtest-1  | HOSTNAME=d9a3789ac47a
argtest-1  | HOME=/root
argtest-1 exited with code 0

Overriding ENV

You can also override ENV at build time; this is slightly different from how ARG is overridden. For example, you cannot supply a key without a value with the ENV instruction, as shown in the following example Dockerfile:

FROM ubuntu:latest
ENV THEENV
RUN echo $THEENV
CMD ["env"]

When we try to build the image, we receive an error:

$ docker build -t envtest .
[+] Building 0.0s (1/1) FINISHED                                                                     docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                 0.0s
 => => transferring dockerfile: 98B                                                                                  0.0s
Dockerfile:3
--------------------
   1 |     FROM ubuntu:latest
   2 |
   3 | >>> ENV THEENV
   4 |     RUN echo $THEENV
   5 |
--------------------
ERROR: failed to solve: ENV must have two arguments

However, we can remove the ENV instruction from the Dockerfile:

FROM ubuntu:latest
RUN echo $THEENV
CMD ["env"]

This allows us to build the image:

$ docker build -t envtest .
<-- SNIP -->
 => [1/2] FROM docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e34a9ab6  0.0s
 => => resolve docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e34a9ab6  0.0s
 => CACHED [2/2] RUN echo $THEENV                                                                                    0.0s
 => exporting to image                                                                                               0.0s
 => => exporting layers                                                                                              0.0s
<-- SNIP -->

Then we can pass an environment variable via the docker run command using the -e flag:

$ docker run --rm -e THEENV=bar envtest
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=638cf682d61f
THEENV=bar
HOME=/root

Although the .env file is usually associated with Docker Compose, it can also be used with docker run.

$ cat .env
THEENV=bar

$ docker run --rm --env-file ./.env envtest
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=59efe1003811
THEENV=bar
HOME=/root

This can also be done using Docker Compose by using the environment key. Note that we use the variable format for the value:

services:
  envtest:
    build:
      context: .
    environment:
      THEENV: ${THEENV}

If we do not supply a value for THEENV, a warning is thrown:

$ docker compose up --build
WARN[0000] The "THEENV" variable is not set. Defaulting to a blank string.
<-- SNIP -->
 => [envtest 1/2] FROM docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04  0.0s
 => => resolve docker.io/library/ubuntu:latest@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e34a9ab6  0.0s
 => CACHED [envtest 2/2] RUN echo ${THEENV}                                                                          0.0s
 => [envtest] exporting to image                                                                                     0.0s
<-- SNIP -->
 ✔ Container dd-envtest-1    Recreated                                                                               0.1s
Attaching to envtest-1
envtest-1  | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
envtest-1  | HOSTNAME=816d164dc067
envtest-1  | THEENV=
envtest-1  | HOME=/root
envtest-1 exited with code 0

The value for our variable can be supplied in several different ways, as follows:

  • On the compose command line:
$ THEENV=bar docker compose up

[+] Running 2/0
 ✔ Synchronized File Shares                                                                                          0.0s
 ✔ Container dd-envtest-1    Recreated                                                                               0.1s
Attaching to envtest-1
envtest-1  | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
envtest-1  | HOSTNAME=20f67bb40c6a
envtest-1  | THEENV=bar
envtest-1  | HOME=/root
envtest-1 exited with code 0
  • In the shell environment on the host system:
$ export THEENV=bar
$ docker compose up

[+] Running 2/0
 ✔ Synchronized File Shares                                                                                          0.0s
 ✔ Container dd-envtest-1    Created                                                                                 0.0s
Attaching to envtest-1
envtest-1  | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
envtest-1  | HOSTNAME=20f67bb40c6a
envtest-1  | THEENV=bar
envtest-1  | HOME=/root
envtest-1 exited with code 0
  • In the special .env file:
$ cat .env
THEENV=bar

$ docker compose up

[+] Running 2/0
 ✔ Synchronized File Shares                                                                                          0.0s
 ✔ Container dd-envtest-1    Created                                                                                 0.0s
Attaching to envtest-1
envtest-1  | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
envtest-1  | HOSTNAME=20f67bb40c6a
envtest-1  | THEENV=bar
envtest-1  | HOME=/root
envtest-1 exited with code 0

Finally, when running services directly using docker compose run, you can use the -e flag to override the .env file.

$ docker compose run -e THEENV=bar envtest

[+] Creating 1/0
 ✔ Synchronized File Shares                                                                                          0.0s
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=219e96494ddd
TERM=xterm
THEENV=bar
HOME=/root

The tl;dr

If you need to access a variable during the build process but not at runtime, use ARG. If you need to access the variable both during the build and at runtime, or only at runtime, use ENV.

To decide between them, consider the following flow (Figure 1):

build process

Both ARG and ENV can be overridden from the command line in docker run and docker compose, giving you a powerful way to dynamically update variables and build flexible workflows.

Learn more

Docker Best Practices: Using Tags and Labels to Manage Docker Image Sprawl

Par : Jay Schmidt
1 octobre 2024 à 12:51

With many organizations moving to container-based workflows, keeping track of the different versions of your images can become a problem. Even smaller organizations can have hundreds of container images spanning from one-off development tests, through emergency variants to fix problems, all the way to core production images. This leads us to the question: How can we tame our image sprawl while still rapidly iterating our images?

A common misconception is that by using the “latest” tag, you are guaranteeing that you are pulling the “latest version of the image. Unfortunately, this assumption is wrong — all latest means is “the last image pushed to this registry.”

Read on to learn more about how to avoid this pitfall when using Docker and how to get a handle on your Docker images.

Docker Best Practices: Using Tags and Labels to Manage Docker Image Sprawl

Using tags

One way to address this issue is to use tags when creating an image. Adding one or more tags to an image helps you remember what it is intended for and helps others as well. One approach is always to tag images with their semantic versioning (semver), which lets you know what version you are deploying. This sounds like a great approach, and, to some extent, it is, but there is a wrinkle.

Unless you’ve configured your registry for immutable tags, tags can be changed. For example, you could tag my-great-app as v1.0.0 and push the image to the registry. However, nothing stops your colleague from pushing their updated version of the app with tag v1.0.0 as well. Now that tag points to their image, not yours. If you add in the convenience tag latest, things get a bit more murky.

Let’s look at an example:

FROM busybox:stable-glibc

# Create a script that outputs the version
RUN echo -e "#!/bin/sh\n" > /test.sh && \
    echo "echo \"This is version 1.0.0\"" >> /test.sh && \
    chmod +x /test.sh

# Set the entrypoint to run the script
ENTRYPOINT ["/bin/sh", "/test.sh"]

We build the above with docker build -t tagexample:1.0.0 . and run it.

$ docker run --rm tagexample:1.0.0
This is version 1.0.0

What if we run it without a tag specified?

$ docker run --rm tagexample
Unable to find image 'tagexample:latest' locally
docker: Error response from daemon: pull access denied for tagexample, repository does not exist or may require 'docker login'.
See 'docker run --help'.

Now we build with docker build . without specifying a tag and run it.

$ docker run --rm tagexample
This is version 1.0.0

The latest tag is always applied to the most recent push that did not specify a tag. So, in our first test, we had one image in the repository with a tag of 1.0.0, but because we did not have any pushes without a tag, the latest tag did not point to an image. However, once we push an image without a tag, the latest tag is automatically applied to it.

Although it is tempting to always pull the latest tag, it’s rarely a good idea. The logical assumption — that this points to the most recent version of the image — is flawed. For example, another developer can update the application to version 1.0.1, build it with the tag 1.0.1, and push it. This results in the following:

$ docker run --rm tagexample:1.0.1
This is version 1.0.1

$ docker run --rm tagexample:latest
This is version 1.0.0

If you made the assumption that latest pointed to the highest version, you’d now be running an out-of-date version of the image.

The other issue is that there is no mechanism in place to prevent someone from inadvertently pushing with the wrong tag. For example, we could create another update to our code bringing it up to 1.0.2. We update the code, build the image, and push it — but we forget to change the tag to reflect the new version. Although it’s a small oversight, this action results in the following:

$ docker run --rm tagexample:1.0.1
This is version 1.0.2

Unfortunately, this happens all too frequently.

Using labels

Because we can’t trust tags, how should we ensure that we are able to identify our images? This is where the concept of adding metadata to our images becomes important.

The first attempt at using metadata to help manage images was the MAINTAINER instruction. This instruction sets the “Author” field (org.opencontainers.image.authors) in the generated image. However, this instruction has been deprecated in favor of the more powerful LABEL instruction. Unlike MAINTAINER, the LABEL instruction allows you to set arbitrary key/value pairs that can then be read with docker inspect as well as other tooling.

Unlike with tags, labels become part of the image, and when implemented properly, can provide a much better way to determine the version of an image. To return to our example above, let’s see how the use of a label would have made a difference.

To do this, we add the LABEL instruction to the Dockerfile, along with the key version and value 1.0.2.

FROM busybox:stable-glibc

LABEL version="1.0.2"

# Create a script that outputs the version
RUN echo -e "#!/bin/sh\n" > /test.sh && \
    echo "echo \"This is version 1.0.2\"" >> /test.sh && \
    chmod +x /test.sh

# Set the entrypoint to run the script
ENTRYPOINT ["/bin/sh", "/test.sh"]

Now, even if we make the same mistake above where we mistakenly tag the image as version 1.0.1, we have a way to check that does not involve running the container to see which version we are using.

$ docker inspect --format='{{json .Config.Labels}}' tagexample:1.0.1
{"version":"1.0.2"}

Best practices

Although you can use any key/value as a LABEL, there are recommendations. The OCI provides a set of suggested labels within the org.opencontainers.image namespace, as shown in the following table:

LabelContent
org.opencontainers.image.createdThe date and time on which the image was built (string, RFC 3339 date-time).
org.opencontainers.image.authorsContact details of the people or organization responsible for the image (freeform string).
org.opencontainers.image.urlURL to find more information on the image (string).
org.opencontainers.image.documentationURL to get documentation on the image (string).
org.opencontainers.image.sourceURL to the source code for building the image (string).
org.opencontainers.image.versionVersion of the packaged software (string).
org.opencontainers.image.revisionSource control revision identifier for the image (string).
org.opencontainers.image.vendorName of the distributing entity, organization, or individual (string).
org.opencontainers.image.licensesLicense(s) under which contained software is distributed (string, SPDX License List).
org.opencontainers.image.ref.nameName of the reference for a target (string).
org.opencontainers.image.titleHuman-readable title of the image (string).
org.opencontainers.image.descriptionHuman-readable description of the software packaged in the image (string).

Because LABEL takes any key/value, it is also possible to create custom labels. For example, labels specific to a team within a company could use the com.myorg.myteam namespace. Isolating these to a specific namespace ensures that they can easily be related back to the team that created the label.

Final thoughts

Image sprawl is a real problem for organizations, and, if not addressed, it can lead to confusion, rework, and potential production problems. By using tags and labels in a consistent manner, it is possible to eliminate these issues and provide a well-documented set of images that make work easier and not harder.

Learn more

10 Docker Myths Debunked

Par : Jay Schmidt
19 septembre 2024 à 13:59

Containers might seem like a relatively recent technological breakthrough, but their origins trace back to the 1970s when Unix systems first used container-like concepts to isolate applications. Fast-forward to 2013, and Docker revolutionized this idea by introducing a portable, user-friendly container platform, sparking widespread adoption. In 2015, Docker was instrumental in creating the Open Container Initiative (OCI) to promote open standards within the container ecosystem. With the stability provided by the OCI, container technology spread throughout the tech world.

Although Docker Desktop is the leading tool for creating containerized applications, Docker remains surrounded by numerous misconceptions. In this article, we’ll debunk the top Docker myths and explain the capabilities and benefits of this transformative technology.

2400x1260 evergreen docker blog e

Myth #1: Docker is no longer open source

Docker consists of multiple components, most of which are open source. The core Docker Engine is open source and licensed under the Apache 2.0 license, so developers can continue to use and contribute to it freely. Other vital parts of the Docker ecosystem, like the Docker CLI and Docker Compose, also remain open source. This allows the community to maintain transparency, contribute improvements, and customize their container solutions.

Docker’s commitment to open source is best illustrated by the Moby Project. In 2017, Moby was spun out of the then-monolithic Docker codebase to provide a set of “building blocks” to create containerized solutions and platforms. Docker uses the Moby project for the free Docker Engine project and our commercial Docker Desktop.

Users can also find Trusted Open Source Content on Docker Hub. These Docker-Sponsored Open Source and Docker Official Images offer trusted versions of open source projects and reliable building blocks for better development.

Docker is a founder and remains a crucial contributor to the OCI, which defines container standards. This initiative ensures that Docker and other container technologies remain interoperable and maintain a commitment to open source principles.

Myth #2: Docker containers are virtual machines 

Docker containers are often mistaken for virtual machines (VMs), but the technologies operate quite differently. Unlike VMs, Docker containers don’t include an entire operating system (OS). Instead, they share the host operating system kernel, making them more lightweight and efficient. VMs require a hypervisor to create virtual hardware for the guest OS, which introduces significant overhead. Docker only packages the application and its dependencies, allowing for faster startup times and minimal performance overhead.

By utilizing the host operating system’s resources efficiently, Docker containers use fewer resources overall than VMs, which need substantial resources to run multiple operating systems concurrently. Docker’s architecture efficiently runs numerous isolated applications on a single host, optimizing infrastructure and development workflows. Understanding this distinction is crucial for maximizing Docker’s lightweight and scalable potential.

However, when running on non-Linux systems, Docker needs to emulate a Linux environment. For example, Docker Desktop uses a fully managed VM to provide a consistent experience across Windows, Mac, and Linux by running its Linux components inside this VM.

Myth #3: Docker Engine vs. Docker Desktop vs. Docker Enterprise Edition — They’re all the same

Considerable confusion surrounds the different Docker options that are available, which include:

  • Mirantis Container Runtime: Docker Enterprise Edition (Docker EE) was sold to Mirantis in 2019 and rebranded as Mirantis Container Runtime. This software, which is managed and sold by Mirantis, is designed for production container deployments and offers a lightweight alternative to existing orchestration tools.
  • Docker Engine: Docker Engine is the fully open source version built from the Moby Project, providing the Docker Engine and CLI.
  • Docker Desktop: Docker Desktop is a commercial offering sold by Docker that combines Docker Engine with additional features to enhance developer productivity. The Docker Business subscription includes advanced security and governance features for enterprises.

All of these variants are OCI-compliant, differing mainly in features and experiences. Docker Engine caters to the open source community, Docker Desktop elevates developer workflows with a comprehensive suite of tools for building and scaling applications, and Mirantis Container Runtime provides a specialized solution for enterprise production environments with advanced management and support. Understanding these distinctions is crucial for selecting the appropriate Docker variant to meet specific project requirements and organizational goals.

Myth #4: Docker is the same thing as Kubernetes

This myth arises from the fact that both Docker and Kubernetes are associated with containerized environments. Although they are both key players in the container ecosystem, they serve different roles.

Kubernetes (K8s) is an orchestration system for managing container instances at scale. This container orchestration tool automates the deployment, scaling, and operations of multiple containers across clusters of hosts. Other orchestration technologies include Nomad, serverless frameworks, Docker’s Swarm mode, and Apache Mesos. Each offers different features for managing containerized workloads.

Docker is primarily a platform for developing, shipping, and running containerized applications. It focuses on packaging applications and their dependencies in a portable container and is often used for local development where scaling is not required. Docker Desktop includes Docker Compose, which is designed to orchestrate multi-container deployments locally

In many organizations, Docker is used to develop applications, and the resulting Docker images are then deployed to Kubernetes for production. To support this workflow, Docker Desktop includes an embedded Kubernetes installation and the Compose Bridge tool for translating Compose format into Kubernetes-friendly code.

Myth #5: Docker is not secure

The belief that Docker is not secure is often a result of misunderstandings around how security is implemented within Docker. To help reduce security vulnerabilities and minimize the attack surface, Docker offers the following measures:

Opt-in security configuration 

Except for a few components, Docker operates on an opt-in basis for security. This approach removes friction for new users, but means Docker can still be configured to be more secure for enterprise considerations and for security-conscious users with sensitive data.

“Rootless” mode capabilities 

Docker Engine can run in rootless mode, where the Docker daemon runs without root permissions. This capability reduces the potential blast radius of malicious code escaping a container and gaining root permissions on the host. Docker Desktop takes security further by offering Enhanced Container Isolation (ECI), which provides advanced isolation features beyond what rootless mode can offer.

Built-in security features

Additionally, Docker security includes built-in features such as namespaces, control groups (cgroups), and seccomp profiles that provide isolation and limit the capabilities of containers.

SOC 2 Type 2 Attestation and ISO 27001 Certification

It’s important to note that, as an open source tool, Docker Engine is not in scope for SOC 2 Type 2 Attestation or ISO 27001 Certification. These certifications pertain to Docker, Inc.’s paid products, which offer additional enterprise-grade security and compliance features. These paid features, outlined in a Docker security blog post, focus on enhancing security and simplifying compliance for SOC 2, ISO 27001, FedRAMP, and other standards.  

Along with these security measures, Docker also provides best practices in the Docker documentation and training materials to help users learn how to secure their containers effectively. Recognizing and implementing these features reduces security risks and ensures that Docker can be a secure platform for containerized applications.

Myth #6: Docker is dead

This myth stems from the rapid growth and changes within the container ecosystem over the past decade. To keep pace with these changes, Docker is actively developed and is also widely adopted. In fact, the Stack Overflow community chose Docker as the most-used and most-desired developer tool in the 2024 Developer Survey for the second year in a row and recognized it as the most-admired developer tool. 

Docker Hub is one of the world’s largest repositories of container images. According to the 2024 Docker State of Application Development Report, tools like Docker Desktop, Docker Scout, Docker Build Cloud, and Docker Debug are integral to more than two-thirds of container development workflows. And, as a founding member of the OCI and steward of the Moby project, Docker continues to play a guiding role in containerization.

In the automation space, Docker is crucial for building OCI images and creating lightweight runners for build queues. With the rise of data science and AI/ML, Docker images facilitate the exchange of models, notebooks, and applications, supported by GPU workload capabilities in Docker Desktop. Additionally, Docker is widely used for quickly and cost-effectively mocking up test scenarios as an alternative to deploying actual hardware or VMs.

Myth #7: Docker is hard to learn

The belief that Docker is difficult to learn often comes from the perceived complexity of container concepts and Docker’s many features. However, Docker is a foundational technology used by more than 20 million developers worldwide, and countless resources are available to make learning Docker accessible.

Docker, Inc. is committed to the developer experience, creating intuitive and user-friendly product design for Docker Desktop and supporting products. Documentation, workshops, training, and examples are accessible through Docker Desktop, the Docker website and blog, and the Docker Navigator newsletter. Additionally, the Docker documentation site offers comprehensive guides and learning paths, and Udemy courses co-produced with Docker help new users understand containerization and Docker usage.

The thriving Docker community also contributes a wealth of content and resources, including video tutorials, how-tos, and in-person talks.

Myth #8: Docker and container technology are only for developers

The idea that Docker is only for developers is a common misconception. Docker and containers are used across various fields beyond development. Docker Desktop’s ability to run containerized workloads on Windows, macOS, or Linux requires minimal technical knowledge from users. Its integration features — synchronized host filesystems, network proxy support, air-gapped containers, and resource controls — ensure administrators can enforce governance and security.

  • Data science: Docker provides consistent environments, enabling data scientists to share models, datasets, and development setups seamlessly.
  • Healthcare: Docker deploys scalable applications for managing patient data and running simulations, such as medical imaging software across different hospital systems.
  • Education: Educators and students use Docker to create reproducible research environments, which facilitate collaboration and simplify coding project setups.

Docker’s versatility extends beyond development, providing consistent, scalable, and secure environments for various applications.

Myth #9: Docker Desktop is just a GUI

The myth that Docker Desktop is merely a graphical user interface (GUI) overlooks its extensive features designed to enhance developer experience, streamline container management, and accelerate productivity, such as:

Cross-platform support

Docker is Linux-based, but most developer workstations run Windows or macOS. Docker Desktop enables these platforms to run Docker tooling inside a fully managed VM integrated with the host system’s networking, filesystem, and resources.

Developer tools

Docker Desktop includes built-in Kubernetes, Docker Scout for supply chain management, Docker Build Cloud for faster builds, and Docker Debug for container debugging.

Security and governance

For administrators, Docker Desktop offers Registry Access Management and Image Access Management, Enhanced Container Isolation, single sign-on (SSO) for authorization, and Settings Management, making it an essential tool for enterprise deployment and management.

Myth #10: Docker containers are for microservices only

Although Docker containers are popular for microservices architectures, they can be used for any type of application. For example, monolithic applications can be containerized, allowing them and their dependencies to be isolated into a versioned image that can run across different environments. This approach enables gradual refactoring into microservices if desired.

Additionally, Docker is excellent for rapid prototyping, allowing quick deployment of minimum viable products (MVPs). Containerized prototypes are easier to manage and refactor compared to those deployed on VMs or bare metal.

Now you know

Now that you have the facts, it’s clear that adopting Docker can significantly enhance productivity, scalability, and security for a variety of use cases. Docker’s versatility, combined with extensive learning resources and robust security features, makes it an indispensable tool in modern software development and deployment. Adopting Docker and its true capabilities can significantly enhance productivity, scalability, and security for your use case.

For more detailed insights, refer to the 2024 Docker State of Application Development Report or dive into Docker Desktop now to start your Docker journey today

Learn more

Zero Trust and Docker Desktop: An Introduction

Par : Jay Schmidt
13 août 2024 à 13:11

Today’s digital landscape is characterized by frequent security breaches resulting in lost revenue, potential legal liability, and loss of customer trust. The Zero Trust model was devised to improve an organization’s security posture and minimize the risk and scope of security breaches.

In this post, we explore Zero Trust security and walk through several strategies for implementing Zero Trust within a Docker Desktop-based development environment. Although this overview is not exhaustive, it offers a foundational perspective that organizations can build upon as they refine and implement their own security strategies.

2400x1260 security column 072024

What is Zero Trust security?

The Zero Trust security model assumes that no entity — inside or outside the network boundary — should be automatically trusted. This approach eliminates automatic trust and mandates rigorous verification of all requests and operations before granting access to resources. Zero Trust significantly enhances security measures by consistently requiring that trust be earned.

The principles and practices of Zero Trust are detailed by the National Institute of Standards and Technology (NIST) in Special Publication 800-207 — Zero Trust Architecture. This document serves as an authoritative guide, outlining the core tenets of Zero Trust, which include strict access control, minimal privileges, and continuous verification of all operational and environmental attributes. For example, Section 2.1 of this publication elaborates on the foundational principles that organizations can adopt to implement a robust Zero Trust environment tailored to their unique security needs. By referencing these guidelines, practitioners can gain a comprehensive understanding of Zero Trust, which aids in strategically implementing its principles across network architectures and strengthening an organization’s security posture.

As organizations transition toward containerized applications and cloud-based architectures, adopting Zero Trust is essential. These environments are marked by their dynamism, with container fleets scaling and evolving rapidly to meet business demands. Unlike traditional security models that rely on perimeter defenses, these modern infrastructures require a security strategy that supports continuous change while ensuring system stability. 

Integrating Zero Trust into the software development life cycle (SDLC) from the outset is crucial. Early adoption ensures that Zero Trust principles are not merely tacked on post-deployment but are embedded within the development process, providing a foundational security framework from the beginning.

Containers and Zero Trust

The isolation of applications and environments from each other via containerization helps with the implementation of Zero Trust by making it easier to apply access controls, apply more granular monitoring and detection rules, and audit the results.

As noted previously, these examples are specific to Docker Desktop, but you can apply the concepts to any container-based environment, including orchestration systems such as Kubernetes.

A solid foundation: Host and network

When applying Zero Trust principles to Docker Desktop, starting with the host system is important. This system should also meet Zero Trust requirements, such as using encrypted storage, limiting user privileges within the operating system, and enabling endpoint monitoring and logging. The host system’s attachment to network resources should require authentication, and all communications should be secured and encrypted.

Principle of least privilege

The principle of least privilege is a fundamental security approach stating that a user, program, or process should have only the minimum permissions necessary to perform its intended function and no more. In terms of working with containers, effectively implementing this principle requires using AppArmor/SELinux, using seccomp (secure computing mode) profiles, ensuring containers do not run as root, ensuring containers do not request or receive heightened privileges, and so on.

Hardened Docker Desktop (available with a Docker Business or Docker Government subscription), however, can satisfy this requirement through the Enhanced Container Isolation (ECI) setting. When active, ECI will do the following:

  • Running containers unprivileged: ECI ensures that even if a container is started with the --privileged flag, the actual processes inside the container do not have elevated privileges within the host or the Docker Desktop VM. This step is crucial for preventing privilege escalation attacks.
  • User namespace remapping: ECI uses a technique where the root user inside a container is mapped to a non-root user outside the container, in the Docker Desktop VM. This approach limits the potential damage and access scope even if a container is compromised.
  • Restricted file system access: Containers run under ECI have limited access to the file system of the host machine. This restriction prevents a compromised container from altering system files or accessing sensitive areas of the host file system.
  • Blocking sensitive system calls: ECI can block or filter system calls from containers that are typically used in attacks, such as certain types of mount operations, further reducing the risk of a breakout.
  • Isolation from the Docker Engine: ECI prevents containers from interacting directly with the Docker Engine’s API unless explicitly allowed, protecting against attacks that target the Docker infrastructure itself.

Network microsegmentation

Microsegmentation offers a way to enhance security further by controlling traffic flow among containers. Through the implementation of stringent network policies, only authorized containers are allowed to interact, which significantly reduces the risk of lateral movement in case of a security breach. For example, a payment processing container may only accept connections from specific parts of the application, isolating it from other, less secure network segments.

The concept of microsegmentation also plays a role for AI systems and workloads. By segmenting networks and data, organizations can apply controls to different parts of their AI infrastructure, effectively isolating the environments used for training, testing, and production. This isolation helps reduce the list of data leakage between environments and can help reduce the blast radius of a security breach.

Docker Desktop’s robust networking provides several ways to address microsegmentation. By leveraging the bridge network for creating isolated networks within the same host or using the Macvlan network driver that allows containers to be treated as physical devices with distinct MAC addresses, administrators can define precise communication paths that align with the least privileged access principles of Zero Trust. Additionally, Docker Compose can easily manage and configure these networks, specifying which containers can communicate on predefined networks. 

This setup facilitates fine-grained network policies at the infrastructure level. It also simplifies the management of container access, ensuring that strict network segmentation policies are enforced to minimize the attack surface and reduce the risk of unauthorized access in containerized environments. Additionally, Docker Desktop supports third-party network drivers, which can also be used to address this concern.

For use cases where Docker Desktop requires containers to have different egress rules than the host, “air-gapped containers” allow for the configuration of granular rules applied to containers. For example, containers can be completely restricted from the internet but allowed on the local network, or they could be proxied/firewalled to a small set of approved hosts.

Note that in Kubernetes, this type of microsegmentation and network traffic management is usually managed by a service mesh.

Authentication and authorization

Implementing strong authentication and role-based access control (RBAC) is crucial in a Docker-based Zero Trust environment. These principles need to be addressed in several different areas, starting with the host and network as noted above.

Single Sign On (SSO) and System for Cross-Domain Identity Management (SCIM) should be enabled and used to manage user authentication to the Docker SaaS. These tools allow for better management of users, including the use of groups to enforce role and team membership at the account level. Additionally, Docker Desktop should be configured to require and enforce login to the Docker organization in use, which prevents users from logging into any other organizations or personal accounts.

When designing, deploying, building, and testing containers locally under Docker Desktop, implementing robust authentication and authorization mechanisms is crucial to align with security best practices and principles. It’s essential to enforce strict access controls at each stage of the container lifecycle.

This approach starts with managing registry and image access, to ensure only approved images are brought into the development process. This can be accomplished by using an internal registry and enforcing firewall rules that block access to other registries. However, an easier approach is to use Registry Access Management (RAM) and Image Access Management (IAM) — features provided by Hardened Docker Desktop — to control images and registries.

The implementation of policies and procedures around secrets management — such as using a purpose-designed secrets store — should be part of the development process. Finally, using Enhanced Container Isolation (as described above) will help ensure that container privileges are managed consistently with best practices.

This comprehensive approach not only strengthens security but also helps maintain the integrity and confidentiality of the development environment, especially when dealing with sensitive or proprietary application data.

Monitoring and auditing

Continuous monitoring and auditing of activities within the Docker environment are vital for early detection of potential security issues. These controls build on the areas identified above by allowing for the auditing and monitoring of the impact of these controls.

Docker Desktop produces a number of logs that provide insight into the operations of the entire application platform. This includes information about the local environment, the internal VM, the image store, container runtime, and more. This data can be redirected and parsed/analyzed by industry standard tooling.

Container logging is important and should be sent to a remote log aggregator for processing. Because the best development approaches require that log formats and log levels from development mirror those used in production, this data can be used not only to look for anomalies in the development process but also to provide operations teams with an idea of what production will look like.

Docker Scout

Ensuring containerized applications comply with security and privacy policies is another key part of continuous monitoring. Docker Scout is designed from the ground up to support this effort. 

Docker Scout starts with the image software bill of materials (SBOM) and continually checks against known and emerging CVEs and security policies. These policies can include detecting high-profile CVEs to be mitigated, validating that approved base images are used, verifying that only valid licenses are being used, and ensuring that a non-root user is defined in the image. Beyond that, the Docker Scout policy engine can be used to write custom policies using the wide array of data points available.  

Immutable containers

The concept of immutable containers, which are not altered after they are deployed, plays a significant role in securing environments. By ensuring that containers are replaced rather than changed, the security of the environment is enhanced, preventing unauthorized or malicious alterations during runtime.

Docker images — more broadly, OCI-compliant images — are immutable by default. When they are deployed as containers, they become writable while they are running via the addition of a “scratch layer” on top of the immutable image. Note that this layer does not persist beyond the life of the container. When the container is removed, the scratch layer is removed as well.

When the immutable flag is added — either by adding the --read-only flag to the docker run command or by adding the read_only: true key value pair in docker compose — Docker will mount the root file system read-only, which prevents writes to the container file system.

In addition to making a container immutable, it is possible to mount Docker volumes as read/write or read-only. Note that you can make the container’s root file system read-only and then use a volume read/write to better manage write access for your container.

Encryption

Ensuring that data is securely encrypted, both in transit and at rest, is non-negotiable in a secure Docker environment. Docker containers should be configured to use TLS for communications both between containers and outside the container environment. Docker images and volumes are stored locally and can benefit from the host system’s disk encryption when they are at rest.

Tool chain updates

Finally, it is important to make sure that Docker Desktop is updated to the most current version, as the Docker team is continually making improvements and mitigating CVEs as they are discovered. For more information, refer to Docker security documentation and Docker security announcements.

Overcoming challenges in Zero Trust adoption

Implementing a Zero Trust architecture with Docker Desktop is not without its challenges. Such challenges include the complexity of managing such environments, potential performance overhead, and the need for a cultural shift within organizations towards enhanced security awareness. However, the benefits of a secure, resilient infrastructure far outweigh these challenges, making the effort and investment in Zero Trust worthwhile.

Conclusion

Incorporating Zero Trust principles into Docker Desktop environments is essential for protecting modern infrastructures against sophisticated cyber threats. By understanding and implementing these principles, organizations can safeguard their applications and data more effectively, ensuring a secure and resilient digital presence.

Learn more

Docker Best Practices: Understanding the Differences Between ADD and COPY Instructions in Dockerfiles

Par : Jay Schmidt
8 août 2024 à 13:30

COPY vs. ADD tl;dr:

When you search for “Dockerfile best practices,” one of the suggestions you will find is that you always use the COPY instruction instead of the ADD instruction when adding files into your Docker image.

This blog post will explore why this suggestion exists by providing additional detail on the functionality of these two instructions. Once you understand these concepts, you may find scenarios where you can benefit from ignoring the suggestion and using the ADD command instead of COPY.

2400x1260 understanding the differences between add and copy instructions in dockerfiles

Understanding file system build context

Before diving into the differences between ADD and COPY, it’s important to understand the concept of build context. The build context is the set of files and directories that are accessible to the Docker engine when building an image. When you run a docker build command, Docker sends the content of the specified context directory (and its subdirectories) to the Docker daemon. This context forms the scope within which the COPY and ADD instructions operate.

COPY instruction

The COPY instruction is straightforward and does exactly what its name implies: It copies files and directories from a source within the build context to a destination layer in the Docker image. This instruction can be used to copy both files and directories, and all paths on the host are relative to the root of the build context.

Syntax:

COPY <src>... <dest>
  • <src>: The source files or directories on the host.
  • <dest>: The destination path inside the Docker image.

Key points

  • Basic functionality: COPY only supports copying files and directories from the host file system. It does not support URLs or automatic unpacking of compressed files.
  • Security: Because COPY only handles local files, it tends to be more predictable and secure than ADD, reducing the risk of unintentionally introducing files from external sources.
  • Use case: Best used when you need to include files from your local build context into the Docker image without any additional processing.

Example:

COPY ./app /usr/src/app
COPY requirements.txt /usr/src/app/

In this example, the contexts of the local app directory are copied into the /usr/src/app directory inside the Docker image being built. The second command copies the requirements.txt file into the /usr/src/app directory as well.

ADD instruction

The ADD instruction provides the same functionality that the COPY instruction does, but it also has additional functionality that, if misunderstood, can introduce complexity and potential security risks.

Syntax:

ADD <src>... <dest>
  • <src>: The source files (directories or URLs).
  • <dest>: The destination path inside the Docker image.

Key points

  • Extended functionality: In addition to copying local files and directories from the build context, ADD provides the following advanced functionality:
    • Handle URLs: When supplied as a source, the file referenced by a URL will be downloaded to the current Docker image layer at the supplied destination path.
    • Extract archives: When supplied as a source, ADD will automatically unpack and expand archives to the current Docker image layer at the supplied destination path.
  • Flexibility vs. security: Although ADD is more flexible, it does introduce risk. Downloading external URLs into the build process may allow malicious code or contents to be brought into the process. Using ADD with archives may result in unintended consequences if you do not understand how it handles archives.
  • Use case: ADD should only be used when you need specific functionality that it provides and are willing to manage the potential security issues arising from this usage.

Example:

ADD https://example.com/file.tar.gz /usr/src/app/
ADD my-archive.tar.gz /usr/src/app/

In this example, the build process first downloads https://example.com/file.tar.gz and extracts the contents into /usr/src/app in the Docker image layer. In the next step, it takes the local file my-archive.tar.gz and extracts it into the Docker image layer under /usr/src/app.

When to use COPY vs. ADD

  • For most use cases, COPY is the better choice due to its simplicity and security. This instruction allows you to transfer files and directories from your local context into the Docker image you are building.
  • Use ADD only when you need the additional capabilities it offers, but be mindful of potential security implications.

Remote contexts

In addition to traditional file system contexts, Docker also supports remote contexts, which can be particularly useful in cloud environments or for building images from code repositories directly. These include:

  • Git repositories: You can specify a Git repository URL as the build context, allowing Docker to clone the repository and use its content as the context.
docker build https://github.com/username/repository.git#branch
  • Remote URLs: Docker can use remote URLs for the build context. This is useful for building images directly from archives available online.
docker build http://example.com/context.tar.gz
  • OCI images: You can use an OCI image as the build context, which is useful for using pre-built images as the starting point for new builds.
docker build oci://registry.example.com/image:tag

How ADD and COPY behave in remote contexts

Note that both ADD and COPY behave slightly differently when used in a remote context.

Using COPY with remote contexts

COPY still operates within the scope of the build context, and can copy files and directories from the cloned repository into the Docker image. For example, when using a Git repository as the build context, COPY can copy files and directories from the cloned repository into the Docker image. It does not support copying files from URLs or other remote sources directly.

Example with Git repository as build context:

# Using a Git repository as build context
COPY ./src /app/src

In this case, COPY will copy the src directory from the Git repository (the build context) to /app/src in the Docker image.

Example with URL build context:

# Using an archive from a URL
COPY ./src /app/src

In this case, COPY will copy the src directory from the extracted archive (the build context) to /app/src in the Docker image.

Example with OCI image as build context:

# Using an OCI image as build context
COPY /path/in/oci/image /app/path

In this case, COPY will copy the contents from the specified path within the OCI image to the specified destination path in the Docker image.

Using ADD with remote contexts

The ADD instruction can still be used to download files and extract archives as well as copy files from the build context. Note that all the caveats provided about the ADD instruction above apply here as well.

Example with Git repository as build context:

# Using a Git repository as build context
ADD https://example.com/data.tar.gz /data
ADD ./src /app/src

In this example, ADD will download and extract data.tar.gz from the URL into the /data directory in the Docker image. It will also copy the src directory from the Git repository (the build context) to /app/src in the Docker image.

Example with URL build context:

# Using an archive from a URL
ADD https://example.com/data.tar.gz /data
ADD ./src /app/src

In this example, ADD will download and extract data.tar.gz from the URL into the /data directory in the Docker image. It will also copy the src directory from the downloaded/unpacked URL (the build context) to /app/src in the Docker image.

Example with OCI image as build context:

# Using an OCI image as build context
ADD https://example.com/data.tar.gz /data
ADD /path/in/oci/image /app/path

In this scenario, ADD will download and extract data.tar.gz from the URL into the /data directory in the Docker image. It will also copy the contents from the specified path within the OCI image to the specified destination path in the Docker image.

COPY vs. ADD tl;dr:

  • Prefer COPY: For most use cases, COPY is the better choice due to its simplicity and security. Use it to transfer files and directories from your local context or a remote context like a Git repository to the Docker image.
  • Use ADD with caution: Opt for ADD only when you need its additional functionalities, like downloading files from URLs or automatically extracting archives (Figure 1). Always be mindful of the potential security implications when using ADD.
Diagram showing concepts that are also explained in the blog post.

Conclusion

Understanding the differences between ADD and COPY instructions in Dockerfiles and how they can be affected by build context can help you build more efficient and secure Docker images. Although COPY offers a straightforward way to include local files, ADD provides additional flexibility with the cost of increased complexity and potential security risks.

Learn more

Docker Best Practices: Choosing Between RUN, CMD, and ENTRYPOINT

Par : Jay Schmidt
15 juillet 2024 à 14:00

Docker’s flexibility and robustness as a containerization tool come with a complexity that can be daunting. Multiple methods are available to accomplish similar tasks, and users must understand the pros and cons of the available options to choose the best approach for their projects.

One confusing area concerns the RUN, CMD, and ENTRYPOINT Dockerfile instructions. In this article, we will discuss the differences between these instructions and describe use cases for each.

2400x1260 choosing between run cmd and entrypoint

RUN

The RUN instruction is used in Dockerfiles to execute commands that build and configure the Docker image. These commands are executed during the image build process, and each RUN instruction creates a new layer in the Docker image. For example, if you create an image that requires specific software or libraries installed, you would use RUN to execute the necessary installation commands.

The following example shows how to instruct the Docker build process to update the apt cache and install Apache during an image build:

RUN apt update && apt -y install apache2

RUN instructions should be used judiciously to keep the image layers to a minimum, combining related commands into a single RUN instruction where possible to reduce image size.

CMD

The CMD instruction specifies the default command to run when a container is started from the Docker image. If no command is specified during the container startup (i.e., in the docker run command), this default is used. CMD can be overridden by supplying command-line arguments to docker run.

CMD is useful for setting default commands and easily overridden parameters. It is often used in images as a way of defining default run parameters and can be overridden from the command line when the container is run. 

For example, by default, you might want a web server to start, but users could override this to run a shell instead:

CMD ["apache2ctl", "-DFOREGROUND"]

Users can start the container with docker run -it <image> /bin/bash to get a Bash shell instead of starting Apache.  

ENTRYPOINT

The ENTRYPOINT instruction sets the default executable for the container. It is similar to CMD but is overridden by the command-line arguments passed to docker run. Instead, any command-line arguments are appended to the ENTRYPOINT command.

Note: Use ENTRYPOINT when you need your container to always run the same base command, and you want to allow users to append additional commands at the end. 

ENTRYPOINT is particularly useful for turning a container into a standalone executable. For example, suppose you are packaging a custom script that requires arguments (e.g., “my_script extra_args”). In that case, you can use ENTRYPOINT to always run the script process (“my_script”) and then allow the image users to specify the “extra_args” on the docker run command line. You can do the following:

ENTRYPOINT ["my_script"]

Combining CMD and ENTRYPOINT

The CMD instruction can be used to provide default arguments to an ENTRYPOINT if it is specified in the exec form. This setup allows the entry point to be the main executable and CMD to specify additional arguments that can be overridden by the user.

For example, you might have a container that runs a Python application where you always want to use the same application file but allow users to specify different command-line arguments:

ENTRYPOINT ["python", "/app/my_script.py"]
CMD ["--default-arg"]

Running docker run myimage --user-arg executes python /app/my_script.py --user-arg.

The following table provides an overview of these commands and use cases.

Command description and use cases

CommandDescriptionUse Case
CMDDefines the default executable of a Docker image. It can be overridden by docker run arguments.Utility images allow users to pass different executables and arguments on the command line.
ENTRYPOINTDefines the default executable. It can be overridden by the “--entrypoint”  docker run arguments.Images built for a specific purpose where overriding the default executable is not desired.
RUNExecutes commands to build layers.Building an image

What is PID 1 and why does it matter?

In the context of Unix and Unix-like systems, including Docker containers, PID 1 refers to the first process started during system boot. All other processes are then started by PID 1, which in the process tree model is the parent of every process in the system. 

In Docker containers, the process that runs as PID 1 is crucial, because it is responsible for managing all other processes inside the container. Additionally, PID 1 is the process that reviews and handles signals from the Docker host. For example, a SIGTERM into the container will be caught and processed by PID 1, and the container should gracefully shut down.

When commands are executed in Docker using the shell form, a shell process (/bin/sh -c) typically becomes PID 1. Still, it does not properly handle these signals, potentially leading to unclean shutdowns of the container. In contrast, when using the exec form, the command runs directly as PID 1 without involving a shell, which allows it to receive and handle signals directly. 

This behavior ensures that the container can gracefully stop, restart, or handle interruptions, making the exec form preferable for applications that require robust and responsive signal handling.

Shell and exec forms

In the previous examples, we used two ways to pass arguments to the RUN, CMD, and ENTRYPOINT instructions. These are referred to as shell form and exec form. 

Note: The key visual difference is that the exec form is passed as a comma-delimited array of commands and arguments with one argument/command per element. Conversely, shell form is expressed as a string combining commands and arguments. 

Each form has implications for executing commands within containers, influencing everything from signal handling to environment variable expansion. The following table provides a quick reference guide for the different forms.

Shell and exec form reference

FormDescriptionExample
Shell FormTakes the form of <INSTRUCTION> <COMMAND>.CMD echo TEST or ENTRYPOINT echo TEST
Exec FormTakes the form of <INSTRUCTION> ["EXECUTABLE", "PARAMETER"].CMD ["echo", "TEST"] or ENTRYPOINT ["echo", "TEST"]

In the shell form, the command is run in a subshell, typically /bin/sh -c on Linux systems. This form is useful because it allows shell processing (like variable expansion, wildcards, etc.), making it more flexible for certain types of commands (see this shell scripting article for examples of shell processing). However, it also means that the process running your command isn’t the container’s PID 1, which can lead to issues with signal handling because signals sent by Docker (like SIGTERM for graceful shutdowns) are received by the shell rather than the intended process.

The exec form does not invoke a command shell. This means the command you specify is executed directly as the container’s PID 1, which is important for correctly handling signals sent to the container. Additionally, this form does not perform shell expansions, so it’s more secure and predictable, especially for specifying arguments or commands from external sources.

Putting it all together

To illustrate the practical application and nuances of Docker’s RUN, CMD, and ENTRYPOINT instructions, along with the choice between shell and exec forms, let’s review some examples. These examples demonstrate how each instruction can be utilized effectively in real-world Dockerfile scenarios, highlighting the differences between shell and exec forms. 

Through these examples, you’ll better understand when and how to use each directive to tailor container behavior precisely to your needs, ensuring proper configuration, security, and performance of your Docker containers. This hands-on approach will help consolidate the theoretical knowledge we’ve discussed into actionable insights that can be directly applied to your Docker projects.

RUN instruction

For RUN, used during the Docker build process to install packages or modify files, choosing between shell and exec form can depend on the need for shell processing. The shell form is necessary for commands that require shell functionality, such as pipelines or file globbing. However, the exec form is preferable for straightforward commands without shell features, as it reduces complexity and potential errors.

# Shell form, useful for complex scripting
RUN apt-get update && apt-get install -y nginx

# Exec form, for direct command execution
RUN ["apt-get", "update"]
RUN ["apt-get", "install", "-y", "nginx"]

CMD and ENTRYPOINT

These instructions control container runtime behavior. Using exec form with ENTRYPOINT ensures that the container’s main application handles signals directly, which is crucial for proper startup and shutdown behavior.  CMD can provide default parameters to an ENTRYPOINT defined in exec form, offering flexibility and robust signal handling.

# ENTRYPOINT with exec form for direct process control
ENTRYPOINT ["httpd"]

# CMD provides default parameters, can be overridden at runtime
CMD ["-D", "FOREGROUND"]

Signal handling and flexibility

Using ENTRYPOINT in exec form and CMD to specify parameters ensures that Docker containers can handle operating system signals gracefully, respond to user inputs dynamically, and maintain secure and predictable operations. 

This setup is particularly beneficial for containers that run critical applications needing reliable shutdown and configuration behaviors. The following table shows key differences between the forms.

Key differences between shell and exec

Shell FormExec Form
FormCommands without [] brackets. Run by the container’s shell, e.g., /bin/sh -c.Commands with [] brackets. Run directly, not through a shell.
Variable SubstitutionInherits environment variables from the shell, such as $HOME and $PATH.Does not inherit shell environment variables but behaves the same for ENV instruction variables.
Shell FeaturesSupports sub-commands, piping output, chaining commands, I/O redirection, etc.Does not support shell features.
Signal Trapping & ForwardingMost shells do not forward process signals to child processes.Directly traps and forwards signals like SIGINT.
Usage with ENTRYPOINTCan cause issues with signal forwarding.Recommended due to better signal handling.
CMD as ENTRYPOINT ParametersNot possible with the shell form.If the first item in the array is not a command, all items are used as parameters for the ENTRYPOINT.

Figure 1 provides a decision tree for using RUN, CMD, and ENTRYPOINT in building a Dockerfile.

2400x1260 run cmd entrypoint
Figure 1: Decision tree — RUN, CMD, ENTRYPOINT.

Figure 2 shows a decision tree to help determine when to use exec form or shell form.

2400x1260 decision tree exec vs shell
Figure 2: Decision tree — exec vs. shell form.

Examples

The following section will walk through the high-level differences between CMD and ENTRYPOINT. In these examples, the RUN command is not included, given that the only decision to make there is easily handled by reviewing the two different formats.

Test Dockerfile

# Use syntax version 1.3-labs for Dockerfile
# syntax=docker/dockerfile:1.3-labs

# Use the Ubuntu 20.04 image as the base image
FROM ubuntu:20.04

# Run the following commands inside the container:
# 1. Update the package lists for upgrades and new package installations
# 2. Install the apache2-utils package (which includes the 'ab' tool)
# 3. Remove the package lists to reduce the image size
#
# This is all run in a HEREDOC; see
# https://www.docker.com/blog/introduction-to-heredocs-in-dockerfiles/
# for more details.
#
RUN <<EOF
apt-get update;
apt-get install -y apache2-utils;
rm -rf /var/lib/apt/lists/*;
EOF

# Set the default command
CMD ab

First build

We will build this image and tag it as ab.

$ docker build -t ab .

[+] Building 7.0s (6/6) FINISHED                                                               docker:desktop-linux
 => [internal] load .dockerignore                                                                              0.0s
 => => transferring context: 2B                                                                                0.0s
 => [internal] load build definition from Dockerfile                                                           0.0s
 => => transferring dockerfile: 730B                                                                           0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                0.4s
 => CACHED [1/2] FROM docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e  0.0s
 => [2/2] RUN <<EOF (apt-get update;...)                                                                       6.5s
 => exporting to image                                                                                         0.0s
 => => exporting layers                                                                                        0.0s
 => => writing image sha256:99ca34fac6a38b79aefd859540f88e309ca759aad0d7ad066c4931356881e518                   0.0s
 => => naming to docker.io/library/ab 

Run with CMD ab

Without any arguments, we get a usage block as expected.

$ docker run ab
ab: wrong number of arguments
Usage: ab [options] [http[s]://]hostname[:port]/path
Options are:
    -n requests     Number of requests to perform
    -c concurrency  Number of multiple requests to make at a time
    -t timelimit    Seconds to max. to spend on benchmarking
                    This implies -n 50000
    -s timeout      Seconds to max. wait for each response
                    Default is 30 seconds
<-- SNIP -->

However, if I run ab and include a URL to test, I initially get an error:

$ docker run --rm ab https://jayschmidt.us
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "https://jayschmidt.us": stat https://jayschmidt.us: no such file or directory: unknown.

The issue here is that the string supplied on the command line — https://jayschmidt.us — is overriding the CMD instruction, and that is not a valid command, resulting in an error being thrown. So, we need to specify the command to run:

$ docker run --rm  ab ab https://jayschmidt.us/
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking jayschmidt.us (be patient).....done


Server Software:        nginx
Server Hostname:        jayschmidt.us
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-ECDSA-AES256-GCM-SHA384,256,256
Server Temp Key:        X25519 253 bits
TLS Server Name:        jayschmidt.us

Document Path:          /
Document Length:        12992 bytes

Concurrency Level:      1
Time taken for tests:   0.132 seconds
Complete requests:      1
Failed requests:        0
Total transferred:      13236 bytes
HTML transferred:       12992 bytes
Requests per second:    7.56 [#/sec] (mean)
Time per request:       132.270 [ms] (mean)
Time per request:       132.270 [ms] (mean, across all concurrent requests)
Transfer rate:          97.72 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       90   90   0.0     90      90
Processing:    43   43   0.0     43      43
Waiting:       43   43   0.0     43      43
Total:        132  132   0.0    132     132

Run with ENTRYPOINT

In this run, we remove the CMD ab instruction from the Dockerfile, replace it with ENTRYPOINT ["ab"], and then rebuild the image.

This is similar to but different from the CMD command — when you use ENTRYPOINT, you cannot override the command unless you use the –entrypoint flag on the docker run command. Instead, any arguments passed to docker run are treated as arguments to the ENTRYPOINT.

$ docker run --rm  ab "https://jayschmidt.us/"
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking jayschmidt.us (be patient).....done


Server Software:        nginx
Server Hostname:        jayschmidt.us
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-ECDSA-AES256-GCM-SHA384,256,256
Server Temp Key:        X25519 253 bits
TLS Server Name:        jayschmidt.us

Document Path:          /
Document Length:        12992 bytes

Concurrency Level:      1
Time taken for tests:   0.122 seconds
Complete requests:      1
Failed requests:        0
Total transferred:      13236 bytes
HTML transferred:       12992 bytes
Requests per second:    8.22 [#/sec] (mean)
Time per request:       121.709 [ms] (mean)
Time per request:       121.709 [ms] (mean, across all concurrent requests)
Transfer rate:          106.20 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       91   91   0.0     91      91
Processing:    31   31   0.0     31      31
Waiting:       31   31   0.0     31      31
Total:        122  122   0.0    122     122

What about syntax?

In the example above, we use ENTRYPOINT ["ab"] syntax to wrap the command we want to run in square brackets and quotes. However, it is possible to specify ENTRYPOINT ab (without quotes or brackets). 

Let’s see what happens when we try that.

$ docker run --rm  ab "https://jayschmidt.us/"
ab: wrong number of arguments
Usage: ab [options] [http[s]://]hostname[:port]/path
Options are:
    -n requests     Number of requests to perform
    -c concurrency  Number of multiple requests to make at a time
    -t timelimit    Seconds to max. to spend on benchmarking
                    This implies -n 50000
    -s timeout      Seconds to max. wait for each response
                    Default is 30 seconds
<-- SNIP -->

Your first thought will likely be to re-run the docker run command as we did for CMD ab above, which is giving both the executable and the argument:

$ docker run --rm ab ab "https://jayschmidt.us/"
ab: wrong number of arguments
Usage: ab [options] [http[s]://]hostname[:port]/path
Options are:
    -n requests     Number of requests to perform
    -c concurrency  Number of multiple requests to make at a time
    -t timelimit    Seconds to max. to spend on benchmarking
                    This implies -n 50000
    -s timeout      Seconds to max. wait for each response
                    Default is 30 seconds
<-- SNIP -->

This is because ENTRYPOINT can only be overridden if you explicitly add the –entrypoint argument to the docker run command. The takeaway is to always use ENTRYPOINT when you want to force the use of a given executable in the container when it is run.

Wrapping up: Key takeaways and best practices

The decision-making process involving the use of RUN, CMD, and ENTRYPOINT, along with the choice between shell and exec forms, showcases Docker’s intricate nature. Each command serves a distinct purpose in the Docker ecosystem, impacting how containers are built, operate, and interact with their environments. 

By selecting the right command and form for each specific scenario, developers can construct Docker images that are more reliable, secure, and optimized for efficiency. This level of understanding and application of Docker’s commands and their formats is crucial for fully harnessing Docker’s capabilities. Implementing these best practices ensures that applications deployed in Docker containers achieve maximum performance across various settings, enhancing development workflows and production deployments.

Learn more

Understanding the Docker USER Instruction

Par : Jay Schmidt
26 juin 2024 à 13:12

In the world of containerization, security and proper user management are crucial aspects that can significantly affect the stability and security of your applications. The USER instruction in a Dockerfile is a fundamental tool that determines which user will execute commands both during the image build process and when running the container. By default, if no USER is specified, Docker will run commands as the root user, which can pose significant security risks. 

In this blog post, we will delve into the best practices and common pitfalls associated with the USER instruction. Additionally, we will provide a hands-on demo to illustrate the importance of these practices. Understanding and correctly implementing the USER instruction is vital for maintaining secure and efficient Docker environments. Let’s explore how to manage user permissions effectively, ensuring that your Docker containers run securely and as intended.

banner docker tips

Docker Desktop 

The commands and examples provided are intended for use with Docker Desktop, which includes Docker Engine as an integrated component. Running these commands on Docker Community Edition (standalone Docker Engine) is possible, but your output may not match that shown in this post. The blog post How to Check Your Docker Installation: Docker Desktop vs. Docker Engine explains the differences and how to determine what you are using.

UID/GID: A refresher

Before we discuss best practices, let’s review UID/GID concepts and why they are important when using Docker. This relationship factors heavily into the security aspects of these best practices.

Linux and other Unix-like operating systems use a numeric identifier to identify each discrete user called a UID (user ID). Groups are identified by a GID (group ID), which is another numeric identifier. These numeric identifiers are mapped to the text strings used for username and groupname, but the numeric identifiers are used by the system internally.

The operating system uses these identifiers to manage permissions and access to system resources, files, and directories. A file or directory has ownership settings including a UID and a GID, which determine which user and group have access rights to it. Users can be members of multiple groups, which can complicate permissions management but offers flexible access control.

In Docker, these concepts of UID and GID are preserved within containers. When a Docker container is run, it can be configured to run as a specific user with a designated UID and GID. Additionally, when mounting volumes, Docker respects the UID and GID of the files and directories on the host machine, which can affect how files are accessed or modified from within the container. This adherence to Unix-like UID/GID management helps maintain consistent security and access controls across both the host and containerized environments. 

Groups

Unlike USER, there is no GROUP directive in the Dockerfile instructions. To set up a group, you specify the groupname (GID) after the username (UID). For example, to run a command as the automation user in the ci group, you would write USER automation:ci in your Dockerfile.

If you do not specify a GID, the list of groups that the user account is configured as part of is used. However, if you do specify a GID, only that GID will be used. 

Current user

Because Docker Desktop uses a virtual machine (VM), the UID/GID of your user account on the host (Linux, Mac, Windows HyperV/WSL2) will almost certainly not have a match inside the Docker VM.

You can always check your UID/GID by using the id command. For example, on my desktop, I am UID 503 with a primary GID of 20:

$ id
uid=503(jschmidt) gid=20(staff) groups=20(staff),<--SNIP-->

Best practices

Use a non-root user to limit root access

As noted above, by default Docker containers will run as UID 0, or root. This means that if the Docker container is compromised, the attacker will have host-level root access to all the resources allocated to the container. By using a non-root user, even if the attacker manages to break out of the application running in the container, they will have limited permissions if the container is running as a non-root user. 

Remember, if you don’t set a USER in your Dockerfile, the user will default to root. Always explicitly set a user, even if it’s just to make it clear who the container will run as.

Specify user by UID and GID

Usernames and groupnames can easily be changed, and different Linux distributions can assign different default values to system users and groups. By using a UID/GID you can ensure that the user is consistently identified, even if the container’s /etc/passwd file changes or is different across distributions. For example:

USER 1001:1001

Create a specific user for the application

If your application requires specific permissions, consider creating a dedicated user for your application in the Dockerfile. This can be done using the RUN command to add the user. 

Note that when we are creating a user and then switching to that user within our Dockerfile, we do not need to use the UID/GID because they are being set within the context of the image via the useradd command. Similarly, you can add a user to a group (and create a group if necessary) via the RUN command.

Ensure that the user you set has the necessary privileges to run the commands in the container. For instance, a non-root user might not have the necessary permissions to bind to ports below 1024. For example:

RUN useradd -ms /bin/bash myuser
USER myuser

Switch back to root for privileged operations

If you need to perform privileged operations in the Dockerfile after setting a non-root user, you can switch to the root user and then switch back to the non-root user once those operations are complete. This approach adheres to the principle of least privilege; only tasks that require administrator privileges are run as an administrator. Note that it is not recommended to use sudo for privilege elevation in a Dockerfile. For example:

USER root
RUN apt-get update && apt-get install -y some-package
USER myuser

Combine USER with WORKDIR

As noted above, the UID/GID used within a container applies both within the container and with the host system. This leads to two common problems:

  • Switching to a non-root user and not having permissions to read or write to the directories you wish to use (for example, trying to create a directory under / or trying to write in /root.
  • Mounting a directory from the host system and switching to a user who does not have permission to read/write to the directory or files in the mount.
USER root
RUN mkdir /app&&chown ubuntu
USER ubuntu
WORKDIR /app

Example

The following example shows you how the UID and GID behave in different scenarios depending on how you write your Dockerfile. Both examples provide output that shows the UID/GID of the running Docker container. If you are following along, you need to have a running Docker Desktop installation and a basic familiarity with the docker command.

Standard Dockerfile

Most people take this approach when they first begin using Docker; they go with the defaults and do not specify a USER.

# Use the official Ubuntu image as the base
FROM ubuntu:20.04

# Print the UID and GID
CMD sh -c "echo 'Inside Container:' && echo 'User: $(whoami) UID: $(id -u) GID: $(id -g)'"

Dockerfile with USER

This example shows how to create a user with a RUN command inside a Dockerfile and then switch to that USER.

# Use the official Ubuntu image as the base
FROM ubuntu:20.04

# Create a custom user with UID 1234 and GID 1234
RUN groupadd -g 1234 customgroup && \
    useradd -m -u 1234 -g customgroup customuser

# Switch to the custom user
USER customuser

# Set the workdir
WORKDIR /home/customuser

# Print the UID and GID
CMD sh -c "echo 'Inside Container:' && echo 'User: $(whoami) UID: $(id -u) GID: $(id -g)'"

Build the two images with:

$ docker build -t default-user-image -f Dockerfile1 .
$ docker build -t custom-user-image -f Dockerfile2 .

Default Docker image

Let’s run our first image, the one that does not provide a USER command. As you can see, the UID and GID are 0/0, so the superuser is root. There are two things at work here. First, we are not defining a UID/GID in the Dockerfile so Docker defaults to the superuser. But how does it become a superuser if my account is not a superuser account? This is because the Docker Engine runs with root permissions, so containers that are built to run as root inherit the permissions from the Docker Engine.

$ docker run --rm default-user-image
Inside Container:
User: root UID: 0 GID: 0
Custom User Docker Image

Custom Docker image

Let’s try to fix this — we really don’t want Docker containers running as root. So, in this version, we explicitly set the UID and GID for the user and group. Running this container, we see that our user is set appropriately.

$ docker run --rm custom-user-image
Inside Container:
User: customuser UID: 1234 GID: 1234

Enforcing best practices

Enforcing best practices in any environment can be challenging, and the best practices outlined in this post are no exception. Docker understands that organizations are continually balancing security and compliance against innovation and agility and is continually working on ways to help with that effort. Our Enhanced Container Isolation (ECI) offering, part of our Hardened Docker Desktop, was designed to address the problematic aspects of having containers running as root.

Enhanced Container Isolation mechanisms, such as user namespaces, help segregate and manage privileges more effectively. User namespaces isolate security-related identifiers and attributes, such as user IDs and group IDs, so that a root user inside a container does not map to the root user outside the container. This feature significantly reduces the risk of privileged escalations by ensuring that even if an attacker compromises the container, the potential damage and access scope remain confined to the containerized environment, dramatically enhancing overall security.

Additionally, Docker Scout can be leveraged on the user desktop to enforce policies not only around CVEs, but around best practices — for example, by ensuring that images run as a non-root user and contain mandatory LABELs.

Staying secure

Through this demonstration, we’ve seen the practical implications and benefits of configuring Docker containers to run as a non-root user, which is crucial for enhancing security by minimizing potential attack surfaces. As demonstrated, Docker inherently runs containers with root privileges unless specified otherwise. This default behavior can lead to significant security risks, particularly if a container becomes compromised, granting attackers potentially wide-ranging access to the host or Docker Engine.

Use custom user and group IDs

The use of custom user and group IDs showcases a more secure practice. By explicitly setting UID and GID, we limit the permissions and capabilities of the process running inside the Docker container, reducing the risks associated with privileged user access. The UID/GID defined inside the Docker container does not need to correspond to any actual user on the host system, which provides additional isolation and security.

User namespaces

Although this post extensively covers the USER instruction in Docker, another approach to secure Docker environments involves the use of namespaces, particularly user namespaces. User namespaces isolate security-related identifiers and attributes, such as user IDs and group IDs, between the host and the containers. 

With user namespaces enabled, Docker can map the user and group IDs inside a container to non-privileged IDs on the host system. This mapping ensures that even if a container’s processes break out and gain root privileges within the Docker container, they do not have root privileges on the host machine. This additional layer of security helps to prevent the escalation of privileges and mitigate potential damage, making it an essential consideration for those looking to bolster their Docker security framework further. Docker’s ECI offering leverages user namespaces as part of its security framework.

Conclusion

When deploying containers, especially in development environments or on Docker Desktop, consider the aspects of container configuration and isolation outlined in this post. Implementing the enhanced security features available in Docker Business, such as Hardened Docker Desktop with Enhanced Container Isolation, can further mitigate risks and ensure a secure, robust operational environment for your applications.

Learn more

How to Check Your Docker Installation: Docker Desktop vs. Docker Engine

Par : Jay Schmidt
20 mai 2024 à 13:11

Docker has become a leader in providing software development solutions, offering tooling that simplifies the process of developing, testing, deploying, and running applications using containers. As such, understanding Docker’s different products, like Docker Desktop, and components, like Docker Engine, and understanding how they work together is essential for developers looking to maximize their productivity and ensure compliance with Docker’s licensing terms. 

This post will clarify the distinctions and similarities between Docker Desktop and Docker Engine, and provide guidance on verifying which one you are currently using so you can make the most out of your experience.

Read on to explore how to distinguish between Docker Desktop and Docker Engine installations, identify when additional Docker tools are in use, understand Docker contexts, and review your Docker usage to ensure it complies with the Docker Licensing Agreement.

banner docker tips

Background

The word “Docker” has become synonymous with containerization to the degree that containerizing an application is increasingly referred to as “dockerizing” an application. Although Docker didn’t create containerization, the company was the first to bring this technology to the developer community in an easily understood set of tooling and artifacts. 

In 2015, Docker took the next step and created the Open Container Initiative (OCI) to define and specify how to build a container image, how to run a container image, and how to share container images. By donating the OCI to the Linux Foundation, Docker provided a level playing field for any application of container technology.

An open source effort headed up by Docker is the Moby Project. Docker created this open framework to assemble specialized container systems without reinventing the wheel. It provides a building block set of dozens of standard components and a framework for assembling them into custom platforms. 

Moby comprises several components, including a container engine, container runtime, networking, storage, and an orchestration system. Both the standalone free Docker Engine (also known Docker Community Edition or Docker CE) and the commercial Docker Desktop originated from the Moby Project. However, Docker Desktop has evolved beyond the Moby Project, with a full product team investing in the features and technology to support individual developers, small teams, and the requirements of large development teams.

Docker Engine vs. Docker Desktop

Docker Desktop is a commercial product sold and supported by Docker, Inc. It includes the Docker Engine and other open source components; proprietary components; and features like an intuitive GUI, synchronized file shares, access to cloud resources, debugging features, native host integration, governance, and security features that support ECI, air-gapped containers, and administrative settings management. To provide a consistent user experience across different operating systems, Docker Desktop uses the host systems’ native virtualization to run and manage a VM for the Docker Engine. This offers developers a turnkey solution for running a containerization toolset on any device or operating system. Users can leverage the Docker Engine at its core on any platform by downloading Docker Desktop.

Docker Engine is a component free to download individually, not as part of Docker Desktop, and runs as a standalone for free. It can run on any supported Linux distribution and includes the Docker CLI to run commands. Docker Engine will not run natively on Windows or macOS and does not come with a GUI or any of the advanced features provided by Docker Desktop.

How can I tell If I’m running Docker Desktop or just the Docker Engine?

You can determine if you’re using Docker Desktop or Docker Engine in a number of different ways. The following section provides guidance for checking from the filesystem and from within the Docker CLI tooling, which is a component in Docker Desktop as well.

1. GUI or icon

If you are using Docker Desktop, you will have either a windowed GUI or a menubar/taskbar icon of a whale (Figure 1).

Screenshot of Docker Desktop menu on macOs.
Figure 1: The macOS menu.

2. Check for an installation

The easiest way to check for Docker Desktop is to look for the installation; this can be automated by scripting or the use of an MDM solution.

Note that the following instructions assume that Docker Desktop is installed in the default location, which may result in false negatives if other installation paths are used.

Docker Desktop on macOS

On macOS, the Docker Desktop application is installed under the /Applications directory and is named Docker (Figure 2).

$ ls -alt /Applications/Docker.app/
total 0
drwxrwxr-x  49 root      admin  1568 Oct 13 09:54 ..
drwxr-xr-x@  9 jschmidt  admin   288 Sep 28 15:36 Contents
drwxr-xr-x@  3 jschmidt  admin    96 Sep  8 02:35 .
Screenshot of Docker application information on macOS, including Date Created, Modified, Version, and more.
Figure 2: Docker application installed on macOS.

Docker Desktop on Windows

On Windows, the Docker Desktop application is installed under the C:\Program Files folder and is named Docker (Figure 3).

C:\Users\qdzlug>dir "c:\Program Files\Docker"
 Volume in drive C has no label.
 Volume Serial Number is DEFE-FC15

 Directory of c:\Program Files\Docker

09/28/2023  02:22 PM    <DIR>          .
09/28/2023  02:22 PM    <DIR>          ..
09/28/2023  02:22 PM    <DIR>          cli-plugins
09/28/2023  02:21 PM    <DIR>          Docker
               0 File(s)              0 bytes
               4 Dir(s)  52,964,356,096 bytes free

C:\Users\qdzlug>
Screenshot of Docker application on Windows, showing file folder and date modified.
Figure 3: Docker application installed on Windows.

Docker Desktop on Linux

On Linux, the Docker Desktop application is installed under /opt/docker-desktop.

$ ls -lat /opt/docker-desktop/
total 208528
drwxr-xr-x 7 root root      4096 Sep 29 10:58  .
drwxr-xr-x 2 root root      4096 Sep 29 10:58  locales
drwxr-xr-x 5 root root      4096 Sep 29 10:58  resources
drwxr-xr-x 2 root root      4096 Sep 29 10:58  share
drwxr-xr-x 2 root root      4096 Sep 29 10:58  linuxkit
drwxr-xr-x 2 root root      4096 Sep 29 10:58  bin
drwxr-xr-x 7 root root      4096 Sep 29 10:57  ..
-rw-r--r-- 1 root root   5313018 Sep 27 12:10  resources.pak
-rw-r--r-- 1 root root    273328 Sep 27 12:10  snapshot_blob.bin
-rw-r--r-- 1 root root    588152 Sep 27 12:10  v8_context_snapshot.bin
-rw-r--r-- 1 root root       107 Sep 27 12:10  vk_swiftshader_icd.json
-rw-r--r-- 1 root root    127746 Sep 27 12:10  chrome_100_percent.pak
-rw-r--r-- 1 root root    179160 Sep 27 12:10  chrome_200_percent.pak
-rwxr-xr-x 1 root root   1254728 Sep 27 12:10  chrome_crashpad_handler
-rwxr-xr-x 1 root root     54256 Sep 27 12:10  chrome-sandbox
-rw-r--r-- 1 root root       398 Sep 27 12:10  componentsVersion.json
-rwxr-xr-x 1 root root 166000248 Sep 27 12:10 'Docker Desktop'
-rw-r--r-- 1 root root  10544880 Sep 27 12:10  icudtl.dat
-rwxr-xr-x 1 root root    252920 Sep 27 12:10  libEGL.so
-rwxr-xr-x 1 root root   2877248 Sep 27 12:10  libffmpeg.so
-rwxr-xr-x 1 root root   6633192 Sep 27 12:10  libGLESv2.so
-rwxr-xr-x 1 root root   4623704 Sep 27 12:10  libvk_swiftshader.so
-rwxr-xr-x 1 root root   6402632 Sep 27 12:10  libvulkan.so.1
-rw-r--r-- 1 root root      1096 Sep 27 12:10  LICENSE.electron.txt
-rw-r--r-- 1 root root   8328249 Sep 27 12:10  LICENSES.chromium.html

Note that the launch icon and location for Docker will depend on the Linux distribution being used.

3. Check a running installation

You can also check a running installation to determine which version of Docker is being used. To do this, you need to use the docker version command; the Server line will indicate the version being used.

Docker Desktop on macOS Arm64

The Server: Docker Desktop 4.24.0 (122432) line indicates using Docker Desktop.

$ docker version
Client:
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:28:49 2023
 OS/Arch:           darwin/arm64
 Context:           default

**Server: Docker Desktop 4.24.0 (122432)**
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:36 2023
  OS/Arch:          linux/arm64
  Experimental:     true
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker Desktop on Windows

The Server: Docker Desktop 4.24.0 (122432) line indicates using Docker Desktop.

C:\Users\qdzlug>docker version
Client:
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:32:48 2023
 OS/Arch:           windows/amd64
 Context:           default

**Server: Docker Desktop 4.24.0 (122432)**
 Engine:
  Version:          dev
  API version:      1.44 (minimum version 1.12)
  Go version:       go1.20.8
  Git commit:       HEAD
  Built:            Tue Sep 26 11:52:32 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

C:\Users\qdzlug>

Docker Desktop on Linux

C:\Users\qdzlug>docker info
Client:
 Version:    24.0.6
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.5
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.2.3
    Path:     C:\Users\qdzlug\.docker\cli-plugins\docker-compose.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)

Docker Engine on Linux

The Server: Docker Engine - Community line indicates using the community edition.

$ docker version
Client: Docker Engine - Community
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           default

**Server: Docker Engine - Community**
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:44 2023
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.6.24
  GitCommit:        61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc:
  Version:          1.1.9
  GitCommit:        v1.1.9-0-gccaecfc
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker contexts

Note that multiple contexts can be installed on a system; this is most often seen in Linux where Docker Desktop and Docker Engine are installed on the same host. To switch between the two, the docker context use command is used. When you are in a context, you communicate with the daemon for that context; thus, in a dual installation situation, you would be switching between the Docker Desktop install and the host install.

To view contexts, you use docker context ls, then switch via docker context use CONTEXTNAME. The following example shows a Linux system with both installed.

$ docker context ls
NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT                                     KUBERNETES ENDPOINT   ORCHESTRATOR
default *           moby                Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                                     
desktop-linux       moby                Docker Desktop                            unix:///home/jschmidt/.docker/desktop/docker.sock               
$ docker version
Client: Docker Engine - Community
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:44 2023
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.6.24
  GitCommit:        61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc:
  Version:          1.1.9
  GitCommit:        v1.1.9-0-gccaecfc
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
$ docker context use desktop-linux
desktop-linux
Current context is now "desktop-linux"
$ docker version
Client: Docker Engine - Community
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.24.0 (122432)
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:32:16 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Other OCI tooling

Because both Docker Engine and Docker Desktop are OCI compliant, a number of solutions are presented and installed as “direct replacements” for Docker. This process usually involves creating helper aliases, scripts, or batch programs to emulate docker commands. 

You can check for aliases by running the command alias docker to see if there is an alias in place. This holds true for Linux and macOS, or a Linux distribution inside WSL2 on Windows.

$ alias docker # Docker aliased to podman
docker='podman'
$ alias docker # No alias present

You can also list the docker binary from the CLI to ensure that it is the official Docker binary:

$ ls -l `which docker` # Docker supplied by Homebrew on the Mac
lrwxr-xr-x  1 jschmidt  admin  34 Apr  2 12:03 /opt/homebrew/bin/docker -> ../Cellar/docker/26.0.0/bin/docker


$ ls -l `which docker` # Official Docker binary on the Mac
lrwxr-xr-x  1 root  wheel  54 Jan 10 16:06 /usr/local/bin/docker -> /Applications/Docker.app/Contents/Resources/bin/docker

Conclusion

To wrap up our exploration, note that there are also several offerings generically referred to as “Docker” available to use as part of your containerization journey. This post focused on Docker Engine and Docker Desktop.

At this point, you should be comfortable distinguishing between a Docker Desktop installation and a Docker Engine installation and be able to identify when other OCI tooling is being used under the docker command name. You should also have a high-level understanding of Docker contexts as they relate to this topic. Finally, you should be able to review your usage against the Docker Licensing Agreement to ensure compliance or simply log in with your company credentials to get access to your procured entitlements..

Learn more

❌
❌